AI-based auxiliary device for pathological detection

Abstract

The invention relates to the technical field of medical treatment, in particular to an AI (artificial intelligence)-based auxiliary device for pathological examination, which comprises a slice acquisition module for acquiring continuous patient tissue slice images; the modeling module is used for constructing three-dimensional cell models one by one and constructing a three-dimensional tissue model; the analysis module comprises a trained convolutional neural network, and the convolutional neural network is used for inputting a patient tissue slice image and a tissue three-dimensional model to identify cell types, growth stages and contours in the patient tissue slice image and the tissue three-dimensional model; and the mechanical analysis module is used for predicting the contour change of each cell in the subsequent growth stage, performing mechanical simulation according to the contour change, and predicting the infiltration, diffusion and proliferation trends of the three-dimensional tissue model under the mechanical action. By adopting the technical scheme of the invention, the three-dimensional structure of the cell is reduced through the section, and the future change of the pathological tissue is predicted based on the mechanical action between the cells through the three-dimensional structure of the section.

Description

Technical Field

[0001] The present invention relates to the field of medical technology, and particularly to an AI-based auxiliary device for pathological detection. Background Art

[0002] AI-based pathological detection of tissue samples is an innovative method for automatically analyzing and diagnosing pathological sections through technologies such as deep learning, image processing, and big data analysis. AI processing can extract key features from a large number of pathological images and use trained models to accurately identify lesion types, grades, and stages, especially performing well in cancer diagnosis, such as breast cancer, lung cancer, and prostate cancer. Compared with traditional pathological diagnosis, AI technology has the advantages of high efficiency, consistency, and auxiliary diagnosis, can significantly improve the diagnosis speed, reduce human errors, and provide reliable second opinions for pathologists.

[0003] For example, in the prior art, Chinese Patent Publication No. CN112508850A discloses a method for detecting malignant regions of thyroid cell pathological sections based on deep learning, which improves the detection accuracy by removing false positives. CN116958666A discloses a method and system for classifying colorectal tissue pathological sections, which can judge cancer tissues by identifying subtle lesions characterized in pathological images. CN118262097B discloses a method for constructing a target detection model based on a large pathological model, which can process different types of pathological section images.

[0004] Secondly, slice cell three-dimensional construction is a new research technology. Generally, a cell is about 10 microns, and a cell slice is 4 microns. That is, three slices can construct a cell three-dimensional model. Through the construction of the cell three-dimensional model, it can provide a basis for further pathological analysis, thereby expanding more information to assist doctors in making judgments. Summary of the Invention

[0005] To solve the above problems, the present invention provides an AI-based auxiliary device for pathological detection, which is used to restore the three-dimensional structure of cells through slices and predict the future changes of pathological tissues based on the mechanical interaction between cells through the three-dimensional structure of the slices.

[0006] To achieve the above object, the technical solution of the present invention is as follows: An AI-based auxiliary device for pathological detection, comprising: A slice acquisition module: used to acquire continuous patient tissue slice images; A modeling module: used to construct three-dimensional models of each cell one by one based on continuous patient tissue slice images and restore the positions of the three-dimensional models of each cell to the patient tissue slice images to construct a tissue three-dimensional model; Analysis module: It includes a trained convolutional neural network. The convolutional neural network is trained based on labeled tissue cell image samples and tissue three-dimensional model samples of different types, growth stages, and contours. The convolutional neural network is used to input a patient's tissue section image and tissue three-dimensional model to identify the cell type, growth stage, and contour within the patient's tissue section image and tissue three-dimensional model. Mechanical analysis module: It is used to predict the contour changes of each cell in the subsequent growth stage based on the type and growth stage of each cell within the tissue three-dimensional model, and perform mechanical simulation according to the contour changes to predict the infiltration, diffusion, and proliferation trends of the tissue three-dimensional model under the action of mechanics.

[0007] The following beneficial effects can be obtained by adopting the above solution: 1. In this solution, the section acquisition module is used to obtain the tissue section image of the patient, and analyze it through the lesion tissue section of the patient to assist the pathological detection process. Conventional pathological sections are two-dimensional structures with fewer characteristic details. By performing three-dimensional reconstruction on the tissue section image through the modeling module, the number of analysis angles available can be increased, and it is beneficial to comprehensively analyze the characteristics of multiple section images combined.

[0008] 2. In this solution, the section contains normal tissue cells and cancer cells. In the prior art, normal tissue cells are often removed or determined as invalid parts to strengthen the observation of the diseased part. However, the pathological effect also includes the influence generated by the interaction between cancer cells and normal tissue cells. For example, the proliferation of cancer cells causes compression or deformation of normal cells. In addition, the growth stages of different cancer cells in the tissue section are different, that is, the development cycle of the lesion is different. It is possible to perform three-dimensional reconstruction and then based on mechanical simulation to predict the infiltration, diffusion, and proliferation trends of the tissue three-dimensional model under the action of mechanics.

[0009] Furthermore, the section acquisition module includes a converter. The converter is used to obtain the tissue section image of the patient scanned by each preset scanning software, and perform cropping, preprocessing, and conversion on the tissue section image of the patient, and output the tissue section image of the patient with a unified format.

[0010] Beneficial effect: The converter can unify the format, make the scanning results from different scanning software match, and facilitate adaptation to different scanning software.

[0011] Furthermore, the number of consecutive tissue section pictures of the patient obtained by the section acquisition module is at least 3.

[0012] Beneficial effect: Generally, a cell is about 10 microns, while a cell section is 4 microns. The number of section pictures of more than 3 sections can meet the construction requirements of most three-dimensional models.

[0013] Further, the modeling module is also used to align cells using an image registration algorithm when constructing a three-dimensional model, and the image registration algorithm is one of Elastix and ANTs.

[0014] Beneficial effects: When constructing a three-dimensional model, there may be offsets or distortions in consecutive patient tissue section images, so an image registration algorithm is needed to align the cells.

[0015] Further, the analysis module is also used to mark each cell type in the patient tissue section image after the convolutional neural network identifies the cell types in the patient tissue section image.

[0016] Beneficial effects: After being trained with tissue cell image samples of different types, growth stages, and contours that have been extensively annotated, the convolutional neural network can effectively mark the patient tissue section images to assist the user in judging various types of cells.

[0017] Further, after marking each cell type, the analysis module is also used to perform gray gradient recognition and judgment, judge the staining information of each cell according to the gray gradient value, and classify the cells according to the staining gradient.

[0018] Beneficial effects: The convolutional neural network can classify cells according to the staining condition, recognize cells under different staining conditions according to the different gray gradients generated by different staining degrees, and classify the cells according to the staining gradient.

[0019] Further, when marking each cell type, the analysis module is also used to collect the internal structure relationship and contour relationship of each cell and generate the cell fingerprint of each cell. When the modeling module performs cell alignment, it judges whether the cell fingerprints of the cells to be aligned in each consecutive patient tissue section image are paired.

[0020] Beneficial effects: When performing cell alignment, in addition to using the image registration algorithm for registration, the cell fingerprint can also be used to assist in the correction during the cell alignment process. The cell fingerprint is the internal structure relationship and contour relationship of the cell, so as to authenticate the cells in different cell sections, thereby reducing the probability of mispairing in the pairing modeling.

[0021] Further, the convolutional neural network is also trained based on patient tissue section image samples and tissue three-dimensional model samples that annotate the characteristics of disease recurrence and metastasis. The convolutional neural network is used to input the patient tissue section image and output the predicted probability of patient disease recurrence and metastasis.

[0022] Beneficial effects: The patient tissue section image can reflect the lesion condition of the patient. Therefore, the convolutional neural network is trained based on a large number of disease recurrence and metastasis characteristics, so as to feedback the probability of patient disease recurrence and metastasis according to the patient tissue section image, assisting the user in pathological analysis.

[0023] Furthermore, the convolutional neural network is also trained based on clinical information samples of different patients and treatment effects under different treatment regimens, and the convolutional neural network is used to comprehensively analyze the clinical information of patients to predict the response of patients to different treatment regimens.

[0024] Beneficial effects: The patient tissue section image can reflect the rehabilitation status of the patient under various treatment regulations. After combining with the clinical information of the patient, it can effectively analyze the suitability between the patient and the treatment regimen. Therefore, the convolutional neural network is trained based on the treatment effects under a large number of treatment regimens, so as to feedback the effect performance of the patient under various treatment regimens according to the patient tissue section image, and assist the user in pathological analysis.

[0025] Furthermore, the mechanical simulation performed by the mechanical analysis module includes the extrusion deformation generated during cell growth, the extrusion deformation generated after cell division and reproduction, the tensile force generated by cell adhesion, and the change in mechanical structure caused by cell invasion and diffusion; The mechanical analysis module is also used to input the three-dimensional tissue model sample after mechanical simulation into the convolutional neural network to predict the recurrence, metastasis probability of the patient's disease and the response to different treatment regimens under the influence of mechanical action.

[0026] Beneficial effects: The mechanical analysis module can predict the later development of the patient tissue section image, so as to expand more information that can be used for prediction. For example, the further expansion trend of cancer cells, the interaction between cancer cells and normal tissue cells, etc. The convolutional network analyzes the three-dimensional tissue model samples after the mechanical structure changes analyzed by the mechanical analysis module respectively, so as to improve the reliability of the analysis of the recurrence, metastasis probability of the patient's disease and the response to different treatment regimens.

[0027] The additional aspects and advantages of the present invention will be partially given in the following description, partially become apparent from the following description, or be understood through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a schematic diagram of the modules of an embodiment of the AI-based auxiliary device for pathological detection of the present invention; Figure 2 It is a schematic logical diagram of an embodiment of the AI-based auxiliary device for pathological detection of the present invention; Figure 3 It is a schematic logical diagram of the slice acquisition module of an embodiment of the AI-based auxiliary device for pathological detection of the present invention; Figure 4 It is a schematic diagram of the training of the convolutional neural network recognition function of an embodiment of the AI-based auxiliary device for pathological detection of the present invention; Figure 5Schematic diagram for training the convolutional neural network prediction function of the AI-based auxiliary device for pathological detection according to an embodiment of the present invention. Detailed implementation manners

[0029] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0030] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. is based on the orientation or positional relationship shown in the accompanying drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. In addition, the terms "first", "second", and "third" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance.

[0031] In the description of the present invention, it should be noted that unless otherwise clearly defined and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0032] The following is a further detailed description through specific implementation manners: Embodiment 1:

[0033] As shown in the Figures 1 - 5 accompanying drawings: An AI-based auxiliary device for pathological detection includes: A slice acquisition module: used to acquire continuous patient tissue slice images, and the patient tissue slice images can be HE slices and immunohistochemical slices. HE slices are used to clearly show the basic structures and morphological features of cells and tissues. Immunohistochemical slices are used to label specific molecules in tissues or cells with specific antibodies, thereby reflecting the presence and expression levels of specific proteins, and can assist in the diagnosis, staging, and prognosis judgment of certain tumor types.

[0034] The slice acquisition module includes a converter, which is used to obtain the patient tissue slice images scanned by each preset scanning software, and crop, preprocess, and convert the patient tissue slice images to output patient tissue slice images with a unified format. The number of consecutive patient tissue slice pictures in the slice acquisition module is at least 3.

[0035] Modeling module: It is used to construct three-dimensional cell models of each cell one by one based on consecutive patient tissue slice images. After the patient tissue slice images are imported, the internal structural relationships and contour relationships of each cell are collected respectively, and the cell fingerprints of each cell are generated. The cell fingerprints can be the relative position of the cell nucleus within the cell wall, the proportion of the cell nucleus within the cell, the wrinkled deformation form of the cell wall, etc. When the modeling module aligns cells, it judges whether the cell fingerprints of the cells to be aligned in each consecutive patient tissue slice image are paired. When the proportion of unpaired cell fingerprints is greater than a preset percentage, an alarm is issued. The image registration algorithm is used to align the cells. The image registration algorithm is one of Elastix and ANTs, and the positions of the three-dimensional cell models are restored to the patient tissue slice images to construct a three-dimensional tissue model. Elastix supports elastic registration and can handle the non-linear deformation between slices. Cells may be deformed during the slicing process due to various reasons such as slice thickness and slice angle. Through elastic registration, these deformations can be corrected, which is convenient for cell reconstruction. ANTs provides a variety of registration algorithms, such as rigid registration, affine registration, and deformable registration, etc., which can align consecutive slices into a common reference frame to construct an accurate three-dimensional cell model.

[0036] Analysis module: It includes a trained convolutional neural network. The number of training samples of the convolutional neural network is greater than 800, and the learning rate of the convolutional neural network is between 0.2 and 0.5. The convolutional neural network is trained based on labeled tissue cell picture samples and three-dimensional tissue model samples of different types, growth stages, and contours. The convolutional neural network is used to input patient tissue slice images and three-dimensional tissue models to identify the cell types, growth stages, and contours within the patient tissue slice images and three-dimensional tissue models. The cell types include tumor cells, immune cells, stromal cells, etc.

[0037] Mechanical analysis module: It is used to predict the contour changes of each cell in the subsequent growth stage based on the types and growth stages of the cells within the three-dimensional tissue model, and perform mechanical simulation according to the contour changes to predict the infiltration, diffusion, and proliferation trends of the three-dimensional tissue model under mechanical action; in the mechanical simulation, each cell is regarded as an individual with material properties, and the material properties of the individual are set according to the different cell types, and the individual shape is based on the three-dimensional cell model. In the mechanical simulation, the simulation content includes the extrusion deformation generated during cell growth, the extrusion deformation generated after cell division and reproduction, the tensile force generated by cell adhesion, and the mechanical structure change caused by cell invasion and diffusion.

[0038] The slice acquisition module is used to obtain tissue slices of a patient, and analyze them through the tissue slices of the patient's lesion to assist the pathological detection process. Conventional pathological slices are two-dimensional structures with few characteristic details. By performing three-dimensional reconstruction on the tissue slice images through the modeling module, the number of analysis angles can be increased, and it is beneficial to comprehensively analyze the characteristics of multiple slice images combined. The converter can unify the format to make the scanning results from different scanning software match, facilitating adaptation to different scanning software for centralized analysis of the patient's tissue slice images. Generally, a single cell is about 10 micrometers, while a cell slice is 4 micrometers. The number of slice images with more than 3 slices can meet the requirements of most three-dimensional model construction.

[0039] During the construction of the three-dimensional model, offsets or distortions may occur in the continuous tissue slice images of the patient. Therefore, it is necessary to use image registration algorithms to align the cells. Elastix and ANTs can adapt to the contours in different slice images to complete slice pairing. However, there may still be cases of mispairing and dislocation. Therefore, in addition to using image registration algorithms for registration, cell fingerprints can also be used to assist in the correction during cell alignment. Cell fingerprints refer to the internal structural relationships and contour relationships of cells, which can identify the cells in different cell slices, thereby reducing the probability of mispairing during paired modeling. Cell fingerprints are the natural relationships of cells. During different slice production processes, since the internal structures of cells are all subject to the same slice production and mechanical relationships, the changes in their internal structural relationships and contour relationships under the same effects are similar. Therefore, cell fingerprints can provide verification capabilities in different slice images.

[0040] The slices contain normal tissue cells and cancer cells. In the prior art, the normal tissue cells are often removed or determined as invalid parts to enhance the observation of the diseased parts. However, the pathological effects also include the impacts generated by the interaction between cancer cells and normal tissue cells. For example, the proliferation of cancer cells compresses or deforms normal cells. When the slice type obtained by the slice acquisition module is an immunohistochemical slice, it can label the antigens in tumor cells through specific antibodies to improve the recognition efficiency of cancer cells, and display the process of cancer cells breaking through the basement membrane or infiltrating into surrounding tissues by labeling epithelial markers or basement membrane components, and label the Ki-67 protein to feedback the proliferation activity of cancer cells. In addition, different cancer cells in the tissue slices are in different growth stages, that is, the development cycle of the lesion is different. It is possible to predict the infiltration, diffusion, and proliferation trends of the three-dimensional tissue model under mechanical action through three-dimensional reconstruction and then based on mechanical simulation.

[0041] Example 2:

[0042] The differences from the above embodiments are as follows: After the convolutional neural network in the analysis module identifies the cell types in the patient's tissue section image, it marks each cell type within the patient's tissue section image. The convolutional neural network is based on a large number of training samples, records and stores the distinguishing features of each type of cell, and records the mapping relationship, so as to complete the identification of cell types using the distinguishing features. During the training of cell type identification, the training sample images can be rotated or mirrored to construct more training samples and improve the stability of the distinguishing feature recognition process.

[0043] After marking each cell type in the analysis module, it performs recognition and judgment on the gray gradient, determines the staining information of each cell according to the gray gradient value, and classifies the cells according to the staining gradient.

[0044] The convolutional neural network is also trained based on the patient tissue section image samples and tissue three-dimensional model samples labeled with disease recurrence and metastasis characteristics. The convolutional neural network is used to input the patient tissue section image and output the predicted probability of the patient's disease recurrence and metastasis.

[0045] After being trained with a large number of tissue cell picture samples of different types, growth stages, and contours after annotation, the convolutional neural network can effectively mark the patient's tissue section image to assist the user in judging various types of cells. The convolutional neural network can classify cells according to the staining condition, recognize cells under different staining conditions according to the different gray gradients generated by different staining degrees, and classify cells according to the staining gradient.

[0046] The patient tissue section image can reflect the lesion condition of the patient. Therefore, the convolutional neural network is trained based on a large number of disease recurrence and metastasis characteristics, so as to feedback the probability of the patient's disease recurrence and metastasis according to the patient tissue section image and assist the user in pathological analysis.

[0047] The convolutional neural network is also trained based on the clinical information samples of different patients and the treatment effects under different treatment plans. The convolutional neural network is used to comprehensively predict the patient's response to different treatment plans based on the patient's clinical information. The treatment plans include targeted drug selection, photothermal therapy, radiotherapy, chemotherapy plans, etc. The convolutional neural network learns the clinical information within various treatment plans, records the performance of patients in different situations under various treatment plans, so as to establish a mapping relationship. By inputting the patient's pathological section image and clinical information such as medical history, physical signs, age, etc., it predicts the pathological changes of the patient after applying different treatment plans, so as to assist the physician in evaluating and selecting appropriate treatment plans.

[0048] Patient tissue section images can reflect the patient's recovery status under various treatment regulations. After combining with the patient's clinical information, the adaptability of the patient and the treatment plan can be effectively analyzed. Therefore, the convolutional neural network is trained based on the treatment effects under a large number of treatment plans, so as to feedback the performance of the patient under various treatment plans according to the patient tissue section image, and assist the user in pathological analysis.

[0049] Example 3:

[0050] The difference from the above embodiment is that the mechanical analysis module is also used to input the three-dimensional tissue model sample after mechanical simulation into the convolutional neural network to predict the recurrence, metastasis probability of the patient's disease and the response to different treatment plans under the influence of mechanical action.

[0051] The mechanical analysis module can predict the later development of the patient tissue section image, so as to expand more information that can be used for prediction. For example, the further expansion trend of cancer cells, the interaction between cancer cells and normal tissue cells, etc. The convolutional network analyzes the three-dimensional tissue model sample after the mechanical structure change analyzed by the mechanical analysis module respectively, so as to improve the reliability of the analysis of the recurrence, metastasis probability of the patient's disease and the response to different treatment plans.

[0052] Obviously, the above embodiments are only examples for clear illustration, rather than limitations on the implementation manners. For those of ordinary skill in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation manners here. And the obvious changes or modifications derived therefrom are still within the protection scope of the present invention.

Claims

1. An AI-based auxiliary device for pathological detection, characterized in that, Including: Slice acquisition module: used to acquire consecutive patient tissue slice images; Modeling module: used to construct three-dimensional cell models one by one based on consecutive patient tissue slice images, and restore the positions of the three-dimensional cell models to the patient tissue slice images to construct a three-dimensional tissue model; Analysis module: including a trained convolutional neural network, which is trained based on labeled tissue cell picture samples and three-dimensional tissue model samples of different types, growth stages, and contours. The convolutional neural network is used to input patient tissue slice images and three-dimensional tissue models to identify cell types, growth stages, and contours in the patient tissue slice images and three-dimensional tissue models; Mechanical analysis module: used to predict the contour changes of each cell in the subsequent growth stage based on the types and growth stages of the cells in the three-dimensional tissue model, and perform mechanical simulation according to the contour changes to predict the infiltration, diffusion, and proliferation trends of the three-dimensional tissue model under mechanical action.

2. The auxiliary device for pathological detection based on AI according to claim 1, wherein, The slice acquisition module includes a converter, which is used to acquire patient tissue slice images scanned by each preset scanning software, and crop, preprocess, and convert the patient tissue slice images to output patient tissue slice images with a unified format.

3. The auxiliary device for pathological detection based on AI according to claim 2, characterized in that, The number of consecutive patient tissue slice pictures acquired by the slice acquisition module is at least 3.

4. The auxiliary device for pathological detection based on AI according to claim 3, characterized in that, The modeling module is also used to align the cells using an image registration algorithm when constructing the three-dimensional model, and the image registration algorithm is one of Elastix and ANTs.

5. The auxiliary device for AI-based pathological detection according to claim 4, characterized in that The analysis module is also used to mark each cell type in the patient tissue slice picture after the convolutional neural network identifies the cell type in the patient tissue slice picture.

6. The auxiliary device for pathological detection based on AI according to claim 5, wherein The analysis module is also used to perform gray gradient recognition and judgment after marking each cell type, judge the staining information of each cell according to the gray gradient value, and classify the cells according to the staining gradient.

7. The auxiliary device for AI-based pathological detection according to claim 6, characterized in that, The analysis module is also used to collect the internal structural relationship and contour relationship of each cell when marking each cell type, and generate a cell fingerprint for each cell. When the modeling module aligns the cells, it judges whether the cell fingerprints of the cells to be aligned in each consecutive patient tissue slice image are paired.

8. The auxiliary device for AI-based pathological detection according to claim 7, characterized in that, The convolutional neural network is also trained based on patient tissue slice image samples and three-dimensional tissue model samples labeled with disease recurrence and metastasis characteristics. The convolutional neural network is used to input patient tissue slice images to output the predicted probability of patient disease recurrence and metastasis.

9. The auxiliary device for AI-based pathological detection according to claim 8, characterized in that, The convolutional neural network is also trained based on clinical information samples of different patients and treatment effects under different treatment plans. The convolutional neural network is used to comprehensively predict the patient's response to different treatment plans based on the patient's clinical information.

10. The auxiliary device for pathological detection based on AI according to claim 9, characterized in that, The mechanical simulation performed by the mechanical analysis module includes the extrusion deformation generated during cell growth, the extrusion deformation generated after cell division and reproduction, the tensile force generated by cell adhesion, and the change in mechanical structure caused by cell invasion and diffusion; The mechanical analysis module is also used to input the three-dimensional tissue model sample after mechanical simulation into the convolutional neural network to predict the probability of patient disease recurrence, metastasis, and the response to different treatment plans under the influence of mechanical action.

Images