Self-supervised cancer detection system based on staining transformation
By using a self-supervised learning method based on staining transformation in pathological analysis, the feature extractor and cancer discriminant classification model are trained, and the time-consuming and labor-intensive problem of artificial pathological analysis is solved, achieving rapid and accurate cancer detection.
Patent Information
- Application Number
- CN202411730234.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-29
- Publication Date
- 2025-04-22
AI Technical Summary
In the prior art, it is time-consuming and labor-intensive to manually perform pathological analysis to detect cancer, and it is difficult to effectively perform pathological analysis to accurately detect cancer in patients.
The self-supervised learning method based on staining transformation is adopted to obtain the pathological original slice image through the image acquisition module. The feature recognition module uses the self-supervised learning method of color transformation to train the feature extractor. The annotation module inputs the pathological feature data and the original image into the cancer discrimination classification model, generates the pathological image after labeling, and performs graphical processing through the auxiliary judgment module to generate auxiliary diagnosis and treatment information.
Effective cancer detection of pathological original images is achieved, and pathological analysis can be performed quickly and accurately, improving the efficiency and accuracy of cancer detection.
Smart Images
Figure CN119863643A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of pathological analysis, and in particular to a self-supervised cancer detection system based on staining transformation. Background Art
[0002] Cancer, also known as malignant tumor, is a "killer" that seriously threatens human life and health.
[0003] Pathological analysis is considered to be the most reliable method for detecting various cancers. However, in actual clinical work, pathological analysis is usually performed manually, which is time-consuming and laborious. How to effectively perform pathological analysis to detect cancer in patients is a long-standing technical problem in this field. Therefore, it is necessary to improve the existing technology and solve this technical problem. Summary of the invention
[0004] The present invention provides a self-supervised cancer detection system based on staining transformation, which can effectively perform pathological analysis to detect cancer in patients
[0005] A self-supervised cancer detection system based on coloring transformation, comprising:
[0006] Image acquisition module: used to obtain original pathological slice images;
[0007] Feature recognition module: used to input the original pathological slice image into the trained feature extractor to determine the pathological feature data of the original pathological slice image; wherein,
[0008] The feature extractor is trained based on a self-supervised learning method of color transformation;
[0009] Annotation module: used to input pathological feature data and original pathological slice images into the trained cancer discrimination classification model to generate annotated pathological images;
[0010] The annotated pathological image includes a pathological data layer and a pathological display layer;
[0011] Auxiliary judgment module: used to graph the annotated pathological images and generate auxiliary diagnosis and treatment information.
[0012] Preferably, the training steps of the feature extractor are as follows:
[0013] A feature recognition model is preset, and the original pathological slice image is processed by the feature recognition model to determine the pathological features;
[0014] Preset the slice image recognition magnification, and perform image segmentation on the original pathological slice image according to the pathological characteristics to obtain a number of polygonal segmentation images;
[0015] The color of several polygon segmentation images is transformed, and the segmentation images with different dyeing are compared and trained to generate a feature extractor;
[0016] SimCLR is used for contrastive learning training.
[0017] Preferably, the preset slice image recognition magnification also includes:
[0018] Under a preset magnification, a sliding window with a polygonal shape is obtained;
[0019] According to the sliding window, several polygon segmentation images are obtained in the pathological image; among them,
[0020] The shape of the polygon segmentation image is adapted to the shape of the sliding window, and the size of the polygon segmentation image is adapted to the size of the sliding window.
[0021] Preferably, the slice image recognition magnification is greater than 10 times and less than 30 times, the polygon is a regular polygon, and the size of the sliding window is 224×224 pixels.
[0022] Preferably, the feature recognition module further includes:
[0023] A transformation unit, used for changing the brightness and / or contrast and / or saturation of the polygon segmentation image to obtain two transformed images;
[0024] The prediction unit is used to predict whether the two transformed images come from the same original image, so as to learn the characteristic information of the original image; wherein,
[0025] The feature information of the original image is feature information that is unrelated to the dyeing characteristics.
[0026] Preferably, the feature recognition module further includes:
[0027] A learning unit that takes different color-transformed views of the same original image as positive sample pairs to construct a self-supervised learning task;
[0028] The training unit is used to train the feature extractor by maximizing the similarity of positive sample pairs and minimizing the similarity of negative sample pairs.
[0029] Preferably, the feature recognition module further includes:
[0030] A fusion unit is used to cascade the feature vectors of the corresponding feature extraction modules to obtain a feature extractor, wherein,
[0031] Each feature extraction module converts the input image into its corresponding feature vector, and forms a fused feature vector by connecting each feature vector in sequence;
[0032] The feature extractor is constructed from the fused feature vector.
[0033] Preferably, the marking module further includes:
[0034] The classification model submodule is used to train a cancer discrimination classification model based on the feature extractor and cancer annotation information of the original pathological image;
[0035] A feature vector submodule, used for inputting the pathological image to be detected into the feature extractor to extract the feature vector of the pathological image to be detected;
[0036] The cancer judgment submodule is used to input the feature vector of the pathological image to be detected into the cancer discrimination classification model to determine whether the pathological image to be detected indicates cancer.
[0037] Preferably, the system further comprises:
[0038] An annotation module, used to provide cancer annotation information of pathological images to train cancer discrimination and classification models;
[0039] A visualization module is used to display the detection results output by the cancer discrimination and classification model in a graphical form;
[0040] A diagnosis report module is used to generate a pathology diagnosis report, which includes the test results and / or the pathology image.
[0041] Preferably, the graphing further comprises:
[0042] The substrate docking technology is used to form a trigger display cursor port at the bottom of the annotated pathological image; wherein,
[0043] The trigger display cursor port includes a trigger port and a spherical suspension ball trigger process;
[0044] With the trigger port as the trigger end, when the user points to any pathological area of the annotated pathological image with the mouse cursor, the corresponding pathological annotation is triggered;
[0045] According to the triggered pathological annotation, the corresponding triggering end retrieves the pathological information of the area currently indicated by the cursor from the pathological data layer, and displays the pathological information through the floating ball.
[0046] The beneficial effects of the present invention are:
[0047] By acquiring a pathological image and performing a color transformation on the original pathological image for comparative learning, a feature extractor is trained, and a cancer discrimination classification model is trained based on the feature extractor and cancer annotation information of the original pathological image to achieve cancer detection in the original pathological image, so that pathological analysis can be effectively performed to detect cancer in patients.
[0048] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.
[0049] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:
[0051] Figure 1 1 is a system composition diagram of a self-supervised cancer detection system based on coloring transformation in an embodiment of the present invention;
[0052] Figure 2 is a training flow chart of the training steps of the feature extractor in an embodiment of the present invention;
[0053] Figure 3 It is a graphical flow chart in an embodiment of the present invention. DETAILED DESCRIPTION
[0054] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.
[0055] Pathological images are images obtained by observing pathological sections under a microscope. Their purpose is to understand the occurrence and development of the disease at the tissue and cellular levels.
[0056] Intelligent pathology is the development direction of modern pathology, which uses computer vision and artificial intelligence technology to assist pathologists in diagnosis and improve the accuracy and efficiency of diagnosis.
[0057] Self-supervised learning is a learning method that does not require a large amount of labeled data. It can learn useful feature representations from unlabeled data and has potential help in the development of intelligent pathology. At present, the mainstream self-supervised learning methods include: predicting future frames (such as Temporal Contrastive Learning), contrastive learning (such as Momentum Contrast, SimCLR), autoencoders (such as Denoising Autoencoder), generative adversarial networks (such as BiGAN), etc. These methods have achieved remarkable results in their respective tasks and provide strong support for the development of self-supervised learning.
[0058] A self-supervised cancer detection system based on coloring transformation, comprising:
[0059] Image acquisition module: used to obtain original pathological slice images;
[0060] Feature recognition module: used to input the original pathological slice image into the trained feature extractor to determine the pathological feature data of the original pathological slice image; wherein,
[0061] The feature extractor is trained based on a self-supervised learning method of color transformation;
[0062] Annotation module: used to input pathological feature data and original pathological slice images into the trained cancer discrimination classification model to generate annotated pathological images;
[0063] The annotated pathological image includes a pathological data layer and a pathological display layer;
[0064] Auxiliary judgment module: used to graph the annotated pathological images and generate auxiliary diagnosis and treatment information.
[0065] The principle of the above technical solution is:
[0066] As attached Figure 1 As shown, the pathological image of the present application can be acquired by a pre-configured image acquisition module. The image acquisition module can be a digital pathology scanner, etc.
[0067] The feature recognition module extracts pathological information based on a pre-trained pathological feature recognition model and analyzes the specific pathological feature data of the original pathological slices.
[0068] The annotation module is used to specifically annotate the pathological information and annotate the specific disease distribution information of different pathological areas, so as to realize the display of pathological information. The annotated pathological image includes a pathological data layer and a pathological display layer; the pathological data layer is used to store the pathological data information of different annotated points, including disease symptoms, cell display status, affected areas, pathological distribution range, etc.;
[0069] The auxiliary judgment module is used to graph the annotated pathological images to assist doctors in viewing the annotated pathological images graphically and converting them into information for auxiliary diagnosis and treatment.
[0070] The beneficial effects of the above technical solution are:
[0071] By acquiring a pathological image and performing a color transformation on the original pathological image for comparative learning, a feature extractor is trained, and a cancer discrimination classification model is trained based on the feature extractor and cancer annotation information of the original pathological image to achieve cancer detection in the original pathological image, so that pathological analysis can be effectively performed to detect cancer in patients.
[0072] Preferably, the training steps of the feature extractor are as follows:
[0073] A feature recognition model is preset, and the original pathological slice image is processed by the feature recognition model to determine the pathological features;
[0074] Preset the slice image recognition magnification, and perform image segmentation on the original pathological slice image according to the pathological characteristics to obtain a number of polygonal segmentation images;
[0075] The color of several polygon segmentation images is transformed, and the segmentation images with different dyeing are compared and trained to generate a feature extractor;
[0076] SimCLR is used for contrastive learning training.
[0077] The principle of the above technical solution is:
[0078] As attached Figure 2 As shown, the feature recognition model of the present application is a pre-trained recognition model, which can perform recognition processing on the original pathological slice image, so as to determine the pathological features of the original pathological slice image; in order to more clearly judge the specific pathological condition, the recognition magnification is switched, and the pathological image is characterized and segmented through a high recognition magnification, so as to determine a number of polygonal segmented images, each segmented area represents a pathological feature, thereby realizing the construction of a feature extractor. The image recognition magnification. It can be 10 times, 20 times or 30 times, with 20 times being preferred. The regular polygon can be a square, a regular hexagon or an equilateral triangle, with a square being preferred. For example only, the size of the sliding window can be 224×224 pixels. The goal of SimCLR is to learn effective visual representations without human supervision, which comes from the paper "A Simple Framework for Contrastive Learning of Visual Representations".
[0079] The beneficial effects of the above technical solution are:
[0080] This application can segment images, quickly identify and train different pathological features, and enhance the results of feature extraction.
[0081] Preferably, the preset slice image recognition magnification also includes:
[0082] Under a preset magnification, a sliding window with a polygonal shape is obtained;
[0083] According to the sliding window, several polygon segmentation images are obtained in the pathological image; among them,
[0084] The shape of the polygon segmentation image is adapted to the shape of the sliding window, and the size of the polygon segmentation image is adapted to the size of the sliding window.
[0085] The principle of the above technical solution is:
[0086] It is used to obtain a plurality of polygon segmentation images in the pathological image according to the sliding window, wherein the shapes of the polygon segmentation images are adapted to the shape of the sliding window, and the sizes of the polygon segmentation images are adapted to the size of the sliding window.
[0087] The beneficial effects of the above technical solution are:
[0088] The present application can realize polygon segmentation of the image through sliding window processing, and adapt to the cutting of the image to a specific size.
[0089] Preferably, the slice image recognition magnification is greater than 10 times and less than 30 times, the polygon is a regular polygon, and the size of the sliding window is 224×224 pixels.
[0090] Preferably, the feature recognition module further includes:
[0091] A transformation unit, used for changing the brightness and / or contrast and / or saturation of the polygon segmentation image to obtain two transformed images;
[0092] The prediction unit is used to predict whether the two transformed images come from the same original image, so as to learn the characteristic information of the original image; wherein,
[0093] The feature information of the original image is feature information that is unrelated to the dyeing characteristics.
[0094] The principle of the above technical solution is:
[0095] The brightness and / or contrast and / or saturation of the polygon segmented image is changed to obtain two transformed images. Specifically, the two transformed images can be obtained by changing the brightness of the original image; the two transformed images can be obtained by changing the contrast of the original image; the two transformed images can be obtained by changing the saturation of the original image; the two transformed images can be obtained by changing the brightness and contrast of the original image; the two transformed images can be obtained by changing the brightness and saturation of the original image; the two transformed images can be obtained by changing the contrast and saturation of the original image; preferably, the two transformed images can be obtained by changing the brightness, contrast and saturation of the original image.
[0096] Predict whether the two transformed images come from the same original image to learn the feature information of the original image. The feature information of the original image is feature information that is unrelated to the staining characteristics. The transformed images are actually different perspectives of the same image and should have similar features. In this way, important features in the original image, i.e., feature information, can be learned, which can be used for various tasks, such as image classification, object detection, etc.
[0097] Preferably, the feature recognition module further includes:
[0098] A learning unit that takes different color-transformed views of the same original image as positive sample pairs to construct a self-supervised learning task;
[0099] The training unit is used to train the feature extractor by maximizing the similarity of positive sample pairs and minimizing the similarity of negative sample pairs.
[0100] The principle of the above technical solution is:
[0101] The learning unit is used to take different color transformed perspectives of the same original image as positive sample pairs to construct a self-supervised learning task. For each original image, multiple different versions can be obtained by changing the color, angle, flipping and other transformations. These transformed versions can be used as different perspectives of the same original image and should have similar features. Therefore, these transformed versions are used as positive sample pairs, that is, they have similar features, so the distance between them should be as close as possible. In this way, the important features in the image can be better learned, because similar features need to be extracted from multiple transformed versions. This self-supervised learning task can help the model generalize better to new data, thereby improving the performance of the model. It is possible to learn image content information that is independent of the staining characteristics, that is, the feature information of the original image is feature information that is independent of the staining characteristics.
[0102] The training unit is used to train the feature extractor by maximizing the similarity of the positive sample pairs and minimizing the similarity of the negative sample pairs. This process can be achieved through a contrast loss function. The contrast loss function is a binary contrast loss that compares a positive sample pair and a negative sample pair. The positive sample pair is composed of different transformed versions of the same original image, while the negative sample pair is composed of different transformed versions of different original images. During the training process, the feature extractor attempts to maximize the similarity of each positive sample pair, that is, to make their respective feature vectors as close as possible. At the same time, the feature extractor also attempts to minimize the similarity of each negative sample pair, that is, to make their respective feature vectors as far away as possible. In this way, the feature extractor can learn how to distinguish different images and extract more meaningful features. Specifically, the loss function includes two parts: one is the contrast loss of the positive sample pair, and the other is the contrast loss of the negative sample pair. During the training process, both losses will be calculated and used to update the parameters of the model. In this way, a more effective feature extractor can be trained so that the model can achieve better performance in various tasks.
[0103] Preferably, the feature recognition module further includes:
[0104] A fusion unit is used to cascade the feature vectors of the corresponding feature extraction modules to obtain a feature extractor, wherein,
[0105] Each feature extraction module converts the input image into its corresponding feature vector, and forms a fused feature vector by connecting each feature vector in sequence;
[0106] The feature extractor is constructed from the fused feature vector.
[0107] The principle of the above technical solution is:
[0108] The method is used to cascade the feature vectors of the feature extraction units corresponding to each other to obtain the feature extractor. Each of the feature extraction modules converts the input image into a feature vector corresponding to each other, and the feature extractor is composed of the fused feature vectors by sequentially connecting the feature vectors. In other words, for each feature extractor, the input image can be converted into a series of feature vectors, which represent various features of the image. Then, these feature vectors are connected one by one to form a longer fused feature vector, which contains the feature information extracted by all feature extractors. In this way, a more comprehensive and accurate image feature representation can be obtained, which can be used for various image processing tasks, such as image classification, target detection, etc. At the same time, by fusing the feature information of multiple feature extractors, the generalization ability and robustness of the model can be improved, so that the model can perform more stably and reliably when facing different tasks and data. According to the fused feature extractor and the cancer annotation information of the pathological image, a cancer discrimination classification model is trained to achieve cancer detection of the pathological image. That is, a fusion feature extractor can be used to extract features from pathological images, and then these features and known cancer annotation information can be used to train a cancer discrimination classification model. This model can learn how to classify cancer based on images.
[0109] Preferably, the marking module further includes:
[0110] The classification model submodule is used to train a cancer discrimination classification model based on the feature extractor and cancer annotation information of the original pathological image;
[0111] A feature vector submodule, used for inputting the pathological image to be detected into the feature extractor to extract the feature vector of the pathological image to be detected;
[0112] The cancer judgment submodule is used to input the feature vector of the pathological image to be detected into the cancer discrimination classification model to determine whether the pathological image to be detected indicates cancer.
[0113] Preferably, the system further comprises:
[0114] An annotation module, used to provide cancer annotation information of pathological images to train cancer discrimination and classification models;
[0115] A visualization module is used to display the detection results output by the cancer discrimination and classification model in a graphical form;
[0116] A diagnosis report module is used to generate a pathology diagnosis report, which includes the test results and / or the pathology image.
[0117] Preferably, the graphing further comprises:
[0118] The substrate docking technology is used to form a trigger display cursor port at the bottom of the annotated pathological image; wherein,
[0119] The trigger display cursor port includes a trigger port and a spherical suspension ball trigger process;
[0120] With the trigger port as the trigger end, when the user points to any pathological area of the annotated pathological image with the mouse cursor, the corresponding pathological annotation is triggered;
[0121] According to the triggered pathological annotation, the corresponding triggering end retrieves the pathological information of the area currently indicated by the cursor from the pathological data layer, and displays the pathological information through the floating ball.
[0122] The principle of the above technical solution is:
[0123] As attached Figure 3 As shown in the figure, when doctors in hospitals observe the pathological images of users, they mainly use their eyes to observe the specific human body information given by the system. In addition, some pathological images require professional analysis. Therefore, in terms of calling information, the corresponding pathological analysis information is called by judging the specific trigger port. The pathological analysis information is displayed through the floating ball, so doctors do not need to call specific pathological images.
[0124] It is a fast pathology information calling mechanism, which can not only realize the fast output of pathology information, but also realize the floating ball display of pathology information.
[0125] When the present invention is specifically implemented:
[0126] The feature vector submodule is used to input the pathological image to be detected into the feature extractor to extract the feature vector of the pathological image to be detected. This submodule extracts features of the pathological image to be detected through the feature extractor, extracts the key features in the image, and uses these feature vectors for subsequent classification and discrimination tasks. This feature extraction process can use deep learning technology to perform operations such as convolution and pooling on the image through a neural network model to extract local and global features in the image. The extracted feature vector can be used for a variety of tasks, such as classification, regression, clustering, etc. In this case, the feature vector is used to train and test a cancer discrimination classification model to predict whether a new pathological image represents cancer. It should be noted that the performance of the feature vector submodule depends on the design of the feature extractor and the quantity and quality of the training data. In order to improve the accuracy of the model, an appropriate feature extractor and an optimized training strategy can be selected.
[0127] The cancer judgment submodule is used to input the feature vector of the pathological image to be detected into the cancer discrimination classification model to determine whether the pathological image to be detected indicates cancer. The cancer discrimination classification model can be a model based on deep learning, using a convolutional neural network (CNN) or other types of neural network structures. The model is trained with a large amount of labeled data, and can judge whether a new pathological image indicates cancer based on the input feature vector. The judgment result is usually a probability value, indicating the probability that the image is a cancer image. If the probability exceeds a predetermined threshold, the image is judged to be a cancer image; otherwise, it is judged to be a non-cancer image. It should be noted that the performance and accuracy of the cancer judgment submodule depend on the design of the cancer discrimination classification model, the quantity and quality of the training data, and the optimization and adjustment of the model. In order to improve the accuracy of the judgment, appropriate models and optimization strategies can be selected, and sufficient testing and verification can be carried out.
[0128] Specifically, the pathological image is first input into the fusion feature extractor to extract the feature vector of the image. These feature vectors are then sent as input to the cancer discrimination classification model. The model will calculate based on these feature vectors and output a prediction result: whether this image contains cancer. In this way, an efficient cancer detection model can be trained to help doctors make accurate diagnoses and treatments.
[0129] The cancer annotation information of the pathological image may be obtained through a pre-configured annotation module, which is used to provide the cancer annotation information of the pathological image to train a cancer discrimination classification model.
[0130] In the self-supervised cancer detection system based on color transformation of the present application, a pathological image is acquired and the original pathological image is subjected to color transformation for contrast learning, a feature extractor is trained, and a cancer discrimination classification model is trained based on the feature extractor and the cancer annotation information of the original pathological image to achieve cancer detection of the original pathological image, and pathological analysis can be effectively performed to detect cancer in patients. Furthermore, by segmenting the pathological image of a preset magnification to obtain a plurality of polygonal segmented images, and color transformation is performed on each of the polygonal segmented images for contrast learning, and the corresponding feature extractors are trained, and a fused feature extractor is obtained based on the corresponding feature extractors, and a cancer discrimination classification model is trained based on the fused feature extractor and the cancer annotation information of the pathological image to achieve cancer detection of the pathological image, with better detection effect.
[0131] The self-supervised cancer detection system based on color transformation can also include:
[0132] The annotation module is used to provide cancer annotation information of the pathological image to train the cancer discrimination classification model. Specifically, the pathological image can be pre-annotated with corresponding cancers manually. In other words, the cancer annotation information of the pathological image can be information pre-configured manually. It should be pointed out that the cancer annotation information of the pathological image is not limited to being pre-configured manually, but can also be pre-configured in other ways, such as information provided by a third-party organization.
[0133] The detection results output by the cancer discrimination classification model are displayed in a graphical form. Specifically, the detection results output by the cancer discrimination classification model can be displayed in a graphical form on a human-computer interface, such as an interface of a display device. The graphical form can be text, or a table, or a picture, or a combination of text and a table, or a combination of text and a picture, or a combination of text, a table, and a picture.
[0134] Generate a pathological diagnosis report, wherein the generated pathological diagnosis report includes the test result and / or the pathological image. Specifically, the generated pathological diagnosis report may include the test result, or may include a combination of the test result and the pathological image.
[0135] As another embodiment, in a self-supervised cancer detection system based on staining transformation disclosed in the present application, a feature extractor capable of extracting features unrelated to staining is trained based on the self-supervised learning method of SimCLR. Specifically, color transformation is performed on pathological sections, for example, a new image is generated by changing the brightness, contrast, saturation, etc. of the image. Then, using the SimCLR algorithm, different color transformation perspectives of the same original image are used as positive sample pairs to construct a self-supervised learning task. By maximizing the similarity of positive sample pairs and minimizing the similarity of negative sample pairs, a feature extractor capable of extracting high-quality features unrelated to staining can be trained.
[0136] As another embodiment, a self-supervised cancer detection system based on staining transformation disclosed in the present application may include the following components: a pathology slice collection system, an annotation system, an algorithm system, a prediction visualization system, and a diagnosis reporting system.
[0137] The pathological slice acquisition system is responsible for acquiring pathological slice images and transmitting the images to the algorithm module.
[0138] The annotation system is used to provide cancer annotation information for pathological slice images, and is used to train a cancer discrimination classification model.
[0139] The algorithm system comprises a feature extractor and a cancer discrimination classification model. The feature extractor is trained by a self-supervised learning method based on color transformation, and the cancer discrimination classification model is trained based on the feature extractor.
[0140] The prediction visualization system is used to display the prediction results of the algorithm system in a graphical form on the diagnosis and treatment interface of the pathologist, so as to assist the doctor in diagnosing the pathological results.
[0141] The diagnostic report system is used to generate a pathological diagnosis report, including prediction results, visual images and other information, for doctors to make final diagnosis.
[0142] According to the self-supervised cancer detection system based on staining transformation in the embodiment of the present application, self-supervised learning is performed through SimCLR to train a feature extractor that can extract features unrelated to staining, and the cancer discrimination classification model is trained in combination with sample annotation. The system can achieve effective cancer detection in pathological images, and assist pathologists in diagnosing pathological results through the visualization of predicted results, so it will have broad application prospects and commercial value in the field of intelligent pathology.
[0143] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. A self-supervised cancer detection system based on coloring transformation, characterized in that include: Image acquisition module: used to obtain original pathological slice images; Feature recognition module: used to input the original pathological slice image into the trained feature extractor to determine the pathological feature data of the original pathological slice image; wherein, The feature extractor is trained based on a self-supervised learning method of color transformation; Annotation module: used to input pathological feature data and original pathological slice images into the trained cancer discrimination classification model to generate annotated pathological images; The annotated pathological image includes a pathological data layer and a pathological display layer; Auxiliary judgment module: used to graph the annotated pathological images and generate auxiliary diagnosis and treatment information.
2. A self-supervised cancer detection system based on coloring transformation as claimed in claim 1, characterized in that: The training steps of the feature extractor are as follows: A feature recognition model is preset, and the original pathological slice image is processed by the feature recognition model to determine the pathological features; Preset the slice image recognition magnification, and perform image segmentation on the original pathological slice image according to the pathological characteristics to obtain a number of polygonal segmentation images; The color of several polygon segmentation images is transformed, and the segmentation images with different dyeing are compared and trained to generate a feature extractor; SimCLR is used for contrastive learning training.
3. A self-supervised cancer detection system based on coloring transformation as claimed in claim 2, characterized in that: The preset slice image recognition magnification also includes: Under a preset magnification, a sliding window with a polygonal shape is obtained; According to the sliding window, several polygon segmentation images are obtained in the pathological image; among them, The shape of the polygon segmentation image is adapted to the shape of the sliding window, and the size of the polygon segmentation image is adapted to the size of the sliding window.
4. The self-supervised cancer detection system based on coloring transformation as claimed in claim 2, characterized in that: The slice image recognition magnification is greater than 10 times and less than 30 times, the polygon is a regular polygon, and the size of the sliding window is 224×224 pixels.
5. The self-supervised cancer detection system based on coloring transformation as claimed in claim 2, characterized in that: The feature recognition module also includes: A transformation unit, used for changing the brightness and / or contrast and / or saturation of the polygon segmentation image to obtain two transformed images; The prediction unit is used to predict whether the two transformed images come from the same original image, so as to learn the characteristic information of the original image; wherein, The feature information of the original image is feature information that is unrelated to the dyeing characteristics.
6. The self-supervised cancer detection system based on coloring transformation as claimed in claim 1, characterized in that: The feature recognition module also includes: A learning unit that takes different color-transformed views of the same original image as positive sample pairs to construct a self-supervised learning task; The training unit is used to train the feature extractor by maximizing the similarity of positive sample pairs and minimizing the similarity of negative sample pairs.
7. The self-supervised cancer detection system based on coloring transformation as claimed in claim 1, characterized in that: The feature recognition module also includes: A fusion unit is used to cascade the feature vectors of the corresponding feature extraction modules to obtain a feature extractor, wherein, Each feature extraction module converts the input image into its corresponding feature vector, and forms a fused feature vector by connecting each feature vector in sequence; The feature extractor is constructed from the fused feature vector.
8. The self-supervised cancer detection system based on coloring transformation as claimed in claim 1, characterized in that: The marking module also includes: The classification model submodule is used to train a cancer discrimination classification model based on the feature extractor and cancer annotation information of the original pathological image; A feature vector submodule, used for inputting the pathological image to be detected into the feature extractor to extract the feature vector of the pathological image to be detected; The cancer judgment submodule is used to input the feature vector of the pathological image to be detected into the cancer discrimination classification model to determine whether the pathological image to be detected indicates cancer.
9. The self-supervised cancer detection system based on coloring transformation as claimed in claim 1, characterized in that: The system further comprises: An annotation module, used to provide cancer annotation information of pathological images to train cancer discrimination and classification models; A visualization module is used to display the detection results output by the cancer discrimination and classification model in a graphical form; The diagnosis report module is used to generate a pathology diagnosis report, wherein the generated pathology diagnosis report includes the detection result and / or the pathology image.
10. The self-supervised cancer detection system based on coloring transformation as claimed in claim 1, characterized in that: The graphics also include: The substrate docking technology is used to form a trigger display cursor port at the bottom of the annotated pathological image; wherein, The trigger display cursor port includes a trigger port and a spherical suspension ball trigger process; With the trigger port as the trigger end, when the user points to any pathological area of the annotated pathological image with the mouse cursor, the corresponding pathological annotation is triggered; According to the triggered pathological annotation, the corresponding triggering end retrieves the pathological information of the area currently indicated by the cursor from the pathological data layer, and displays the pathological information through the floating ball.