Self-supervised cancer auxiliary detection system and method based on pathological image staining reconstruction

Abstract

The invention provides a self-supervised cancer auxiliary detection system and method based on pathological image staining reconstruction, and the system comprises a staining reconstruction subsystem which is used for carrying out the staining reconstruction of an obtained pathological section image, and obtaining an input image; the self-supervised learning subsystem is used for acquiring a training target and training a first cancer detection model according to the training target based on a self-supervised learning technology; the fine tuning subsystem is used for performing fine tuning on the first cancer detection model according to the obtained labeling information to obtain a second cancer detection model; and the result output subsystem is used for inputting the input image into a second cancer detection model to obtain a cancer auxiliary detection result. According to the self-supervised cancer auxiliary detection system and method based on pathological image staining reconstruction, the pathological image is stained and reconstructed, and staining is more standard; the second cancer detection model is obtained by introducing a self-supervised learning technology and annotation information, the training cost is lower, and further, the auxiliary detection result is more suitable.

Description

Technical Field

[0001] The present invention relates to the technical field of cancer detection, and in particular to a self-supervised cancer auxiliary detection system and method based on pathological image staining reconstruction. Background Art

[0002] Pathological images are images obtained by observing pathological sections under a microscope. After staining, the morphology, structure and distribution of tissue cells can be clearly observed. Pathological images are of great significance for pathological diagnosis and can help doctors accurately diagnose diseases and evaluate treatment effects. Self-supervised learning is an unsupervised learning technology that trains neural networks to extract useful feature representations by utilizing the intrinsic structural information of unlabeled data. Self-supervised learning tasks usually include two stages: pre-training and fine-tuning. The pre-training stage learns the unsupervised feature representations related to the task, and the fine-tuning stage uses labeled data to train the supervised model related to the task.

[0003] There are many ways to stain pathological images, and different staining methods may result in different characteristics of tissue cells such as color, shape, and texture. This staining diversity poses certain challenges to pathological diagnosis, because staining differences may make it difficult for the algorithm to extract effective features that are not related to the staining characteristics of pathological sections. Self-supervised learning can learn effective feature representations on unlabeled data and improve the generalization ability of the model. In the field of smart pathology, self-supervised learning can help extract features that are not related to the staining characteristics of pathological sections, thereby reducing the impact of staining differences on pathological diagnosis and improving the accuracy of auxiliary doctors' diagnosis.

[0004] The invention patent with application number: CN202110731127.0 discloses an AI-based analysis method and system for the prognosis of postoperative recurrence of early hepatocellular carcinoma, wherein the method includes: obtaining a full-field digital slice (WSI) image of the specimen to be analyzed; extracting a foreground region of interest (ROI) from the WSI image. The above invention identifies different areas of digital pathological images through deep learning. The risk of postoperative recurrence and prognosis of early hepatocellular carcinoma can be effectively analyzed directly based on HE-stained sections. While identifying a variety of different cell regions, its accuracy is also very high, which is of great guiding significance for patient recurrence.

[0005] However, the above-mentioned prior art directly analyzes based on HE-stained sections. When different staining methods are used, the staining is not standard. At the same time, the conventional deep learning method for identifying different areas of digital pathological images requires a large amount of labeled data, and the acquisition process of a large amount of labeled data is not easy and the training cost is high. Based on the above-mentioned prerequisites, prognostic analysis is not suitable.

[0006] In view of this, there is an urgent need for a high-frequency quantitative trading method and system based on deep learning to at least solve the above-mentioned deficiencies. Summary of the invention

[0007] One of the purposes of the present invention is to provide a self-supervised cancer auxiliary detection system based on pathological image staining reconstruction. The pathological image obtained by staining reconstruction is used as the input image, which improves the standardization of staining. At the same time, self-supervised learning technology is introduced to train the first cancer detection model, and annotation information is introduced to obtain the second cancer detection model after fine-tuning. The intelligent model is trained with as little annotation data as possible, which reduces the training cost. The cancer auxiliary detection result is determined according to the input image and the second cancer detection model, which is more appropriate.

[0008] The self-supervised cancer auxiliary detection system based on pathological image staining reconstruction provided by the embodiment of the present invention includes:

[0009] Pathological section image acquisition subsystem, used for acquiring pathological section images;

[0010] The staining and reconstruction subsystem is used to perform staining and reconstruction on the pathological section image to obtain an input image;

[0011] A self-supervised learning subsystem, for retaining the pathological section image before preprocessing as a training target, and training a first cancer detection model according to the training target based on the self-supervised learning technology;

[0012] Annotation information acquisition subsystem, used to acquire annotation information;

[0013] A fine-tuning subsystem, used for fine-tuning the first cancer detection model according to the labeled information to obtain a second cancer detection model;

[0014] The result output subsystem is used to input the input image into the second cancer detection model to obtain cancer auxiliary detection results.

[0015] Preferably, the self-supervised learning subsystem includes:

[0016] An enhanced data acquisition module is used to perform data enhancement operations on training targets and obtain enhanced data;

[0017] A feature representation extraction module is used to extract feature representations of enhanced data based on a preset CNN model;

[0018] Self-supervised task construction module, used to construct self-supervised tasks;

[0019] The first cancer detection model determination module is used to perform model training according to the self-supervised task and feature representation, and obtain the first cancer detection model when the first loss function of the self-supervised task is minimized.

[0020] Preferably, the annotation information acquisition subsystem includes:

[0021] A data attribute determination module, used to obtain a first data providing node of a data service provider and determine data attributes of data provided by the first data providing node;

[0022] A data acquisition requirement acquisition module is used to acquire data acquisition requirements;

[0023] A second data providing node acquisition module, used for acquiring a second data providing node that provides annotation information according to data acquisition requirements and data attributes;

[0024] The annotation information acquisition module is used to send a data acquisition request to the second data providing node and obtain the annotation information replied by the second data providing node.

[0025] Preferably, the fine-tuning subsystem comprises:

[0026] A labeling task acquisition module is used to acquire labeling tasks of labeling information;

[0027] A second loss function determination module, used to determine a second loss function of the first cancer detection model according to the labeling task;

[0028] The optimization module is used to optimize the first cancer detection model according to the second loss function to obtain an optimized second cancer detection model.

[0029] Preferably, the staining reconstruction subsystem comprises:

[0030] The target image library acquisition module is used to acquire the target image library; wherein the target image library includes: a plurality of one-to-one corresponding original black-and-white images and standard color images;

[0031] A processing image pair determination module is used to process the original black-and-white image and the standard color image in the target image library according to OpenCV to determine the processing image pair;

[0032] A dyeing and reconstruction model building module is used to extract a first color feature of the processed image pair, and build a dyeing and reconstruction model according to the first color feature and a preset generative adversarial model;

[0033] A grayscale image acquisition module is used to grayscale the pathological slice image based on image processing technology to obtain a grayscale image;

[0034] The input image acquisition module is used to perform staining reconstruction on the grayscale image according to the staining reconstruction model to obtain the input image.

[0035] Preferably, the staining reconstruction model building module comprises:

[0036] A second color feature determination submodule, used to determine a second color feature according to the processed image pair and a preset generator;

[0037] A third color feature discriminating submodule, used for training a discriminator to discriminate a third color feature corresponding to the second color feature based on the adversarial training technology and according to the first color feature, the second color feature and a preset discriminator;

[0038] A target discriminator acquisition submodule, used to optimize the discriminant parameter of the discriminator by minimizing the difference between the third color feature and the first color feature corresponding to the third color feature, so as to obtain the target discriminator;

[0039] The dyeing reconstruction model determination submodule is used to use the generator and the target discriminator together as the dyeing reconstruction model.

[0040] Preferably, the target image library acquisition module includes:

[0041] A standard coloring rule acquisition submodule is used to acquire standard coloring rules;

[0042] The construction submodule is used to construct a target image library according to standard staining rules.

[0043] Preferably, the standard dyeing rule acquisition submodule includes:

[0044] A slice part acquisition unit, used for acquiring a slice part of a pathological slice image;

[0045] A detection requirement acquisition unit, used for acquiring detection requirements;

[0046] A target staining substance determination unit, used to determine the target staining substance according to the section location and detection requirements;

[0047] A dyeing method determination unit, used to determine the dyeing method according to the target dyeing substance and the detection requirements;

[0048] A historical dyeing process guidance record acquisition unit, used for acquiring a historical dyeing process guidance record of a dyeing method;

[0049] A dyeing process semantic determination unit, used to determine the dyeing process semantics according to the historical dyeing process guidance records;

[0050] A dyeing process specification determination unit is used to determine the dyeing process specification according to the dyeing process semantics;

[0051] The standard dyeing rule determination unit is used to take the detection requirements, target dyeing substances, dyeing methods and dyeing process specifications as standard dyeing rules.

[0052] Preferably, the dyeing method determination unit comprises:

[0053] A determination result determination subunit, for determining a determination result of the availability of an antibody for a target staining substance according to the target staining substance and the detection requirements;

[0054] The first staining method determination subunit is used to obtain a first candidate staining method corresponding to the target staining substance if the result of the determination of the antibody availability is that the antibody is available, and determine the staining method according to the use conditions and detection requirements of the first candidate staining method;

[0055] A synthetic literature acquisition subunit, for acquiring synthetic literature of available antibodies corresponding to the target staining substance if the result of the determination of the antibody availability is that the antibody is unavailable;

[0056] The preparation method determination subunit is used to determine the preparation method of the available antibodies based on the synthesis literature;

[0057] The preparation scheduling subunit is used to schedule the staff to prepare the corresponding target dyeing substance according to the preparation method to obtain the preparation product;

[0058] A credibility verification subunit, used to verify the credibility of the second to-be-verified dyeing method for preparing the product recorded in the synthesis document according to the experimental data and the prepared product of the corresponding target dyeing substance recorded in the synthesis document;

[0059] The second dyeing method determination subunit is used to use the corresponding second dyeing method to be verified as the dyeing method corresponding to the prepared product if the credibility is greater than or equal to a preset credibility threshold.

[0060] The self-supervised cancer auxiliary detection method based on pathological image staining reconstruction provided by the embodiment of the present invention includes:

[0061] Step 1: Obtain pathological slice images;

[0062] Step 2: Stain and reconstruct the pathological section image to obtain an input image;

[0063] Step 3: retain the pathological section image before preprocessing as a training target, and train the first cancer detection model based on the training target based on the self-supervised learning technology;

[0064] Step 4: Get annotation information;

[0065] Step 5: Fine-tune the first cancer detection model according to the labeled information to obtain a second cancer detection model;

[0066] Step 6: Input the input image into the second cancer detection model to obtain cancer auxiliary detection results.

[0067] The beneficial effects of the present invention are:

[0068] The present invention obtains pathological images through staining reconstruction and uses them as input images, thereby improving the standardization of staining. At the same time, a self-supervised learning technology is introduced to train a first cancer detection model, and annotated information is introduced to obtain a fine-tuned second cancer detection model. The intelligent model is trained with as little annotated data as possible, thereby reducing the training cost. The auxiliary cancer detection result is determined based on the input image and the second cancer detection model, which is more appropriate.

[0069] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0070] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0072] Figure 1 Schematic diagram of a self-supervised cancer auxiliary detection system based on pathological image staining reconstruction in an embodiment of the present invention;

[0073] Figure 2 Schematic diagram of a self-supervised cancer auxiliary detection method based on pathological image staining reconstruction in an embodiment of the present invention. DETAILED DESCRIPTION

[0074] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0075] The embodiment of the present invention provides a self-supervised cancer auxiliary detection system based on pathological image staining reconstruction, such as Figure 1 As shown, including:

[0076] The pathological slice image acquisition subsystem 1 is used to acquire the pathological slice image; wherein the pathological slice image is: an image obtained by observing the pathological slice through a microscope;

[0077] The staining reconstruction subsystem 2 is used to perform staining reconstruction on the pathological slice image to obtain an input image; wherein the staining reconstruction is: performing staining simulation on the digitized pathological slice image through computer image processing technology to restore the visual effect after staining; the input image is: the pathological slice image after staining reconstruction;

[0078] The self-supervised learning subsystem 3 is used to retain the pathological section image before preprocessing as a training target, and based on the self-supervised learning technology, train the first cancer detection model according to the training target; wherein, based on the self-supervised learning technology, the first cancer detection model is trained according to the training target: according to the training target, the cancer-related features are automatically detected and identified and trained to obtain the first cancer detection model, and the first cancer detection model is: an intelligent model for replacing manual cancer detection based on pathological images;

[0079] The annotation information acquisition subsystem 4 is used to acquire the annotation information; wherein the annotation information is: a pathological section image with manual annotation of tissue structure, cell nucleus, lesion, marker point and staining;

[0080] A fine-tuning subsystem 5 is used to fine-tune the first cancer detection model according to the annotation information to obtain a second cancer detection model; wherein, the first cancer detection model is fine-tuned according to the annotation information to obtain the second cancer detection model by loading the parameters of the first cancer detection model as initial parameters into the second cancer detection model, which will cause the second model to be fine-tuned from the pre-trained weights of the first model. During the fine-tuning process, the second cancer detection model is trained using the annotation information. During the training process, the model parameters are optimized by minimizing a loss function, wherein the loss function may be a cross entropy loss function or a D ice loss function, etc.;

[0081] The result output subsystem 6 is used to input the input image into the second cancer detection model to obtain the cancer auxiliary detection result. The cancer auxiliary detection result is, for example, what kind of cancer cells are in what area.

[0082] The working principle and beneficial effects of the above technical solution are:

[0083] The pathological image obtained by staining reconstruction in this application is used as the input image, which improves the standardization of staining. At the same time, self-supervised learning technology is introduced to train the first cancer detection model, and the second cancer detection model after fine-tuning is obtained by introducing annotation information. The intelligent model is trained with as little annotated data as possible, which reduces the training cost. It is more appropriate to determine the cancer auxiliary detection results based on the input image and the second cancer detection model.

[0084] In specific applications, the pathological section images are input into the system, and the system automatically stains and reconstructs the pathological section images and trains the cancer detection model, and determines the cancer auxiliary detection results based on the input images obtained by staining and reconstruction and the trained second cancer detection model.

[0085] In one embodiment, the self-supervised learning subsystem includes:

[0086] The enhanced data acquisition module is used to perform data enhancement operations on the training target to obtain enhanced data; wherein the data enhancement operations are, for example, random rotation, mirror flipping, and brightness and contrast adjustment, etc.; the enhanced data are, for example, pathological slice images after random rotation;

[0087] The feature representation extraction module is used to extract the feature representation of the enhanced data based on a preset CNN model; wherein the CNN model is: a convolutional neural network model; the feature representation is: a feature representation of the enhanced data;

[0088] A self-supervised task building module is used to build self-supervised tasks. For example, a self-supervised task is to identify a certain type of cancer cell. The self-supervised task is built based on the feature representation of the enhanced data.

[0089] The first cancer detection model determination module is used to perform model training according to the self-supervised task and feature representation, and obtain the first cancer detection model when the first loss function of the self-supervised task is minimized. The first loss function is, for example, a cross entropy loss function; the first cancer detection model is: an intelligent model for cancer detection trained according to the training target based on the self-supervised learning technology;

[0090] The working principle and beneficial effects of the above technical solution are:

[0091] This application introduces a data enhancement operation to perform data enhancement on the training target to obtain enhanced data, and extracts feature representations of the enhanced data based on the CNN model, thereby determining a self-supervised task, introducing a first loss function for the supervised task, and determining a first cancer detection model with the smallest first loss function, thereby improving the comprehensiveness and suitability of the training of the first cancer detection model.

[0092] In one embodiment, the annotation information acquisition subsystem includes:

[0093] The data attribute determination module is used to obtain the first data providing node of the data service provider and determine the data attributes of the data provided by the first data providing node; wherein the data service provider is: a cloud service platform for medical data; the first data providing node is: a communication node of the data service provider; the provided data is, for example: what kind of annotated pathological slice image; the data attributes are: the type and value range of the data, for example: the slice part of the pathological slice image, the annotated doctor and the annotated content, etc.;

[0094] The data acquisition requirement acquisition module is used to acquire data acquisition requirements; wherein the data acquisition requirements are: what kind of annotated pathological slice images need to be acquired;

[0095] A second data providing node acquisition module is used to acquire a second data providing node that provides annotation information according to data acquisition requirements and data attributes; wherein, when acquiring the second data providing node that provides annotation information according to data acquisition requirements and data attributes, the required data attributes and the data attributes are determined according to the data acquisition requirements to perform attribute matching, and if the attribute matching is met, the first data providing node corresponding to the required data attributes that match the matching is used as the second data providing node;

[0096] The annotation information acquisition module is used to send a data acquisition request to the second data providing node and obtain the annotation information replied by the second data providing node.

[0097] The working principle and beneficial effects of the above technical solution are:

[0098] The present application introduces data attributes of the first data providing node, and determines the second data providing node to obtain annotation information according to the acquired data acquisition requirements, the data acquisition requirements and the data attributes, thereby improving the suitability of the acquisition of annotation information.

[0099] In one embodiment, the fine-tuning subsystem includes:

[0100] A labeling task acquisition module is used to acquire the labeling task of the labeling information; wherein the labeling task is determined according to the labeling features of the labeling information, and the labeling features are, for example, labeling cell types and labeling methods;

[0101] A second loss function determination module is used to determine a second loss function of the first cancer detection model according to the labeling task; wherein the second loss function is: a cross entropy loss function of the first cancer detection model corresponding to the labeling task;

[0102] The optimization module is used to optimize the first cancer detection model according to the second loss function to obtain the optimized second cancer detection model. The optimization of the first cancer detection model is: setting the model parameters of the first cancer detection model so that the corresponding second loss function changes in the direction of small value; the second cancer detection model is: when the second loss function corresponding to the first cancer detection model is the minimum, the corresponding model parameter setting of the first cancer detection model.

[0103] The working principle and beneficial effects of the above technical solution are:

[0104] This application determines the second loss function of the first cancer detection model based on the labeling task, and optimizes the first cancer detection model based on the second loss function of the first cancer detection model to obtain the second cancer detection model, thereby further improving the accuracy of the cancer detection model detection.

[0105] In one embodiment, the staining reconstruction subsystem includes:

[0106] The target image library acquisition module is used to acquire the target image library; wherein the target image library includes: a plurality of one-to-one corresponding original black-and-white images and standard color images;

[0107] A processing image pair determination module is used to process the original black-and-white image and the standard color image in the target image library according to OpenCV to determine the processing image pair; wherein OpenCV is a widely used computer vision and image processing library; when the original black-and-white image and the standard color image in the target image library are processed according to OpenCV, image denoising and image enhancement and other processing are performed;

[0108] The dyeing reconstruction model construction module is used to extract the first color feature of the processed image pair, and construct the dyeing reconstruction model according to the first color feature and the preset generative adversarial model; wherein the first color feature is: color texture feature and color gradient feature, etc.; the adversarial model is a deep learning model, which is composed of two networks, a generator and a discriminator, and generates realistic data samples through adversarial training; the dyeing reconstruction model is: an intelligent model for reconstructing the color of the image input into the dyeing reconstruction model and obtaining an image with a unified standard color;

[0109] A grayscale image acquisition module is used to grayscale the pathological slice image based on image processing technology to obtain a grayscale image;

[0110] The input image acquisition module is used to perform staining reconstruction on the grayscale image according to the staining reconstruction model to obtain the input image.

[0111] The working principle and beneficial effects of the above technical solution are:

[0112] This application introduces a target image library, processes the original black and white images and standard color images in the target image library according to OpenCV, determines a processed image pair, introduces a first color feature of the processed image pair, and constructs a staining reconstruction model based on the first color feature and the introduced generative adversarial model, and inputs the grayscale image into the staining reconstruction model to obtain an input image with staining reconstruction completed.

[0113] In one embodiment, the staining reconstruction model building module includes:

[0114] The adversarial feature determination submodule is used to generate an adversarial feature according to the first color feature; wherein the adversarial feature is a color feature similar to the grayscale color feature in the first color feature (the color texture feature and the color gradient feature of the grayscale image in the processed image pair, etc.);

[0115] A second color feature determination submodule is used to determine a second color feature based on an adversarial training technique according to the grayscale color feature, the adversarial feature and a preset generator in the first color feature; wherein the preset generator is: a part of the preset generative adversarial model used to generate features of the stained reconstructed image; the second color feature is: a color feature output by the generator corresponding to the grayscale color feature and the adversarial feature, such as: a color texture feature and a color gradient feature, etc.;

[0116] A third color feature discrimination submodule, used for training a discriminator to discriminate the coloring color feature in the second color feature according to the coloring color feature in the first color feature, the second color feature and a preset discriminator;

[0117] The third color feature acquisition submodule is used to acquire the color feature not identified by the discriminator and use it as the third color feature;

[0118] An unrecognized grayscale color feature determination submodule, used to obtain a grayscale color feature corresponding to the third color feature and use it as an unrecognized grayscale color feature;

[0119] A fourth color feature determination submodule, used to determine, based on the unrecognized grayscale color feature, that the unrecognized grayscale color feature corresponds to the second color feature of the generator, and use the second color feature as the fourth color feature;

[0120] A target discriminator acquisition submodule is used to optimize the generation parameters of the generator by minimizing the difference between the third color feature and the fourth color feature corresponding to the third color feature, so as to obtain a target generator; wherein the generation parameters are: weights and bias parameters in the generator network;

[0121] A dye reconstruction model determination submodule is used to connect the target generator and the discriminator as a dye reconstruction model;

[0122] Among them, the adversarial feature determination submodule includes:

[0123] An average eigenvalue determining unit is used to determine an average eigenvalue corresponding to each feature dimension according to the first color feature; wherein the feature dimension is, for example, color texture, or color gradient;

[0124] The difference value acquisition unit is used to traverse the first color feature in sequence, and during each traversal, determine the feature dimension of the first color feature currently being traversed, and obtain the difference value between the current feature value and the average feature value of the feature dimension; wherein the current feature value is, for example, what is the color gradient of the first color feature being traversed; and the average feature value is, for example, what is the average color gradient of all first color features;

[0125] A sub-adversarial feature determination unit is used to use half of the difference value as a disturbance range selection value, and determine the sub-adversarial feature of the first color feature being traversed according to the disturbance range selection value and the current feature value; wherein, when determining the sub-adversarial feature of the first color feature being traversed, the current feature value is reduced by the corresponding disturbance range selection value as the lower limit of the disturbance range of the current feature dimension, and the current feature value is increased by the corresponding disturbance range selection value as the upper limit of the disturbance range of the current feature dimension, and when determining the sub-adversarial feature according to the lower limit and upper limit of the disturbance range of each feature dimension, the value of the sub-adversarial feature corresponding to each feature dimension is a normal distribution with the current feature value of the corresponding feature dimension as the center of the normal distribution, and the value does not exceed the range determined by the lower limit and upper limit of the disturbance range;

[0126] The sub-adversarial feature integration unit is used to integrate all sub-adversarial features to obtain adversarial features.

[0127] The working principle and beneficial effects of the above technical solution are:

[0128] There are many methods for staining pathological images. Different staining methods may cause different characteristics such as color, shape and texture of tissue cells. Therefore, this application introduces a staining reconstruction model. Specifically, this application generates adversarial features based on the first color feature. When generating adversarial features, half of the difference between the current eigenvalue and the average eigenvalue of the feature dimension is introduced as the disturbance range selection value. The adversarial feature is determined according to the dynamic range selection value and the normal distribution function, thereby improving the rationality of the adversarial feature determination process. At the same time, adversarial training technology is introduced to determine the second color feature generated by the generator, and the unrecognized grayscale color feature is determined according to the coloring color feature and the second color feature in the first color feature. By minimizing the unrecognized grayscale color feature corresponding to the fourth color feature of the generator and the coloring color feature not discriminated by the discriminator, the generation parameters of the optimized generator are determined and the generator is optimized. Then, the staining reconstruction model is determined according to the optimized target generator and the discriminator, thereby improving the generalization ability of the staining reconstruction model.

[0129] In one embodiment, the target image library acquisition module includes:

[0130] The standard staining rule acquisition submodule is used to acquire the standard staining rule; wherein the standard staining rule is: which pathological section uses which staining method;

[0131] The construction submodule is used to construct a target image library according to the standard staining rules. When constructing the target image library according to the standard staining rules, the pathological image pairs stained according to the standard staining rules are obtained through the big data medical service platform and stored in the target image library.

[0132] The working principle and beneficial effects of the above technical solution are:

[0133] This application introduces standard coloring rules and constructs a target image library based on the standard coloring rules, thereby improving the accuracy of subsequent adversarial training.

[0134] In one embodiment, the standard coloring rule acquisition submodule includes:

[0135] A slice part acquisition unit, used to acquire a slice part of a pathological slice image; wherein the slice part is: a body part corresponding to the slice tissue of the pathological slice image;

[0136] A detection requirement acquisition unit is used to acquire detection requirements; wherein the detection requirements are: what kind of diseased cells need to be detected;

[0137] The target staining substance determination unit is used to determine the target staining substance according to the section location and the detection requirements; wherein the target staining substance is, for example, the nucleus of a certain cell;

[0138] A staining method determination unit is used to determine a staining method according to a target staining substance and a detection requirement; wherein the staining method is, for example, H&E staining, DAPI staining, and PI staining;

[0139] A historical staining process guidance record acquisition unit is used to acquire a historical staining process guidance record of a staining method; the historical staining process guidance record is a record in which an experienced doctor guides an inexperienced doctor to stain a pathological section based on the staining method, and the record may be recorded by video or in text;

[0140] The dyeing process semantic determination unit is used to determine the semantics of the dyeing process according to the historical dyeing process guidance records; wherein the semantics of the dyeing process is obtained through semantic analysis technology, which is the conversation semantics of the experienced doctor's dyeing operation guidance obtained from the historical dyeing process analysis;

[0141] The dyeing process specification determination unit is used to determine the dyeing process specification according to the dyeing process semantics; wherein the dyeing process specification is: the dyeing operation should be performed... and the dyeing operation should not be performed...;

[0142] The standard dyeing rule determination unit is used to take the detection requirements, target dyeing substances, dyeing methods and dyeing process specifications as standard dyeing rules.

[0143] The working principle and beneficial effects of the above technical solution are:

[0144] When the staining standards are different, the constructed staining reconstruction models are also different, and the results of the staining reconstruction are also different. Therefore, this application determines the target staining substance according to the slice location and detection requirements, determines the staining method according to the target staining substance and detection requirements, introduces the historical staining process guidance record of the staining method to determine the staining process semantics, and determines the staining process specifications based on the staining process semantics. The detection requirements, target staining substances, staining methods and staining process specifications are collectively used as standard staining rules, which improves the standardization of the formulation of standard staining rules.

[0145] In one embodiment, the dyeing method determination unit comprises:

[0146] The determination result determination subunit is used to determine the determination result of the antibody availability of the target staining substance according to the target staining substance and the detection requirements; wherein the antibody availability is: whether a reliable and effective antibody suitable for the target staining substance can be obtained under specific experimental conditions; the determination result is: the antibody is available and the antibody is not available;

[0147] The first staining method determination subunit is used to obtain a first candidate staining method corresponding to the target staining substance if the result of the determination of the antibody availability is that the antibody is available, and determine the staining method according to the use conditions and detection requirements of the first candidate staining method;

[0148] A synthetic literature acquisition subunit is used to acquire synthetic literature of available antibodies corresponding to the target staining substance if the result of determining the availability of the antibody is that the antibody is unavailable; wherein, when acquiring the synthetic literature, literature retrieval can be performed based on the target staining substance;

[0149] The preparation method determination subunit is used to determine the preparation method of the usable antibody according to the synthesis literature; wherein the usable antibody is: a substance that can make the target dye substance develop color; the preparation method is: a method for preparing the usable antibody;

[0150] The preparation scheduling subunit is used to schedule the staff to prepare the corresponding target dyeing substance according to the preparation method to obtain the preparation product; wherein the preparation product is: the substance prepared according to the preparation method;

[0151] A credibility verification subunit is used to verify the credibility of the second to-be-verified dyeing method for preparing the product recorded in the synthesis document according to the experimental data of the corresponding target dyeing substance and the prepared product recorded in the synthesis document; wherein the second to-be-verified dyeing method is: a method for dyeing the target dyeing substance based on the prepared product; when verifying the credibility of the second to-be-verified dyeing method for preparing the product recorded in the synthesis document, sample the prepared product for antibody staining inspection to determine whether the sampled prepared product is an antibody suitable for dyeing the target dyeing substance;

[0152] The second dyeing method determination subunit is used to use the corresponding second dyeing method to be verified as the dyeing method corresponding to the prepared product if the credibility is greater than or equal to a preset credibility threshold. The preset credibility threshold is manually preset.

[0153] The working principle and beneficial effects of the above technical solution are:

[0154] Generally, when determining the staining method of a staining substance, it is necessary to obtain the corresponding antibody. However, not all staining substances have ready-made antibodies available. Therefore, the present application determines the result of the determination of the antibody availability of the target staining substance based on the target staining substance and the detection requirements. When the result of the determination of the antibody availability is that the antibody is available, the first candidate staining method for the target staining substance is obtained, and the staining method is determined based on the use conditions and detection requirements of the first candidate staining method; when the result of the determination of the antibody availability is that the antibody is unavailable, a synthetic document is introduced, and the preparation method of the available antibody is determined based on the synthetic document and the preparation product is prepared. The credibility of the preparation product is verified according to the experimental data of the corresponding target staining substance recorded in the synthetic document and the preparation product. When the credibility verification is passed, the staining method is obtained, which improves the comprehensiveness and suitability of the staining method acquisition.

[0155] The embodiment of the present invention provides a self-supervised cancer auxiliary detection method based on pathological image staining reconstruction, such as Figure 2 As shown, including:

[0156] Step 1: Obtain pathological slice images;

[0157] Step 2: Stain and reconstruct the pathological section image to obtain an input image;

[0158] Step 3: retain the pathological section image before preprocessing as a training target, and train the first cancer detection model based on the training target based on the self-supervised learning technology;

[0159] Step 4: Get annotation information;

[0160] Step 5: Fine-tune the first cancer detection model according to the labeled information to obtain a second cancer detection model;

[0161] Step 6: Input the input image into the second cancer detection model to obtain cancer auxiliary detection results.

[0162] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A self-supervised cancer-assisted detection system based on pathological image staining reconstruction, characterized in that: include: Pathological section image acquisition subsystem, used for acquiring pathological section images; The staining and reconstruction subsystem is used to perform staining and reconstruction on the pathological section image to obtain an input image; A self-supervised learning subsystem, for retaining the pathological section image before preprocessing as a training target, and training a first cancer detection model according to the training target based on the self-supervised learning technology; Annotation information acquisition subsystem, used to acquire annotation information; A fine-tuning subsystem, used for fine-tuning the first cancer detection model according to the labeled information to obtain a second cancer detection model; The result output subsystem is used to input the input image into the second cancer detection model to obtain cancer auxiliary detection results.

2. The self-supervised cancer auxiliary detection system based on pathological image staining reconstruction as claimed in claim 1, characterized in that: Self-supervised learning subsystem, including: An enhanced data acquisition module is used to perform data enhancement operations on training targets and obtain enhanced data; A feature representation extraction module is used to extract feature representations of enhanced data based on a preset CNN model; Self-supervised task construction module, used to construct self-supervised tasks; The first cancer detection model determination module is used to perform model training according to the self-supervised task and feature representation, and obtain the first cancer detection model when the first loss function of the self-supervised task is minimized.

3. The self-supervised cancer auxiliary detection system based on pathological image staining reconstruction as claimed in claim 1, characterized in that: The annotation information acquisition subsystem includes: A data attribute determination module, used to obtain a first data providing node of a data service provider and determine data attributes of data provided by the first data providing node; A data acquisition requirement acquisition module is used to acquire data acquisition requirements; A second data providing node acquisition module, used for acquiring a second data providing node that provides annotation information according to data acquisition requirements and data attributes; The annotation information acquisition module is used to send a data acquisition request to the second data providing node and obtain the annotation information replied by the second data providing node.

4. The self-supervised cancer auxiliary detection system based on pathological image staining reconstruction as claimed in claim 1, characterized in that: Fine-tuning subsystems, including: A labeling task acquisition module is used to acquire labeling tasks of labeling information; A second loss function determination module, used to determine a second loss function of the first cancer detection model according to the labeling task; The optimization module is used to optimize the first cancer detection model according to the second loss function to obtain an optimized second cancer detection model.

5. The self-supervised cancer auxiliary detection system based on pathological image staining reconstruction as claimed in claim 1, characterized in that: Stain reconstruction subsystem, including: The target image library acquisition module is used to acquire the target image library; wherein the target image library includes: a plurality of one-to-one corresponding original black-and-white images and standard color images; A processing image pair determination module is used to process the original black-and-white image and the standard color image in the target image library according to OpenCV to determine the processing image pair; A dyeing and reconstruction model building module is used to extract a first color feature of the processed image pair, and build a dyeing and reconstruction model according to the first color feature and a preset generative adversarial model; A grayscale image acquisition module is used to grayscale the pathological slice image based on image processing technology to obtain a grayscale image; The input image acquisition module is used to perform staining reconstruction on the grayscale image according to the staining reconstruction model to obtain the input image.

6. The self-supervised cancer auxiliary detection system based on pathological image staining reconstruction as claimed in claim 5, characterized in that: Staining reconstruction model building blocks, including: An adversarial feature determination submodule, used to generate an adversarial feature according to the first color feature; A second color feature determination submodule, used to determine a second color feature based on an adversarial training technique according to the grayscale color feature in the first color feature, the adversarial feature and a preset generator; A third color feature discrimination submodule, used for training a discriminator to discriminate the coloring color feature in the second color feature according to the coloring color feature in the first color feature, the second color feature and a preset discriminator; The third color feature acquisition submodule is used to acquire the color feature not identified by the discriminator and use it as the third color feature; An unrecognized grayscale color feature determination submodule, used to obtain a grayscale color feature corresponding to the third color feature and use it as an unrecognized grayscale color feature; A fourth color feature determination submodule, used to determine, based on the unrecognized grayscale color feature, that the unrecognized grayscale color feature corresponds to the second color feature of the generator, and use the second color feature as the fourth color feature; A target discriminator acquisition submodule, used to optimize the generation parameters of the generator by minimizing the difference between the third color feature and the fourth color feature corresponding to the third color feature, so as to obtain a target generator; The dyeing reconstruction model determination submodule is used to connect the target generator and the discriminator as a dyeing reconstruction model.

7. The self-supervised cancer auxiliary detection system based on pathological image staining reconstruction as claimed in claim 5, characterized in that: The target image library acquisition module includes: A standard coloring rule acquisition submodule is used to acquire standard coloring rules; The construction submodule is used to construct a target image library according to standard staining rules.

8. The self-supervised cancer auxiliary detection system based on pathological image staining reconstruction as claimed in claim 7, characterized in that: Standard coloring rule acquisition submodule, including: A slice part acquisition unit, used for acquiring a slice part of a pathological slice image; A detection requirement acquisition unit, used for acquiring detection requirements; A target staining substance determination unit, used to determine the target staining substance according to the section location and detection requirements; A dyeing method determination unit, used to determine the dyeing method according to the target dyeing substance and the detection requirements; A historical dyeing process guidance record acquisition unit, used for acquiring a historical dyeing process guidance record of a dyeing method; A dyeing process semantic determination unit, used to determine the dyeing process semantics according to the historical dyeing process guidance records; A dyeing process specification determination unit is used to determine the dyeing process specification according to the dyeing process semantics; The standard dyeing rule determination unit is used to take the detection requirements, target dyeing substances, dyeing methods and dyeing process specifications as standard dyeing rules.

9. The self-supervised cancer auxiliary detection system based on pathological image staining reconstruction as claimed in claim 8, characterized in that: The dyeing method determination unit includes: A determination result determination subunit, for determining a determination result of the availability of an antibody for a target staining substance according to the target staining substance and the detection requirements; The first staining method determination subunit is used to obtain a first candidate staining method corresponding to the target staining substance if the result of the determination of the antibody availability is that the antibody is available, and determine the staining method according to the use conditions and detection requirements of the first candidate staining method; A synthetic literature acquisition subunit, for acquiring synthetic literature of available antibodies corresponding to the target staining substance if the result of the determination of the antibody availability is that the antibody is unavailable; The preparation method determination subunit is used to determine the preparation method of the available antibodies based on the synthesis literature; The preparation scheduling subunit is used to schedule the staff to prepare the corresponding target dyeing substance according to the preparation method to obtain the preparation product; A credibility verification subunit, used to verify the credibility of the second to-be-verified dyeing method for preparing the product recorded in the synthesis document according to the experimental data and the prepared product of the corresponding target dyeing substance recorded in the synthesis document; The second dyeing method determination subunit is used to use the corresponding second dyeing method to be verified as the dyeing method corresponding to the prepared product if the credibility is greater than or equal to a preset credibility threshold.

10. A self-supervised cancer-assisted detection method based on pathological image staining reconstruction, characterized in that: include: Step 1: Obtain pathological slice images; Step 2: Stain and reconstruct the pathological section image to obtain an input image; Step 3: retain the pathological section image before preprocessing as a training target, and train the first cancer detection model based on the training target based on the self-supervised learning technology; Step 4: Get annotation information; Step 5: Fine-tune the first cancer detection model according to the labeled information to obtain a second cancer detection model; Step 6: Input the input image into the second cancer detection model to obtain cancer auxiliary detection results.

Images