Dual-task recurrent self-supervised nuclei segmentation method based on staining similarity reconstruction

Abstract

The application discloses a double-task cycle self-supervision nucleus segmentation method based on dyeing similarity reconstruction, and comprises the following steps: establishing a prior knowledge guided self-supervision learning framework, generating reliable initial pseudo labels through self-supervision learning of dyeing prior knowledge and image prior knowledge; the prior knowledge guided self-supervision learning framework comprises a contrast learning model and a dyeing reconstruction model to help the framework learn image and dyeing prior knowledge; training a downstream double-task nucleus segmentation model, using inter-task consistency and intra-task self-refinement strategies to improve the performance of contrast learning and dyeing reconstruction, and realizing nucleus segmentation. The application realizes the initialization of pseudo labels through self-supervision learning and dyeing reconstruction of the prior knowledge of histopathology images; the double-task model is trained to realize nucleus segmentation, the inter-task consistency and intra-task self-refinement strategies are used to make up for the information gap, and the iterative refinement of the two tasks is maintained.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computational pathology, and in particular to a dual-task cycle self-supervised nuclei segmentation method based on staining similarity reconstruction. BACKGROUND

[0002] Instance segmentation of nuclei in histopathological images plays a vital role in digital pathology workflow. It allows researchers to analyze morphological features of individual nuclei, count mitotic cells, explore spatial distribution of different types of nuclei, and achieve accurate cancer diagnosis and prognosis prediction simultaneously. Currently, fully supervised learning methods are widely used in nuclei instance segmentation due to their powerful feature representation ability. However, deep learning methods require densely annotated datasets to achieve good performance. To reduce the annotation workload, a large number of works replace dense pixel-level annotation with point-based annotation. To bridge the gap between point annotation and pixel-level annotation, existing (Qu, H., Wu, P., Huang, Q., Yi, J., Yan, Z., Li, K., Riedlinger, G. M., De, S., Zhang, S., Metaxas, D. N., 2020. Weakly supervised deep nuclei segmentation using partial points annotation in histopathology images. IEEE transactions on medical imaging 39, 3655-3666.) introduce a dual-task strategy to train two consecutive nuclei detection and segmentation task models. This strategy not only maximizes the value of point annotation, but also correlates the results of the two tasks to separate attached nuclei. However, due to the huge clinical workload, even with more efficient annotation methods, annotating millions of nuclei still puts a huge burden on pathologists. Recently, self-supervised learning methods provide a solution to segment nuclei without any manual annotation. The model idea is to use a large amount of unannotated data to perform self-supervised learning through different proxy tasks, generate initial pseudo-annotation for training downstream nuclei detection and segmentation tasks.

[0003] Unsupervised nuclei segmentation technology for pathological images. Fully supervised learning methods have achieved remarkable results in nuclei segmentation, but require a large amount of pixel-level annotation, which requires a lot of manpower, financial resources and professional knowledge of pathologists.

[0004] Currently, the full supervision method is limited by the current situation of complex cell structure and extremely high image resolution in the background of histopathology images, and the annotation process is time-consuming and laborious. Unlike natural images, due to the requirement of clinical professional knowledge, accurate annotations cannot be obtained quickly by crowdsourcing for cell nucleus segmentation.

[0005] Currently, the unlabeled cell nucleus segmentation method, due to the unsuitability of the pixel mean, rotation degree and similarity in the nuclear segmentation proxy task to nuclear segmentation, results in unclear feature representation of the nucleus and inaccurate initial pseudo-labeling. The errors introduced by the pseudo-labeling may accumulate in the step-by-step training process. SUMMARY

[0006] In order to reduce the annotation workload, the present application proposes a self-supervised learning method for unlabeled cell nucleus segmentation method, which realizes the initialization of pseudo-labeling by self-supervised learning and staining reconstruction of the prior knowledge of histopathology images; trains a double-task model to realize nuclear segmentation, uses inter-task consistency and intra-task self-refinement strategies to make up for the information gap, and maintains the iterative refinement of the two tasks, which can be used as a good alternative solution when the annotation budget is tight or even without annotation budget.

[0007] The present application is realized by at least one of the following technical solutions.

[0008] The double-task cyclic self-supervised cell nucleus segmentation method based on staining similarity reconstruction comprises the following steps:

[0009] An prior knowledge guided self-supervised learning framework is established, and reliable initial pseudo-labels are generated through self-supervised learning of staining prior knowledge and image prior knowledge; the prior knowledge guided self-supervised learning framework comprises a contrast learning model and a staining reconstruction model to help the framework learn the image and staining prior knowledge;

[0010] A downstream double-task nuclear segmentation model is trained to alleviate the huge information gap between pseudo-labels and dense prediction, and two strategies of inter-task consistency and intra-task self-refinement are used to improve the performance of contrast learning and staining reconstruction, and finally reliable nuclear segmentation is realized.

[0011] Further, cooperation with medical institutions or research institutions is established to obtain a training set composed of fully annotated and authorized medical panoramic image data;

[0012] The annotation includes manual annotation of pathological changes and tissue structure by pathologists, or automatic annotation algorithm using a trained deep model, and manual review and modification based on the annotation;

[0013] Before constructing the training set, data cleaning and preprocessing are needed, and the preprocessing includes image denoising, color correction, cropping and scaling operations.

[0014] Further, the establishment of the contrast learning model comprises:

[0015] From the training set, a pair of original images X is randomly selected, and a random image enhancement operation is performed, including translation, scaling, flipping, cropping, etc., to obtain a transformed image, which is called an enhanced view; the enhanced view is regarded as a positive sample X + , and another pair of images randomly selected from the training set is regarded as a negative sample X - ); a contrast learning model f CL is used to extract the feature embedding model of the sample, the original image X, the positive sample X + , and the negative sample X - , to obtain corresponding image sample features F, F + , and F - , which are modeled as:

[0016] F, F + , and F - = f CL (X, X + , X - ) (2)

[0017] The triplet loss L triplet is calculated to minimize the feature embedding distance between the original image and the positive sample and maximize the distance between the original image and the negative sample:

[0018] L triplet = max{d(F, F + )-d(F, F - )+α, 0} (2)

[0019] Where max(*), d(*) and α represent the maximum function of the distance between elements in the triplet, the distance function and the minimum threshold, respectively.

[0020] Further, the staining reconstruction comprises:

[0021] According to the Beer-Lambert law, the intensity maps of two dyes, including hematoxylin map (H) and eosin map (E), are extracted from the original image X, a generative adversarial network (GAN) based image generation model is established to reconstruct the original image X, and a GAN loss L GAN is introduced to ensure the fidelity of the generated image X′:

[0022] L GAN = E X pdata(X) [logD(X)]+E X pdata(x) [log(1-D(f stain (X)))] (3)

[0023] where D(X) and f stain (X) represent the generator and discriminator operation in GAN network respectively; pdata(X) denotes the real sample distribution in training set, E X pdata(x) is the probability of corresponding sample X in training set;

[0024] Meanwhile, considering the similarity of H-staining and E-staining between the original image and the image generated by the GAN model can be used as an additional constraint, the staining loss L stain is introduced in the GAN model to guide the model:

[0025] L stain =∑(||H-H′||1+||E-E′||1) (4)

[0026] where H and E are the H-staining intensity map and E-staining intensity map extracted from the input image X, H' and E' represent the H-staining intensity map and E-staining intensity map extracted from the generated image X', and ||*||1 is the L1 regularization loss;

[0027] Pseudo-labels are generated from the contrastive learning model and the staining reconstruction model. Given the original image X, the contrastive activation map A is generated:

[0028] A=λ*A CL +(1-λ)*A stain (5)

[0029] where λ controls the contribution weight of each activation mapping, A CL and A stain represent the mappings generated by the above-mentioned contrastive learning model f CL and the GAN model respectively. Finally, the contrastive activation map A is used to set a threshold for threshold segmentation, and then the watershed algorithm is used to separate the attached nuclei to obtain the initial pseudo-label M init of the nuclei:

[0030]

[0031] where t represents the threshold.

[0032] Further, the downstream dual-task nucleus segmentation model includes a nucleus detection model and a nucleus segmentation model, and the outputs of the nucleus detection model and the nucleus segmentation model are used as mutual guidance.

[0033] Further, the nucleus detection model is composed of ResUNet, and a mean square loss L mse and a cross-entropy loss L ce are constructed, and the two losses are passed through the first round of prediction results and the pseudo-label P init or Minit , which is calculated by the following equation:

[0034]

[0035]

[0036] where o i represents the i-th pixel of the model output, p i and m i are the pixels corresponding to the pseudo-labels, Ω represents the set of foreground pixels, ω i represents the weight assigned to the corresponding pixel, which is used to address the imbalance between the marked points and background pixels.

[0037] Further, after obtaining the initial pseudo-label M init , an extended Gaussian mask generator is used to generate the pseudo-point label P init for the initialization of the cell nucleus detection model, and the conversion formula is as follows:

[0038]

[0039] where D is the distance from the pixel to the nearest marked point, σ is the root mean square width of the Gaussian under the standard deviation, r1 and r2 represent the minimum and maximum nuclear radii respectively, and the two parameters are set through experiments and experience.

[0040] Further, in the n-th round of training, the shallow features of the first n-1 rounds of training are used to supervise the n-th round of training, and the K-means clustering method is used to generate a clustering map extracted from the cell nucleus detection model. The cell nucleus detection model separates the best feature representation of the nucleus and the background through three clustering, and the pseudo-point label is obtained by using the pseudo-point label The voting strategy is used to select the core representative cluster, and the cluster with the highest hit rate of point labeling is mapped as the nucleus mapping, and the other two clusters are defined as the background. Further, an extended Gaussian mask generator is used to obtain the pseudo-point label and train the cell nucleus detection model, and the loss function L det-intra is defined as:

[0041]

[0042] where, is the output result of the n-th round of cell nucleus detection model.

[0043] Further, for the training of the cell nucleus segmentation model, the cross-entropy loss L seg-intr is used in the intra-task self-refinement loss L ce :

[0044]

[0045] wherein, is the prediction result of the nucleus segmentation model of the nth round, is the cluster mask;

[0046] For the nucleus segmentation model, the detection model output result of the nth round is used The segmentation model is trained, and in order to fit the segmentation model, the detection result is converted into a Voronoi diagram V n , using the cross-entropy loss L ce The task consistency loss L of the nucleus segmentation model is constructed seg-inter :

[0047]

[0048] wherein is the output of the nucleus segmentation model;

[0049] The nucleus segmentation model is used to generate pseudo point labels The detection model output result of the nth round is combined The task consistency loss L of the nucleus detection model is constructed by the mean square loss Lmse formula det-inter Guide the training of the nucleus detection model:

[0050]

[0051] Further, the overall objective function of the nucleus detection and segmentation model is defined as:

[0052] L det = beta * L det-inter + (1-beta) * L det-intra (14)

[0053] L seg = beta * L seg-inter + (1-beta) * L seg-intra (15)

[0054] Wherein beta is the weight of the intra-task loss and the inter-task loss, L seg-inter is the task consistency loss of the nucleus segmentation model, L det-inter is the task consistency loss of the nucleus detection model.

[0055] Compared with the prior art, the beneficial effects of the present application are:

[0056] The proposed method relies on good model design and strategy, and outperforms existing self-supervised nucleus segmentation methods on two publicly available datasets. It even outperforms some previous weakly supervised learning models. When the medical image dataset used for training has a tight label budget or even no label, it can be used as a good solution. BRIEF DESCRIPTION OF DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0058] Figure 1 is a flowchart of the two-task cycle self-supervised nucleus segmentation method of the embodiment based on staining similarity reconstruction;

[0059] Figure 2 is a flowchart of two proxy tasks of the embodiment;

[0060] Figure 3 is a comparison result graph of various methods of the embodiment. DETAILED DESCRIPTION

[0061] The embodiments of the present application will be described in detail below, and examples of the embodiments are shown in the drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and are intended to explain the embodiments of the present application, and cannot be understood as a limitation of the present application.

[0062] To solve the problem that artificial labeling is difficult to obtain, the embodiment proposes a label-free self-supervised learning nucleus segmentation framework. Since there is no artificial labeling, how to generate reliable pseudo-labels and how to bridge the information gap become the focus of research. As shown in Figure 1 The method of the present application is a two-stage progressive model. The first stage proposes a prior knowledge guided self-supervised learning framework, which realizes the initialization of pseudo-labels by self-supervised learning and staining reconstruction of the prior knowledge of histopathology images. In the second stage, a dual-task model is trained to realize nucleus segmentation. The inter-task consistency and intra-task self-refinement strategy are proposed to bridge the information gap and maintain the iterative refinement of the two tasks.

[0063] As Figure 1The method includes a priori knowledge guided self-supervised learning framework and a downstream dual-task nuclear segmentation model. The model can be applied to the field of medical image analysis, such as tissue section image analysis in pathology. It can help automatically extract nuclear information to support tumor diagnosis, disease research, and drug development. In addition, the model can be used in biological research for cell behavior analysis, cell life cycle research, and other fields to promote the understanding of the internal processes of cells.

[0064] (1) First stage

[0065] Initial pseudo-label generation is a basic step for unlabelled nuclear segmentation, which affects the quality of subsequent model training. In this stage, a priori knowledge guided self-supervised learning framework is proposed to learn deep image priori knowledge from a large number of unlabelled images without human intervention and generate pseudo-labels. To achieve this, stage one introduces two proxy tasks (as shown in Figure 1 ), including a contrast learning model and a staining reconstruction model, to help the framework learn image and staining priori knowledge.

[0066] Cooperation with medical institutions or research institutions is established to obtain fully annotated and authorized medical panoramic images (WSI) data, which constitute the training set. Annotation can include manual annotation of pathological changes, tissue structure, etc. by pathologists, or annotation using automatic algorithms. The quality of annotation directly affects the training effect of the model. At the same time, the data usually has high resolution and complex structure, and may have noise, artifacts, etc. Before building the training set, data cleaning and preprocessing are needed to ensure data quality and consistency. Preprocessing includes image denoising, color correction, cropping, and scaling operations to improve the robustness and generalization ability of the model. Through the above steps, a high-quality medical image training set can be constructed to provide reliable data support for the training and application of deep learning models.

[0067] For proxy task 1 (contrast learning), from the above training set, an original image X is randomly selected, and random image enhancement operations are performed on it, including translation, scaling, flipping, cropping, etc. to obtain the transformed image, which is called the enhanced view. The enhanced view is regarded as the positive sample X + , and another image randomly selected from the training set is regarded as the negative sample X - ); a contrast learning model f CL is used to extract the feature embedding of the sample. The original image X, the positive sample X + , and the negative sample X - can obtain the corresponding image sample features F, F + , and F- , is modeled as:

[0068] F, F + , F - = f CL (X, X + , X - ) (3)

[0069] The triplet loss L triplet is calculated to minimize the distance between the feature embedding of the original image and the positive sample and maximize the distance between the original image and the negative sample:

[0070] L triplet = max{d(F, F + )-d(F, F - )+a, 0} (2)

[0071] where max(*), d(*) and a represent the maximum function of the distance between elements in the triplet, the distance function and the minimum threshold, respectively.

[0072] For the proxy task 2 (stain reconstruction), according to the Beer-Lambert law, the intensity maps of two dyes, including hematoxylin (H) and eosin (E), are extracted from the original image X, an image generation model based on generative adversarial nets (GAN) is established to reconstruct the original image X, and a GAN loss L GAN is introduced to ensure the fidelity of the generated image X':

[0073] L GAN = E X pdata(X) [log D(X)] + E X pdata(x) [log (1-D(f stain (X)))] (3)

[0074] where D(X) and f stain (X) represent the generator and discriminator operations in the GAN network, respectively. Pdata(X) represents the distribution of real samples in the training set, and E X pdata(x) is the probability of the corresponding sample X in the training set.

[0075] At the same time, the similarity of H staining and E staining of the original image and the image generated by the GAN model can be used as an additional constraint. In the GAN model, a staining loss L stain is introduced to guide the model:

[0076] L stain = å (||H-H' ||1+||E-E' ||1) (4)

[0077] where H and E are the H-stain intensity map and E-stain intensity map extracted from the input image X, H' and E' represent the H-stain intensity map and E-stain intensity map extracted from the generated image X', ||*||1 is the L1 regularization loss. The relationship and implementation steps of the two-agent task are as shown in (a) and (b) of Figure 2 .

[0078] The pseudo-labels are generated from the contrast learning model and the staining reconstruction model, and given the original image X, the contrast activation map A is generated:

[0079] A = λ * A cL + (1-λ) * A stain (5)

[0080] where λ controls the contribution weight of each activation mapping, A cL and A stain represent the mappings generated by the contrast learning model f CL and the GAN model respectively. Finally, the contrast activation map A is used to perform threshold segmentation by setting a threshold t, and then the watershed algorithm is used to separate the attached nuclei to obtain the initial pseudo-label M init of the nuclei.

[0081]

[0082] After obtaining the initial pseudo-label M init , an extended Gaussian mask generator is further used to generate a pseudo-point label P init for the initialization of the nucleus detection model, and the conversion formula is as follows:

[0083]

[0084] where D is the distance from the pixel to the nearest labeled point, σ is the root mean square width of the Gaussian under the standard deviation. r1 and r2 represent the minimum and maximum nuclear radii respectively, and the two parameters are set through experiments and experience.

[0085] (2) Second stage

[0086] Given the pseudo-labels generated by the contrast learning model and the staining reconstruction model trained in the first stage, the present application introduces a double-task model (as shown in Figure 1 ) including the nucleus detection task and the nucleus segmentation task. In order to alleviate the huge information gap between the pseudo-labels and the dense prediction, it is proposed to use the inter-task consistency and intra-task self-refinement strategies to improve the performance of the two tasks, and finally realize reliable nucleus segmentation.

[0087] Specifically, in order to initialize the nucleus detection model, the pseudo-label P init and M initTrain the cell nucleus detection model, which is composed of ResUNet and constructs the mean square loss L mse With cross entropy loss L ce , the two losses are passed through the first round prediction results of the given detection model With pseudo label P init or M init , calculated using the following formula:

[0088]

[0089]

[0090] Among them i Represents the i-th pixel output by the model. p i and m i is the pixel corresponding to the pseudo label. Ω represents the set of foreground pixels. i represents the weight assigned to the corresponding pixel, which is used to resolve the imbalance between the marker and background pixels.

[0091] After the first round of model initialization, this embodiment uses inter-task consistency and intra-task self-refinement strategies to further bridge the information gap and improve the original cell nucleus segmentation model and cell nucleus detection model. Figure 1 As shown in (c). Here, this embodiment defines n as an indicator of the training round. Since the cell nucleus detection task and the cell nucleus segmentation task have similar but different goals. Constructing a relationship between the two tasks may be beneficial to the model training of the two tasks and improve the generalization ability and feature representation ability of the model. Similar to the semi-supervised learning method of constructing consistency between labeled features and unlabeled features, this embodiment introduces the consistency between the two tasks as inter-task consistency. To achieve this, the outputs of the cell nucleus detection model and the cell nucleus segmentation model are used to guide the other model. For the cell nucleus segmentation model, the output result of the cell nucleus detection model of the nth round is used To guide the training of the segmentation model, the detection results are converted into the Voronoi diagram V n .in is the output of the kernel segmentation model, also using the cross entropy loss L ce Inter-task consistency loss L for constructing cell nucleus segmentation model seg-inter :

[0092]

[0093] The prediction template of the segmentation model is also used to generate pseudo point labels Combined output results of the cell nucleus detection model for the nth round Through the mean square loss L mseThe task consistency loss L of the formula constructing the nucleus detection model det-inte Guiding the training of the nucleus detection model:

[0094]

[0095] In order to prevent the model from being finally guided by noise labels, an intra-task self-refinement strategy is proposed to introduce self-intervention in the early stage of model training to alleviate the performance decline caused by overfitting. For the detection task, in the nth round of training, the shallow features of the first n-1 rounds of training are used to supervise the nth round of training.

[0096] As an embodiment, the K-means clustering method is used to generate the cluster map The shallow feature map extracted from the detection model. In practical applications, the nucleus detection model separates the best feature representation of the nucleus and the background through three clusters. Because there is no manual annotation and intervention, the cluster task cannot perceive the cluster of the cluster representing the nucleus. By using the pseudo point label The cluster representing the core is selected by voting strategy. The cluster with the highest hit rate of point labeling is called nucleus mapping. The other two clusters are defined as background. Further use the extended Gaussian mask generator to obtain the pseudo point label And train the nucleus detection model, the loss function L det-intr is defined as:

[0097]

[0098] Wherein, is the prediction result of the detection model. Similar to the detection model, the shallow features extracted in the last round are used to guide the intra-task self-refining process. The cluster mapping is generated from the model in the last round. Then the cluster mask is generated by voting the pseudo point label As the pseudo mask of the segmentation network.

[0099] For the training of the nucleus segmentation model, the cross entropy loss L ce is used in the intra-task self-refining loss L seg-intr :

[0100]

[0101] Wherein, is the prediction result of the nth round of segmentation model. After the initialization of the first round of model, the intra-task self-refining strategy is activated in the second round of training to perform early self-intervention to alleviate the overfitting on the noise label. The overall objective function of the nucleus detection and segmentation model is defined as:

[0102] L det = β*L det-inter+ (1 - β) * L det-intra (14)

[0103] L seg = β * L seg-inte + (1 - β) * L seg-intra (15)

[0104] wherein β is a weight of the intra-task loss and the inter-task loss.

[0105] As an embodiment, the embodiment defines β = 0.5 as the weight of the intra-task self-refinement loss and the inter-task consistency loss. Experiments are performed on the scientific research public data sets MoNuSeg and CPM-17, and the segmentation results of the embodiment are as shown in FIG. 6. Figure 3

[0106] The above embodiments are the preferred embodiments of the present application, but the embodiments of the present application are not limited to the above embodiments, and any changes, modifications, substitutions, combinations, simplifications made without departing from the spirit and principles of the present application should be equivalent replacement methods, and are all included in the protection scope of the present application.​

Claims

1. A dual-task cyclic self-supervised nuclei segmentation method based on stain similarity reconstruction, characterized in that, Comprising the following steps: An a priori knowledge guided self-supervised learning framework is established to generate reliable initial pseudo labels through self-supervised learning of dyeing a priori knowledge and image a priori knowledge; the a priori knowledge guided self-supervised learning framework comprises a contrast learning model and a dyeing reconstruction model to help the framework learn image and dyeing a priori knowledge; The dyeing reconstruction comprises: According to the Beer-Lambert law, the intensity maps of two dyes are extracted from the original image X, including the hematoxylin map and the eosin map, an image generation model based on the generative adversarial network (GAN) is established, the original image X is reconstructed, and the GAN loss L GAN to ensure the fidelity of the generated image X'. where D(X) and f stain (X) represent the generator and discriminator operations in the GAN network, respectively; pdata(X) denotes the real sample distribution in the training set, E X pdata(X) is the probability of the corresponding sample X in the training set. Meanwhile, considering the similarity of H staining and E staining of the original image and the image generated by the GAN network, the staining loss L stain To guide the model: L stain =∑(||H-H'||1+||E-E'||1) (4) Wherein H and E are H dyeing intensity map and E dyeing intensity map extracted from input image X, H' and E' represent H dyeing intensity map and E dyeing intensity map extracted from generated image X', ||*||1 is L1 regularization loss; Pseudo labels are generated from the contrast learning model and the dyeing reconstruction model, and given the original image X, the contrast activation map A is generated: A = λ * A CL + (1 - λ) * A stain (5) where λ controls the contribution weight of each activation map, A CL and A stain represent the above-mentioned contrast learning model f CL and the map generated by the GAN network, respectively. Finally, the contrast activation map A is used to perform threshold segmentation by setting a threshold, and then the watershed algorithm is used to separate the attached nuclei to obtain the initial pseudo-labeling M of the nuclei init : Wherein t represents the threshold value; A downstream dual-task nuclear segmentation model is trained to alleviate the huge information gap between pseudo labels and dense prediction, and two strategies of inter-task consistency and intra-task self-refinement are used to improve the performance of contrast learning and dyeing reconstruction, and finally realize nuclear segmentation; the downstream dual-task nuclear segmentation model comprises a cell nucleus detection model and a cell nucleus segmentation model, and the outputs of the cell nucleus detection model and the cell nucleus segmentation model serve as mutual guidance.

2. The dual-task cyclic self-supervised nuclei segmentation method based on staining similarity reconstruction according to claim 1, wherein, Cooperation with medical institutions or research institutions is established to obtain a training set composed of fully labeled and authorized medical panoramic image data; Labeling includes manual labeling of pathologists on lesion areas and tissue structures, or automatic labeling algorithm using a trained deep model, and manual review and modification based on the labeling. Before constructing the training set, data cleaning and preprocessing are needed, and preprocessing includes image denoising, color correction, cropping and scaling operations.

3. The dual-task cyclic self-supervised nuclei segmentation method based on staining similarity reconstruction according to claim 1, wherein, The establishment of the contrast learning model comprises: From the training set, a pair of original images X is randomly selected, and a random image enhancement operation is performed on it, including translation, scaling, flipping, and cropping to obtain a transformed image, which is called an enhanced view; the enhanced view is regarded as a positive sample X + , while another image randomly selected from the training set is regarded as a negative sample X - ; a contrast learning model f CL is used to extract the feature embedding model of the sample, and the original image X, the positive sample X + , and the negative sample X - respectively obtain corresponding image sample features F, F + , and F - , which are modeled as: F, F + F, F - = f CL (X, X + , X - ) (1) Compute the triple loss L triplet minimize the distance between the feature embedding of the original image and the positive sample and maximize the distance between the original image and the negative sample: L triplet = max{d(F, F + )- d(F, F - )+ a, 0} (2) where max(*), d(*) and a denote the maximum function of the distance between elements in the triplet, the distance function and the minimum threshold, respectively, F, F + ,F - are the original image X, the positive sample X + , the negative sample X - corresponding image sample features.

4. The dyadic similarity reconstruction based dual-task cyclic self-supervised nuclei segmentation method according to claim 1, wherein, The nucleus detection model is composed of ResUNet, and a mean square loss L is constructed mse With cross-entropy loss L ce , two losses are calculated by the first round of prediction results of the passing nucleus detection model With pseudo label P init Or M init , calculated by the following formula: where o i represents the i-th pixel of the model output, p i and m i is the pixel corresponding to the pseudo label, Ω represents the set of foreground pixels, ω i represents the weight assigned to the corresponding pixel, which is used to address the imbalance between the marked points and the background pixels.

5. The dual-task cyclic self-supervised nuclei segmentation method based on staining similarity reconstruction according to claim 4, wherein, Obtain the initial pseudo-label M init After that, the extended Gaussian mask generator is used to generate the pseudo point label P init For the initialization of the cell nucleus detection model, the conversion formula is as follows: Wherein D is the distance from the pixel to the nearest labeled point, σ is the root mean square width of Gaussian under standard deviation, r1 and r2 represent the minimum and maximum nuclear radii respectively, and the two parameters are set through experiments and experience.

6. The dual-task cyclic self-supervised nuclei segmentation method based on staining similarity reconstruction according to claim 5, wherein, In the nth round of training, the shallow features of the first n-1 rounds of training are used to supervise the nth round of training, and the K-means clustering method is used to generate a clustering map The shallow feature map extracted from the cell nucleus detection model, the cell nucleus detection model separates the best feature representation of the nucleus and the background through three clustering, and the pseudo point label is obtained by using the pseudo point label The voting strategy is used to select the cluster representing the core, the cluster with the highest hit rate of point labeling is mapped as the nucleus mapping, and the other two clusters are defined as the background. Further, an extended Gaussian mask generator is used to obtain a pseudo point label And train the cell nucleus detection model, the loss function L det-intra is defined as: wherein, is the output result of the nth round of the nucleus detection model.

7. The dyadic similarity reconstruction-based dual-task cyclic self-supervised nuclei segmentation method according to any one of claims 3-6, characterized in that, For training of the nucleus segmentation model, an in-task self-refinement loss L seg-intra cross-entropy loss L ce : wherein, is the prediction result of the nth round of the nucleus segmentation model, is the cluster mask; For the nucleus segmentation model, the output results of the nth round of nucleus detection model are used The segmentation model is trained, and the detection results are converted into a Voronoi diagram V in order to fit the segmentation model n , using cross-entropy loss L ce The task consistency loss L of the nucleus segmentation model is constructed seg-inter : wherein is the output of the nucleus segmentation model; Generating pseudo-point labels using a cell nucleus segmentation model Combining the output results of the nth round of cell nucleus detection model Through the mean square loss L mse Formulating the inter-task consistency loss L of the cell nucleus detection model det-inter Guiding the training of the cell nucleus detection model:

8. The dyadic similarity reconstruction based dual-task cyclic self-supervised nuclei segmentation method according to claim 1, wherein, The overall objective function of the nuclear detection and segmentation model is defined as: L det = β * L det-inter + (1 - β) * L det-intra (14) L seg = β * L seg-inter + (1 - β) * L seg-intra (15) where β is the weight of intra-task loss and inter-task loss, L seg-inter is the inter-task consistency loss for the nucleus segmentation model, L deg-inter is the inter-task consistency loss for the nucleus detection model.

Images