Cell nucleus image segmentation model training method and cell nucleus image segmentation method

Abstract

The invention provides a training method of a cell nucleus image segmentation model and a cell nucleus image segmentation method, and relates to the technical field of image segmentation. The method comprises the following steps: performing data enhancement processing on an unlabeled sample cell nucleus image to obtain a first sample cell nucleus image and a second sample cell nucleus image which are different; inputting the first sample cell nucleus image into a student model to obtain a first phenotype activation feature of the first sample cell nucleus image, and inputting the second sample cell nucleus image into a teacher model to obtain a second phenotype activation feature of the second sample cell nucleus image; determining a phenotype consistency loss based on a difference between the first phenotype activation feature and the second phenotype activation feature; and based on the phenotype consistency loss, adjusting model parameters of the student model to obtain a trained cell nucleus image segmentation model. The method can reduce the training cost of the model, improves the model training effect of the cell nucleus image segmentation model, and improves the image segmentation accuracy of the model.

Description

Technical Field

[0001] The present invention relates to the technical field of image segmentation, and in particular, to a training method for a cell nucleus image segmentation model and a cell nucleus image segmentation method. Background Art

[0002] Cell nucleus image segmentation is to segment the foreground of a cell nucleus image to separate the cell nucleus region and the background region, and further separate each cell nucleus instance. And cell nucleus image segmentation is the basis for many downstream tasks such as immunohistochemical analysis and tumor microenvironment analysis. Therefore, it is necessary to implement cell nucleus image segmentation.

[0003] Currently, a cell nucleus image segmentation model is trained through a fully supervised training method. However, the fully supervised training method highly depends on the refined cell nucleus image segmentation result labels, that is, highly depends on accurate label annotation work. And the cell nucleus images to be segmented usually contain hundreds or thousands of cell nuclei, and even doctors are needed to perform accurate annotation, resulting in a high cost of label annotation, and thus a high training cost of the cell nucleus image segmentation model. Summary of the Invention

[0004] The present invention provides a training method for a cell nucleus image segmentation model and a cell nucleus image segmentation method, which are used to solve the defect of high training cost of the cell nucleus image segmentation model in the prior art, realize a training method for a cell nucleus image segmentation model with low cost, and train a high-performance cell nucleus image segmentation model.

[0005] The present invention provides a training method for a cell nucleus image segmentation model, including: Performing data augmentation processing on the unlabeled sample cell nucleus images to obtain different first sample cell nucleus images and second sample cell nucleus images; Inputting the first sample cell nucleus image into a student model to obtain a first phenotypic activation feature of the first sample cell nucleus image, and inputting the second sample cell nucleus image into a teacher model to obtain a second phenotypic activation feature of the second sample cell nucleus image; Determining a phenotypic consistency loss based on the difference between the first phenotypic activation feature and the second phenotypic activation feature; Adjusting the model parameters of the student model based on the phenotypic consistency loss to obtain a trained cell nucleus image segmentation model; Among them, the unlabeled sample cell nucleus image is the sample cell nucleus image of the segmentation result of the unlabeled cell nucleus image; both the student model and the teacher model are used to perform cell nucleus image segmentation on the input cell nucleus image; the first phenotypic activation feature is obtained by fusing the first phenotypic vector of the first sample cell nucleus image and the first feature map of the first sample cell nucleus image, and the first phenotypic vector is the model parameter that can be learned by the student model; the second phenotypic activation feature is obtained by fusing the second phenotypic vector of the second sample cell nucleus image and the second feature map of the second sample cell nucleus image, and the second phenotypic vector is the model parameter that can be learned by the teacher model; both the first phenotypic vector and the second phenotypic vector are used to characterize the phenotypic features of the cell nucleus.

[0006] According to a training method of a cell nucleus image segmentation model provided by the present invention, the step of inputting the first sample cell nucleus image into the student model to obtain the first phenotypic activation feature of the first sample cell nucleus image includes: Input the first sample cell nucleus image into the student model, obtain the first feature map output by the encoding layer in the student model, and obtain K first phenotypic vectors of the first sample cell nucleus image; K is the preset number of cell nucleus categories; Perform feature fusion on the K first phenotypic vectors and the first feature map respectively to obtain first phenotypic activation sub-features of K cell nucleus categories; Based on the K first phenotypic activation sub-features, determine the first phenotypic activation feature of the first sample cell nucleus image.

[0007] According to a training method of a cell nucleus image segmentation model provided by the present invention, before inputting the first sample cell nucleus image into the student model, obtaining the first feature map output by the encoding layer in the student model, and obtaining K first phenotypic vectors of the first sample cell nucleus image, the method further includes: For any one of the K cell nucleus categories, input the first labeled sample cell nucleus image corresponding to the cell nucleus category into the student model, obtain the third feature map output by the encoding layer in the student model, and obtain the target first phenotypic vector of the first labeled sample cell nucleus image; the first labeled sample cell nucleus image is the sample cell nucleus image that labels the segmentation result of the cell nucleus image corresponding to the cell nucleus category, and the target first phenotypic vector is used to characterize the phenotypic features of the cell nucleus corresponding to the cell nucleus category; Based on the cell nucleus image segmentation result label corresponding to the first labeled sample cell nucleus image, determine a number of target features from the third feature map; any one of the target features is the feature corresponding to the cell nucleus region belonging to the cell nucleus category; Generate a target feature vector based on the several target features; Determine a first loss based on the difference between the target first phenotype vector and the target feature vector; Adjust the model parameters of the student model based on the first loss.

[0008] According to a training method of a cell nucleus image segmentation model provided by the present invention, the step of respectively performing feature fusion on the K first phenotype vectors and the first feature map to obtain first phenotype activator features of K cell nucleus categories includes: Perform a channel dimension transformation on the first feature map to obtain an intermediate feature map with the same dimension as the K first phenotype vectors; Respectively perform a pixel-by-pixel dot product process on the K first phenotype vectors and the intermediate feature map to obtain first phenotype activator features of K cell nucleus categories.

[0009] According to a training method of a cell nucleus image segmentation model provided by the present invention, the step of determining a phenotype consistency loss based on the difference between the first phenotype activation feature and the second phenotype activation feature includes: Obtain a first predicted cell nucleus image segmentation result corresponding to the second sample cell nucleus image output by the teacher model; Input the first predicted cell nucleus image segmentation result into a segmentation quality scoring model to obtain a comprehensive quality scoring map output by the segmentation quality scoring model; the comprehensive quality scoring map includes comprehensive quality scoring values of each region in the second sample cell nucleus image; Based on the comprehensive quality scoring map, perform a weighted process on the difference between the first phenotype activation feature and the second phenotype activation feature to obtain a phenotype consistency loss; Wherein, the segmentation quality scoring model is used to evaluate the cell nucleus image segmentation quality of the input cell nucleus image segmentation result.

[0010] According to a training method of a cell nucleus image segmentation model provided by the present invention, the segmentation quality scoring model is trained based on the following method: Input a sample cell nucleus image segmentation result into an initial quality scoring layer to obtain a predicted quality scoring map with multiple dimensions output by the initial quality scoring layer; any dimension of the predicted quality scoring map includes quality scoring values of each region in the cell nucleus image before segmentation corresponding to the sample cell nucleus image segmentation result; Determine a second loss based on the quality scoring map labels with multiple dimensions corresponding to the sample cell nucleus image segmentation result and the predicted quality scoring map with multiple dimensions; Based on the second loss, train the initial quality scoring layer to obtain the segmentation quality scoring model based on the trained quality scoring layer and the comprehensive scoring layer; The comprehensive scoring layer is used to fuse multiple dimensions of quality scoring maps output by the quality scoring layer to obtain a comprehensive quality scoring map.

[0011] According to a method for training a cell nucleus image segmentation model provided by the present invention, the segmentation result of the sample cell nucleus image is obtained based on the following method: Based on the second labeled sample cell nucleus image, train the initial student model to obtain the student model; the second labeled sample cell nucleus image is a sample cell nucleus image with the segmentation result of the labeled cell nucleus image; Input the unlabeled sample cell nucleus image into the student model to obtain the segmentation result of the sample cell nucleus image output by the student model.

[0012] According to a method for training a cell nucleus image segmentation model provided by the present invention, before adjusting the model parameters of the student model based on the phenotypic consistency loss, the method further includes: Input the third labeled sample cell nucleus image into the student model to obtain the second predicted cell nucleus image segmentation result output by the student model; the third labeled sample cell nucleus image is a sample cell nucleus image with the segmentation result of the labeled cell nucleus image, and the number of samples of the third labeled sample cell nucleus image is less than the number of samples of the unlabeled sample cell nucleus image; Based on the difference between the second predicted cell nucleus image segmentation result and the cell nucleus image segmentation result label corresponding to the third labeled sample cell nucleus image, determine the third loss; Correspondingly, adjusting the model parameters of the student model based on the phenotypic consistency loss includes: Based on the phenotypic consistency loss and the third loss, adjust the model parameters of the student model.

[0013] According to a method for training a cell nucleus image segmentation model provided by the present invention, after adjusting the model parameters of the student model based on the phenotypic consistency loss, the method further includes: Based on the current model parameters of the teacher model and the adjusted model parameters of the student model, update the model parameters of the teacher model; The model structure of the student model is the same as the model structure of the teacher model.

[0014] According to a training method of a cell nucleus image segmentation model provided by the present invention, before inputting the first sample cell nucleus image into the student model to obtain the first phenotypic activation feature of the first sample cell nucleus image and inputting the second sample cell nucleus image into the teacher model to obtain the second phenotypic activation feature of the second sample cell nucleus image, the method further includes: Based on the second annotated sample cell nucleus image, adjusting the model parameters of the initial student model to obtain the student model; the second annotated sample cell nucleus image is a sample cell nucleus image annotating the segmentation result of the cell nucleus image; Synchronizing the model parameters of the adjusted initial student model to the teacher model; Wherein, the model structure of the student model is the same as that of the teacher model.

[0015] The present invention also provides a cell nucleus image segmentation method, including: Inputting the cell nucleus image to be segmented into the cell nucleus image segmentation model to obtain the target cell nucleus image segmentation result output by the cell nucleus image segmentation model; Wherein, the cell nucleus image segmentation model is trained based on the training method of any one of the above-mentioned cell nucleus image segmentation models.

[0016] The present invention also provides a training device for a cell nucleus image segmentation model, including: A data enhancement module, configured to perform data enhancement processing on the unannotated sample cell nucleus image to obtain different first sample cell nucleus images and second sample cell nucleus images; An image input module, configured to input the first sample cell nucleus image into the student model to obtain the first phenotypic activation feature of the first sample cell nucleus image, and input the second sample cell nucleus image into the teacher model to obtain the second phenotypic activation feature of the second sample cell nucleus image; A loss determination module, configured to determine a phenotypic consistency loss based on the difference between the first phenotypic activation feature and the second phenotypic activation feature; A parameter adjustment module, configured to adjust the model parameters of the student model based on the phenotypic consistency loss to obtain the trained cell nucleus image segmentation model; Among them, the unlabeled sample cell nucleus image is the sample cell nucleus image of the segmentation result of the unlabeled cell nucleus image; both the student model and the teacher model are used to perform cell nucleus image segmentation on the input cell nucleus image; the first phenotypic activation feature is obtained by fusing the first phenotypic vector of the first sample cell nucleus image with the first feature map of the first sample cell nucleus image, and the first phenotypic vector is the model parameter that can be learned by the student model; the second phenotypic activation feature is obtained by fusing the second phenotypic vector of the second sample cell nucleus image with the second feature map of the second sample cell nucleus image, and the second phenotypic vector is the model parameter that can be learned by the teacher model; both the first phenotypic vector and the second phenotypic vector are used to characterize the phenotypic features of the cell nucleus.

[0017] The present invention also provides a cell nucleus image segmentation device, including: An image segmentation module, configured to input the cell nucleus image to be segmented into the cell nucleus image segmentation model, and obtain the target cell nucleus image segmentation result output by the cell nucleus image segmentation model; Among them, the cell nucleus image segmentation model is trained by the training method of any one of the above-mentioned cell nucleus image segmentation models.

[0018] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the training method of any one of the above-mentioned cell nucleus image segmentation models or implements the cell nucleus image segmentation method of any one of the above-mentioned.

[0019] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the training method of any one of the above-mentioned cell nucleus image segmentation models or implements the cell nucleus image segmentation method of any one of the above-mentioned.

[0020] The present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the training method of any one of the above-mentioned cell nucleus image segmentation models or implements the cell nucleus image segmentation method of any one of the above-mentioned.

[0021] The training method and nuclear image segmentation method of the nuclear image segmentation model provided by the present invention perform data augmentation processing on unlabeled sample nuclear images to obtain different first sample nuclear images and second sample nuclear images. The first sample nuclear images are input into the student model to obtain the first phenotypic activation features of the first sample nuclear images, and the second sample nuclear images are input into the teacher model to obtain the second phenotypic activation features of the second sample nuclear images. Based on the differences between the first phenotypic activation features and the second phenotypic activation features, the phenotypic consistency loss is determined. Based on the phenotypic consistency loss, the model parameters of the student model are adjusted. The unlabeled sample nuclear images are sample nuclear images of the segmentation results of unlabeled nuclear images. Thus, the nuclear image segmentation model can be trained based on the unlabeled sample nuclear images, thereby reducing the label annotation cost, that is, reducing the training cost of the nuclear image segmentation model. At the same time, based on the differences between the first phenotypic activation features and the second phenotypic activation features, the phenotypic consistency loss is determined. The first phenotypic activation feature is obtained by fusing the first phenotypic vector of the first sample nuclear image and the first feature map of the first sample nuclear image. The first phenotypic vector is a model parameter that can be learned by the student model. The second phenotypic activation feature is obtained by fusing the second phenotypic vector of the second sample nuclear image and the second feature map of the second sample nuclear image. The second phenotypic vector is a model parameter that can be learned by the teacher model. Both the first phenotypic vector and the second phenotypic vector are used to characterize the phenotypic features of the nucleus. Thus, based on the phenotypic consistency loss, the classification and discrimination ability of the nuclear image segmentation model for different types of nuclei is enhanced. That is, learnable phenotypic vectors are set in both the student model and the teacher model to be used for mining the fine-grained phenotypic features of different types of nuclei to perform more fine-grained consistency constraints on the phenotypic features generated during the model, thereby improving the model training effect of the nuclear image segmentation model and further improving the image segmentation accuracy of the trained nuclear image segmentation model. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0023] Figure 1 is one of the flowcharts of the training method of the nuclear image segmentation model provided by the present invention.

[0024] Figure 2 is the second flowchart of the training method of the nuclear image segmentation model provided by the present invention.

[0025] Figure 3 It is the third flow schematic diagram of the training method of the nucleus image segmentation model provided by the present invention.

[0026] Figure 4 It is the fourth flow schematic diagram of the training method of the nucleus image segmentation model provided by the present invention.

[0027] Figure 5 It is the fifth flow schematic diagram of the training method of the nucleus image segmentation model provided by the present invention.

[0028] Figure 6 It is the sixth flow schematic diagram of the training method of the nucleus image segmentation model provided by the present invention.

[0029] Figure 7 It is the flow schematic diagram of the nucleus image segmentation method provided by the present invention.

[0030] Figure 8 It is the structural schematic diagram of the training device of the nucleus image segmentation model provided by the present invention.

[0031] Figure 9 It is the structural schematic diagram of the nucleus image segmentation device provided by the present invention.

[0032] Figure 10 It is the structural schematic diagram of the electronic device provided by the present invention. Specific Embodiments

[0033] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without making creative efforts based on the embodiments in the present invention belong to the scope of protection of the present invention.

[0034] Currently, a nucleus image segmentation model is trained through a fully supervised training method. However, the fully supervised training method highly depends on the fine nucleus image segmentation result labels, that is, it highly depends on accurate label annotation work. And the nucleus images to be segmented usually contain hundreds or thousands of nuclei, and even doctors are required for accurate annotation, resulting in a high cost of label annotation, and thus a high training cost of the nucleus image segmentation model.

[0035] In the prior art, even if a nucleus image segmentation model is trained by a semi-supervised training method, the nucleus itself is small and the number of instances is large, and the semi-supervised algorithm in natural images is difficult to be applied to nucleus image segmentation, that is, the image segmentation accuracy of the nucleus image segmentation model trained by the existing semi-supervised training method is not high. Specifically, it is mainly trained by constructing a consistency constraint loss in a teacher-student model. However, there are two problems with this method: one is that the consistency constraint loss actually uses the output of the teacher model to supervise the training of the student model, but the prediction of the model is different from the true label and there are often incorrect predictions. Therefore, noise will be introduced in the process of constructing the consistency constraint, resulting in a reduction in the model training effect, and further resulting in low image segmentation accuracy of the trained nucleus image segmentation model; the other is that the existing consistency constraint is relatively loose, only requiring the teacher model and the student model to be consistent in the overall output, resulting in a reduction in the model training effect, and further resulting in low image segmentation accuracy of the trained nucleus image segmentation model.

[0036] In view of the above problems, the present invention proposes the following embodiments. The following combines Figures 1 - 7 to describe the training method of the nucleus image segmentation model and the nucleus image segmentation method of the present invention.

[0037] Figure 1 is one of the flow diagrams of the training method of the nucleus image segmentation model provided by the present invention. As Figure 1 shown, the training method of the nucleus image segmentation model includes the following steps 110, 120, 130, and 140.

[0038] Step 110: Perform data augmentation processing on the unlabeled sample nucleus image to obtain different first sample nucleus images and second sample nucleus images.

[0039] Among them, the unlabeled sample nucleus image is a sample nucleus image without the segmentation result of the nucleus image being labeled.

[0040] In one embodiment, two different data augmentation processes are performed on the same unlabeled sample nucleus image to obtain different first sample nucleus images and second sample nucleus images. That is, both the first sample nucleus image and the second sample nucleus image are unlabeled sample nucleus images after data augmentation processing.

[0041] In another embodiment, the same unlabeled sample cell nucleus image is subjected to data augmentation processing once to obtain an unlabeled sample cell nucleus image after data augmentation processing. The unlabeled sample cell nucleus image after data augmentation processing is determined as the first sample cell nucleus image, and the unlabeled sample cell nucleus image is determined as the second sample cell nucleus image, or the unlabeled sample cell nucleus image after data augmentation processing is determined as the second sample cell nucleus image, and the unlabeled sample cell nucleus image is determined as the first sample cell nucleus image. That is, only one of the first sample cell nucleus image and the second sample cell nucleus image needs to be the unlabeled sample cell nucleus image after data augmentation processing.

[0042] It should be noted that different first sample cell nucleus images and second sample cell nucleus images are obtained, that is, different perturbations are applied to the unlabeled sample cell nucleus images and input into the teacher model and the student model respectively, so as to use the output of the teacher model to supervise the update of the model parameters of the student model.

[0043] Step 120: Input the first sample cell nucleus image into the student model to obtain the first phenotypic activation feature of the first sample cell nucleus image, and input the second sample cell nucleus image into the teacher model to obtain the second phenotypic activation feature of the second sample cell nucleus image.

[0044] Wherein, the first phenotypic activation feature is obtained by fusing the first phenotypic vector of the first sample cell nucleus image and the first feature map of the first sample cell nucleus image, and the first phenotypic vector is a model parameter that can be learned by the student model. This first phenotypic vector is the model parameter (network layer parameter) of the student model, and this first phenotypic vector can store the phenotypic features of the cell nucleus learned by the student model.

[0045] Wherein, the second phenotypic activation feature is obtained by fusing the second phenotypic vector of the second sample cell nucleus image and the second feature map of the second sample cell nucleus image, and the second phenotypic vector is a model parameter that can be learned by the teacher model. This second phenotypic vector is the model parameter (network layer parameter) of the teacher model, and this second phenotypic vector can store the phenotypic features of the cell nucleus learned by the teacher model.

[0046] Wherein, both the first phenotypic vector and the second phenotypic vector are used to characterize the phenotypic features of the cell nucleus.

[0047] Wherein, both the student model and the teacher model are used to perform cell nucleus image segmentation on the input cell nucleus image.

[0048] Step 130: Determine the phenotypic consistency loss based on the difference between the first phenotypic activation feature and the second phenotypic activation feature.

[0049] In a specific embodiment, the consistency loss function for determining the loss of phenotypic consistency may be the mean squared error loss function, and of course, it may also be other loss functions.

[0050] Exemplarily, the consistency loss function is as follows: ; In the formula, represents the phenotypic consistency loss, represents the mean squared error loss function, represents the first phenotypic activation feature, represents the model parameters of the student model, represents the first sample cell nucleus image, represents the second phenotypic activation feature, represents the model parameters of the teacher model, represents the second sample cell nucleus image.

[0051] Step 140: Based on the phenotypic consistency loss, adjust the model parameters of the student model to obtain a trained cell nucleus image segmentation model.

[0052] It should be understood that the number of unlabeled sample cell nucleus images is multiple. Therefore, based on multiple phenotypic consistency losses, continuously iterate and adjust the model parameters of the student model, and finally obtain a trained cell nucleus image segmentation model.

[0053] The training method of the cell nucleus image segmentation model provided by the embodiments of the present invention performs data augmentation on the unlabeled sample cell nucleus images to obtain different first sample cell nucleus images and second sample cell nucleus images. The first sample cell nucleus images are input into the student model to obtain the first phenotypic activation features of the first sample cell nucleus images, and the second sample cell nucleus images are input into the teacher model to obtain the second phenotypic activation features of the second sample cell nucleus images. Based on the differences between the first phenotypic activation features and the second phenotypic activation features, the phenotypic consistency loss is determined. Based on the phenotypic consistency loss, the model parameters of the student model are adjusted. The unlabeled sample cell nucleus images are the sample cell nucleus images of the unlabeled cell nucleus image segmentation results. Thus, the cell nucleus image segmentation model can be trained based on the unlabeled sample cell nucleus images, thereby reducing the label annotation cost, that is, reducing the training cost of the cell nucleus image segmentation model. At the same time, based on the differences between the first phenotypic activation features and the second phenotypic activation features, the phenotypic consistency loss is determined. The first phenotypic activation feature is obtained by fusing the first phenotypic vector of the first sample cell nucleus image and the first feature map of the first sample cell nucleus image. The first phenotypic vector is the model parameter that can be learned by the student model. The second phenotypic activation feature is obtained by fusing the second phenotypic vector of the second sample cell nucleus image and the second feature map of the second sample cell nucleus image. The second phenotypic vector is the model parameter that can be learned by the teacher model. Both the first phenotypic vector and the second phenotypic vector are used to characterize the phenotypic features of the cell nucleus. Thus, based on the phenotypic consistency loss, the classification and discrimination ability of the cell nucleus image segmentation model for different types of cell nuclei is enhanced. That is, learnable phenotypic vectors are set in both the student model and the teacher model to be used to mine the fine-grained phenotypic features of different types of cell nuclei to perform more fine-grained consistency constraints on the phenotypic features generated during the model, thereby improving the model training effect of the cell nucleus image segmentation model, and further improving the image segmentation accuracy of the trained cell nucleus image segmentation model.

[0054] Based on any of the above embodiments, Figure 2 is the second schematic flow chart of the training method of the cell nucleus image segmentation model provided by the present invention. As Figure 2 shown, in the above step 120, inputting the first sample cell nucleus image into the student model to obtain the first phenotypic activation features of the first sample cell nucleus image includes: step 121, step 122, and step 123.

[0055] Step 121, input the first sample cell nucleus image into the student model, obtain the first feature map output by the encoding layer in the student model, and obtain K first phenotypic vectors of the first sample cell nucleus image.

[0056] Here, the student model includes an encoding layer and a decoding layer connected in sequence. The encoding layer is used to extract the features of the first sample cell nucleus image, and the decoding layer is used to perform cell nucleus image segmentation based on the features extracted by the encoding layer.

[0057] Where K is the preset number of cell nucleus categories. The number of cell nucleus categories can be the number of cell nucleus categories seen in the sample cell nucleus image set. Use K learnable first phenotypic vectors to store the phenotypic features of the cell nucleus learned by the student model based on the first sample cell nucleus image. Exemplarily, the K first phenotypic vectors are denoted as , denotes the th first phenotypic vector.

[0058] Step 122: Perform feature fusion on the K first phenotypic vectors and the first feature map respectively to obtain the first phenotypic activation sub-features of K cell nucleus categories.

[0059] In a specific embodiment, perform a pixel-by-pixel dot product operation on the K first phenotypic vectors and the first feature map respectively to obtain the first phenotypic activation sub-features of K cell nucleus categories.

[0060] To facilitate the feature fusion of the K first phenotypic vectors and the first feature map, perform a channel dimension transformation on the first feature map to obtain an intermediate feature map with the same dimension as the K first phenotypic vectors.

[0061] Step 123: Determine the first phenotypic activation feature of the first sample cell nucleus image based on the K first phenotypic activation sub-features.

[0062] Here, the first phenotypic activation feature can be a first phenotypic activation map, and the channel dimension of the first phenotypic activation map is also K.

[0063] In the training method of the cell nucleus image segmentation model provided by the embodiment of the present invention, the first sample cell nucleus image is input into the student model to obtain the first feature map output by the encoding layer in the student model, and K first phenotypic vectors of the first sample cell nucleus image are obtained, and K is the preset number of cell nucleus categories. Thus, use K learnable first phenotypic vectors to store the phenotypic features of the cell nucleus learned by the student model based on the first sample cell nucleus image, and then perform more fine-grained consistency constraints on the phenotypic features generated in the middle of the model, thereby improving the model training effect of the cell nucleus image segmentation model, and further improving the image segmentation accuracy of the trained cell nucleus image segmentation model.

[0064] Based on any of the above embodiments, Figure 3 is the third flowchart of the training method of the cell nucleus image segmentation model provided by the present invention, as shown in Figure 3As shown, before the above step 121, the method further includes: steps 310 to 350.

[0065] Step 310, for any one of the K types of cell nuclei, input the first labeled sample cell nucleus image corresponding to the type of cell nucleus into the student model, obtain the third feature map output by the encoding layer in the student model, and obtain the target first phenotype vector of the first labeled sample cell nucleus image.

[0066] Wherein, the first labeled sample cell nucleus image is a sample cell nucleus image annotating the cell nucleus image segmentation result corresponding to the type of cell nucleus, and the target first phenotype vector is used to characterize the phenotypic characteristics of the cell nucleus corresponding to the type of cell nucleus.

[0067] It should be understood that the first labeled sample cell nucleus image should include the cell nucleus corresponding to the type of cell nucleus, and the annotated cell nucleus image segmentation result should also cover the image segmentation result of the cell nucleus corresponding to the type of cell nucleus.

[0068] It should be noted that in order to continuously learn the phenotype vectors corresponding to different types of cell nuclei, the student model can be continuously iteratively trained based on the first labeled sample cell nucleus images corresponding to the K types of cell nuclei.

[0069] The number of samples of the first labeled sample cell nucleus image is much smaller than the number of unlabeled sample cell nucleus images, so that a cell nucleus image segmentation model can be trained based on a small number of labeled sample cell nucleus images and unlabeled sample cell nucleus images, thereby reducing the label annotation cost, that is, reducing the training cost of the cell nucleus image segmentation model.

[0070] Step 320, based on the cell nucleus image segmentation result label corresponding to the first labeled sample cell nucleus image, determine a number of target features from the third feature map.

[0071] Wherein, any one of the target features is a feature corresponding to the cell nucleus region belonging to the type of cell nucleus.

[0072] Considering that the third feature map includes the features of all regions in the first labeled sample cell nucleus image, that is, includes the features of the foreground cell nucleus and the features of the background region, based on this, the target features related to the type of cell nucleus are determined from the third feature map based on the cell nucleus image segmentation result label.

[0073] To facilitate the determination of the target features from the third feature map, the cell nucleus image segmentation result label is downsampled to align with the feature map size of the third feature map.

[0074] Step 330, generate a target feature vector based on the number of target features.

[0075] In a specific embodiment, average pooling is performed on a number of target features to obtain a target feature vector.

[0076] Step 340: Determine a first loss based on the difference between the target first phenotype vector and the target feature vector.

[0077] To facilitate the determination of the first loss, the target first phenotype vector is first mapped to the same feature dimension as the target feature vector. For example, the target first phenotype vector is mapped to the same feature dimension as the target feature vector through a fully connected layer.

[0078] In a specific embodiment, the first loss is determined based on the Kullback-Leibler Divergence loss function.

[0079] Step 350: Adjust the model parameters of the student model based on the first loss.

[0080] Based on this, the student model is supervised to learn the phenotype vector based on the first loss, so as to better learn the phenotype vectors corresponding to different cell nucleus types.

[0081] Exemplarily, for the labeled sample cell nucleus image, in order to better learn the phenotype vectors corresponding to different cell nucleus types by using the segmentation annotation (the label of the cell nucleus image segmentation result), the segmentation annotation corresponding to the cell nucleus category is downsampled and aligned to the feature map size of the third feature map output by the encoding layer of the student model , and the third feature map The features at the positions corresponding to 1 (that is, the cell nucleus region corresponding to this cell nucleus category) are taken out for average pooling to obtain a target feature vector , where , and the specific process is shown in the following formula: ; ; In the formula, represents the coordinates in the third feature map, represents average pooling.

[0082] After that, the corresponding target first phenotype vector is mapped to the same feature dimension as the target feature vector through a fully connected layer, and then the KL divergence between it and the target feature vector is calculated as the loss function for supervising the learning of the phenotype vector.

[0083] Further, the model structure of the student model is the same as that of the teacher model. After adjusting the model parameters of the student model, the adjusted model parameters of the student model are synchronized to the teacher model, which is equivalent to adjusting the model parameters of the teacher model based on the first loss to better use the K learnable second phenotypic vectors to store the phenotypic features of the cell nuclei learned by the teacher model based on the second sample cell nucleus images, thereby imposing a more fine-grained consistency constraint on the phenotypic features generated during the model, so as to improve the model training effect of the cell nucleus image segmentation model, and further improve the image segmentation accuracy of the trained cell nucleus image segmentation model.

[0084] The training method of the cell nucleus image segmentation model provided by the embodiment of the present invention obtains the first loss through the above method, and based on the first loss, supervises the student model's learning of the phenotypic vectors, so as to better learn the phenotypic vectors corresponding to different cell nucleus types, and better use the K learnable first phenotypic vectors to store the phenotypic features of the cell nuclei learned by the student model based on the first sample cell nucleus images, thereby imposing a more fine-grained consistency constraint on the phenotypic features generated during the model, so as to improve the model training effect of the cell nucleus image segmentation model, and further improve the image segmentation accuracy of the trained cell nucleus image segmentation model.

[0085] Based on any of the above embodiments, in this method, step 122 includes: step 1221 and step 1222.

[0086] Step 1221, perform a channel dimension transformation on the first feature map to obtain an intermediate feature map with the same dimension as the K first phenotypic vectors.

[0087] In a specific embodiment, the first feature map is input into a projection layer to obtain the intermediate feature map output by this projection layer. This projection layer may include a convolutional layer composed of 1x1 convolutional kernels.

[0088] Step 1222, perform a pixel-by-pixel dot product process on each of the K first phenotypic vectors and the intermediate feature map to obtain the first phenotypic activation sub-features of K cell nucleus categories.

[0089] Exemplarily, the K first phenotypic vectors are represented as , represents the th first phenotypic vector, , the first feature map output by the encoder of the student model passes through a projection layer composed of 1x1 convolutional kernels to convert the channel dimension of the first feature map into an intermediate feature map , where the first phenotypic activation feature The feature map of each channel represents the activation intensity of the corresponding phenotype at each position.

[0090] In the training method of the nucleus image segmentation model provided by the embodiments of the present invention, the channel dimension of the first feature map is converted to obtain an intermediate feature map with the same dimension as the K first phenotypic vectors, so as to better perform pixel-by-pixel dot product processing of the K first phenotypic vectors and the intermediate feature map respectively, to obtain the first phenotypic activation sub-features of K nucleus categories, thereby improving the determination accuracy of the first phenotypic activation feature, and further performing more accurate consistency constraints on the phenotypic features generated in the middle of the model, thereby improving the model training effect of the nucleus image segmentation model, and further improving the image segmentation accuracy of the trained nucleus image segmentation model.

[0091] Based on any of the above embodiments, in this method, in step 120 above, inputting the second sample nucleus image into the teacher model to obtain the second phenotypic activation feature of the second sample nucleus image includes: step 124, step 125, and step 126.

[0092] Step 124: Input the second sample nucleus image into the teacher model to obtain a second feature map output by the encoding layer in the teacher model, and obtain K second phenotypic vectors of the second sample nucleus image.

[0093] Here, the teacher model includes an encoding layer and a decoding layer connected in sequence. The encoding layer is used to extract the features of the second sample nucleus image, and the decoding layer is used to perform nucleus image segmentation based on the features extracted by the encoding layer.

[0094] Among them, K is the preset number of nucleus categories. The number of nucleus categories can be the number of nucleus categories seen in the sample nucleus image set. K learnable second phenotypic vectors are used to store the phenotypic features of the nucleus learned by the teacher model based on the second sample nucleus image. Exemplarily, the K second phenotypic vectors are expressed as , denotes the th second phenotypic vector.

[0095] Step 125: Perform feature fusion of the K second phenotypic vectors and the second feature map respectively to obtain the second phenotypic activation sub-features of K nucleus categories.

[0096] In a specific embodiment, perform pixel-by-pixel dot product processing of the K second phenotypic vectors and the second feature map respectively to obtain the second phenotypic activation sub-features of K nucleus categories.

[0097] To facilitate the feature fusion between the K second phenotypic vectors and the second feature map respectively, the channel dimension of the second feature map is transformed to obtain an intermediate feature map with the same dimension as the K second phenotypic vectors.

[0098] Step 126: Determine the second phenotypic activation feature of the second sample cell nucleus image based on the K second phenotypic activation sub-features.

[0099] Here, the second phenotypic activation feature can be a second phenotypic activation map, and the channel dimension of the second phenotypic activation map is also K.

[0100] In the training method of the cell nucleus image segmentation model provided by the embodiments of the present invention, the second sample cell nucleus image is input into the teacher model to obtain the second feature map output by the encoding layer in the teacher model, and K second phenotypic vectors of the second sample cell nucleus image are obtained, where K is the preset number of cell nucleus categories. Thus, the K learnable second phenotypic vectors are used to store the phenotypic features of the cell nucleus learned by the teacher model based on the second sample cell nucleus image, and further, a more fine-grained consistency constraint is imposed on the phenotypic features generated in the middle of the model, thereby improving the model training effect of the cell nucleus image segmentation model, and further improving the image segmentation accuracy of the trained cell nucleus image segmentation model.

[0101] Based on any of the above embodiments, in this method, the above step 125 includes: step 1251 and step 1252.

[0102] Step 1251: Transform the channel dimension of the second feature map to obtain an intermediate feature map with the same dimension as the K second phenotypic vectors.

[0103] In a specific embodiment, the second feature map is input into a projection layer to obtain the intermediate feature map output by this projection layer. This projection layer can include a convolutional layer composed of 1x1 convolutional kernels.

[0104] Step 1252: Perform a pixel-by-pixel dot product process between the K second phenotypic vectors and the intermediate feature map respectively to obtain the second phenotypic activation sub-features of K cell nucleus categories.

[0105] Exemplarily, the K second phenotypic vectors are represented as ， Denote the th second phenotypic vector, , the second feature map output by the encoding layer (encoder) of the teacher model passes through a projection layer composed of 1x1 convolutional kernels to transform the channel dimension of the second feature map into an intermediate feature map , the K learnable second phenotypic vectors are respectively subjected to pixel-by-pixel dot product with the intermediate feature map, and finally the second phenotypic activation features corresponding to the teacher model are obtained. , where the second phenotypic activation features The feature maps of each channel of represent the activation intensity of the corresponding phenotype at each position.

[0106] The training method of the nucleus image segmentation model provided by the embodiment of the present invention performs channel dimension conversion on the second feature map to obtain an intermediate feature map with the same dimension as the K second phenotypic vectors, so as to better perform pixel-by-pixel dot product processing on the K second phenotypic vectors and the intermediate feature map respectively, obtain the second phenotypic activation sub-features of K nucleus categories, thereby improving the determination accuracy of the second phenotypic activation features, and further performing more accurate consistency constraints on the phenotypic features generated in the middle of the model, thereby improving the model training effect of the nucleus image segmentation model, and further improving the image segmentation accuracy of the trained nucleus image segmentation model.

[0107] Based on any of the above embodiments, Figure 4 is the fourth flow diagram of the training method of the nucleus image segmentation model provided by the present invention. As Figure 4 shown, the above step 130 includes: step 131, step 132 and step 133.

[0108] Step 131, obtaining the first predicted nucleus image segmentation result corresponding to the second sample nucleus image output by the teacher model.

[0109] After inputting the second sample nucleus image into the teacher model, the first predicted nucleus image segmentation result output by the teacher model can be obtained.

[0110] Step 132, inputting the first predicted nucleus image segmentation result into the segmentation quality scoring model to obtain the comprehensive quality scoring map output by the segmentation quality scoring model.

[0111] Among them, the comprehensive quality scoring map includes the comprehensive quality scoring values of each region in the second sample nucleus image. Exemplarily, the comprehensive quality scoring map is represented by a feature map, and this comprehensive quality scoring map can be expressed as .

[0112] Among them, the segmentation quality scoring model is used to evaluate the nucleus image segmentation quality of the input nucleus image segmentation result.

[0113] The segmentation quality scoring model is trained based on the segmentation results of sample cell nucleus images and the corresponding comprehensive quality scoring map labels. Further, the comprehensive quality scoring map labels can include quality scoring map labels in multiple dimensions; the quality scoring map labels in any dimension include the quality scoring values of each region in the cell nucleus image before segmentation corresponding to the segmentation result of the sample cell nucleus image.

[0114] Step 133: Based on the comprehensive quality scoring map, perform weighted processing on the difference between the first phenotypic activation feature and the second phenotypic activation feature to obtain a phenotypic consistency loss.

[0115] Specifically, use the comprehensive quality scoring map as the attention weight of the phenotypic consistency loss function, and based on this attention weight, perform weighted processing on the difference between the first phenotypic activation feature and the second phenotypic activation feature to obtain a phenotypic consistency loss.

[0116] Exemplarily, the consistency loss function is as follows: ; In the formula, represents the phenotypic consistency loss, represents the comprehensive quality scoring map, represents the mean square error loss function, represents the first phenotypic activation feature, represents the model parameters of the student model, represents the first sample cell nucleus image, represents the second phenotypic activation feature, represents the model parameters of the teacher model, represents the second sample cell nucleus image.

[0117] It should be understood that in order to suppress the noise in the process of phenotypic consistency loss, a segmentation quality scoring model is used to distinguish the quality of the cell nucleus image segmentation in the teacher model, so as to assign a higher attention weight to the high-quality cell nucleus segmentation result, thereby suppressing the noise introduced in the process of constructing the phenotypic consistency loss. In other words, by using the segmentation quality scoring model to assign a high attention weight to the high-quality prediction region, the noise introduced in the process of constructing the phenotypic consistency constraint is suppressed.

[0118] The training method of the cell nucleus image segmentation model provided by the embodiment of the present invention obtains the first predicted cell nucleus image segmentation result corresponding to the second sample cell nucleus image output by the teacher model, inputs the first predicted cell nucleus image segmentation result into the segmentation quality scoring model, and obtains the comprehensive quality scoring map output by the segmentation quality scoring model. The comprehensive quality scoring map includes the comprehensive quality scoring values of each region in the second sample cell nucleus image. Based on the comprehensive quality scoring map, the difference between the first phenotypic activation feature and the second phenotypic activation feature is weighted to obtain the phenotypic consistency loss. Considering that the prediction of the teacher model is not necessarily accurate, in order to suppress the noise in the process of the phenotypic consistency loss, a segmentation quality scoring model is used to distinguish the quality of the cell nucleus image segmentation in the teacher model, so as to assign a higher attention weight to the high-quality cell nucleus segmentation result, thereby suppressing the noise introduced in the process of constructing the phenotypic consistency loss, improving the training effect of the cell nucleus image segmentation model, and further improving the image segmentation accuracy of the trained cell nucleus image segmentation model, that is, improving the image segmentation performance of the cell nucleus image segmentation model.

[0119] Based on any of the above embodiments, Figure 5 is the fifth flowchart of the training method of the cell nucleus image segmentation model provided by the present invention, as Figure 5 shown, the segmentation quality scoring model is trained based on the following steps: Step 510, Step 520, and Step 530.

[0120] Step 510, input the sample cell nucleus image segmentation result into the initial quality scoring layer, and obtain the predicted quality scoring maps of multiple dimensions output by the initial quality scoring layer.

[0121] Among them, any dimension of the predicted quality scoring map includes the quality scoring values of each region in the cell nucleus image before segmentation corresponding to the sample cell nucleus image segmentation result.

[0122] Here, the initial quality scoring layer is used to evaluate the quality of the cell nucleus image segmentation for the input sample cell nucleus image segmentation result.

[0123] Here, multiple dimensions may include but are not limited to at least two of the following: cell nucleus perimeter, cell nucleus area, cell nucleus contour curvature, cell nucleus smoothness, cell nucleus solidity, average brightness, etc.

[0124] Further, to improve the training effect of the segmentation quality scoring model, the sample cell nucleus image segmentation result includes low-quality cell nucleus image segmentation results and high-quality cell nucleus image segmentation results.

[0125] Step 520, based on the quality scoring map labels of multiple dimensions corresponding to the sample cell nucleus image segmentation result, and the predicted quality scoring maps of multiple dimensions, determine the second loss.

[0126] Here, the quality score map label in any dimension includes the quality score values of each region in the sample cell nucleus image.

[0127] Specifically, based on the differences between the quality score map labels (true values) in multiple dimensions corresponding to the sample cell nucleus image segmentation result and the predicted quality score maps (predicted values) in multiple dimensions, the second loss is determined.

[0128] Considering that the structural characteristics of cell nuclei are usually relatively fixed and different types of cell nuclei have specific patterns, based on this, the segmentation quality evaluation score can be obtained by calculating the manual morphological features of cell nuclei, and then the quality score map labels in multiple dimensions corresponding to the sample cell nucleus image segmentation result can be determined. Specifically, the true cell nucleus segmentation morphology can be extracted from the sample cell nucleus image, and manual features such as the cell nucleus perimeter, area, contour curvature, smoothness, solidity, and average brightness can be calculated. Then, the probability density function of these manual features can be obtained using kernel density estimation, which reflects the characteristic distribution of the true cell nucleus segmentation morphology. For the cell nucleus segmentation result predicted by the model (the sample cell nucleus image segmentation result), the corresponding manual morphological features can also be calculated and substituted into the above-obtained probability density function to calculate the corresponding probability. This value reflects the possibility that the cell nucleus morphology segmented by the model is close to the true cell nucleus morphology. The higher the possibility, the more similar the segmented cell nucleus is to the true cell nucleus.

[0129] Furthermore, the above quality scoring strategy can be used to obtain the quality scores of each cell nucleus, and then the quality score map labels in multiple dimensions corresponding to the sample cell nucleus image segmentation result can be constructed. Specifically, a map with the same size as the input sample cell nucleus image is created , where is the number of manual features. The value of the background region in the quality score map is 0, and the value of the corresponding cell nucleus region is the probability value calculated from the corresponding manual features, representing the quality score of the corresponding manual feature. In this way, each sample cell nucleus image segmentation result and its corresponding quality score map labels in multiple dimensions form a sample pair for training the segmentation quality scoring model.

[0130] Step 530, based on the second loss, train the initial quality scoring layer to obtain the segmentation quality scoring model based on the trained quality scoring layer and the comprehensive scoring layer.

[0131] Wherein, the comprehensive scoring layer is used to fuse the quality score maps in multiple dimensions output by the quality scoring layer to obtain a comprehensive quality score map.

[0132] In one embodiment, the comprehensive scoring layer can be an average pooling layer.

[0133] Here, the segmentation quality scoring model includes a quality scoring layer and a comprehensive scoring layer connected in sequence. The quality scoring layer is used to predict quality scoring maps of various dimensions.

[0134] It should be understood that the output of the teacher model is directly embedded after the training of the quality scoring layer is completed, and the model parameters of the quality scoring layer are frozen. When training the cell nucleus image segmentation model subsequently, the model parameters of the quality scoring layer do not need to be adjusted.

[0135] The training method of the cell nucleus image segmentation model provided by the embodiments of the present invention can predict predicted quality scoring maps of multiple dimensions through the segmentation quality scoring model trained in the above manner, so as to perform a more comprehensive segmentation quality evaluation, improve the accuracy of the image segmentation quality evaluation, that is, suppress the noise introduced in the process of constructing the phenotypic consistency loss, thereby improving the training effect of the cell nucleus image segmentation model, and further improving the prediction accuracy of the comprehensive quality scoring map, and finally improving the image segmentation accuracy of the trained cell nucleus image segmentation model, that is, improving the image segmentation performance of the cell nucleus image segmentation model.

[0136] Based on any of the above embodiments, in this method, the segmentation result of the sample cell nucleus image is obtained based on the following method: Based on the second labeled sample cell nucleus image, an initial student model is trained to obtain the student model; the second labeled sample cell nucleus image is a sample cell nucleus image with a labeled cell nucleus image segmentation result. The unlabeled sample cell nucleus image is input into the student model to obtain the segmentation result of the sample cell nucleus image output by the student model.

[0137] The number of samples of the second labeled sample cell nucleus image is much smaller than the number of samples of the unlabeled sample cell nucleus image, so that a cell nucleus image segmentation model can be trained based on a small number of labeled sample cell nucleus images and unlabeled sample cell nucleus images, thereby reducing the label annotation cost, that is, reducing the training cost of the cell nucleus image segmentation model. In addition, the second labeled sample cell nucleus image and the above first labeled sample cell nucleus image can be data in the same sample labeled dataset.

[0138] Specifically, the second labeled sample cell nucleus image is input into the initial student model to obtain the predicted cell nucleus image segmentation result output by the initial student model, and the initial student model is trained based on the predicted cell nucleus image segmentation result and the cell nucleus image segmentation result label corresponding to the second labeled sample cell nucleus image.

[0139] It should be understood that the training of the nuclear image segmentation model is divided into two stages. In the first stage, a small number of second-labeled sample nuclear images are used to train the initial student model to obtain a student model with a certain nuclear image segmentation ability, so that in the second stage, the unlabeled sample nuclear images can be input into the student model with a certain ability to obtain the sample nuclear image segmentation results output by the student model.

[0140] It should be understood that since the initial student model is only trained based on a small number of second-labeled sample nuclear images, the nuclear image segmentation quality of the student model is not necessarily high. Based on this, the obtained sample nuclear image segmentation results include low-quality and high-quality nuclear image segmentation results, thereby improving the training effect of the segmentation quality scoring model, further improving the prediction accuracy of the comprehensive quality scoring map, and ultimately improving the image segmentation accuracy of the trained nuclear image segmentation model, that is, improving the image segmentation performance of the nuclear image segmentation model.

[0141] The training method of the nuclear image segmentation model provided by the embodiments of the present invention trains the initial student model based on the second-labeled sample nuclear images to obtain a student model with a certain nuclear image segmentation ability, so that the unlabeled sample nuclear images can be input into the student model with a certain ability to obtain the sample nuclear image segmentation results output by the student model, thereby initially training the initial student model based on the labeled images and then better obtaining the sample nuclear image segmentation results based on the unlabeled images, thereby improving the training effect of the segmentation quality scoring model, further improving the prediction accuracy of the comprehensive quality scoring map, and ultimately improving the image segmentation accuracy of the trained nuclear image segmentation model, that is, improving the image segmentation performance of the nuclear image segmentation model.

[0142] Based on any of the above embodiments, before the above step 140, the method further includes: Inputting the third-labeled sample nuclear images into the student model to obtain the second predicted nuclear image segmentation results output by the student model; Determining the third loss based on the difference between the second predicted nuclear image segmentation results and the nuclear image segmentation result labels corresponding to the third-labeled sample nuclear images.

[0143] Correspondingly, the above step 140 includes: adjusting the model parameters of the student model based on the phenotypic consistency loss and the third loss.

[0144] Among them, the third labeled sample cell nucleus image is a sample cell nucleus image for the segmentation result of the labeled cell nucleus image. The number of samples of the third labeled sample cell nucleus image is less than that of the unlabeled sample cell nucleus image, so that a cell nucleus image segmentation model can be trained based on a small number of labeled sample cell nucleus images and unlabeled sample cell nucleus images, thereby reducing the label annotation cost, that is, reducing the training cost of the cell nucleus image segmentation model. In addition, the third labeled sample cell nucleus image and the above first labeled sample cell nucleus image or second labeled sample cell nucleus image can be data in the same sample labeled dataset.

[0145] Exemplarily, the sample cell nucleus image set for training is as follows: ; Among them, the sample cell nucleus image set includes data, among which sample cell nucleus images contain the annotation of the cell nucleus image segmentation result, and the remaining sample cell nucleus images do not contain the annotation of the cell nucleus image segmentation result, and , that is, the unlabeled sample cell nucleus images are much larger than the labeled sample cell nucleus images.

[0146] In a specific embodiment, the loss function for determining the third loss can be the cross-entropy loss function, and of course, it can also be other loss functions.

[0147] Exemplarily, the loss function of the third loss is as follows: ; In the formula, represents the third loss, represents the cross-entropy loss function, represents the second predicted cell nucleus image segmentation result, represents the model parameters of the student model, represents the cell nucleus image segmentation result label corresponding to the third labeled sample cell nucleus image.

[0148] That is to say, aiming at the problem of lack of annotation for pathological image cell nucleus instance segmentation, through the above method, based on the semi-supervised training method of phenotypic consistency, using a small number of labeled sample cell nucleus images and unlabeled sample cell nucleus images lacking annotation, the cell nucleus image segmentation model can achieve a performance approaching full annotation, that is, the semi-supervised training method based on a small number of labels has higher cost performance.

[0149] The training method of the cell nucleus image segmentation model provided by the embodiment of the present invention, through the above method, inputs the labeled sample cell nucleus images and unlabeled sample cell nucleus images into the student model for semi-supervised training at the same time, so as to improve the training effect of the cell nucleus image segmentation model and reduce the label annotation cost, that is, reduce the training cost of the cell nucleus image segmentation model.

[0150] Based on any of the above embodiments, after the above step 140, the method further includes: Updating the model parameters of the teacher model based on the current model parameters of the teacher model and the model parameters of the adjusted student model.

[0151] Wherein, the model structure of the student model is the same as that of the teacher model. Based on this, the model parameters of the teacher model can be directly updated based on the model parameters of the student model.

[0152] It should be noted that the teacher model does not perform backpropagation to update the model parameters, that is, the teacher model never participates in gradient backpropagation.

[0153] Here, the current model parameters are the model parameters of the teacher model in the previous round of iteration.

[0154] Here, the model parameters of the adjusted student model are obtained by adjusting the model parameters of the student model based on the phenotypic consistency loss. Further, the model parameters of the adjusted student model are obtained by adjusting the model parameters of the student model based on the phenotypic consistency loss, the above first loss, and / or the above third loss.

[0155] Further, considering that the weights of the current model parameters of the teacher model and the model parameters of the adjusted student model are different, based on this, updating the model parameters of the teacher model based on the current model parameters of the teacher model and their corresponding first weights, and the model parameters of the adjusted student model and their second weights.

[0156] Exemplarily, the model parameters of the teacher model are updated by exponential moving average, as shown in the following formula: ; In the formula, represents the updated model parameters of the teacher model, is the current model parameters of the teacher model, is the model parameters of the adjusted student model, is the smoothing coefficient (first weight), is the second weight.

[0157] The training method of the cell nucleus image segmentation model provided by the embodiment of the present invention updates the model parameters of the teacher model based on the current model parameters of the teacher model and the adjusted model parameters of the student model, so as to first adjust the model parameters of the student model, and then adjust the model parameters of the teacher model based on the model parameters of the student model, improve the update accuracy of the model parameters of the teacher model, and the teacher model is also used to supervise the training of the student model, thereby improving the training effect of the cell nucleus image segmentation model, and finally improving the image segmentation accuracy of the trained cell nucleus image segmentation model, that is, improving the image segmentation performance of the cell nucleus image segmentation model; and the teacher model does not perform backpropagation to update the model parameters, thereby reducing the model training cost.

[0158] Based on any of the above embodiments, before the above step 120, the method further includes: Based on the second labeled sample cell nucleus image, adjust the model parameters of the initial student model to obtain the student model; the second labeled sample cell nucleus image is a sample cell nucleus image labeled with the cell nucleus image segmentation result; Synchronize the adjusted model parameters of the initial student model to the teacher model.

[0159] Wherein, the model structure of the student model is the same as that of the teacher model.

[0160] Further, the number of samples of the second labeled sample cell nucleus image is much smaller than the number of unlabeled sample cell nucleus images, so that a cell nucleus image segmentation model can be trained based on a small number of labeled sample cell nucleus images and unlabeled sample cell nucleus images, thereby reducing the label annotation cost, that is, reducing the training cost of the cell nucleus image segmentation model.

[0161] It should be understood that the training of the cell nucleus image segmentation model is divided into two stages. The first stage is to train the initial student model with a small number of second labeled sample cell nucleus images to obtain a student model with certain cell nucleus image segmentation capabilities, so that in the second stage, the unlabeled sample cell nucleus images are input into the student model with certain capabilities to obtain the sample cell nucleus image segmentation result output by the student model; and synchronize the adjusted model parameters of the initial student model to the teacher model to obtain a teacher model with certain cell nucleus image segmentation capabilities, so that in the second stage, the unlabeled sample cell nucleus images are input into the teacher model with certain capabilities.

[0162] The training method of the cell nucleus image segmentation model provided by the embodiment of the present invention adjusts the model parameters of the initial student model based on the second annotated sample cell nucleus image in the above manner to obtain the student model, and then synchronizes the model parameters of the adjusted initial student model to the teacher model. Thus, only the initial student model needs to be trained based on the annotated sample cell nucleus image, and the teacher model can synchronize the model parameters, thereby reducing the training cost of the cell nucleus image segmentation model and improving the training efficiency of the cell nucleus image segmentation model.

[0163] To facilitate the understanding of the above embodiments, a specific embodiment is described here. As Figure 6 shown, perform data augmentation on the unannotated sample cell nucleus image to obtain different first sample cell nucleus images and second sample cell nucleus images; input the first sample cell nucleus image into the student model, obtain the first feature map output by the encoding layer in the student model, and obtain K first phenotypic vectors of the first sample cell nucleus image. Input the first feature map into the projection layer to perform channel dimension conversion to obtain an intermediate feature map with the same dimension as the K first phenotypic vectors. Perform pixel-by-pixel dot product processing on the K first phenotypic vectors and the intermediate feature map respectively to obtain the first phenotypic activation sub-features of K cell nucleus categories, and determine the first phenotypic activation feature of the first sample cell nucleus image based on the K first phenotypic activation sub-features; input the second sample cell nucleus image into the teacher model, obtain the second feature map output by the encoding layer in the teacher model, and obtain K second phenotypic vectors of the second sample cell nucleus image. Input the second feature map into the projection layer to perform channel dimension conversion to obtain an intermediate feature map with the same dimension as the K second phenotypic vectors. Perform pixel-by-pixel dot product processing on the K second phenotypic vectors and the intermediate feature map respectively to obtain the second phenotypic activation sub-features of K cell nucleus categories, and determine the second phenotypic activation feature of the second sample cell nucleus image based on the K second phenotypic activation sub-features; determine the phenotypic consistency loss based on the difference between the first phenotypic activation feature and the second phenotypic activation feature, and this phenotypic consistency loss is used to adjust the model parameters of the student model.

[0164] Further, as Figure 6 shown, obtain the first predicted cell nucleus image segmentation result corresponding to the second sample cell nucleus image output by the decoding layer in the teacher model; input the first predicted cell nucleus image segmentation result into the segmentation quality scoring model to obtain the comprehensive quality scoring map output by the segmentation quality scoring model, and perform weighted processing on the difference between the first phenotypic activation feature and the second phenotypic activation feature based on the comprehensive quality scoring map to obtain the phenotypic consistency loss.

[0165] Further, as Figure 6As shown in the figure, the labeled sample nucleus image is input into the encoding layer in the student model, and the second predicted nucleus image segmentation result output by the decoding layer in the student model is obtained; based on the difference between the second predicted nucleus image segmentation result and the nucleus image segmentation result label corresponding to the labeled sample nucleus image, a third loss is determined, and this third loss is used to adjust the model parameters of the student model.

[0166] Further, as Figure 6 shown in the figure, for any one of the K nucleus categories, the labeled sample nucleus image corresponding to the nucleus category is input into the student model, the third feature map output by the encoding layer in the student model is obtained, and the target first phenotype vector of the labeled sample nucleus image is obtained; based on the nucleus image segmentation result label corresponding to the labeled sample nucleus image, several target features are determined from the third feature map; average pooling is performed on the several target features to obtain a target feature vector; the target first phenotype vector is input into a fully connected layer to map the target first phenotype vector to the same feature dimension as the target feature vector, and then based on the difference between the mapped target first phenotype vector and the target feature vector, a first loss is determined, and this first loss is used to adjust the model parameters of the student model.

[0167] Based on the above embodiments, the present invention proposes a semi-supervised nucleus segmentation method based on phenotypic consistency for the problem of lack of labeled pathological image nucleus instance segmentation. Through two teacher models and student models with the same structure, and a small number of labeled sample nucleus images are directly input into the student model for gradient direction propagation to update the model. For unlabeled sample nucleus images lacking labels, different perturbations are applied and input into the teacher model and the student model respectively. The output of the teacher model is used to supervise the gradient update of the student model, and the model parameters of the teacher model are updated by exponential moving average.

[0168] In other words, the present invention realizes more strict and finer-grained consistency constraints, performs phenotypic consistency loss constraints, and performs segmentation quality evaluation, thereby reducing the noise in the process of constructing consistency constraints. That is, it improves the nucleus segmentation ability of the semi-supervised model under limited labeled data, thereby reducing the dependence on labeled data, and is expected to provide segmentation algorithm support for tasks such as immunohistochemical analysis and tumor microenvironment analysis.

[0169] In addition, the training of the cell nucleus image segmentation model is divided into two stages. The first stage is the Warmup stage, and the second stage is the Semi stage. Specifically, in the Warmup stage, the labeled sample cell nucleus images are input into the student model, and full-supervised training is performed according to the labeled data to enable the model to have a certain segmentation ability and reduce the noise in the process of constructing the phenotypic consistency constraint by inputting the unlabeled sample cell nucleus images. In the Semi stage, the model parameters of the student model are synchronized to the teacher model, and the segmentation quality scoring model is trained. After the training is completed, the model parameters of the segmentation quality scoring model are frozen and embedded after the output of the teacher model. Then, the labeled sample cell nucleus images and the unlabeled sample cell nucleus images are input into the model for semi-supervised training.

[0170] Based on any of the above embodiments, the present invention also provides a method for segmenting cell nucleus images. Figure 7 It is a schematic flowchart of the method for segmenting cell nucleus images provided by the present invention. As Figure 7 shown, the method for segmenting cell nucleus images includes the following step 710.

[0171] Step 710: Input the cell nucleus image to be segmented into the cell nucleus image segmentation model to obtain the target cell nucleus image segmentation result output by the cell nucleus image segmentation model.

[0172] Among them, the cell nucleus image segmentation model is trained by the training method of the cell nucleus image segmentation model according to any of the above embodiments.

[0173] The nuclear image segmentation method provided by the embodiments of the present invention inputs the nuclear image to be segmented into a nuclear image segmentation model, and obtains the target nuclear image segmentation result output by the nuclear image segmentation model. Since the nuclear image segmentation model is trained by the training method of the nuclear image segmentation model as described in any of the above embodiments, during the training process of the nuclear image segmentation model, data augmentation processing is performed on the unlabeled sample nuclear image to obtain different first sample nuclear images and second sample nuclear images. The first sample nuclear image is input into the student model to obtain the first phenotypic activation feature of the first sample nuclear image, and the second sample nuclear image is input into the teacher model to obtain the second phenotypic activation feature of the second sample nuclear image. Based on the difference between the first phenotypic activation feature and the second phenotypic activation feature, the phenotypic consistency loss is determined. Based on the phenotypic consistency loss, the model parameters of the student model are adjusted, and the unlabeled sample nuclear image is a sample nuclear image of the unlabeled nuclear image segmentation result. Thus, the nuclear image segmentation model can be trained based on the unlabeled sample nuclear image, thereby reducing the label annotation cost, that is, reducing the training cost of the nuclear image segmentation model. At the same time, based on the difference between the first phenotypic activation feature and the second phenotypic activation feature, the phenotypic consistency loss is determined. The first phenotypic activation feature is obtained by fusing the first phenotypic vector of the first sample nuclear image and the first feature map of the first sample nuclear image. The first phenotypic vector is a model parameter that can be learned by the student model. The second phenotypic activation feature is obtained by fusing the second phenotypic vector of the second sample nuclear image and the second feature map of the second sample nuclear image. The second phenotypic vector is a model parameter that can be learned by the teacher model. Both the first phenotypic vector and the second phenotypic vector are used to characterize the phenotypic features of the nucleus. Thus, based on the phenotypic consistency loss, the classification and discrimination ability of the nuclear image segmentation model for different types of nuclei is enhanced. That is, learnable phenotypic vectors are set in both the student model and the teacher model to be used to mine the fine-grained phenotypic features of different types of nuclei, so as to perform more fine-grained consistency constraints on the phenotypic features generated during the model, thereby improving the model training effect of the nuclear image segmentation model, and further improving the image segmentation accuracy of the trained nuclear image segmentation model.

[0174] The training device of the nuclear image segmentation model provided by the present invention will be described below. The training device of the nuclear image segmentation model described below can be correspondingly referred to the training method of the nuclear image segmentation model described above.

[0175] Figure 8 is a structural schematic diagram of the training device of the nuclear image segmentation model provided by the present invention, as Figure 8As shown in the figure, the training device for the nuclear image segmentation model includes a data augmentation module 810, an image input module 820, a loss determination module 830, and a parameter adjustment module 840.

[0176] The data augmentation module 810 is configured to perform data augmentation processing on the unlabeled sample nuclear images to obtain different first sample nuclear images and second sample nuclear images.

[0177] The image input module 820 is configured to input the first sample nuclear image into the student model to obtain the first phenotypic activation feature of the first sample nuclear image, and input the second sample nuclear image into the teacher model to obtain the second phenotypic activation feature of the second sample nuclear image.

[0178] The loss determination module 830 is configured to determine the phenotypic consistency loss based on the difference between the first phenotypic activation feature and the second phenotypic activation feature.

[0179] The parameter adjustment module 840 is configured to adjust the model parameters of the student model based on the phenotypic consistency loss to obtain the trained nuclear image segmentation model.

[0180] Wherein, the unlabeled sample nuclear image is a sample nuclear image without the segmentation result of the labeled nucleus; both the student model and the teacher model are used for nuclear image segmentation of the input nuclear image; the first phenotypic activation feature is obtained by fusing the first phenotypic vector of the first sample nuclear image and the first feature map of the first sample nuclear image, and the first phenotypic vector is the model parameter that can be learned by the student model; the second phenotypic activation feature is obtained by fusing the second phenotypic vector of the second sample nuclear image and the second feature map of the second sample nuclear image, and the second phenotypic vector is the model parameter that can be learned by the teacher model; both the first phenotypic vector and the second phenotypic vector are used to characterize the phenotypic features of the nucleus.

[0181] Next, the nuclear image segmentation device provided by the present invention will be described. The nuclear image segmentation device described below can be mutually referred to the nuclear image segmentation method described above.

[0182] Figure 9 It is a schematic structural diagram of the nuclear image segmentation device provided by the present invention. As Figure 9 shown, the nuclear image segmentation device includes an image segmentation module 910.

[0183] The image segmentation module 910 is configured to input the nuclear image to be segmented into the nuclear image segmentation model to obtain the target nuclear image segmentation result output by the nuclear image segmentation model.

[0184] Among them, the cell nucleus image segmentation model is trained by the training method of the cell nucleus image segmentation model described in any of the above embodiments.

[0185] Figure 10 An example of the physical structure diagram of an electronic device is shown as Figure 10 shown. The electronic device may include: a processor 1010, a communication interface 1020, a memory 1030, and a communication bus 1040. Among them, the processor 1010, the communication interface 1020, and the memory 1030 communicate with each other through the communication bus 1040. The processor 1010 can call the logical instructions in the memory 1030 to execute the training method of the cell nucleus image segmentation model. The method includes: performing data augmentation processing on the unlabeled sample cell nucleus image to obtain different first sample cell nucleus images and second sample cell nucleus images; inputting the first sample cell nucleus image into the student model to obtain the first phenotypic activation feature of the first sample cell nucleus image, and inputting the second sample cell nucleus image into the teacher model to obtain the second phenotypic activation feature of the second sample cell nucleus image; determining the phenotypic consistency loss based on the difference between the first phenotypic activation feature and the second phenotypic activation feature; adjusting the model parameters of the student model based on the phenotypic consistency loss to obtain the trained cell nucleus image segmentation model; where the unlabeled sample cell nucleus image is a sample cell nucleus image without the segmentation result of the cell nucleus image labeled; both the student model and the teacher model are used for segmenting the cell nucleus image of the input; the first phenotypic activation feature is obtained by fusing the first phenotypic vector of the first sample cell nucleus image and the first feature map of the first sample cell nucleus image, and the first phenotypic vector is the model parameter that can be learned by the student model; the second phenotypic activation feature is obtained by fusing the second phenotypic vector of the second sample cell nucleus image and the second feature map of the second sample cell nucleus image, and the second phenotypic vector is the model parameter that can be learned by the teacher model; both the first phenotypic vector and the second phenotypic vector are used to characterize the phenotypic features of the cell nucleus. Or execute the cell nucleus image segmentation method, which includes: inputting the cell nucleus image to be segmented into the cell nucleus image segmentation model to obtain the target cell nucleus image segmentation result output by the cell nucleus image segmentation model; where the cell nucleus image segmentation model is trained by the training method of the cell nucleus image segmentation model described in any of the above embodiments.

[0186] In addition, when the logical instructions in the above-mentioned memory 1030 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs.

[0187] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the training method of the cell nucleus image segmentation model provided by each of the above methods. The method includes: performing data augmentation processing on the unlabeled sample cell nucleus images to obtain different first sample cell nucleus images and second sample cell nucleus images; inputting the first sample cell nucleus images into a student model to obtain first phenotypic activation features of the first sample cell nucleus images, and inputting the second sample cell nucleus images into a teacher model to obtain second phenotypic activation features of the second sample cell nucleus images; determining a phenotypic consistency loss based on the difference between the first phenotypic activation features and the second phenotypic activation features; adjusting the model parameters of the student model based on the phenotypic consistency loss to obtain a trained cell nucleus image segmentation model; wherein, the unlabeled sample cell nucleus images are sample cell nucleus images without labeled cell nucleus image segmentation results; both the student model and the teacher model are used for performing cell nucleus image segmentation on the input cell nucleus images; the first phenotypic activation features are obtained by fusing a first phenotypic vector of the first sample cell nucleus images with a first feature map of the first sample cell nucleus images, and the first phenotypic vector is a model parameter that can be learned by the student model; the second phenotypic activation features are obtained by fusing a second phenotypic vector of the second sample cell nucleus images with a second feature map of the second sample cell nucleus images, and the second phenotypic vector is a model parameter that can be learned by the teacher model; both the first phenotypic vector and the second phenotypic vector are used to characterize the phenotypic features of the cell nucleus. Or execute the cell nucleus image segmentation method provided by each of the above methods. The method includes: inputting the cell nucleus image to be segmented into the cell nucleus image segmentation model to obtain the target cell nucleus image segmentation result output by the cell nucleus image segmentation model; wherein, the cell nucleus image segmentation model is trained based on the training method of the cell nucleus image segmentation model according to any one of the above embodiments.

[0188] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the training method of the cell nucleus image segmentation model provided by the above-mentioned various methods. The method includes: performing data augmentation processing on the unlabeled sample cell nucleus images to obtain different first sample cell nucleus images and second sample cell nucleus images; inputting the first sample cell nucleus images into a student model to obtain first phenotypic activation features of the first sample cell nucleus images, and inputting the second sample cell nucleus images into a teacher model to obtain second phenotypic activation features of the second sample cell nucleus images; determining a phenotypic consistency loss based on the difference between the first phenotypic activation features and the second phenotypic activation features; adjusting the model parameters of the student model based on the phenotypic consistency loss to obtain a trained cell nucleus image segmentation model; wherein the unlabeled sample cell nucleus images are sample cell nucleus images without labeled cell nucleus image segmentation results; both the student model and the teacher model are used for performing cell nucleus image segmentation on the input cell nucleus images; the first phenotypic activation features are obtained by fusing a first phenotypic vector of the first sample cell nucleus images with a first feature map of the first sample cell nucleus images, and the first phenotypic vector is a model parameter that can be learned by the student model; the second phenotypic activation features are obtained by fusing a second phenotypic vector of the second sample cell nucleus images with a second feature map of the second sample cell nucleus images, and the second phenotypic vector is a model parameter that can be learned by the teacher model; both the first phenotypic vector and the second phenotypic vector are used to characterize the phenotypic features of the cell nucleus. Or when executed, it implements the cell nucleus image segmentation method provided by the above-mentioned various methods. The method includes: inputting the cell nucleus image to be segmented into the cell nucleus image segmentation model to obtain the target cell nucleus image segmentation result output by the cell nucleus image segmentation model; wherein the cell nucleus image segmentation model is trained based on the training method of the cell nucleus image segmentation model according to any one of the above embodiments.

[0189] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0190] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solutions, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0191] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of each embodiment of the present invention.

Claims

1. A training method for a cell nucleus image segmentation model, characterized in that, Including: Performing data augmentation on the unlabeled sample cell nucleus images to obtain different first sample cell nucleus images and second sample cell nucleus images; Inputting the first sample cell nucleus images into the student model to obtain first phenotypic activation features of the first sample cell nucleus images, and inputting the second sample cell nucleus images into the teacher model to obtain second phenotypic activation features of the second sample cell nucleus images; Determining a phenotypic consistency loss based on the difference between the first phenotypic activation features and the second phenotypic activation features; Adjusting the model parameters of the student model based on the phenotypic consistency loss to obtain a trained cell nucleus image segmentation model; Wherein, the unlabeled sample cell nucleus images are sample cell nucleus images without labeled cell nucleus image segmentation results; both the student model and the teacher model are used for performing cell nucleus image segmentation on the input cell nucleus images; the first phenotypic activation features are obtained by fusing a first phenotypic vector of the first sample cell nucleus images with a first feature map of the first sample cell nucleus images, and the first phenotypic vector is a model parameter that can be learned by the student model; the second phenotypic activation features are obtained by fusing a second phenotypic vector of the second sample cell nucleus images with a second feature map of the second sample cell nucleus images, and the second phenotypic vector is a model parameter that can be learned by the teacher model; both the first phenotypic vector and the second phenotypic vector are used to characterize the phenotypic features of the cell nucleus.

2. The training method of the cell nucleus image segmentation model according to claim 1, wherein The step of inputting the first sample cell nucleus images into the student model to obtain first phenotypic activation features of the first sample cell nucleus images includes: Inputting the first sample cell nucleus images into the student model to obtain a first feature map output by the encoding layer in the student model, and obtaining K first phenotypic vectors of the first sample cell nucleus images; K is the preset number of cell nucleus categories; Respectively performing feature fusion on the K first phenotypic vectors and the first feature map to obtain first phenotypic activation sub-features of K cell nucleus categories; Determining the first phenotypic activation features of the first sample cell nucleus images based on the K first phenotypic activation sub-features.

3. The training method of the cell nucleus image segmentation model according to claim 2, wherein Before inputting the first sample cell nucleus images into the student model to obtain a first feature map output by the encoding layer in the student model and obtaining K first phenotypic vectors of the first sample cell nucleus images, the method further includes: For any one of the K cell nucleus categories, inputting the first labeled sample cell nucleus image corresponding to the cell nucleus category into the student model to obtain a third feature map output by the encoding layer in the student model, and obtaining a target first phenotypic vector of the first labeled sample cell nucleus image; the first labeled sample cell nucleus image is a sample cell nucleus image labeled with the cell nucleus image segmentation result corresponding to the cell nucleus category, and the target first phenotypic vector is used to characterize the phenotypic features of the cell nucleus corresponding to the cell nucleus category. Based on the nucleus image segmentation result labels corresponding to the first labeled sample nucleus images, determine a number of target features from the third feature map; any one of the target features is a feature corresponding to the nucleus region belonging to the nucleus category; Generate a target feature vector based on the number of target features; Determine a first loss based on the difference between the target first phenotype vector and the target feature vector; Adjust the model parameters of the student model based on the first loss.

4. The training method of the cell nucleus image segmentation model according to claim 2, wherein The step of respectively performing feature fusion on the K first phenotype vectors and the first feature map to obtain first phenotype activation sub-features of K nucleus categories includes: Perform a channel dimension transformation on the first feature map to obtain an intermediate feature map with the same dimension as the K first phenotype vectors; Perform a pixel-by-pixel dot product process on the K first phenotype vectors and the intermediate feature map respectively to obtain first phenotype activation sub-features of K nucleus categories.

5. The training method of the cell nucleus image segmentation model according to any one of claims 1 to 4, characterized in that The step of determining the phenotype consistency loss based on the difference between the first phenotype activation feature and the second phenotype activation feature includes: Obtain a first predicted nucleus image segmentation result corresponding to the second sample nucleus image output by the teacher model; Input the first predicted nucleus image segmentation result into a segmentation quality scoring model to obtain a comprehensive quality scoring map output by the segmentation quality scoring model; the comprehensive quality scoring map includes comprehensive quality scoring values of each region in the second sample nucleus image; Based on the comprehensive quality scoring map, perform a weighted process on the difference between the first phenotype activation feature and the second phenotype activation feature to obtain the phenotype consistency loss; Wherein, the segmentation quality scoring model is used to evaluate the quality of the input nucleus image segmentation result.

6. The training method of the cell nucleus image segmentation model according to claim 5, characterized in that The segmentation quality scoring model is trained in the following manner: Input the sample nucleus image segmentation result into an initial quality scoring layer to obtain predicted quality scoring maps of multiple dimensions output by the initial quality scoring layer; any dimension of the predicted quality scoring map includes the quality scoring values of each region in the nucleus image before segmentation corresponding to the sample nucleus image segmentation result; Determine a second loss based on the quality scoring map labels of multiple dimensions corresponding to the sample nucleus image segmentation result and the predicted quality scoring maps of multiple dimensions; Train the initial quality scoring layer based on the second loss to obtain the segmentation quality scoring model based on the trained quality scoring layer and comprehensive scoring layer; Wherein, the comprehensive scoring layer is used to fuse the quality scoring maps of multiple dimensions output by the quality scoring layer to obtain a comprehensive quality scoring map.

7. The training method of the cell nucleus image segmentation model according to claim 6, characterized in that, The sample nucleus image segmentation result is obtained in the following manner: Train an initial student model based on the second labeled sample nucleus images to obtain the student model; the second labeled sample nucleus images are sample nucleus images with labeled nucleus image segmentation results; Input the unlabeled sample nucleus images into the student model to obtain the sample nucleus image segmentation result output by the student model.

8. The training method of the cell nucleus image segmentation model according to any one of claims 1 to 4, characterized in that Before adjusting the model parameters of the student model based on the phenotypic consistency loss, the method further includes: Inputting the third labeled sample nuclear image into the student model to obtain a second predicted nuclear image segmentation result output by the student model; the third labeled sample nuclear image is a sample nuclear image with a labeled nuclear image segmentation result, and the number of samples of the third labeled sample nuclear image is less than the number of samples of the unlabeled sample nuclear image; Determining a third loss based on the difference between the second predicted nuclear image segmentation result and the nuclear image segmentation result label corresponding to the third labeled sample nuclear image; Accordingly, adjusting the model parameters of the student model based on the phenotypic consistency loss includes: Adjusting the model parameters of the student model based on the phenotypic consistency loss and the third loss.

9. The training method of the cell nucleus image segmentation model according to any one of claims 1 to 4, characterized in that After adjusting the model parameters of the student model based on the phenotypic consistency loss, the method further includes: Updating the model parameters of the teacher model based on the current model parameters of the teacher model and the adjusted model parameters of the student model; Wherein, the model structure of the student model is the same as the model structure of the teacher model.

10. According to the method for training a nuclear image segmentation model according to any one of claims 1 to 4, before inputting the first sample nuclear image into the student model to obtain the first phenotypic activation feature of the first sample nuclear image and inputting the second sample nuclear image into the teacher model to obtain the second phenotypic activation feature of the second sample nuclear image, the method further includes: Adjusting the model parameters of the initial student model based on the second labeled sample nuclear image to obtain the student model; The second labeled sample nuclear image is a sample nuclear image with a labeled nuclear image segmentation result; Synchronizing the adjusted model parameters of the initial student model to the teacher model; Wherein, the model structure of the student model is the same as the model structure of the teacher model.

11. A method for nuclear image segmentation, characterized in that, Including: Inputting the nuclear image to be segmented into the nuclear image segmentation model to obtain a target nuclear image segmentation result output by the nuclear image segmentation model; Wherein, the nuclear image segmentation model is trained based on the method for training a nuclear image segmentation model according to any one of claims 1 to 10.

12. A training device for a cell nucleus image segmentation model, characterized in that, Including: A data augmentation module for performing data augmentation processing on the unlabeled sample nuclear image to obtain different first sample nuclear images and second sample nuclear images; An image input module for inputting the first sample nuclear image into the student model to obtain the first phenotypic activation feature of the first sample nuclear image and inputting the second sample nuclear image into the teacher model to obtain the second phenotypic activation feature of the second sample nuclear image; A loss determination module for determining a phenotypic consistency loss based on the difference between the first phenotypic activation feature and the second phenotypic activation feature; A parameter adjustment module, configured to adjust model parameters of the student model based on the phenotypic consistency loss to obtain a trained nucleus image segmentation model; Wherein, the unlabeled sample nucleus image is a sample nucleus image without a labeled nucleus image segmentation result; both the student model and the teacher model are used for nucleus image segmentation of an input nucleus image; the first phenotypic activation feature is obtained by fusing the first phenotypic vector of the first sample nucleus image with the first feature map of the first sample nucleus image, and the first phenotypic vector is a model parameter that can be learned by the student model; the second phenotypic activation feature is obtained by fusing the second phenotypic vector of the second sample nucleus image with the second feature map of the second sample nucleus image, and the second phenotypic vector is a model parameter that can be learned by the teacher model; both the first phenotypic vector and the second phenotypic vector are used to characterize the phenotypic features of the nucleus.

13. A device for nuclear image segmentation, characterized in that, It includes: An image segmentation module, configured to input a nucleus image to be segmented into the nucleus image segmentation model to obtain a target nucleus image segmentation result output by the nucleus image segmentation model; Wherein, the nucleus image segmentation model is trained based on the training method of the nucleus image segmentation model according to any one of claims 1 to 10.

14. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the training method of the nucleus image segmentation model according to any one of claims 1 to 10 or implements the nucleus image segmentation method according to claim 11.

15. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the training method of the nucleus image segmentation model according to any one of claims 1 to 10 or implements the nucleus image segmentation method according to claim 11.

16. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the training method of the nucleus image segmentation model according to any one of claims 1 to 10 or implements the nucleus image segmentation method according to claim 11.

Images