Histopathological image segmentation method, device, equipment, medium and product

Abstract

The present invention relates to the field of image processing technology, and in particular to a method, device, equipment, medium and product for segmenting histopathological images, wherein the method comprises: training a semi-supervised segmentation model according to labeled data and unlabeled data, wherein the number of samples of the labeled data is less than the number of samples of the unlabeled data, learning different feature representations between classes according to the prototype semantic comparison of each class, calculating the contrast loss according to the different feature representations between classes, and obtaining prototype pseudo-labels using the prototype semantic similarity between labeled samples and unlabeled samples; updating the weight parameters of the student model according to the total loss of the training process, transferring the weight parameters of the student model to the teacher model, and using the semi-supervised segmentation model to segment the histopathological images after the training to obtain the segmentation results. Thus, the problems of how to better use global information for prototype contrast learning and reduce the noise of pseudo-labels are solved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, device, equipment, medium and product for segmenting a tissue pathology image. Background Art

[0002] Histopathology images are images obtained by observing specially processed tissue slices under a microscope. They are used to study the microstructure of cells, the structure of tissues, and possible pathological changes. In histopathology image analysis, accurate image segmentation is of great significance for the diagnosis, research, and treatment of diseases. These images usually contain complex cell structures and diverse tissue types, making automatic segmentation a challenging task. Although deep learning methods have made significant progress in image segmentation, most of these methods rely on a large amount of accurately labeled data to train the model, and in the field of histopathology, obtaining such data is both expensive and time-consuming.

[0003] Semi-supervised segmentation is a machine learning technique widely used in medical image processing, computer vision and other fields. It aims to train the model by combining a small amount of labeled data with a large amount of unlabeled data, so as to achieve the purpose of accurate image segmentation. This type of method is particularly suitable for situations where the cost of obtaining labeled data is high or it is difficult to obtain a large number of labeled samples. Semi-supervised segmentation enhances the generalization ability of the model by utilizing the potential information in unlabeled data and reduces the dependence on a large amount of manually labeled data. It usually includes self-training, generative adversarial networks, graph models, contrastive learning and other technologies. Contrastive learning methods enable the model to learn richer and more robust feature representations by comparing the similarities or differences between data points.

[0004] However, traditional semi-supervised learning methods, such as pseudo-labeling and consistency regularization, can utilize unlabeled data, but they usually rely on the quality of the initial output of the model. If the initial prediction error is large, these methods are prone to accumulate errors, resulting in degraded model performance. Summary of the invention

[0005] The present invention provides a histopathological image segmentation method, device, equipment, medium and product to solve the problems of how to better utilize global information for prototype comparative learning and reduce the noise of pseudo labels.

[0006] The first aspect of the present invention provides a method for segmenting a tissue pathology image, comprising the following steps: generating labeled data and unlabeled data according to a tissue pathology image data set, wherein the labeled data includes a plurality of labeled samples, the unlabeled data includes a plurality of unlabeled samples, and the number of labeled data samples is less than the number of unlabeled data samples; training a semi-supervised segmentation model according to the labeled data and the unlabeled data, wherein the semi-supervised segmentation model uses Mean The Teacher architecture includes a student model and a teacher model, obtains the supervision loss of the student model training process, the consistency loss of the student model and the teacher model in the joint training process, and the feature vector in the feature space of the labeled sample; determines the prototype semantics of each class according to the feature vector, learns different feature representations between classes according to the prototype semantic comparison, calculates the contrast loss according to the different feature representations between classes, obtains the prototype pseudo-label by using the prototype semantic similarity between the labeled sample and the unlabeled sample, and calculates the prototype loss according to the prototype pseudo-label and the pseudo-label predicted by the student model; calculates the total loss of the student model according to the supervision loss, the consistency loss, the contrast loss and the prototype loss, updates the weight parameters of the student model according to the total loss, passes the weight parameters of the student model to the teacher model, and after the training is completed, uses the semi-supervised segmentation model to segment the tissue pathology image to obtain the segmentation result.

[0007] Optionally, the supervision loss of the student model training process, the consistency loss of the joint training process of the student model and the teacher model, and the feature vector in the feature space of the labeled samples are obtained, including: inputting multiple labeled samples into the student model, the student model outputs a first prediction map for each labeled sample, and the supervision loss is calculated according to the first prediction map and the label map of each labeled sample; inputting multiple unlabeled samples into the student model and the teacher model respectively, the student model outputs a second prediction map for each unlabeled sample, the teacher model outputs a third prediction map for each unlabeled sample, and the consistency loss is calculated according to the second prediction map and the third prediction map; and obtaining the feature vector in the feature space of the labeled samples through a projection head set after the student model.

[0008] Optionally, the projection head includes a multilayer perceptron of a first linear layer, a multilayer perceptron of a second linear layer, an activation function after the first linear layer, and a batch normalization layer.

[0009] Optionally, the different feature representations between classes include positive samples with the same category as the anchor point and positive samples with different categories from the anchor point, and the contrast loss is calculated based on the different feature representations between classes, including: generating positive and negative sample pairs based on the positive samples and the negative samples; and calculating the contrast loss based on the positive and negative sample pairs.

[0010] Optionally, the contrast loss is calculated as:

[0011]

[0012] Among them, u and ν are positive sample pairs, ν i represents the pixels contained in the negative sample set, τ represents the temperature parameter, Represents a feature map containing negative samples, N represents the number of sampled negative voxels, and i represents a natural number less than N-1.

[0013] Optionally, the prototype pseudo-label is obtained by using the prototype semantic similarity between the labeled sample and the unlabeled sample, including: obtaining the centroid of each class prototype; calculating the similarity between any point of each unlabeled sample and the centroid of each class prototype; and selecting the prototype semantic with the highest similarity as the prototype semantic prototype pseudo-label.

[0014] The second aspect of the present invention provides a segmentation device for a tissue pathology image, comprising: a generation module, used to generate labeled data and unlabeled data according to a tissue pathology image data set, wherein the labeled data includes a plurality of labeled samples, the unlabeled data includes a plurality of unlabeled samples, and the number of samples of the labeled data is less than the number of samples of the unlabeled data; a training module, used to train a semi-supervised segmentation model according to the labeled data and the unlabeled data, wherein the semi-supervised segmentation model uses Mean The Teacher architecture includes a student model and a teacher model, and obtains the supervision loss of the student model training process, the consistency loss of the student model and the teacher model in the joint training process, and the feature vector in the feature space of the labeled sample; the contrastive learning module is used to determine the prototype semantics of each class according to the feature vector, compare and learn different feature representations between classes according to the prototype semantics, calculate the contrastive loss according to the different feature representations between classes, obtain the prototype pseudo-label by using the prototype semantic similarity between the labeled sample and the unlabeled sample, and calculate the prototype loss according to the prototype pseudo-label and the pseudo-label predicted by the student model; the parameter transfer module is used to calculate the total loss of the student model according to the supervision loss, consistency loss, contrastive loss and prototype loss, update the weight parameters of the student model according to the total loss, and transfer the weight parameters of the student model to the teacher model. After the training is completed, the semi-supervised segmentation model is used to segment the tissue pathology image to obtain the segmentation result.

[0015] A third aspect of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for segmenting a tissue pathology image as described in the above embodiment.

[0016] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program or instructions stored thereon, and the computer program or instructions are executed by a processor to implement a method for segmenting a tissue pathology image as described in the above embodiment.

[0017] A fifth aspect of the present invention provides a computer program, including a computer program or instructions, which, when executed, are used to implement the method for segmenting a tissue pathology image as described in the above embodiment.

[0018] Thus, the present invention includes the following beneficial effects:

[0019] The embodiment of the present invention can use the feature vector of each category to define the prototype semantics of the category; by calculating the prototype similarity between labeled samples and unlabeled samples, high-quality prototype pseudo-labels are generated for unlabeled samples; by comparing the prototype pseudo-labels with the labels predicted by the model through the prototype loss, the reliability of the pseudo-labels and the segmentation accuracy of the overall model are further improved, thereby promoting the model to learn the unique feature representation of each category by introducing a prototype contrast learning module, enhancing the compactness within the class and the separability between classes, and thus providing an effective strategy for semi-supervised learning, which is particularly suitable for improving the segmentation performance of tissue pathology images when the labeled data is limited.

[0020] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0022] Figure 1 A flowchart of a method for segmenting a tissue pathology image according to an embodiment of the present invention;

[0023] Figure 2 An example diagram of a semi-supervised segmentation model for histopathological images provided according to an embodiment of the present invention;

[0024] Figure 3 An exemplary diagram of a device for segmenting a tissue pathology image according to an embodiment of the present invention;

[0025] Figure 4 FIG. 4 is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0027] Semi-supervised learning methods in related technologies often ignore the feature differences between and within classes, especially when the class boundaries are not obvious in pathological images, the effectiveness of this method is limited. Therefore, how to better utilize global information for prototype contrast learning and reduce the noise of pseudo labels is an urgent problem to be solved.

[0028] In order to overcome the potential noise influence of pseudo-labels and the separability of classes in feature space in the related art, the present invention provides a semi-supervised segmentation method for histopathology images based on prototype contrast learning. By introducing a prototype contrast learning module, the model learns different feature representations between classes, thereby having better intra-class compactness and inter-class discreteness; in addition, the prototype semantics of the corresponding class is obtained through each class of feature vectors, and the prototype pseudo-label is obtained by using the prototype similarity between labeled and unlabeled samples; finally, the prototype loss is calculated with the predicted pseudo-label to improve the reliability of the pseudo-label.

[0029] The following is an explanation of the semi-supervised segmentation method of histopathology images based on prototype contrast learning, as follows:

[0030] Figure 1 The present invention provides a flowchart of a method for segmenting a tissue pathology image.

[0031] like Figure 1 As shown, the segmentation method of the tissue pathology image includes the following steps:

[0032] In step S101, labeled data and unlabeled data are generated according to a tissue pathology image dataset, wherein the labeled data includes a plurality of labeled samples, the unlabeled data includes a plurality of unlabeled samples, and the number of labeled data samples is less than the number of unlabeled data samples.

[0033] It is understandable that the embodiment of the present invention can preprocess tissue pathology images to form a data set, which includes labeled data with pixel-level annotations and a large amount of unlabeled data, wherein the labeled samples include labeled images and corresponding labels, and the unlabeled samples only include labeled images.

[0034] It should be noted that since the annotation of tissue pathology images is very time-consuming, there are fewer annotated images, resulting in the number of samples of labeled data being much smaller than the number of samples of unlabeled data. Semi-supervised technology is mainly used to alleviate the problem of few labeled samples. The embodiments of the present invention can use semi-supervised technology to improve the segmentation performance of tissue pathology images when there are few labeled samples.

[0035] Specifically, the histopathology image dataset consists of histopathology images and their corresponding pixel-level label maps, in which different pixel values ​​represent a class. First, the embodiment of the present invention can uniformly crop the labeled images and labels of the histopathology image dataset in the histopathology image dataset into a preset size using a sliding window. The preset size, such as 256×256, is not specifically limited. Second, the entire histopathology image dataset is divided into two data sources: labeled data and labeled data. Unlabeled data Among them, x i is the i-th labeled image, y i is the label map corresponding to the i-th labeled image, D l < <D u .

[0036] It should be noted that before executing step S102, an embodiment of the present invention can construct a semi-supervised segmentation network framework, use the Mean Teacher architecture and initialize the student model and the teacher model. Specifically: use the Mean Teacher architecture, select the deeplabv3plus segmentation model as the network architecture of the student model and the teacher model, and add a projection head after each model; use the same weights to initialize the student model and the teacher model at the beginning of training, so that the teacher model gradually learns from the student model and provides guidance.

[0037] In step S102, a semi-supervised segmentation model is trained according to labeled data and unlabeled data. The semi-supervised segmentation model uses a Mean Teacher architecture, including a student model and a teacher model, to obtain the supervision loss of the student model training process, the consistency loss of the student model and the teacher model joint training process, and the feature vector in the feature space of the labeled samples.

[0038] It can be understood that the embodiments of the present invention can put labeled data into student model training, calculate the supervised loss with the label map corresponding to the labeled image, and obtain the sample feature vector; for unlabeled data, it is sent to the student model and the teacher model respectively, two prediction maps are generated, and the consistency loss is calculated, so that the labeled data can be used for supervised training and the unlabeled data can be used for unsupervised training, and the labeled data can be used to help the unlabeled data for unsupervised training, and the high-performance training of the semi-supervised segmentation model can be completed when the labeled samples are limited, thereby improving the segmentation performance of the semi-supervised segmentation model for tissue pathology images, saving training time, and improving the training efficiency of the semi-supervised segmentation model.

[0039] In an embodiment of the present invention, the supervision loss of the student model training process, the consistency loss of the student model and the teacher model joint training process, and the feature vector in the feature space of the labeled sample are obtained, including: inputting multiple labeled samples into the student model, the student model outputs a first prediction map for each labeled sample, and the supervision loss is calculated according to the first prediction map and the label map of each labeled sample; inputting multiple unlabeled samples into the student model and the teacher model respectively, the student model outputs a second prediction map for each unlabeled sample, the teacher model outputs a third prediction map for each unlabeled sample, and the consistency loss is calculated according to the second prediction map and the third prediction map; and obtaining the feature vector in the feature space of the labeled sample through a projection head set behind the student model.

[0040] The projection head includes a multilayer perceptron of the first linear layer, a multilayer perceptron of the second linear layer, an activation function after the first linear layer, and a batch normalization layer. The type of activation function can be specifically set or selected according to actual needs, for example, a ReLU (Rectified Linear Unit) activation function can be selected.

[0041] Specifically, the semi-supervised segmentation model of histopathology images based on prototype contrastive learning is as follows Figure 2 As shown in the figure, the semi-supervised segmentation model of histopathology images includes a student model and a teacher model. The training and loss calculation of the semi-supervised segmentation model of histopathology images will be described as follows:

[0042] (1) For labeled data D l , put it into the student model M s Get the prediction graph Through the prediction graph Corresponding to the label map Y, the supervision loss of each pixel in the map is calculated using cross entropy and Dice loss, and then the supervision loss L of each image is obtained seg , the calculation formula is as follows:

[0043]

[0044] Where N is the total number of pixels in the image, y n,i is the label of the true category of the nth pixel in the i-th category, p n,i is the probability that the model predicts that the pixel belongs to the i-th category.

[0045] (2) A projection head is set after the segmentation model, and the feature vector obtained by the model is converted into a new feature space through the projection head to obtain a new feature vector. Specifically: the projection head contains two linear layers of MLP (Multilayer Perceptron), the first linear layer is followed by a ReLU activation function and a batch normalization layer, and the last linear layer output does not use an activation function. The input and output dimensions are the first dimension and the second dimension, respectively, where the second dimension is larger than the first dimension, for example, the first dimension can be 16 and the second dimension can be 128, etc. A queue is established as a memory bank, and the pixels with higher prediction confidence in the prediction map output by the student model are stored in the memory bank, including the category and feature vector of the pixel; if the confidence of the prediction result is greater than the preset confidence, such as 0.7 or 0.8, etc., it can be considered that the confidence is high, and thus it is used as the source domain of the prototype.

[0046] (3) For unlabeled data, we feed them into the student model and the teacher model to obtain two prediction graphs P. s and P w ,For the two prediction images obtained, the consistency loss is calculated using the mean square error. The calculation process is as follows:

[0047]

[0048] Where N is the total number of pixels in the image, C is the total number of categories, and are the probabilities that the nth pixel predicted by the student model and the teacher model belongs to the cth category.

[0049] In step S103, the prototype semantics of each class is determined according to the feature vector, different feature representations between classes are compared and learned according to the prototype semantics, the contrast loss is calculated according to the different feature representations between classes, the prototype pseudo-label is obtained by using the prototype semantic similarity between the labeled samples and the unlabeled samples, and the prototype loss is calculated according to the prototype pseudo-label and the pseudo-label predicted by the student model.

[0050] It can be understood that the embodiments of the present invention can introduce a semantic prototype contrast learning model, obtain the prototype of each class through the class centroid, and calculate the contrast loss; use the prototype similarity between the labeled and unlabeled samples to obtain the prototype pseudo-label, so that by introducing the prototype contrast learning module, the model can learn different feature representations between classes, and thus have better intra-class compactness and inter-class discreteness, and obtain the prototype semantics of the corresponding class through each class feature vector, and use the prototype similarity between the labeled and unlabeled samples to obtain the prototype pseudo-label, generate high-quality prototype pseudo-labels for unlabeled samples, and compare the prototype pseudo-labels with the labels predicted by the model through the prototype loss, thereby further improving the reliability of the pseudo-labels and the segmentation accuracy of the overall model.

[0051] In the embodiment of the present invention, different feature representations between classes include positive samples of the same category as the anchor point and positive samples of different categories from the anchor point, and calculating contrast loss according to different feature representations between classes includes: generating positive and negative sample pairs according to positive samples and negative samples; and calculating contrast loss according to the positive and negative sample pairs. The anchor point is the sampled pixel point.

[0052] Specifically, the embodiment of the present invention can obtain the feature vector of each anchor point by using the projection head, and then obtain the prototype of each class by using the class centroid. For a certain class prototype, the centroid is used to obtain the prototype, and the calculation process is as follows:

[0053]

[0054] Where N is the total number of feature vectors of this type, x i is the feature vector of the i-th pixel of this category;

[0055] The positive and negative sample sampling strategy of contrastive learning in the embodiment of the present invention is: prototypes of the same category as the anchor point are used as positive samples, and prototypes of different categories are used as negative samples. The purpose of contrastive learning is to narrow the distance between the same category (positive sample pairs) and push away the feature representations of different categories (negative sample pairs), thereby improving the generalization and robustness of the model. According to the positive and negative sample pairs, the contrast loss is calculated using the InfoNCE function. For a specific pixel, the contrast loss calculation process is as follows:

[0056]

[0057] Among them, u and ν are positive sample pairs v i represents the pixels or feature vectors contained in the negative sample set, and τ is the temperature parameter used to control the smoothness of the similarity distribution.

[0058] In an embodiment of the present invention, a prototype pseudo-label is obtained by using the prototype semantic similarity between a labeled sample and an unlabeled sample, including: obtaining the centroid of each class prototype; calculating the similarity between any point of each unlabeled sample and the centroid of each class prototype; and selecting the prototype semantic with the highest similarity as the prototype semantic prototype pseudo-label.

[0059] Specifically, for an unlabeled sample, obtain its prototype pseudo label P p , calculate the similarity between the point and the centroid of each category through the cosine distance, and select the highest similarity as the pseudo label. The calculation process is as follows:

[0060]

[0061] The prototype loss L is calculated using the cross entropy loss through the pseudo labels generated by the student model and the prototype pseudo labels. pro .

[0062] In step S104, the total loss of the student model is calculated based on the supervision loss, consistency loss, contrast loss and prototype loss, the weight parameters of the student model are updated according to the total loss, and the weight parameters of the student model are passed to the teacher model. After the training is completed, the semi-supervised segmentation model is used to segment the tissue pathology image to obtain the segmentation result.

[0063] It can be understood that the overall loss function of the network is the sum of the loss of labeled data and the loss of unlabeled data: supervision loss, consistency loss and contrast loss. The student network model is updated according to the overall loss, and the weight parameters of the student model are passed to the teacher model using the EMA (Exponential Moving Average) strategy until the end of training. The semi-supervised segmentation model is used to segment the tissue pathology image to obtain the segmentation result. Among them, the segmentation result is obtained through the student model, and the teacher model assists the student model to improve the segmentation accuracy of the student model.

[0064] Specifically, the overall loss is used to back-propagate and update the parameters of the student model. The overall loss includes supervision loss, consistency loss, and contrast loss, which is calculated as follows:

[0065] L total =L seg +λ1L cons +λ2L cont +λ3L pro

[0066] Among them, λ1, λ2 and λ3 are the weights of the corresponding losses.

[0067] The parameters of the teacher model are updated through the EMA strategy of the parameters of the student model, that is, the weight parameters of the teacher model are updated through the EMA strategy after each update iteration of the student model. The specific process is as follows:

[0068] Assume that at each training step t, the parameters of the student model are θ t , the parameters of the teacher model are The teacher model parameter update is calculated as follows:

[0069] φ t =αφ t-1 +(1-α)θ t

[0070] Among them, α is a smoothing coefficient with a value range of [0,1], which is used to control the retention amount of the previous model parameters.

[0071] According to the segmentation method of histopathological images proposed in the embodiment of the present invention, the feature vector of each category can be used to define the prototype semantics of the category; by calculating the prototype similarity between labeled samples and unlabeled samples, high-quality prototype pseudo-labels are generated for unlabeled samples; by comparing the prototype pseudo-labels with the labels predicted by the model through the prototype loss, the reliability of the pseudo-labels and the segmentation accuracy of the overall model are further improved, thereby promoting the model to learn the unique feature representation of each category by introducing a prototype contrast learning module, enhancing the compactness within the class and the separability between classes, and thus providing an effective strategy for semi-supervised learning, which is particularly suitable for improving the segmentation performance of histopathological images when the labeled data is limited.

[0072] Next, a tissue pathology image segmentation device according to an embodiment of the present invention will be described with reference to the accompanying drawings.

[0073] Figure 3 4 is a block diagram of a device for segmenting a tissue pathology image according to an embodiment of the present invention.

[0074] like Figure 3 As shown, the tissue pathology image segmentation device 10 includes: a generation module 100, a training module 200, a comparison learning module 300 and a parameter transmission module 400.

[0075] The generation module 100 is used to generate labeled data and unlabeled data according to the tissue pathology image data set, wherein the labeled data includes multiple labeled samples, the unlabeled data includes multiple unlabeled samples, and the number of labeled data samples is less than the number of unlabeled data samples; the training module 200 is used to train the semi-supervised segmentation model according to the labeled data and the unlabeled data, and the semi-supervised segmentation model uses Mean The teacher architecture includes a student model and a teacher model, and obtains the supervision loss of the student model training process, the consistency loss of the student model and the teacher model in the joint training process, and the feature vector in the feature space of the labeled sample; the contrast learning module 300 is used to determine the prototype semantics of each class according to the feature vector, compare and learn different feature representations between classes according to the prototype semantics, calculate the contrast loss according to the different feature representations between classes, obtain the prototype pseudo-label by using the prototype semantic similarity between the labeled sample and the unlabeled sample, and calculate the prototype loss according to the prototype pseudo-label and the pseudo-label predicted by the student model; the parameter transmission module 400 is used to calculate the total loss of the student model according to the supervision loss, consistency loss, contrast loss and prototype loss, update the weight parameters of the student model according to the total loss, and pass the weight parameters of the student model to the teacher model. After the training is completed, the semi-supervised segmentation model is used to segment the tissue pathology image to obtain the segmentation result.

[0076] It should be noted that the above explanation of the embodiment of the histopathological image segmentation method is also applicable to the histopathological image segmentation device of this embodiment, and will not be repeated here.

[0077] According to the segmentation device of histopathological images proposed in the embodiment of the present invention, the feature vector of each category can be used to define the prototype semantics of the category; by calculating the prototype similarity between labeled samples and unlabeled samples, high-quality prototype pseudo-labels are generated for unlabeled samples; by comparing the prototype pseudo-labels with the labels predicted by the model through prototype loss, the reliability of the pseudo-labels and the segmentation accuracy of the overall model are further improved, thereby promoting the model to learn the unique feature representation of each category by introducing a prototype contrast learning module, enhancing the compactness within the class and the separability between classes, and thus providing an effective strategy for semi-supervised learning, which is particularly suitable for improving the segmentation performance of histopathological images when the labeled data is limited.

[0078] Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device may include:

[0079] Memory 401 , processor 402 , and a computer program stored in the memory 401 and executable on the processor 402 .

[0080] When the processor 402 executes the program, the histopathological image segmentation method provided in the above embodiment is implemented.

[0081] Furthermore, the electronic device further comprises:

[0082] The communication interface 403 is used for communication between the memory 401 and the processor 402 .

[0083] The memory 401 is used to store computer programs that can be executed on the processor 402 .

[0084] The memory 401 may include a high-speed RAM (Random Access Memory) memory, and may also include a non-volatile memory, such as at least one disk memory.

[0085] If the memory 401, the processor 402 and the communication interface 403 are implemented independently, the communication interface 403, the memory 401 and the processor 402 can be connected to each other through a bus and communicate with each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0086] Optionally, in a specific implementation, if the memory 401, the processor 402 and the communication interface 403 are integrated on a chip, the memory 401, the processor 402 and the communication interface 403 can communicate with each other through an internal interface.

[0087] The processor 402 may be a CPU (Central Processing Unit), or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention.

[0088] An embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for segmenting a tissue pathology image as described in the above embodiment.

[0089] An embodiment of the present invention provides a computer-readable storage medium on which a computer program or instruction is stored. The computer program or instruction is executed by a processor to implement the method for segmenting a tissue pathology image as described in the above embodiment.

[0090] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0091] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "N" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0092] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

[0093] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, the steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array, a field programmable gate array, etc.

[0094] A person of ordinary skill in the art may understand that all or part of the steps carried by the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the above-mentioned program may be stored in a computer-readable storage medium, which, when executed, includes one of the steps of the method embodiment or a combination thereof.

[0095] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A method for segmenting a tissue pathology image, characterized in that: The following steps are involved: Generate labeled data and unlabeled data according to the tissue pathology image data set, wherein the labeled data includes a plurality of labeled samples, the unlabeled data includes a plurality of unlabeled samples, and the number of samples of the labeled data is less than the number of samples of the unlabeled data; A semi-supervised segmentation model is trained according to the labeled data and the unlabeled data, wherein the semi-supervised segmentation model uses a Mean Teacher architecture, including a student model and a teacher model, and a supervision loss of the student model training process, a consistency loss of the student model and the teacher model in a joint training process, and a feature vector in a feature space of the labeled sample is obtained; Determine the prototype semantics of each class according to the feature vector, compare and learn different feature representations between classes according to the prototype semantics, calculate the contrast loss according to the different feature representations between the classes, obtain the prototype pseudo-label by using the prototype semantic similarity between the labeled samples and the unlabeled samples, and calculate the prototype loss according to the prototype pseudo-label and the pseudo-label predicted by the student model; The total loss of the student model is calculated according to the supervised loss, the consistency loss, the contrast loss and the prototype loss, the weight parameters of the student model are updated according to the total loss, and the weight parameters of the student model are passed to the teacher model. After the training is completed, the tissue pathology image is segmented using the semi-supervised segmentation model to obtain the segmentation result.

2. The method for segmenting a tissue pathology image according to claim 1, characterized in that: The obtaining of the supervision loss of the student model training process, the consistency loss of the student model and the teacher model joint training process, and the feature vector in the feature space of the labeled sample includes: Input the multiple labeled samples into the student model, the student model outputs a first prediction map for each labeled sample, and calculates the supervision loss according to the first prediction map and the label map of each labeled sample; Input the multiple unlabeled samples into the student model and the teacher model respectively, the student model outputs a second prediction map for each unlabeled sample, the teacher model outputs a third prediction map for each unlabeled sample, and the consistency loss is calculated according to the second prediction map and the third prediction map; The feature vector in the feature space of the labeled sample is obtained through a projection head arranged behind the student model.

3. The method for segmenting a tissue pathology image according to claim 2, characterized in that: The projection head includes a multilayer perceptron of a first linear layer, a multilayer perceptron of a second linear layer, an activation function after the first linear layer, and a batch normalization layer.

4. The method for segmenting a tissue pathology image according to claim 1, characterized in that: The different feature representations between classes include positive samples of the same category as the anchor point and positive samples of different categories from the anchor point, and the calculation of the contrast loss according to the different feature representations between classes includes: Generate positive and negative sample pairs according to the positive samples and negative samples; The contrast loss is calculated based on the positive and negative sample pairs.

5. The method for segmenting a tissue pathology image according to claim 4, characterized in that: The calculation formula of the contrast loss is: Among them, u and ν are positive sample pairs, ν i represents the pixels contained in the negative sample set, τ represents the temperature parameter, represents a feature map containing negative samples, N represents the number of sampled negative voxels, and i represents a natural number less than N-1.

6. The method for segmenting a tissue pathology image according to claim 1, characterized in that: The method of obtaining a prototype pseudo label by using the prototype semantic similarity between the labeled sample and the unlabeled sample includes: Get the centroid of each class prototype; Calculate the similarity between any point of each unlabeled sample and the centroid of each class prototype; The prototype semantics with the highest similarity is selected as the prototype pseudo-label.

7. A device for segmenting a tissue pathology image, characterized in that: include: A generating module, configured to generate labeled data and unlabeled data according to a tissue pathology image data set, wherein the labeled data includes a plurality of labeled samples, the unlabeled data includes a plurality of unlabeled samples, and the number of samples of the labeled data is less than the number of samples of the unlabeled data; A training module, used to train a semi-supervised segmentation model according to the labeled data and the unlabeled data, wherein the semi-supervised segmentation model uses a Mean Teacher architecture, including a student model and a teacher model, and obtains a supervision loss during the training process of the student model, a consistency loss during the joint training process of the student model and the teacher model, and a feature vector in the feature space of the labeled sample; a contrastive learning module, used to determine the prototype semantics of each class according to the feature vector, contrastively learn different feature representations between classes according to the prototype semantics, calculate contrast loss according to the different feature representations between classes, obtain prototype pseudo-labels using the prototype semantic similarity between labeled samples and unlabeled samples, and calculate prototype loss according to the prototype pseudo-labels and the pseudo-labels predicted by the student model; The parameter transfer module is used to calculate the total loss of the student model according to the supervision loss, the consistency loss, the contrast loss and the prototype loss, update the weight parameters of the student model according to the total loss, and transfer the weight parameters of the student model to the teacher model. After the training is completed, the semi-supervised segmentation model is used to segment the tissue pathology image to obtain the segmentation result.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for segmenting a tissue pathology image according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed, the method for segmenting a tissue pathology image according to any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed, the method for segmenting a tissue pathology image according to any one of claims 1 to 6 is implemented.

Images