A semi-supervised medical image segmentation method and device

Abstract

The application discloses a kind of semi-supervised medical image segmentation method and device, method includes: constructing double branch medical image segmentation network, including: the same structure of teacher subnetwork and student subnetwork;Based on segmentation network output segmentation graph;Design entropy-weighted consistency loss, construct consistency constraint in the uncertainty area of segmentation graph;The segmentation graph of student subnetwork output is one-hot coding to generate pseudo label, construct dynamic weighted pseudo-supervision loss, more confident pixel in the pseudo label is given higher weight, to provide more effective supervision for certainty area;Cross-entropy loss and dice loss are combined into supervision loss, entropy-weighted consistency loss and dynamic weighted pseudo-supervision loss are combined into unsupervised loss, the overall loss function of network is obtained by jointing supervision loss and unsupervised loss;Based on overall loss function training finally output segmented medical image.The device includes: processor and memory.The application provides more reliable supervision signal for network, and finally realizes the improvement of medical image segmentation performance.

Description

Technical Field

[0001] This invention relates to the fields of deep learning and image segmentation, and in particular to a semi-supervised medical image segmentation method and apparatus. Background Technology

[0002] With advancements in medical imaging technologies such as computed tomography (CT) and magnetic resonance imaging (MRI), medical image segmentation has attracted widespread attention from researchers. This technology aims to accurately segment organs or lesion regions from medical images, providing effective support for subsequent diagnosis by physicians. In recent years, with the innovation of deep learning technology, deep learning-based medical image segmentation methods have made significant progress. Among them, supervised learning-based medical image segmentation methods, trained on large-scale data with real-world annotations, have achieved excellent segmentation results. However, a large amount of unlabeled data exists in real-world medical scenarios. Manually annotating large-scale unlabeled medical image data is not only time-consuming and labor-intensive but also requires a high level of expertise from the annotators, which greatly limits the practical application of supervised learning-based medical image segmentation methods.

[0003] To alleviate reliance on large-scale labeled data, researchers have begun exploring semi-supervised learning-based medical image segmentation methods. These methods aim to improve segmentation performance in label-constrained scenarios by utilizing a small amount of labeled data and a large amount of unlabeled data. Based on how unlabeled data is utilized, existing semi-supervised medical image segmentation methods can be categorized into consistency regularization methods and entropy minimization methods. Consistency regularization methods typically add different random perturbations to the same unlabeled data and use it as network input. By encouraging the network to produce consistent predictions for inputs with different perturbations, they achieve effective utilization of unlabeled data. For example, Xu et al. proposed a consistency regularization method based on fuzzy theory. This method uses an average teacher model to ensure consistency in the outputs of the teacher and student networks in uncertain regions, improving the segmentation accuracy of the left atrium and brain tumors. However, this type of method only calculates the loss in uncertain regions, failing to fully utilize information from deterministic regions to supervise network learning. In contrast, entropy minimization methods can fully utilize information from deterministic regions, thus providing more effective supervision for the network. Specifically, these methods typically train an initial segmentation model on labeled data, then generate pseudo-labels based on the deterministic regions in the segmentation results output by the model, and finally use the generated pseudo-labels to optimize network learning, thereby achieving effective utilization of unlabeled data. For example, Xu et al. proposed an adaptive threshold-adjusted entropy minimization method, which dynamically adjusts the threshold of the entropy value during training to generate pseudo-labels with higher confidence, thereby improving medical image segmentation results. Although the above-mentioned consistency regularization methods and entropy minimization methods achieve effective utilization of information from uncertain and deterministic regions, respectively, these methods usually treat all pixels within uncertain or deterministic regions equally, ignoring the differences between different pixels. This leads to insufficient attention to pixels at blurred edges or small branches within uncertain regions, and insufficient utilization of high-confidence pixels within deterministic regions, resulting in unreliable supervision signals provided by the network and ultimately affecting segmentation performance. Summary of the Invention

[0004] While existing semi-supervised medical image segmentation methods have made some progress, these methods typically treat all pixels within uncertain or deterministic regions equally, ignoring the differences between different pixels. This leads to insufficient attention to pixels at blurred edges or small branches within uncertain regions, and inadequate utilization of high-confidence pixels within deterministic regions, resulting in unreliable supervision signals provided by the network and ultimately affecting segmentation performance. Therefore, this invention proposes a semi-supervised medical image segmentation method and apparatus. By fully exploiting the differences between different pixels within uncertain and deterministic regions, it adaptively assigns weights to different pixels when constructing constraints, thereby providing the network with more reliable supervision signals and ultimately improving the performance of medical image segmentation. Details are described below:

[0005] A first aspect: a semi-supervised medical image segmentation method, the method comprising:

[0006] A bi-branch medical image segmentation network is constructed, comprising a teacher sub-network and a student sub-network with identical structures; a segmentation map is output based on the segmentation network;

[0007] The design uses entropy-weighted consistency loss to construct consistency constraints in the uncertain regions of the segmented graph;

[0008] One-hot encoding is performed on the segmentation map output by the student subnetwork to generate pseudo-labels. A dynamically weighted pseudo-supervision loss is constructed to assign higher weights to the more confident pixels in the pseudo-labels, so as to provide more effective supervision for deterministic regions.

[0009] The cross-entropy loss and Dice loss are combined to form the supervised loss, and the entropy-weighted consistency loss and the dynamically weighted pseudo-supervised loss are combined to form the unsupervised loss. The overall loss function of the network is obtained by combining the supervised loss and the unsupervised loss.

[0010] The final output is a segmented medical image trained based on the overall loss function.

[0011] The entropy-weighted consistency loss is as follows:

[0012]

[0013]

[0014] in, The weighted graph representing the region of uncertainty, L c The entropy-weighted consistency loss represents the Sigmoid function, and the mean squared error function represents the mean squared error function. The entropy-weighted consistency loss makes the network pay more attention to challenging pixels with low confidence in uncertain regions, thereby improving segmentation accuracy.

[0015] The dynamically weighted pseudo-supervision loss is as follows:

[0016]

[0017]

[0018] Among them, L d This represents the dynamically weighted pseudo-monitoring loss. Weighted graph representing deterministic regions This indicates that the loss is calculated only in deterministic regions, and Dice(·) represents the Dice loss; the dynamically weighted pseudo-supervision loss is used to assign higher weights to more confident pixels in pseudo-labels to improve the supervision accuracy of deterministic regions.

[0019] The overall loss function is:

[0020]

[0021] L u =L c +L d

[0022]

[0023] Among them, L total L represents the overall loss function. s L represents the monitoring loss. u This represents the unsupervised loss, where t represents the current training step. max L represents the total number of training steps; c L represents the entropy-weighted consistency loss. d This represents the pseudo-monitoring loss, which is dynamically weighted. The input image with labeled data is processed by the student subnetwork Net. s The predicted segmentation result, Y i represents the true value of the corresponding segmentation graph, and CE(·) represents the cross-entropy loss; the multi-loss function mechanism is used to make more efficient use of labeled and unlabeled data, and improve the supervision accuracy of labeled and unlabeled data.

[0024] In a second aspect, a semi-supervised medical image segmentation apparatus, the apparatus comprising: a processor and a memory, the memory storing program instructions, the processor calling the program instructions stored in the memory to cause the apparatus to perform the method described in any of the first aspects.

[0025] Third aspect, a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method described in any one of the first aspects.

[0026] The beneficial effects of the technical solution provided by this invention are:

[0027] 1) This invention proposes a semi-supervised medical image segmentation method, which explores the differences between different pixels in a region to adaptively assign weights to different pixels when constructing constraints, thereby providing a more reliable supervision signal for the network and ultimately improving the performance of medical image segmentation;

[0028] 2) This invention designs an entropy-weighted consistency loss, which establishes a numerically decreasing mapping relationship between the entropy value and the loss weight in the uncertain region, encouraging the network to pay more attention to the challenging pixels, thereby providing more effective supervision for the uncertain region;

[0029] 3) This invention designs a dynamically weighted pseudo-supervised loss, which dynamically adjusts the weights of different pixels in the pseudo-labels based on the change in confidence, providing a more reliable pseudo-supervised signal for network learning, thereby further improving the segmentation performance of the model. Attached Figure Description

[0030] Figure 1 This is a flowchart of a semi-supervised medical image segmentation method. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below.

[0032] Based on the above analysis, this invention proposes a semi-supervised medical image segmentation method. This method effectively mines the differences between different pixels within a region to encourage the network to focus more on challenging pixels such as edges and fine branches within uncertain regions. Furthermore, by increasing the weights of pixels with higher confidence in pseudo-labels within deterministic regions, a more reliable supervisory signal is provided to the network, thereby improving the network's segmentation performance.

[0033] I. Constructing a Two-Branch Medical Image Segmentation Network

[0034] This invention constructs a two-branch medical image segmentation network, which adopts a teacher-student network architecture, including a teacher subnetwork and a student subnetwork with identical structures. The two-branch medical image segmentation network takes an unlabeled image as input and outputs two different segmentation maps. Specifically, the input unlabeled image... Data augmentation converts the image into two augmented images I′. iand I″ i Then, they are respectively sent to the teacher subnetwork Net. t and student subnetwork Net s In this process, two segmentation maps S′ are obtained. i and S″i 。 Both the teacher and student subnetworks consist of an encoder and a decoder. The encoder contains multiple 3D convolutional layers (e.g., five 3D convolutional layers), each accompanied by a 3D max-pooling layer. Through convolution and pooling operations, the encoder can progressively reduce the spatial dimension of the feature map and increase the number of channels, enabling the extraction of multi-scale features from the image. The decoder contains multiple 3D transposed convolutional layers (e.g., four 3D transposed convolutional layers) to upsample the feature map, thereby progressively restoring the image resolution. Furthermore, skip connections are established between shallow and deep features in both the encoder and decoder to achieve effective fusion of features from different levels.

[0035] II. Design Entropy Weighted Consistency Loss

[0036] Segmentation map S′ output by a two-branch medical image segmentation network i and S″ i In this embodiment of the invention, an entropy-weighted consistency loss is designed for the segmented graph S′. i and S″ i Consistency constraints are constructed based on the uncertain region. Specifically, firstly, based on S′ i and S″ i A joint counting matrix was constructed to represent S′. i and S″ i Inconsistencies in segmentation category predictions are then addressed. The joint count matrix is ​​then normalized to obtain the uncertainty graph. In uncertainty diagram In this context, a pixel value of 1 indicates that the segmentation result corresponding to that pixel is reliable, while a pixel value of 0 indicates that the segmentation result corresponding to that pixel is unreliable.

[0037] Meanwhile, based on the teacher subnetwork Net t Output segmentation image S′ i Calculate its corresponding entropy value E′ i The formula for this process is as follows:

[0038]

[0039] Where T represents the total number of pixels in the image, b represents each pixel, c represents the predicted class, and S′ i,c S′ i The predicted value for category c. E′ i Each pixel in the array takes a value between 0 and 1, and its value represents S′.i The entropy value of the corresponding pixel. A higher entropy value indicates a lower confidence level; a lower entropy value indicates a higher confidence level.

[0040] Generally, pixels at edges and small branches in an image have low confidence levels, posing a greater challenge to segmentation and having a greater impact on the final segmentation performance. Therefore, to improve segmentation performance, an entropy-weighted consistency loss is designed to encourage the network to focus more on these challenging pixels. The formula for this loss is as follows:

[0041]

[0042]

[0043] in, The weighted graph representing the region of uncertainty, L c The denoting factor is entropy-weighted consistency loss, where Sigmoid(·) represents the Sigmoid function and MSE(·) represents the mean squared error function. Since the Sigmoid function is monotonically increasing, it can assign greater weights to pixels with lower confidence (i.e., higher entropy values), thus allowing the network to focus more on challenging pixels with low confidence in uncertain regions, thereby improving segmentation accuracy.

[0044] III. Designing a dynamically weighted pseudo-supervised loss

[0045] To further achieve effective supervision of deterministic regions, this embodiment of the invention designs a dynamically weighted pseudo-supervision loss. Specifically, based on the student subnetwork Net... s Output segmentation image S″ i First, it is one-hot encoded to generate the pseudo-label P. i The formula for the above process is expressed as follows:

[0046] P i =argmax(S″ i )

[0047] Here, argmax(·) represents the operation of selecting the index corresponding to the maximum value.

[0048] At the same time, calculate S″ i The corresponding entropy graph E″ i :

[0049]

[0050] Among them, E″ i Each pixel in the image takes a value between 0 and 1, and these values ​​directly reflect S″. iThe entropy value of the corresponding pixel in the graph represents the confidence level of that pixel to a certain extent.

[0051] Considering the pseudo-label P i With segmentation image S″ i The correspondence, E″ i It can also achieve P i This provides a precise representation of the confidence level of each pixel. Intuitively, the higher the confidence level of a pixel, the more accurate its corresponding pseudo-label, and the higher its weight should be assigned when used as a supervisory signal. Based on this, this embodiment of the invention constructs a dynamically weighted pseudo-supervision loss, the formula of which is as follows:

[0052]

[0053]

[0054] Among them, L d This represents the dynamically weighted pseudo-monitoring loss. Weighted graph representing deterministic regions This indicates that the loss is calculated only in the deterministic region; Dice(·) represents the Dice loss. Because... As entropy decreases (i.e. confidence increases), pixels that are more confident in pseudo-labels can be assigned higher weights, which helps improve the accuracy of supervision of deterministic regions.

[0055] IV. Constructing a multi-loss function mechanism

[0056] For semi-supervised segmentation tasks, the training process requires the joint use of labeled and unlabeled data. Therefore, this invention constructs a multi-loss function mechanism, combining supervised and unsupervised losses to form an overall loss function, achieving efficient utilization of both types of data. Specifically, the supervised loss includes cross-entropy loss and Dice loss. The supervised loss L... s The formula is expressed as follows:

[0057]

[0058] Among them, L s Indicates monitoring losses, The input image with labeled data is processed by the student subnetwork Net. s The predicted segmentation result, Y i represents the truth value of the corresponding segmentation graph, and CE(·) represents the cross-entropy loss.

[0059] Furthermore, the entropy-weighted consistency loss L c And dynamically weighted pseudo-supervision loss L d Together they constitute the unsupervised loss. Unsupervised loss Lu The formula is expressed as follows:

[0060] L u =L c +L d

[0061] Finally, by combining supervised and unsupervised losses, the overall loss function of the network is defined as follows:

[0062]

[0063] Where t represents the current training step number, t max This represents the total number of training steps.

[0064] V. Training a Semi-Supervised Medical Image Segmentation Network

[0065] The embodiments of the present invention use the overall loss function L total Optimize student subnetwork Net s During training, an exponential moving average strategy is used for the teacher subnetwork Net. t The process is updated to integrate information from different training steps. The formula for this process is as follows:

[0066] θ′ t =αθ t +(1-α)θ s

[0067] Where, θ′ t Represents the updated teacher subnetwork Net t The parameter, θ t and θ s These represent the teacher subnetwork Net before the update. t and student subnetwork Net s The parameter α represents the smoothing coefficient of the exponential moving average. In this embodiment of the invention, α is set to 0.99.

[0068] This invention example uses only the trained teacher subnetwork Net. t Testing is conducted to obtain the final semi-supervised medical image segmentation results. Specifically, the medical image to be tested is fed into the trained teacher sub-network Net. t This is done to obtain the final segmentation result. The formula for the above process is expressed as follows:

[0069]

[0070] Where S represents the result of semi-supervised segmentation of any medical image I in the test set. Represents the trained teacher subnetwork Net t Its network parameters are θ′t The final output of the proposed semi-supervised medical image segmentation network is the segmentation result S described above, which can be used to assist doctors in diagnosis and improve diagnostic efficiency.

[0071] A semi-supervised medical image segmentation apparatus includes a processor and a memory, wherein the memory stores program instructions, and the processor invokes the program instructions stored in the memory to cause the apparatus to perform the following method steps:

[0072] A bi-branch medical image segmentation network is constructed, comprising a teacher sub-network and a student sub-network with identical structures; a segmentation map is output based on the segmentation network;

[0073] The design uses entropy-weighted consistency loss to construct consistency constraints in the uncertain regions of the segmented graph;

[0074] One-hot encoding is performed on the segmentation map output by the student subnetwork to generate pseudo-labels. A dynamically weighted pseudo-supervision loss is constructed to assign higher weights to the more confident pixels in the pseudo-labels, so as to provide more effective supervision for deterministic regions.

[0075] The cross-entropy loss and Dice loss are combined to form the supervised loss, and the entropy-weighted consistency loss and the dynamically weighted pseudo-supervised loss are combined to form the unsupervised loss. The overall loss function of the network is obtained by combining the supervised loss and the unsupervised loss.

[0076] The final output is a segmented medical image trained based on the overall loss function.

[0077] The entropy-weighted consistency loss is:

[0078]

[0079]

[0080] in, The weighted graph representing the region of uncertainty, L c The entropy-weighted consistency loss represents the Sigmoid function, and the mean squared error function represents the mean squared error function. The entropy-weighted consistency loss makes the network pay more attention to challenging pixels with low confidence in uncertain regions, thereby improving segmentation accuracy.

[0081] The dynamically weighted pseudo-supervision loss is:

[0082]

[0083]

[0084] Among them, L d This represents the dynamically weighted pseudo-monitoring loss. Weighted graph representing deterministic regions This indicates that the loss is calculated only in deterministic regions, and Dice(·) represents the Dice loss; the dynamically weighted pseudo-supervision loss is used to assign higher weights to more confident pixels in pseudo-labels to improve the supervision accuracy of deterministic regions.

[0085] The overall loss function is:

[0086]

[0087] L u =L c +L d

[0088]

[0089] Among them, L total L represents the overall loss function. s L represents the monitoring loss. u This represents the unsupervised loss, where t represents the current training step. max L represents the total number of training steps; c L represents the entropy-weighted consistency loss. d This represents the pseudo-monitoring loss, which is dynamically weighted. The input image with labeled data is processed by the student subnetwork Net. s The predicted segmentation result, Y i represents the true value of the corresponding segmentation graph, and CE(·) represents the cross-entropy loss; the multiple loss function mechanism is used to make more efficient use of labeled and unlabeled data, and improve the supervision accuracy of labeled and unlabeled data.

[0090] It should be noted that the device descriptions in the above embodiments correspond to the method descriptions in the embodiments, and the embodiments of the present invention will not be repeated here.

[0091] The execution entities of the aforementioned processor and memory can be devices with computing functions such as computers, microcontrollers, and single-chip microcomputers. In specific implementations, the embodiments of the present invention do not limit the execution entities and can select them according to the needs of actual applications.

[0092] Data signals are transmitted between the memory and the processor via a bus, which will not be elaborated upon in this embodiment of the invention.

[0093] Based on the same inventive concept, embodiments of the present invention also provide a computer-readable storage medium, the storage medium including a stored program, which, when the program is running, controls the device where the storage medium is located to execute the method steps in the above embodiments.

[0094] The computer-readable storage medium includes, but is not limited to, flash memory, hard disk, solid-state drive, etc.

[0095] It should be noted that the description of the readable storage medium in the above embodiments corresponds to the description of the method in the embodiments, and the embodiments of the present invention will not be repeated here.

[0096] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of the present invention is generated.

[0097] A computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in or transmitted through a computer-readable storage medium. A computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be magnetic or semiconductor, etc.

[0098] Unless otherwise specified, the model numbers of the various devices in this embodiment of the invention are not limited, and any device that can perform the above functions is acceptable.

[0099] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0100] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A semi-supervised medical image segmentation method, characterized in that, The method includes: Construct a two-branch medical image segmentation network, including: a teacher subnetwork and a student subnetwork with identical structures; and outputting segmentation maps based on the segmentation network. and We design an entropy-weighted consistency loss and construct consistency constraints in the uncertain regions of the segmented graph. based on and A joint counting matrix was constructed, and the joint counting matrix was normalized to obtain the uncertainty graph. In the uncertainty diagram In the diagram, a pixel value of 1 indicates that the segmentation result corresponding to that pixel is reliable, while a pixel value of 0 indicates that the segmentation result corresponding to that pixel is unreliable. Based on teacher subnetwork Output segmentation map Calculate its corresponding entropy value graph One-hot encoding is performed on the segmentation map output by the student subnetwork to generate pseudo-labels. A dynamically weighted pseudo-supervision loss is constructed to assign higher weights to the more confident pixels in the pseudo-labels, so as to provide more effective supervision for deterministic regions. The cross-entropy loss and Dice loss are combined to form the supervised loss, and the entropy-weighted consistency loss and the dynamically weighted pseudo-supervised loss are combined to form the unsupervised loss. The overall loss function of the network is obtained by combining the supervised loss and the unsupervised loss. The final output is a segmented medical image trained based on the overall loss function. The entropy-weighted consistency loss is as follows: ； ； in, Weighted graph representing regions of uncertainty This represents the entropy-weighted consistency loss. This represents the Sigmoid function. The mean squared error function is represented by the entropy-weighted consistency loss, which makes the network pay more attention to challenging pixels with low confidence in uncertain regions, thereby improving segmentation accuracy. The dynamically weighted pseudo-supervision loss is as follows: Based on student subnetwork Output segmentation map First, it is one-hot encoded to generate pseudo-tags. ,as follows: ； in, This indicates the operation of selecting the index corresponding to the maximum value; Simultaneously, calculation Corresponding entropy diagram The formula for dynamically weighted pseudo-supervised loss is: ； ； in, This represents the dynamically weighted pseudo-monitoring loss. Weighted graph representing deterministic regions This indicates that the loss is calculated only in the deterministic region. This represents the Dice loss; the dynamically weighted pseudo-supervision loss improves the supervision accuracy of deterministic regions by assigning higher weights to pixels that are more confident in the pseudo-labels. The overall loss function is: ； ； ； in, Represents the overall loss function. This indicates the extent of the monitoring loss. This indicates unsupervised loss. Indicates the current training steps. This represents the total number of training steps; This represents the entropy-weighted consistency loss. This represents the pseudo-monitoring loss, which is dynamically weighted. The input image with labeled data is processed by the student subnetwork. The predicted segmentation results This represents the corresponding truth value of the segmentation graph. This represents the cross-entropy loss; the multiple loss function mechanism is used to make more efficient use of labeled and unlabeled data, thereby improving the accuracy of supervision of labeled and unlabeled data.

2. A semi-supervised medical image segmentation device, characterized in that, The device includes a processor and a memory, the memory storing program instructions, the processor calling the program instructions stored in the memory to cause the device to perform the method of claim 1.

3. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method of claim 1.

Images