Semi-supervised 3D medical image segmentation training method and system based on double teachers

Abstract

The invention provides a semi-supervised 3D medical image segmentation training method and system based on double teachers. The method comprises the following steps: initializing a student network and two teacher networks; respectively inputting the labeled data into a student network and an SAM2 teacher network, and calculating supervision loss; inputting the label-free data into a student network and two teacher networks respectively, and quantifying an uncertainty map of the student network; calculating semi-supervised loss based on the uncertainty graph; taking the weighted sum of the supervised loss and the semi-supervised loss as the total loss, and performing back propagation based on the total loss to update the parameters of the student network and the parameters of the CPE module and the adapter module in the SAM2 teacher network; and updating the parameters of the EMA teacher network by adopting an index moving average mode according to the updated parameters of the student network. The image segmentation network trained by the method can realize high-precision 3D medical image segmentation.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a semi-supervised 3D medical image segmentation training method and system based on dual teachers. Background Technology

[0002] Medical image segmentation is a crucial step in computer-aided diagnosis, enabling the precise depiction of anatomical structures or pathological regions within images. In recent years, deep learning methods, represented by convolutional neural networks, have achieved significant success in medical image segmentation tasks, with performance comparable to or even surpassing that of human experts on certain benchmarks. However, the success of these methods heavily relies on large-scale, finely annotated training datasets. For 3D medical images (such as CT and MRI), the manual annotation process, involving each voxel, is extremely time-consuming, costly, and requires intensive expert knowledge, constituting a major bottleneck for the widespread application of this technology.

[0003] To address the problem of data scarcity, semi-supervised learning has emerged as a promising solution. It trains models using massive amounts of unlabeled data and a small subset of labeled data. A core challenge in applying semi-supervised learning to 3D medical image segmentation is efficiently processing and utilizing the contextual information of 3D space. While pure 2D networks are computationally efficient, their slice-by-slice processing ignores the continuity between slices; conversely, pure 3D networks, while capturing complete spatial context, face significant computational overhead and the risk of overfitting. To balance these two approaches, researchers have proposed various hybrid methods known as 2.5D. Early practices, such as the work of Zhang et al. and Han et al., involved stacking adjacent 2D slices as multi-channel inputs before feeding them into a 2D network for processing. To more effectively fuse cross-slice information, subsequent research has proposed attention-based models. For example, Hung et al. proposed a cross-slice attention module that captures global information by applying attention across semantic, positional, and slice dimensions. Similarly, CSA-Net proposed by Kumar et al. combines cross-slice attention for learning pixel-level correlations between slices with self-attention for learning pixel-level correlations within slices.

[0004] Existing semi-supervised 3D medical image segmentation techniques face two main challenges: insufficient efficiency in utilizing 3D contextual information and the effectiveness of introducing and transferring external knowledge sources. Specifically, in terms of 3D context modeling, to avoid the high computational cost of pure 3D networks, 2.5D or cross-slice attention methods proposed by researchers have become mainstream. However, most of these methods rely on modular or slicing operations to aggregate information from adjacent slices. This approach makes it difficult to establish accurate pixel-level correspondences, resulting in insufficient preservation of fine topological structures in anatomical structures and leading to discontinuities and artifacts at object boundaries in the segmentation results. Regarding the introduction of external knowledge sources, although using large-scale visual foundational models such as SAM as "expert teachers" can break the performance bottleneck of the "internal loop" in traditional semi-supervised learning, it also brings new technical challenges. Existing methods mainly transfer knowledge from SAM to student models through knowledge distillation, but there are two key limitations: First, in terms of the prompting mechanism, existing methods usually require points or boxes as prompts to guide SAM in generating pseudo-labels. This approach is extremely sensitive to the quality of prompts; incorrect prompts will severely reduce the reliability of pseudo-labels, thereby affecting the training effect of student models. Secondly, in the process of knowledge transfer, traditional knowledge distillation forces student networks to "passively" imitate the teacher's output in its entirety. This mechanism inevitably allows student networks to learn the noise and errors inherent in the teacher's predictions, especially when the teacher's model is uncertain about certain ambiguous or difficult areas, these errors will be amplified. Summary of the Invention

[0005] To address the problem of low segmentation accuracy caused by insufficient utilization of 3D contextual information and ineffective introduction and transfer of external knowledge sources in existing semi-supervised 3D medical image segmentation techniques, this invention provides a dual-teacher-based semi-supervised 3D medical image segmentation training method and system.

[0006] In a first aspect, the present invention provides a semi-supervised 3D medical image segmentation training method based on dual teachers, comprising: Step 1: Initialize one student network and two teacher networks; the student network includes an encoder and two decoders sharing the encoder, with a hierarchical hybrid attention mechanism module connected between the encoder and each decoder; one teacher network is a SAM2 teacher network, and the other teacher network is an exponential moving average (EMA) teacher network; the SAM2 teacher network includes a pre-trained SAM2 model and a class-aware cue embedding (CPE) module, and an adapter module is added to the encoder and decoder in the SAM2 model, wherein the input medical image slices are processed by the SAM2 model encoder to obtain image features, the image features are input to the CPE module to generate feature cues, and the feature cues are fed into the SAM2 model decoder to generate pseudo-labels; the EMA teacher network includes an encoder and a decoder, with a hierarchical hybrid attention mechanism module connected between the decoder and the encoder; Step 2: Input three consecutive labeled medical image slices simultaneously into the student network and the SAM2 teacher network, respectively, and calculate the supervised loss based on the outputs of the student network and the SAM2 teacher network; Step 3: Input three unlabeled consecutive medical image slices simultaneously into the student network and the two teacher networks respectively. Quantize the output of the student network to obtain its own uncertainty graph. Calculate the hybrid distillation loss based on the pseudo-labels generated by the SAM2 teacher network and the uncertainty graph to guide the student network to learn from the SAM2 teacher network. Calculate the consistency loss between the output of the student network and the pseudo-labels generated by the EMA teacher network. Use the weighted sum of the hybrid distillation loss and the consistency loss as the semi-supervised loss. Step 4: Take the weighted sum of the supervised loss and the semi-supervised loss as the total loss, and perform backpropagation based on the total loss to update the parameters of the student network and the parameters of the CPE module and adapter module in the SAM2 teacher network; Step 5: Update the parameters of the EMA teacher network using an exponential moving average method based on the updated student network parameters; Step 6: Repeat steps 2 through 5 until the training stop condition is met.

[0007] Furthermore, the hierarchical hybrid attention mechanism module includes multiple context-focus dual-path attention modules and a pixel-level deep semantic alignment module; wherein, each encoding layer in the encoder, except for the last layer, is connected to a context-focus dual-path attention module and a corresponding decoding layer in the decoder; the last encoding layer is connected to the corresponding decoding layer via the pixel-level deep semantic alignment module. The context-focus dual-path attention module includes a cascaded semantic attention module and a positional attention module; The semantic attention module comprises a pooling layer, a multilayer perceptron, a feature fusion layer, a concatenation layer, and a sigmoid activation layer connected in sequence. The pooling layer includes two parallel pooling modules, each comprising two parallel max-pooling and average-pooling layers. The two pooling modules share the multilayer perceptron. The feature fusion layer includes two parallel element-wise addition operations. Correspondingly, the processing of the semantic attention module includes: inputting multilayer slice stacked features from the SAM2 model encoder into one pooling module, inputting the center slice features from the SAM2 model encoder into the other pooling module, and passing the outputs of the two pooling modules through the multilayer perceptron. For the output of each pooling module, the feature fusion layer performs an element-wise addition operation on the two pooling results after passing through the multilayer perceptron. In the concatenation layer, the outputs after the two element-wise addition operations are concatenated. The concatenated result is processed by the sigmoid activation layer to obtain the output feature map. The positional attention module includes a pooling layer, a concatenation layer, a convolutional layer, and a sigmoid activation layer connected in sequence. Correspondingly, the processing of the positional attention module includes: inputting the multi-layer slice stacked features from the semantic attention module into one of the pooling modules, and inputting the center slice features from the semantic attention module into another pooling module; in the concatenation layer, concatenating the outputs of the two pooling modules; the concatenated result is then processed by the convolutional layer and the sigmoid activation layer to obtain the output feature map. The pixel-level deep semantic alignment module includes two parallel axial cross attentions and one planar self attention, as well as a concatenation layer and a 1×1 convolutional layer. Correspondingly, the input feature map is first processed in parallel using three attentions, then the outputs of the three attentions are fused through the concatenation layer, and then the fused result is passed through the 1×1 convolutional layer to finally obtain the output feature map.

[0008] Furthermore, the CPE module includes: an input end, a multi-scale feature interaction layer composed of multiple stacked bidirectional attention modules, and an output end; wherein, the processing of the input end includes: given a learnable initial query vector, the initial query vector first passes through a shared multilayer perceptron to extract a common semantic pattern shared by all categories; the output of the shared multilayer perceptron enters an N× category-specific adapter to obtain an optimized query vector carrying specific anatomical structure identity information, where N represents the number of categories; The multi-scale interaction layer process includes: multi-scale feature input from the SAM2 encoder is fed into multiple bidirectional attention modules for layer-by-layer interaction, so as to fuse the optimized query vector with image features; The output processing includes: the post-interaction features from the multi-scale interaction layer are passed through a shared multilayer perceptron again; The output of the shared multilayer perceptron is fed into the N× category calibration module to scale and offset the features to generate feature cues.

[0009] Furthermore, in step 2, the formula for calculating the supervision loss is: in, and The probability graph is the output of two decoders in the student network; This represents a label that contains labeled data; This indicates that the segmentation loss of the student network is calculated using Dice loss. This represents the independent optimization loss of the adapter module on labeled data in the SAM2 teacher network. ; This represents the output of the SAM2 teacher network.

[0010] Further, in step 3, the uncertainty graph of the student network itself is obtained based on the output quantization of the student network, specifically including: Where C is the total number of categories, and This is a probability graph of the outputs of two decoders in a student network. This is an uncertain graph.

[0011] Furthermore, the formula for calculating the mixed distillation loss is as follows: in, This represents the uncertainty-guided hybrid distillation loss of a single decoder i in the student network. ; Represents cross-entropy loss, Denotes KL divergence, and These represent the hard pseudo-labels and soft pseudo-labels output by the SAM2 teacher network, respectively. This represents the raw predicted value output by the student network decoder. This represents the total mixed distillation loss. This indicates element-wise multiplication.

[0012] Furthermore, the formula for calculating the consistency loss is as follows: in, and This is a probability graph of the outputs of two decoders in a student network. Hard pseudo-labels output by the EMA teacher network, This represents the cross-entropy loss.

[0013] Secondly, the present invention provides a semi-supervised 3D medical image segmentation training system based on dual teachers, comprising: A segmentation network construction module is used to initialize a student network and two teacher networks. The student network includes an encoder and two decoders sharing the encoder, with a hierarchical hybrid attention mechanism module connected between the encoder and each decoder. One teacher network is a SAM2 teacher network, and the other is an exponential moving average (EMA) teacher network. The SAM2 teacher network includes a pre-trained SAM2 model and a class-aware cue embedding (CPE) module, with adapter modules added to the encoder and decoder of the SAM2 model. Input medical image slices are processed by the SAM2 model encoder to obtain image features, which are then input to the CPE module to generate feature cues. These feature cues are then fed into the SAM2 model decoder to generate pseudo-labels. The EMA teacher network includes an encoder and a decoder, with a hierarchical hybrid attention mechanism module connected between the decoder and the encoder. The supervised loss calculation module is used to simultaneously input three consecutive labeled medical image slices into the student network and the SAM2 teacher network, respectively, and calculate the supervised loss based on the outputs of the student network and the SAM2 teacher network. A semi-supervised loss calculation module is used to simultaneously input three unlabeled consecutive medical image slices into the student network and two teacher networks, respectively; quantize the output of the student network to obtain its own uncertainty graph; calculate a hybrid distillation loss based on the pseudo-labels generated by the SAM2 teacher network and the uncertainty graph to guide the student network to learn from the SAM2 teacher network; calculate the consistency loss between the output of the student network and the pseudo-labels generated by the EMA teacher network; and use the weighted sum of the hybrid distillation loss and the consistency loss as the semi-supervised loss. The total loss calculation module is used to calculate the total loss by weighting the supervised loss and the semi-supervised loss. The network update module is used to perform backpropagation based on the total loss to update the parameters of the student network and the parameters of the CPE module and adapter module in the SAM2 teacher network; and to update the parameters of the EMA teacher network using an exponential moving average method based on the updated parameters of the student network.

[0014] Thirdly, the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method as described in the first aspect.

[0015] Fourthly, the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the method described in the first aspect.

[0016] The beneficial effects of this invention are as follows: The image segmentation network of this invention consists of a student network and two collaborative teacher networks. The student network and the EMA teacher network integrate a hierarchical hybrid attention mechanism module (where shallow attention focuses on feature enhancement and deep attention focuses on semantic alignment) to efficiently and precisely fuse cross-slice contextual information while maintaining the computational efficiency of the 2D network, thereby improving segmentation accuracy. A class-aware cueable embedding (CPE) module is also designed in the teacher network to automatically generate high-quality, category-specific cues for the SAM2 model, overcoming the limitation of the SAM2 model relying on manually designed or generic cues, thus significantly improving the reliability of pseudo-labels generated by the SAM2 teacher network. An uncertainty-guided hybrid distillation (UHD) strategy is proposed, enabling the student network to adaptively learn knowledge from the SAM2 teacher network based on its own predicted uncertainty (reinforcement learning in the confidence zone and fine-tuning learning in the uncertainty zone), effectively solving the reliability and efficiency problems of external knowledge sources in the introduction and transfer process, and realizing a paradigm shift from "passive indoctrination" to "active inquiry." Attached Figure Description

[0017] Figure 1 A schematic diagram of the image segmentation network architecture provided in an embodiment of the present invention; Figure 2 A flowchart of a semi-supervised 3D medical image segmentation training method based on dual teachers provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of the hierarchical hybrid attention mechanism module provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of the context-focus dual-path attention module provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the pixel-level depth semantic alignment module provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of a class-aware prompting embedding module provided in an embodiment of the present invention; Figure 7A structural diagram of a semi-supervised 3D medical image segmentation training system based on dual teachers provided in an embodiment of the present invention; Figure 8 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the embodiments of this invention will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0019] Semi-supervised learning (SSL) is a machine learning paradigm that uses a small amount of labeled data and a large amount of unlabeled data for training.

[0020] Combination Figure 1 and Figure 2 As shown, this embodiment of the invention provides a semi-supervised 3D medical image segmentation training method based on dual teachers, including the following steps: S101: Initialize a student network and two teacher networks.

[0021] Specifically, the student network includes an encoder and two decoders (referred to as decoder A and decoder B, respectively) that share the encoder. A hierarchical hybrid attention (HHA) module is connected between the encoder and each of the decoders. The HHA module is used to efficiently fuse cross-slice contextual information.

[0022] One teacher network is the SAM2 teacher network, and the other is the Exponential Moving Average (EMA) teacher network. The SAM2 teacher network includes a pre-trained SAM2 (Segment Anything Model 2) model and a Class-aware Promptable Embedding (CPE) module; adapter modules are added to the encoder and decoder of the SAM2 model. The EMA teacher network includes an encoder and a decoder, connected by a hierarchical hybrid attention mechanism module; the CPE module is used to automatically generate high-quality, class-specific cues for the base model.

[0023] In the SAM2 teacher network, the input medical image first passes through the image encoder of the SAM2 model to extract image features; the CPE module generates optimized feature prompts containing clear category identity information based on the image features encoded by the SAM2 model; the feature prompts generated by the CPE module are sent to the SAM2 decoder, and the decoder generates pseudo-labels.

[0024] Because the parameters of the EMA teacher network are updated by smoothing the parameters of the student model through exponential moving averages, without direct training via backpropagation, it is called an EMA teacher network. This parameter update method provides a more stable supervision signal for the student network.

[0025] S102: Input three consecutive labeled medical image slices simultaneously into the student network and the SAM2 teacher network, respectively, and calculate the supervised loss based on the outputs of the student network and the SAM2 teacher network.

[0026] Specifically, a labeled dataset is pre-built. Dataset Only a small amount of labeled data is needed. For labeled datasets... Each sample (x, y) in the dataset, where x represents a labeled data sample (three consecutive medical image slices) and y represents the label of the three consecutive medical image slices, is input into the student network. The student network performs forward propagation, passing through a shared encoder, an HHA module, and a dual decoder. Each decoder outputs a probability map. and .

[0027] Monitoring losses The calculation formula is as follows: (1) in, and The probability graph is the output of two decoders in the student network; This represents a label that contains labeled data; This indicates that the segmentation loss of the student network is calculated using Dice loss. This represents the independent optimization loss of the adapter module on labeled data in the SAM2 teacher network. ; This represents the output of the SAM2 teacher network.

[0028] S103: Three unlabeled consecutive medical image slices are simultaneously input into the student network and the two teacher networks, respectively. An uncertainty graph of the student network is obtained by quantizing its output. A hybrid distillation loss is calculated based on the pseudo-labels generated by the SAM2 teacher network and the uncertainty graph to guide the student network in learning from the SAM2 teacher network. A consistency loss is calculated between the output of the student network and the pseudo-labels generated by the EMA teacher network. The weighted sum of the hybrid distillation loss and the consistency loss is used as the semi-supervised loss. This step specifically includes the following sub-steps: S1031: Student Network Forward Propagation and Uncertainty Quantification.

[0029] Specifically, unlabeled datasets Each unlabeled 3D medical image sample Input the student network and obtain two probability graphs. and Subsequently, pixel-level L1 distances were calculated based on the two probabilistic graphs to quantify the uncertainty graph of the student network itself. As shown in formula (2): (2) Where C is the total number of categories.

[0030] S1032: SAM2 teacher network generates pseudo-labels.

[0031] Specifically, using the CPE module and the student network to predict and generate bounding box cues, the SAM2 teacher network processes unlabeled samples. Perform inference to generate high-quality hard pseudo-labels. and soft fake tags The so-called box cue refers to the bounding box information (usually a rectangle) generated by the student network that surrounds the detected target, that is, the location information of the detected target in the image. By inputting this box cue into the SAM2 teacher network, the SAM2 teacher network can be guided to focus on the region in the image corresponding to this location information.

[0032] S1033: Calculate the total mixed distillation loss .

[0033] Specifically, this loss is used to guide the student network in learning from the SAM2 teacher network, and is determined by the uncertainty graph. As dynamic weights, the model is composed of uncertainty-guided hybrid distillation (UHD) loss, which is derived from the uncertainty of the two decoders in the student network. This loss aims to guide the model to adaptively transfer knowledge based on the uncertainty of the student network itself. Specifically, in regions with high confidence in the student network, hard pseudo-labels are used for supervision to reinforce existing knowledge; in regions with high uncertainty, soft knowledge distillation is used to learn the complete probability distribution of the teacher model. This achieves a shift from "passive imitation" to "active inquiry," effectively avoiding the accumulation of erroneous information and making the knowledge transfer process more robust and efficient.

[0034] For the output of any decoder in the student network ( ), its UHD loss It consists of two parts: hard label supervision loss and soft label distillation loss, as shown in formula (3): (3) in, It is cross-entropy loss. It is the KL divergence. This represents the raw predicted value output by the student network decoder, i.e., the score before Softmax / Sigmoid normalization. This indicates element-wise multiplication.

[0035] Ultimately, the total mixed distillation loss is: (4) S1034: EMA teacher-generated pseudo-labels and consistency loss calculation.

[0036] Specifically, unlabeled data Input the EMA teacher network to generate stable hard pseudo-labels. Calculate the consistency loss between the average predictions of the student network and the pseudo-labels of the EMA teacher network. As shown in formula (5): (5) S1035: Calculate the semi-supervised loss .

[0037] Specifically, the loss is a weighted sum of the two teacher guidance signals, as shown in formula (6): (6) in, and It is a hyperparameter for balancing weights.

[0038] S104: Calculate the total loss and update the network; this includes the following sub-steps: S1041: Calculate the total loss for the current batch .

[0039] Specifically, total loss It is composed of a weighted average of supervised loss and semi-supervised loss, as shown in formula (7): (7) in, It is a ramp-up function that varies with the training progress t, used to smoothly introduce semi-supervised signals.

[0040] S1042: Backpropagation and parameter update.

[0041] Specifically, calculations are performed through backpropagation. The gradient is calculated, and the parameters of the student network are updated. The parameters of the adapter module and CPE module in the SAM2 teacher network.

[0042] S105: Update EMA teacher network parameters.

[0043] Specifically, in each student network parameter Updated parameters for the EMA teacher network The update is performed using the following exponential moving average method, as shown in formula (8): (8) in, It is the attenuation coefficient.

[0044] S106: Repeat steps S102 to S105 until the model training converges.

[0045] In one embodiment, such as Figure 3 As shown, the hierarchical hybrid attention mechanism module includes multiple context-focus dual-path attention modules and a pixel-level deep semantic alignment module; wherein, each encoding layer in the encoder except the last layer is connected to a context-focus dual-path attention module and the corresponding decoding layer in the decoder; the last encoding layer is connected to the corresponding decoding layer via the pixel-level deep semantic alignment module.

[0046] Specifically, different attention modules are deployed in the shallow and deep layers of the student network. A context-focus dual-path attention module is used for the shallow output of the encoder to efficiently capture the contextual information of adjacent slices, while simultaneously enhancing the feature representation of the central slice, such as... Figure 4 As shown; a pixel-level deep semantic alignment module is used at network bottlenecks to finely reconstruct the 3D topology, such as... Figure 5 As shown, this allows for the fine-grained fusion of contextual information while maintaining computational efficiency.

[0047] like Figure 4 As shown, the context-focus dual-path attention module includes a cascaded semantic attention module and a positional attention module; Figure 4 The input on the left represents a three-slice stacked feature, typically referring to [previous slice, current slice, next slice]. This design allows the model to be as efficient as a 2D network while also utilizing inter-layer information like a 3D network.

[0048] The semantic attention module comprises a pooling layer, a multilayer perceptron, a feature fusion layer, a concatenation layer, and a sigmoid activation layer connected in sequence. The pooling layer includes two parallel pooling modules, each comprising two parallel max-pooling and average-pooling layers. The two pooling modules share the multilayer perceptron. The feature fusion layer includes two parallel element-wise addition operations. Correspondingly, the processing of the semantic attention module includes: inputting multilayer slice stacked features from the SAM2 model encoder into one pooling module, inputting the center slice features from the SAM2 model encoder into the other pooling module, and passing the outputs of the two pooling modules through the multilayer perceptron. For the output of each pooling module, the feature fusion layer performs an element-wise addition operation on the two pooling results after passing through the multilayer perceptron. In the concatenation layer, the outputs after the two element-wise addition operations are concatenated. The concatenated result is processed by the sigmoid activation layer to obtain the output feature map. The positional attention module includes a pooling layer, a concatenation layer, a convolutional layer, and a sigmoid activation layer connected in sequence. Correspondingly, the processing of the positional attention module includes: inputting the multi-layer slice stacked features from the semantic attention module into one of the pooling modules, and inputting the center slice features from the semantic attention module into another pooling module; in the concatenation layer, concatenating the outputs of the two pooling modules; after the concatenation result is processed by the convolutional layer and the sigmoid activation layer, the output feature map is obtained.

[0049] Specifically, features from the SAM2 encoder first enter the semantic attention module for processing. After feature refinement (weighting) by the semantic attention module, the output is directly used as the input to the right-hand position attention module. This is a "channel (semantic) first, space (position) second" processing strategy. The model first identifies "what to look at" (i.e., which channel features are important), and then identifies "where to look" (i.e., which spatial location in the image is important).

[0050] In the semantic attention module, a dual-path parallel processing design is adopted to capture channel-dimensional dependencies. The input is divided into two paths: a multi-layer slice stacked feature path (also known as the Stack path) and a center slice feature path (also known as the Center path). Specifically: Pooling Layer: Both inputs are processed in parallel using max pooling and average pooling to compress spatial dimensional information. Multilayer Perceptron: Pooled features share the same multilayer perceptron for processing to extract non-linear feature relationships. Feature Fusion: For each path (Stack or Center), the results of Max Pooling and AvgPooling after MLP are added element-wise. Path Concatenation: The results of the Stack path and Center path are concatenated. Activation and Weighting: The concatenated features are processed using the Sigmoid activation function to generate a semantic attention weight map, which is then multiplied element-wise with the original input features to complete the semantic feature relabeling.

[0051] The location attention module aims to establish pixel-level spatial correspondences while retaining attention to context and center slices. Specifically: Pooling Layer: Features are processed using both max pooling and average pooling. Pooling is typically performed along the channel dimension to compress channel information and preserve spatial features. Concatenation: The results of max pooling and average pooling are concatenated. Convolutional Layer: The concatenated feature map is passed through a 1×1 convolutional layer. This step fuses information and reduces the number of channels. Activation and Weighting: A spatial attention weight map is generated using the sigmoid activation function, and finally, element-wise multiplication is performed with the input features to output a double-refined feature map.

[0052] like Figure 5 As shown, the pixel-level deep semantic alignment module includes two parallel axial cross attentions and a planar self-attention, as well as a concatenation layer and a 1×1 convolutional layer. Correspondingly, the input feature map is first processed in parallel using three attentions, then the outputs of the three attentions are fused through the concatenation layer, and then the fused result is passed through the 1×1 convolutional layer to finally obtain the output feature map.

[0053] Specifically, axial cross attention is an existing attention mechanism and will not be elaborated upon here. Planar self-attention applies self-attention computation to the two-dimensional spatial plane formed by the feature maps of a slice in the height and width dimensions; planar attention aims to capture long-range dependencies within a single slice. It is called "planar" because its attention scope is limited to the two-dimensional spatial plane of the slice and does not involve the depth dimension.

[0054] In one embodiment, such as Figure 6 As shown, the CPE module includes an input end, a multi-scale feature interaction layer composed of multiple stacked bidirectional attention modules, and an output end; wherein, the processing of the input end includes: Preprocessing: Given a learnable initial query vector, the initial query vector is first passed through a shared multilayer perceptron to extract common semantic patterns shared by all categories; Category adaptation: The output of the shared multilayer perceptron is fed into N×class-specific adapters to obtain an optimized query vector carrying specific anatomical structure identity information. N represents the number of categories, that is, each category has an independent adapter to transform the query vector into an optimized query vector carrying specific anatomical structure identity information. The processing of the multi-scale interaction layer includes: Lateral connectivity: Multi-scale features from the SAM2 encoder are input to multiple bidirectional attention modules for layer-by-layer interaction. The number of bidirectional attention modules corresponds to the number of multi-scale features output by the SAM2 encoder. As shown in the figure, the multi-scale feature interaction layer consists of three stacked bidirectional attention modules, and the image features extracted by the SAM2 encoder are fed into these three modules. In the figure, the arrows from the SAM2 encoder point to different heights, representing the layer-by-layer interaction between the CPE module and the multi-scale features (feature maps at different levels) of the SAM2 encoder. Through the bidirectional attention mechanism, the optimized query vector is fused with the image features to perceive specific target regions in the image.

[0055] The output processing includes: Post-processing: The post-interaction features from the multi-scale interaction layer are passed through a shared multilayer perceptron again; Calibration: The output of the shared multilayer perceptron is fed into the N×Class CalibrationModules to scale and offset the features to generate high-quality feature cues.

[0056] Ultimately, the feature cues generated by the CPE module and the box cues from the student network output together constitute the complete cues input to the SAM2 decoder.

[0057] Specifically, the CPE module can automatically generate high-quality, category-specific hints for the SAM2 model. It overcomes the limitations of the SAM2 model's reliance on manual or generic hints by injecting category knowledge into the query vector and fine-tuning the final embeddedness through a "two-sided" category calibration mechanism, significantly improving the quality and reliability of pseudo-labels.

[0058] This invention conducted extensive experiments on a series of publicly available and challenging 3D medical image segmentation datasets to comprehensively test the performance of the proposed method, particularly its effectiveness in semi-supervised learning scenarios with limited labeled data. The experiments covered three different types of medical segmentation tasks: ACDC heart segmentation, COVID-19 lesion segmentation, and PROMISE12 prostate segmentation. All experiments were conducted with a challengingly low proportion of labeled data. Specifically, on the ACDC heart segmentation and COVID-19 lesion segmentation datasets, training was tested using only 5% and 10% of labeled data, respectively; while on the PROMISE12 prostate segmentation dataset, training was tested using 10% and 20% of labeled data, respectively. The experimental results are shown in Tables 1, 2, and 3. On all tested datasets, and at different proportions of labeled data, the proposed method achieved the highest segmentation accuracy compared to several current mainstream and state-of-the-art baseline methods.

[0059] Table 1. Comparison with other semi-supervised medical image segmentation methods on the ACDC dataset. Table 2 Comparison with other semi-supervised medical image segmentation methods on the COVID19 dataset Table 3 Comparison with other semi-supervised medical image segmentation methods on the PROMISE12 dataset. In summary, whether in multi-class and structurally complex cardiac segmentation tasks or in lesion segmentation tasks with irregular shapes and blurred boundaries, this invention achieves state-of-the-art results in core evaluation metrics (such as Dice similarity coefficient and IoU), significantly outperforming state-of-the-art methods including SAMT-PCL and Corrmatch. These results strongly demonstrate that this invention effectively overcomes the challenges posed by data scarcity, exhibiting superior accuracy, robustness, and generalization ability across various medical applications.

[0060] Based on the same inventive concept, such as Figure 7 As shown, this embodiment of the invention provides a semi-supervised 3D medical image segmentation training system based on dual teachers, including a segmentation network construction module, a supervised loss calculation module, a semi-supervised loss calculation module, a total loss calculation module, and a network update module.

[0061] Specifically, the segmentation network construction module initializes a student network and two teacher networks. The student network includes an encoder and two decoders sharing the encoder, with a hierarchical hybrid attention mechanism module connected between the encoder and each decoder. One teacher network is a SAM2 teacher network, and the other is an exponential moving average (EMA) teacher network. The SAM2 teacher network includes a pre-trained SAM2 model and a class-aware cue embedding (CPE) module, with adapter modules added to the encoder and decoder of the SAM2 model. Input medical image slices are processed by the SAM2 model encoder to obtain image features, which are then input to the CPE module to generate feature cues. These feature cues are then fed into the SAM2 model decoder to generate pseudo-labels. The EMA teacher network includes an encoder and a decoder, with a hierarchical hybrid attention mechanism module connected between the decoder and the encoder. The supervised loss calculation module simultaneously inputs three consecutive labeled medical image slices into the student network and the SAM2 teacher network, respectively, and calculates the supervised loss based on the outputs of the student network and the SAM2 teacher network. The semi-supervised loss calculation module simultaneously inputs three unlabeled consecutive medical image slices into the student network and the two teacher networks, respectively. Based on the output of the student network, it quantizes to obtain the uncertainty graph of the student network itself. It calculates a hybrid distillation loss based on the pseudo-labels generated by the SAM2 teacher network and the uncertainty graph to guide the student network in learning from the SAM2 teacher network. It calculates the consistency loss between the output of the student network and the pseudo-labels generated by the EMA teacher network. The weighted sum of the hybrid distillation loss and the consistency loss is used as the semi-supervised loss. The total loss calculation module uses the weighted sum of the supervised loss and the semi-supervised loss as the total loss. The network update module performs backpropagation based on the total loss to update the parameters of the student network and the parameters of the CPE module and adapter module in the SAM2 teacher network. It also updates the parameters of the EMA teacher network using an exponential moving average method based on the updated parameters of the student network.

[0062] It should be noted that the semi-supervised 3D medical image segmentation training system based on dual teachers provided in this embodiment of the invention is for implementing the above method. Its specific functions can be referred to in the above method embodiments, and will not be repeated here.

[0063] The present invention provides a semi-supervised 3D medical image segmentation training system based on dual teachers, which effectively solves the problem of insufficient utilization of contextual information in 3D medical image segmentation. By efficiently fusing cross-slice information, it significantly improves the accuracy of the segmentation model and its ability to preserve fine topological structures. It also significantly improves the effectiveness and reliability of using the large-scale base model SAM2 in semi-supervised learning. By automatically generating high-quality guidance prompts, it effectively bridges the domain gap between general models and medical images. Furthermore, it proposes a more robust and efficient knowledge transfer paradigm, enabling student networks to learn dynamically from teacher models based on their own uncertainties, thereby making full use of external knowledge while avoiding the accumulation of erroneous information.

[0064] Figure 8 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 8As shown, the electronic device may include: a processor 801, a communication interface 802, a memory 803, and a communication bus 804, wherein the processor 801, the communication interface 802, and the memory 803 communicate with each other through the communication bus 804. Processor 801 can call logic instructions in memory 803 to execute a semi-supervised 3D medical image segmentation training method based on dual teachers. This method includes: initializing a student network and two teacher networks; the student network includes an encoder and two decoders sharing the encoder, with a hierarchical hybrid attention mechanism module connected between the encoder and each decoder; one teacher network is a SAM2 teacher network, and the other is an exponential moving average (EMA) teacher network; the SAM2 teacher network includes a pre-trained SAM2 model and a class-aware cue embedding (CPE) module, and an adapter module is added to the encoder and decoder in the SAM2 model. The input medical image slices are processed by the SAM2 model encoder to obtain image features, which are then input to the CPE module to generate feature cues, which are then fed into the SAM2 model decoder to generate pseudo-labels; the EMA teacher network includes an encoder and a decoder, with a hierarchical hybrid attention mechanism module connected between the decoder and the encoder; and three consecutive labeled medical image slices are simultaneously processed... The data is input to the student network and the SAM2 teacher network, and a supervised loss is calculated based on the outputs of the student network and the SAM2 teacher network. Three consecutive unlabeled medical image slices are simultaneously input to the student network and the two teacher networks, respectively. The uncertainty graph of the student network is obtained by quantizing the output of the student network. A hybrid distillation loss is calculated based on the pseudo-labels generated by the SAM2 teacher network and the uncertainty graph to guide the student network to learn from the SAM2 teacher network. A consistency loss is calculated between the output of the student network and the pseudo-labels generated by the EMA teacher network. The weighted sum of the hybrid distillation loss and the consistency loss is used as the semi-supervised loss. The weighted sum of the supervised loss and the semi-supervised loss is used as the total loss. Backpropagation is performed based on the total loss to update the parameters of the student network and the parameters of the CPE module and the adapter module in the SAM2 teacher network. The parameters of the EMA teacher network are updated using an exponential moving average method according to the updated parameters of the student network. The processing of labeled and unlabeled data is repeated until the training stopping condition is met.

[0065] Furthermore, when the logical instructions in the aforementioned memory 803 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0066] This invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer can execute a semi-supervised 3D medical image segmentation training method based on dual teachers provided in the above-described method embodiments.

[0067] This invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a dual-teacher-based semi-supervised 3D medical image segmentation training method provided in the above-described method embodiments.

[0068] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0069] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A dual-teacher based semi-supervised 3D medical image segmentation training method, characterized in that, The method comprises the following steps: Step 1: initializing a student network and two teacher networks; the student network comprises an encoder and two decoders sharing the encoder, and a hierarchical hybrid attention mechanism module is connected between the encoder and each decoder; one of the teacher networks is a SAM2 teacher network, and the other is an exponential moving average (EMA) teacher network; the SAM2 teacher network comprises a pre-trained SAM2 model and a class perception prompt embedding (CPE) module, and an adapter module is added in the encoder and the decoder of the SAM2 model, wherein an input medical image slice is encoded by the SAM2 model encoder to obtain image features, the image features are input into the CPE module to generate feature prompts, and the feature prompts are sent into the SAM2 model decoder to generate pseudo labels; the EMA teacher network comprises an encoder and a decoder, and a hierarchical hybrid attention mechanism module is connected between the decoder and the encoder; Step 2: simultaneously inputting three labeled consecutive medical image slices into the student network and the SAM2 teacher network respectively, and calculating a supervised loss based on the outputs of the student network and the SAM2 teacher network; Step 3: simultaneously inputting three unlabeled consecutive medical image slices into the student network and the two teacher networks respectively, quantifying the uncertainty map of the student network based on the output of the student network; calculating a hybrid distillation loss based on the pseudo labels generated by the SAM2 teacher network and the uncertainty map to guide the student network to learn from the SAM2 teacher network; calculating a consistency loss between the output of the student network and the pseudo labels generated by the EMA teacher network; and taking the weighted sum of the hybrid distillation loss and the consistency loss as a semi-supervised loss; Step 4: taking the weighted sum of the supervised loss and the semi-supervised loss as a total loss, and updating the parameters of the student network and the parameters of the CPE module and the adapter module in the SAM2 teacher network based on the total loss; Step 5: updating the parameters of the EMA teacher network in an exponential moving average manner according to the updated parameters of the student network; Step 6: repeating steps 2 to 5 until a training stopping condition is reached. 2.The dual-teacher based semi-supervised 3D medical image segmentation training method of claim 1, wherein, The hierarchical hybrid attention mechanism module comprises a plurality of context-focus dual-path attention modules and a pixel-level deep semantic alignment module; wherein a context-focus dual-path attention module is connected between each encoding layer in the encoder except the last layer and the corresponding decoding layer in the decoder; and the pixel-level deep semantic alignment module is connected between the last encoding layer and the corresponding decoding layer. The context-focus dual-path attention module comprises a semantic attention module and a position attention module connected in series. The semantic attention module comprises, which are connected in sequence, a pooling layer, a multi-layer perception, a feature fusion layer, a splicing layer and a Sigmoid activation layer; the pooling layer comprises two parallel pooling modules, each of which comprises two parallel maximum pooling layers and average pooling layers; the two pooling modules share the multi-layer perception; the feature fusion layer comprises two parallel element-wise addition operations; correspondingly, the processing process of the semantic attention module comprises: inputting multi-layer slice stacking features from a SAM2 model encoder into one of the pooling modules, inputting center slice features from the SAM2 model encoder into the other pooling module, and outputting the two pooling modules through the multi-layer perception; for the output of each pooling module, the two pooling results after the multi-layer perception are subjected to element-wise addition operation in the feature fusion layer; the outputs after the element-wise addition operation are spliced in the splicing layer; after the splicing result is processed through the Sigmoid activation layer, an output feature map is obtained; The position attention module comprises, which are connected in sequence, a pooling layer, a splicing layer, a convolution layer and a Sigmoid activation layer; correspondingly, the processing process of the position attention module comprises: inputting multi-layer slice stacking features from the semantic attention module into one of the pooling modules, inputting center slice features from the semantic attention module into the other pooling module; the outputs of the two pooling modules are spliced in the splicing layer; after the splicing result is processed through the convolution layer and the Sigmoid activation layer in sequence, an output feature map is obtained; The pixel-level depth semantic alignment module comprises two parallel axial cross-attention and one plane self-attention, and a splicing layer and a 1×1 convolution layer; correspondingly, the input feature map is processed in parallel by using the three attentions first, then the outputs of the three attentions are fused through the splicing layer, and finally the fused result is processed through the 1×1 convolution layer to obtain an output feature map. 3.The dual-teacher based semi-supervised 3D medical image segmentation training method of claim 1, wherein, The CPE module comprises: an input end, a multi-scale feature interaction layer composed of a plurality of stacked bidirectional attention modules, and an output end; wherein the processing process of the input end comprises: given a learnable initial query vector, the initial query vector is first passed through a shared multi-layer perception to extract general semantic patterns common to all categories; the output of the shared multi-layer perception enters an N×category-specific adapter to obtain an optimized query vector carrying specific anatomical structure identity information, N represents the number of categories; The processing process of the multi-scale interaction layer comprises: multi-scale features from the SAM2 encoder are input into the plurality of bidirectional attention modules for layer-by-layer interaction to fuse the optimized query vector with the image features; The processing process of the output end comprises: the interacted features from the multi-scale interaction layer are again passed through a shared multi-layer perception; The output of the shared multi-layer perception enters an N×category calibration module to scale and offset the features to generate feature hints. 4.The dual-teacher based semi-supervised 3D medical image segmentation training method of claim 1, wherein, In step 2, the calculation formula of the supervised loss is: wherein, and are probability maps outputted by two decoders in the student network; denotes a label of the labeled data; denotes a segmentation loss of the student network calculated by the Dice loss, denotes an independent optimization loss of the adapter module in the SAM2 teacher network on the labeled data, ; denotes an output of the SAM2 teacher network. 5.The dual-teacher based semi-supervised 3D medical image segmentation training method of claim 1, wherein, In step 3, the uncertainty map of the student network itself is quantified based on the output of the student network, specifically including: where C is the total number of classes, and is the probability map output by the two decoders in the student network, is the uncertainty map.

6. The semi-supervised 3D medical image segmentation training method based on double teachers according to claim 5, characterized in that, The calculation formula of the mixed distillation loss is: wherein, represents the uncertainty-guided hybrid distillation loss for a single decoder i in the student network, ; represents the cross-entropy loss, represents the KL divergence, and represent the hard and soft pseudo labels output by the SAM2 teacher network, respectively, represents the original predicted numerical values output by the student network decoders, represents the total hybrid distillation loss, represents the element-wise multiplication.

7. The dual-teacher based semi-supervised 3D medical image segmentation training method according to claim 1, wherein, The calculation formula of the consistency loss is: wherein, and is the probability map output by two decoders in the student network, is the hard pseudo-label output by the EMA teacher network, is the cross-entropy loss.

8. A dual-teacher based semi-supervised 3D medical image segmentation training system, characterized in that, The method comprises the following steps: The segmentation network construction module is configured to initialize a student network and two teacher networks; the student network comprises an encoder and two decoders sharing the encoder, and a hierarchical mixed attention mechanism module is connected between the encoder and each decoder; one of the teacher networks is a SAM2 teacher network, and the other is an exponential moving average (EMA) teacher network; the SAM2 teacher network comprises a pre-trained SAM2 model and a class perception prompt embedding (CPE) module, and an adapter module is added to the encoder and the decoder in the SAM2 model, wherein the input medical image slice is encoded by the SAM2 model encoder to obtain image features, the image features are input into the CPE module to generate feature prompts, and the feature prompts are sent into the SAM2 model decoder to generate pseudo labels; the EMA teacher network comprises an encoder and a decoder, and a hierarchical mixed attention mechanism module is connected between the decoder and the encoder; The supervised loss calculation module is configured to simultaneously input three labeled consecutive medical image slices into the student network and the SAM2 teacher network, and calculate a supervised loss based on the outputs of the student network and the SAM2 teacher network; The semi-supervised loss calculation module is configured to simultaneously input three unlabeled consecutive medical image slices into the student network and the two teacher networks, quantize the uncertainty map of the student network itself based on the output of the student network, calculate a mixed distillation loss based on the pseudo labels generated by the SAM2 teacher network and the uncertainty map to guide the student network to learn from the SAM2 teacher network, calculate a consistency loss between the output of the student network and the pseudo labels generated by the EMA teacher network, and take the weighted sum of the mixed distillation loss and the consistency loss as a semi-supervised loss; The total loss calculation module is configured to take the weighted sum of the supervised loss and the semi-supervised loss as a total loss. The network updating module is configured to perform back propagation based on the total loss to update the parameters of the student network and the parameters of the CPE module and the adapter module in the SAM2 teacher network, and update the parameters of the EMA teacher network in an exponential moving average manner according to the updated parameters of the student network.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the method of any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1 to 7.

Images