An efficient deep supervision distillation method and device applied to medical image segmentation

Abstract

This invention relates to the field of medical image segmentation technology, specifically to an efficient deep supervised distillation method and apparatus for medical image segmentation, comprising a pre-trained encoding module, an efficient multi-scale fusion block, a multi-scale grouping attention gate, an efficient frequency domain boundary extraction block, an efficient fusion upsampling block, and knowledge-aligned multi-scale distillation. This invention effectively captures and fuses multi-scale features while enhancing feature representation capabilities in skip connections. Furthermore, by utilizing frequency domain information to improve the model's boundary prediction capability and gradually aligning the multi-scale knowledge of the teacher and student models during the encoding-decoding stage, the lightweight student model maintains excellent segmentation performance while significantly reducing computational costs. Moreover, this invention possesses strong compatibility, adapting to multi-layer visual encoders of different scales and to the output of different encoder levels, effectively reducing computational and parameter requirements while ensuring high segmentation performance.

Description

Technical Field

[0001] This invention relates to the field of medical image segmentation technology, and in particular to an efficient depth-supervised distillation method and apparatus for medical image segmentation. Background Technology

[0002] Pre-trained visual encoders are core components of many visual tasks, such as classification and segmentation. Early encoders were primarily based on convolutional neural networks, such as AlexNet and VGG, which effectively extracted image features through convolutional layers. ResNet introduced residual connections to solve the vanishing gradient problem in deep networks, significantly improving the training efficiency and accuracy of the network. MobileNets and its improved version, MobileNetV2, employed depthwise separable convolutions, successfully reducing model parameters and computational cost. EfficientNet proposed a novel scaling method that balances network depth, width, and resolution, achieving even higher efficiency and performance. Although CNNs have achieved success in many visual tasks, their inherent local receptive fields make it difficult to effectively capture long-range dependencies in images, which limits their effectiveness in some complex tasks.

[0003] In recent years, the introduction of Vision Transformers (ViTs) has broadened the application of Transformers in visual tasks, effectively learning long-range dependencies of images through the Self-Attention (SA) mechanism. However, the high computational complexity and large data requirements of the original ViTs limit their direct application. Subsequent research has focused on two main improvements: combining CNNs and optimizing the SA mechanism. MaxViT introduces a multi-axis attention mechanism and combines the advantages of CNNs and Transformers to build a hybrid model. Swin Transformer uses a moving window mechanism to restrict Self-Attention to a local window. SegFormer effectively aggregates information from different layers by combining the Transformer with a lightweight multilayer perceptron (MLP) decoder. PVT applies the Transformer to dense prediction tasks by progressively shrinking the pyramid structure and spatially reducing the attention layers. PVTv2 further improves the attention mechanism, patch embedding method, and feedforward network, enhancing model efficiency and performance. Although ViTs have significant advantages in capturing long-range dependencies, they still have shortcomings in capturing local spatial relationships.

[0004] Medical image segmentation aims to perform accurate pixel-level classification of complex anatomical structures and lesion regions, and is a key task in medical image analysis. Early methods, such as FCN, achieved end-to-end segmentation, greatly promoting the application of deep learning in segmentation tasks. U-Net proposed a classic encoder-decoder architecture, using skip connections to combine features at different levels, becoming the foundational structure for medical image segmentation. Numerous improved methods based on the U-shaped architecture have emerged, such as UNet++, which introduced a design with dense skip connections to further develop this structure. UNet 3+ further improved the integration of multi-scale features through comprehensive skip connections. 3D-UNet extends the application of the U-shaped structure in 3D medical image segmentation.

[0005] With the development of Vision Transformers (ViTs), breakthroughs have been achieved in medical image segmentation. TransUnet combines CNNs with transformers, enhancing the model's ability to extract both local and global features. UNETR utilizes a pure Transformer as the encoder, effectively capturing global multi-scale information. Swin-Unet uses Swin Transformer blocks in both the encoder and decoder stages, constructing a pure transformer U-shaped segmentation network. These methods demonstrate the advantages of Transformers in modeling global information, but they still have shortcomings in capturing fine-grained local features.

[0006] Meanwhile, the application of frequency domain information has gradually gained attention, aiming to enhance the local and global context extraction capabilities of segmentation models. FCANet generalizes preprocessing using global evaluation pooling in the frequency domain, thereby realizing a frequency domain channel attention mechanism. FRUNet studies the combination of FCA and U-Net, and incorporates residual units to improve segmentation accuracy. MADGNet refines the spatial feature extraction process by combining multi-frequency and multi-scale features. However, these methods typically focus on improving segmentation accuracy but are insufficient in terms of computational efficiency, making it difficult to meet the needs of practical applications.

[0007] In view of this, this application aims to provide an efficient depth-supervised distillation method and apparatus for medical image segmentation, so as to better solve the above-mentioned technical problems. Summary of the Invention

[0008] To address the aforementioned issues, this invention provides an efficient deep-supervised distillation method and apparatus for medical image segmentation. This method can adapt to pre-trained encoders of different sizes, effectively refine input features and enhance the representation of important features. By combining deep supervision strategies and knowledge-aligned multi-scale distillation, the representational power of the model is further improved, thereby increasing the accuracy of medical image segmentation.

[0009] The technical solution adopted in this invention is:

[0010] An efficient depth-supervised distillation method for medical image segmentation includes the following steps:

[0011] First, a pre-trained encoder is used to obtain hierarchical multi-scale feature maps, and then the feature maps are passed to the decoder part through skip connections;

[0012] The decoder has a multi-layered structure, the number of which is determined by the number of layers in the pre-trained encoder. Each layer consists of an efficient multi-scale fusion block, a multi-scale grouped attention gate, and an efficient frequency domain boundary extraction block. Upsampling is performed from the lower to the higher layers using an efficient fusion upsampling block.

[0013] Finally, based on knowledge alignment, multi-scale distillation is performed on the student model using the soft labels output by the teacher model. Through a multi-level distillation mechanism, the multi-scale features and predictions of the teacher and student models are aligned layer by layer in the encoding-decoding stage, and the multi-scale features and predictions between the teacher and student models are aligned, thereby achieving high-performance medical image segmentation.

[0014] Furthermore, the efficient multi-scale fusion block takes as input the output feature map of the pre-trained encoder and the feature map enhanced and fused by the multi-scale grouping attention gate and the efficient fusion upsampling block. It uses parallel multi-scale deep convolution to extract and fuse contextual information at different scales, and improves the interaction between channels through channel shuffling. Finally, it uses another deep convolution and batch normalization to normalize the feature distribution, so as to achieve feature extraction and feature fusion and enhance the expression of important features.

[0015] Furthermore, the specific implementation steps of the efficient multi-scale fusion block are as follows:

[0016] First, a depthwise convolution with a kernel size of 1×1 is applied to the input feature map. This operation is followed by a batch normalization layer and the ReLU6 activation function to improve non-linear expressive power while maintaining lightweightness.

[0017] Next, we use deep convolutions with kernel sizes of {1,3,5,7} to perform multi-scale context capture and fusion.

[0018] Then, the feature channel relationships are shuffled and recombined through channel shuffling. Subsequently, another depthwise convolution with a kernel size of 1×1 and a batch normalization layer are used to normalize the feature distribution. Finally, skip connections are used to enhance the feature map and the input feature map to obtain the final output.

[0019] Furthermore, the multi-scale grouping attention gate effectively fuses and refines the upsampled fused features with the skip connection features through the multi-scale grouping attention mechanism, strengthens the expression of relevant features, and suppresses irrelevant or redundant features, thereby comprehensively improving the diversity and robustness of feature extraction.

[0020] Furthermore, the specific implementation steps of the multi-scale grouping attention gate are as follows:

[0021] First, the input features are processed by multi-scale grouped convolution with kernel size {1,3,5,7}. After each convolution operation, batch normalization is performed to stabilize the training and standardize the feature distribution. These processed features are then fused through element-wise multiplication to form a unified multi-scale feature representation.

[0022] Next, a 1×1 kernel convolution and batch normalization are used to further compress the feature dimension to obtain a single-channel feature map, which is then used to generate attention coefficients through the Sigmoid activation function.

[0023] Finally, the attention coefficients are multiplied element-wise with the input feature map to obtain the multi-scale gated attention feature output.

[0024] Furthermore, the efficient frequency domain boundary extraction block takes the output of the efficient multi-scale fusion block as its input. Through adaptive average pooling downsampling, the input feature map is first compressed to a specific resolution, and then multiplied with a set of high-frequency filters to extract the high-frequency information of the feature map. These high-frequency feature maps are then restored to their original resolution and fused with other features. Afterward, channel attention blocks and spatial attention blocks are used to enhance the attention of the obtained feature maps at the channel and spatial levels, respectively. Finally, a convolution is performed to obtain the final boundary prediction result. By combining high-frequency information with spatial and channel attention mechanisms, it outputs an accurate boundary prediction map and performs deep supervision in conjunction with the feature map in the decoding stage to prevent the model from losing key information during the upsampling process.

[0025] Furthermore, the specific implementation steps of the high-efficiency frequency domain boundary extraction block are as follows:

[0026] A frequency selection strategy is used to extract high-frequency components. For a given number of channels, a set of high-frequency indices is selected for each frequency index, and a two-dimensional DCT filter is constructed for each frequency index. These filters are then multiplied with the input feature map to extract high-frequency information from the input feature map.

[0027] The obtained high-frequency features are further enhanced by applying deep convolution, batch normalization, and ReLU6, and then enhanced by channel attention blocks and spatial attention blocks. Finally, a boundary prediction map is output through a convolution with a kernel size of 1×1.

[0028] The channel attention block first performs adaptive average pooling and adaptive max pooling operations on the input feature map to extract important information from the feature map. Then, for the two pooled feature maps, pointwise convolution is used to reduce the number of channels to 1 / 16 of the original number, and the ReLU activation function is applied for non-linear transformation. Next, another pointwise convolution is used to restore the original number of channels, and the final attention weights are obtained through the Sigmoid activation function.

[0029] The spatial attention block first gathers the maximum and average values ​​of the input feature map along the channel dimension and concatenates them. Next, a 7×7 kernel convolution is used to enhance the local contextual relationships between features. Finally, attention weights are generated through the Sigmoid activation function.

[0030] Furthermore, the efficient fusion upsampling block uses the features of fused skip connections as input to the multi-scale group attention gate, and further combines the output of the multi-scale group attention gate to provide an enhanced feature flow to the efficient multi-scale fusion block;

[0031] The specific implementation steps are as follows:

[0032] First, the feature map was upsampled using a scaling factor of 2. Then, the feature map was divided into two branches and subjected to depthwise convolution with a kernel size of 3×3 and batch normalization. The amplified feature map was then enhanced using the ReLU6 activation function. Finally, the feature map was fused by element-wise multiplication and channel shuffling, and then a convolution with a kernel size of 1×1 was used to reduce the number of channels to adapt to the next stage.

[0033] Furthermore, during the training and optimization of efficient deep supervised distillation, mean squared error loss function, standard knowledge distillation loss function, binary cross-entropy loss function, and Dice coefficient loss function are used to optimize segmentation accuracy.

[0034] Based on the same inventive concept, this application also provides a high-efficiency depth-supervised distillation apparatus for medical image segmentation, comprising:

[0035] The pre-trained encoder module is used to obtain hierarchical multi-scale feature maps through the pre-trained encoder, and then pass the feature maps to the decoder part through skip connections;

[0036] A high-efficiency multi-scale fusion block is used to extract and fuse features during the decoding process, enhancing the representation of important features;

[0037] The multi-scale grouping attention gate effectively integrates refined features and skip connection features through a multi-scale grouping attention mechanism, thereby comprehensively improving the diversity and robustness of feature extraction.

[0038] The efficient frequency domain boundary extraction block combines high-frequency information with spatial and channel attention mechanisms to output accurate boundary prediction maps. It also performs deep supervision by combining feature maps from the decoding stage to prevent the model from losing key information during the upsampling process.

[0039] The efficient fusion upsampling block integrates the features of skip connections as input to the multi-scale grouping attention gate, and further combines the output of the multi-scale grouping attention gate to provide an enhanced feature flow to the efficient multi-scale fusion block;

[0040] Knowledge-aligned multi-scale distillation can align multi-scale features and predictions between teacher and student models, significantly improving model segmentation performance.

[0041] The beneficial effects of this invention are as follows:

[0042] This invention provides a highly efficient deep-supervised distillation device for medical image segmentation. It combines an efficient multi-scale fusion block and a multi-scale grouped attention gate, effectively capturing and fusing multi-scale features while enhancing feature representation in skip connections. Furthermore, an efficient frequency domain boundary extraction block, combined with deep supervision, improves the model's boundary prediction ability by utilizing frequency domain information. Based on knowledge alignment, multi-scale distillation progressively aligns the multi-scale knowledge of the teacher and student models during the encoding-decoding stage, enabling the lightweight student model to maintain excellent segmentation performance while significantly reducing computational costs. This invention possesses strong compatibility, adapting to multi-layer visual encoders of different scales (e.g., ResNet18, ResNet34, PVTv2-B0, PVTv2-B2, etc.) and adapting to the output of different encoder levels, effectively reducing computational and parameter requirements while ensuring high segmentation performance. Attached Figure Description

[0043] Figure 1 This is a structural diagram of the efficient depth-monitored distillation framework in an embodiment of the present invention.

[0044] in, Figure 1 (d) is a schematic diagram of the structure of a high-efficiency multi-scale fusion block.

[0045] in, Figure 1 (c) is a schematic diagram of the structure of a multi-scale grouped attention gate.

[0046] in, Figure 1 (e) is a schematic diagram of the structure of the high-efficiency frequency domain boundary extraction block.

[0047] in, Figure 1 (f) is a schematic diagram of the structure of the high-efficiency fusion upsampling block;

[0048] Figure 2 for Figure 1 A magnified view of a portion of region (d);

[0049] Figure 3 for Figure 1 A magnified view of a portion of region (c);

[0050] Figure 4 for Figure 1 A magnified view of a portion of region (e);

[0051] Figure 5 for Figure 1 A magnified view of a portion of region (f);

[0052] Figure 6 This is a schematic diagram of knowledge-aligned multi-scale distillation in an embodiment of the present invention.

[0053] Figure 7 This is a schematic diagram comparing the segmentation results in an embodiment of the present invention. Detailed Implementation

[0054] To facilitate understanding of the present invention, it will be described more fully below through embodiments, and preferred embodiments of the present invention are given below. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. Any other implementation schemes obtained by modifying or equivalently substituting the technical solutions of the present invention without inventive step are all within the protection scope of the present invention.

[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0056] The numerical values ​​disclosed in the embodiments of this invention are approximate values, not definitive values. Where error or experimental conditions permit, all values ​​within the error range may be included, and the specific numerical values ​​disclosed in the embodiments of this invention are not limited to those specified.

[0057] like Figures 1-6 As shown, this embodiment provides a high-efficiency depth-supervised distillation device for medical image segmentation, comprising:

[0058] The encoder part of the efficient deep supervised distillation framework uses a pre-trained encoder to obtain hierarchical multi-scale feature maps, which are then passed to the decoder part of the efficient deep supervised distillation framework through skip connections.

[0059] The decoder part of the high-efficiency deep supervised distillation framework has a multi-level structure. The number of its layers is determined by the number of layers of the pre-trained encoder. Each layer consists of a high-efficiency multi-scale fusion block, a multi-scale grouped attention gate, and a high-efficiency frequency domain boundary extraction block. Upsampling is performed from the lower to the higher layers using a high-efficiency fusion upsampling block.

[0060] Among them, the high-efficiency multi-scale fusion block is used to enhance the features of the encoder output;

[0061] Specifically, the core of the efficient multi-scale fusion block lies in using parallel multi-scale deep convolutions to extract and fuse contextual information at different scales, and improving the interaction between channels through channel shuffling operations, thereby further enhancing features;

[0062] The specific implementation steps are as follows:

[0063] Feature activation: First, a depthwise convolution with a kernel size of 1×1 is applied to the input feature map. This is followed by a batch normalization layer and the ReLU6 activation function to improve non-linear expressiveness while maintaining lightweight operation, as shown in the following equation:

[0064] F act =R6(BN(DWC) 1×1 (F in )))

[0065] Multi-scale convolutional fusion: Next, depthwise convolutions with kernel sizes of {1, 3, 5, 7} are used for multi-scale context capture and fusion. The specific fusion method is element-wise dot product, as shown in the following formula:

[0066]

[0067] Channel shuffling and restoration: Since depthwise convolution ignores the relationships between channels, a channel shuffling operation is used to shuffle and recombine the feature channel relationships. Then, another depthwise convolution with a kernel size of 1×1 and a batch normalization layer are used to normalize the feature distribution. Finally, a skip connection is used to enhance the feature map and the input feature map to obtain the final output. Here, g is the greatest common divisor (gcd) of the number of input and output channels. The residual connection F_in helps improve regularization performance, and the specific process is shown in the following equation:

[0068] F out =BN(DWC) 1×1 (CS(F fusion ,g)))+F in ,g=gcd(C in C out )

[0069] Among them, the multi-scale grouped attention gate uses a gated attention mechanism to refine the feature map after upsampling and skip connection fusion;

[0070] Specifically, the multi-scale grouping attention gate generates gating signals in a multi-scale manner, thereby effectively controlling the information flow of the network and improving segmentation performance.

[0071] The specific implementation steps are as follows:

[0072] First, the input features are processed using multi-scale grouped convolutions with kernel sizes {1, 3, 5, 7}. Each convolution operation is followed by batch normalization to stabilize training and standardize the feature distribution. These processed features are then fused through element-wise multiplication to form a unified multi-scale feature representation, as shown in the following equation:

[0073]

[0074] Next, a 1×1 convolution and batch normalization are used to further compress the feature dimension, resulting in a single-channel feature map. Then, attention coefficients are generated using the Sigmoid activation function. Finally, the attention coefficients are multiplied element-wise with the input feature map to obtain the multi-scale gated attention feature output, as shown in the following equation:

[0075] MSGAG(F in ) = F in ⊙σ(BN(Conv 1×1 (F fusion )))

[0076] Among them, the high-efficiency frequency domain boundary extraction block aims to enhance the model's ability to perceive target edges and details through frequency domain information;

[0077] Specifically, the efficient frequency domain boundary extraction block compresses the input feature map to a specific resolution through adaptive average pooling downsampling. This downsampled feature map is then multiplied by a set of high-frequency filters to extract high-frequency information. Next, these high-frequency features are upsampled to the original resolution and fused with other features. The high-frequency features are then subjected to a 3×3 depthwise convolution and batch normalization, and further enhanced using the ReLU6 activation function. To further optimize feature representation, the efficient frequency domain boundary extraction block utilizes channel attention blocks and spatial attention blocks to enhance the obtained feature map at the channel and spatial levels, respectively. To ensure the stability of the training process, the feature map is finally added to the input feature map via a residual connection and then subjected to a 1×1 convolution to obtain the final boundary prediction result.

[0078] The specific implementation steps are as follows:

[0079] First, a frequency selection strategy is employed to extract high-frequency components. For a given number of channels, a set of high-frequency indices is selected, and for each frequency index, a two-dimensional DCT filter is constructed. By multiplying these filters with the input feature map, high-frequency information can be extracted from the input feature map. The specific process is shown in the following equation:

[0080]

[0081] F (i) =AAP(F in ,(d h ,d w ))⊙W (i)

[0082]

[0083] Then, the obtained high-frequency features are further enhanced by applying depthwise convolution, batch normalization, and ReLU6, and further enhanced by channel attention blocks and spatial attention blocks. Finally, a boundary prediction map is output through a convolution with a kernel size of 1×1. The specific calculation process is shown below:

[0084] F enhanced =R6(BN(DWC) 1×1 (F HF )))

[0085] F CA =F enhanced ⊙CAB(F enhanced )

[0086] F SA =F enhanced ⊙SAB(F enhanced )

[0087] Output = Conv 1×1 (F CA ⊙F SA +F in )

[0088] Channel Attention Block: First, adaptive average pooling and adaptive max pooling are performed on the input feature map to extract important information. Then, for the two pooled feature maps, pointwise convolution is used to reduce the number of channels to 1 / 16 of the original, and the ReLU activation function is applied for non-linear transformation. Next, another pointwise convolution is used to restore the original number of channels, and the final attention weights are obtained through the Sigmoid activation function. The specific calculation process is as follows:

[0089] CAB(F in )=σ(C2(R(C1(AMP(F in ))))+C2(R(C1(AAP(F in )))))

[0090] Spatial attention block: First, the maximum and average values ​​of the input feature map are pooled along the channel dimension and concatenated. Next, a 7×7 convolution is used to enhance the local contextual relationships between features. Finally, attention weights are generated using the Sigmoid activation function, the specific calculation process of which is shown below:

[0091] SAB(x)=σ(Conv 7×7 ([Ch max (F in ),Ch avg (F in )]))

[0092] Among them, the efficient fusion upsampling block fuses and enhances the feature map while upsampling, and passes the processing result to the input and output of the multi-scale grouping attention gate;

[0093] Specifically, the features of the efficient fusion upsampling block fused with skip connections are used as input to the multi-scale grouped attention gate, and the output of the multi-scale grouped attention gate is further combined to provide an enhanced feature flow to the efficient frequency domain boundary extraction block.

[0094] The specific implementation steps are as follows:

[0095] First, the feature map was upsampled using a scaling factor of 2. Then, a 3×3 depthwise convolution with batch normalization was applied to the feature map in two branches, and the amplified feature map was enhanced using the ReLU6 activation function. Finally, element-wise multiplication and fusion were performed, followed by channel shuffling, and then a 1×1 convolution was used to reduce the number of channels to accommodate the next stage. The specific calculation process is shown below:

[0096] F k =BN(R6(DWC(Up) ×2 (F in ),k)))

[0097] F output =PWC(CS(F1⊙F2))

[0098] Knowledge-aligned multi-scale distillation significantly enhances the performance and robustness of student models by aligning multi-scale features of teacher and student models during the encoding-decoding phase.

[0099] Specifically, knowledge-aligned multi-scale distillation improves the robustness of the student model by utilizing multi-scale intermediate layer features during the encoding stage, and further aligns the knowledge between the teacher and student models by using the multi-scale output of the teacher model as soft labels through deep supervision during the decoding stage. Finally, the final distillation loss function is computed by jointly using the ground truth mask and boundary information.

[0100] Specifically, in the training and optimization process of the above distillation framework, the mean squared error loss function, the standard knowledge distillation loss function, the binary cross-entropy loss function, and the Dice coefficient loss function are used. This process can be expressed by the following formula:

[0101]

[0102] L total =L inter +T 2 L soft +L m +L b

[0103] Where w1 = w2 = w3 = w4 = 0.25; F s and F t These are the encoder intermediate layer features of the student model and the teacher model, respectively; L kd It uses the standard knowledge distillation loss function, while employing temperature T (T=4) to smooth the prediction distribution of the teacher model; m s and m GT These are the student model output mask and the ground truth mask, respectively, while b s and b GT These are the student model's predicted boundary and the ground truth boundary, respectively. The loss function without knowledge-aligned multi-scale distillation can be described as L' total =L m +L b .

[0104] In the experiments, the framework used the ISIC2018 dataset, CVC-ClinicDB dataset, Kvasir-seg dataset, and MSD dataset. The Dice similarity coefficient (DSC), Intersection over Union (IoU), and 95% Hausdorff Distance (HD95) were used as evaluation metrics.

[0105] See the table below for details. The bottom three rows of the table, from top to bottom, are: an efficient deep supervised distillation framework using ImageNet pre-trained PVTv2-b0 as the encoder, an efficient deep supervised distillation framework using ImageNet pre-trained PVTv2-b0 as the encoder (with knowledge-aligned multi-scale distillation), and an efficient deep supervised distillation framework using ImageNet pre-trained PVTv2-b2 as the encoder.

[0106]

[0107] Based on the above experimental results, it can be seen that the efficient deep supervised distillation framework provided in this embodiment significantly outperforms other models on four public medical image segmentation datasets, as shown in the table above. The three specifications of the efficient deep supervised distillation framework achieved average DSC of 81.10%, 83.89%, and 84.65% on the four datasets, demonstrating excellent segmentation performance.

[0108] The segmentation results in this embodiment are visualized as follows: Figure 7 As shown (the methods proposed in this embodiment are labeled 18, 19, and 20). The traditional U-Net (2) relies solely on skip connections and a progressive encoder-decoder structure, resulting in unstable predictions and oversegmentation and undersegmentation issues on the ISIC2018, Kvasir-SEG, and MSD datasets. Similarly, Attention U-Net (4) and U-Net++ (5), despite introducing some structural improvements, still suffer from similar inaccurate segmentation problems. In contrast, CE-Net (8) performs more accurately in boundary prediction through multi-scale information fusion, but it is easily affected by noise in dermoscopy data, leading to oversegmentation. For lightweight segmentation models (such as BiSeNetV2 (9), CMUneXt (12), Segformer (16), and UneXt (17)), although computationally efficient, the prediction results are unreliable on difficult samples in dermoscopy and colonoscopy data, and obvious missegmentation regions are prone to appear. On the other hand, EMCAD (14) utilizes a multi-scale convolutional attention mechanism, while MADGNet (15) combines multi-scale and multi-frequency domain information, achieving certain advantages in boundary prediction. However, when dealing with complex samples from dermatoscopy and colonoscopy, it is still impossible to completely avoid the generation of erroneous segmentation regions. In contrast, this invention, through joint optimization of multi-scale information fusion, feature enhancement, frequency domain information-enhanced boundary extraction, and deep supervision, performs well in various complex scenarios. In particular, ESDDF-T (20) can depict the boundary closest to the real label. Furthermore, with the help of knowledge-aligned multi-scale distillation, ESDDF-S effectively reduces oversegmentation (as shown in the comparison results of (18) and (19) in the CVC-ClinicDB dataset), further verifying the role of knowledge-aligned multi-scale distillation in improving segmentation stability and accuracy.

[0109] In summary, the efficient deep-supervised distillation device for medical image segmentation provided by this invention combines an efficient multi-scale fusion block and a multi-scale grouped attention gate, which can effectively capture and fuse multi-scale features while enhancing the feature representation ability in skip connections. Furthermore, the efficient frequency domain boundary extraction block, combined with deep supervision, improves the model's boundary prediction ability by utilizing frequency domain information. Based on knowledge alignment, multi-scale distillation gradually aligns the multi-scale knowledge of the teacher model and the student model during the encoding-decoding stage, enabling the lightweight student model to maintain excellent segmentation performance while significantly reducing computational costs. This invention possesses strong compatibility, adapting to multi-layer visual encoders of different scales (e.g., ResNet18, ResNet34, PVTv2-B0, PVTv2-B2, etc.) and adapting to the output of different levels of the encoder, effectively reducing computational and parameter requirements while ensuring high segmentation performance.

[0110] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. An efficient deep supervision distillation method applied to medical image segmentation, characterized in that, The method comprises the steps of: First, a pre-trained encoder is used to obtain a hierarchical multi-scale feature map, and then the feature map is transmitted to the decoder part through a jump connection; The decoder part has a multi-level structure, the number of layers of which is determined by the number of layers of the pre-trained encoder, and each layer of the structure is composed of an efficient multi-scale fusion block, a multi-scale grouping attention gate, and an efficient frequency domain boundary extraction block, and from the low layer to the high layer, an efficient fusion upsampling block is used for upsampling; Finally, based on the knowledge alignment multi-scale distillation, the soft labels output by the teacher model are used to distill the student model, and through a multi-level distillation mechanism, the multi-scale features and predictions of the teacher and student models are aligned layer by layer in the encoding-decoding stage, the multi-scale features and predictions of the teacher and student models are aligned, and high-performance medical image segmentation is realized; The input of the efficient multi-scale fusion block is the output feature map of the pre-trained encoder and the feature map enhanced and fused through the multi-scale grouping attention gate and the efficient fusion upsampling block, the parallel multi-scale deep convolution extraction and fusion of different scale context information are used, the channel interaction is improved through the channel shuffle operation, and finally another deep convolution and batch normalization are used to standardize the feature distribution, so as to realize feature extraction and fusion of features and enhance important feature expression; The multi-scale grouping attention gate effectively fuses the features after the upsampling fusion and the jump connection features through the multi-scale grouping attention mechanism, strengthens the expression of related features, and at the same time suppresses irrelevant or redundant features, so as to comprehensively improve the diversity and robustness of feature extraction; The input of the efficient frequency domain boundary extraction block is the output of the efficient multi-scale fusion block, the input feature map is compressed to a specific resolution through adaptive average pooling downsampling, then it is multiplied by a group of high-frequency filters to extract the high-frequency information of the feature map, then the high-frequency feature map is restored to the original resolution and fused with other features, then the channel attention block and the spatial attention block are used to enhance the channel and spatial attention of the obtained feature map, and finally a convolution is performed to obtain the final boundary prediction result, which combines high-frequency information with spatial and channel attention mechanisms to output an accurate boundary prediction map, and performs deep supervision on the feature map in the decoding stage to prevent the loss of key information in the upsampling process of the model; The efficient fusion upsampling block fuses the features of the jump connection as the input of the multi-scale grouping attention gate, and further combines the output of the multi-scale grouping attention gate to provide the enhanced feature stream to the efficient multi-scale fusion block; The specific implementation steps are as follows: First, the feature map is upsampled by a scale factor of 2, then a depth convolution with a kernel size of 3*3 and batch normalization are applied to the feature map in two branches, and the amplified feature map is enhanced by using the activation function ReLU6, finally, the element-wise multiplication fusion and channel shuffle operation are performed, and then a convolution with a kernel size of 1*1 is used to reduce the number of channels to adapt to the next stage.

2. The efficient deep supervision distillation method applied to medical image segmentation according to claim 1, characterized in that, The specific implementation steps of the efficient multi-scale fusion block are as follows: First, a depth convolution with a kernel size of 1x1 is used on the input feature map, which is followed by a batch normalization layer and a ReLU6 activation function to maintain lightweight while improving nonlinear expression capability; Next, a depth convolution with a kernel size of {1, 3, 5, 7} is used for multi-scale context capture and fusion; Then, the feature channel relationship is disrupted and recombined through a channel shuffle operation, followed by another depth convolution with a kernel size of 1x1 and a batch normalization layer to regulate the feature distribution, and finally the enhanced feature map is connected with the input feature map through a skip connection to obtain the final output.

3. The efficient deep supervision distillation method applied to medical image segmentation of claim 1, wherein, The specific implementation steps of the multi-scale grouping attention gate are as follows: First, the input features are processed using multi-scale grouping convolution with kernel sizes of {1, 3, 5, 7}, and batch normalization is applied after each convolution operation to stabilize training and regulate feature distribution. These processed features are fused through element-wise multiplication to form a unified multi-scale feature representation; Next, a kernel size of 1x1 is used for convolution and batch normalization to further compress the feature dimension, resulting in a single-channel feature map, followed by a Sigmoid activation function to generate attention coefficients; Finally, the attention coefficients are multiplied element-wise with the input feature map to obtain the multi-scale gated attention feature output.

4. The efficient deep supervision distillation method for medical image segmentation of claim 1, wherein, The specific implementation steps of the efficient frequency domain boundary extraction block are as follows: A frequency selection strategy is used to extract high-frequency components. For a given number of channels, a set of high-frequency indices is selected for each frequency index, and a two-dimensional DCT filter is constructed for each frequency index. By multiplying these filters with the input feature map, high-frequency information is extracted from the input feature map; The obtained high-frequency features are further enhanced using depth convolution and batch normalization and ReLU6, and are enhanced through a channel attention block and a spatial attention block, and finally output a boundary prediction map through a kernel size of 1x1 convolution; The channel attention block first performs adaptive average pooling and adaptive max pooling on the input feature map to extract important information from the feature map. Then, for the two pooled feature maps, a point-wise convolution is used to reduce the channel number to 1 / 16 of the original, and a ReLU activation function is applied for nonlinear transformation. Next, another point-wise convolution is used to restore the original channel number, and a Sigmoid activation function is used to obtain the final attention weight. The spatial attention block first collects the maximum and average values along the channel dimension of the input feature map and concatenates them. Next, a 7x7 kernel size convolution is used to enhance the local context relationship between features. Finally, a Sigmoid activation function is used to generate the attention weight.

5. The efficient deep supervision distillation method for medical image segmentation of claim 1, wherein, In the training and optimization process of efficient deep supervision distillation, the mean square error loss function, the standard knowledge distillation loss function, the binary cross entropy loss function, and the Dice coefficient loss function are used to optimize the segmentation accuracy.

6. Distillation apparatus for use in the distillation process according to any one of claims 1 to 5, characterized in that It includes: A pre-training encoding module is used to obtain hierarchical multi-scale feature maps through a pre-training encoder, and then the feature maps are passed to the decoder part through a skip connection; Efficient multi-scale fusion block is used to extract and fuse features in the decoding process, and enhance the expression of important features; Multi-scale grouping attention gate effectively fuses refined features and skip connection features through multi-scale grouping attention mechanism, thereby comprehensively improving the diversity and robustness of feature extraction; Efficient frequency domain boundary extraction block combines high-frequency information with spatial and channel attention mechanisms to output accurate boundary prediction maps and jointly supervise the feature maps in the decoding stage to prevent key information loss during the upsampling process; Efficient fusion upsampling block fuses the features of the skip connection as the input of the multi-scale grouping attention gate and further combines the output of the multi-scale grouping attention gate to provide enhanced feature flow to the efficient multi-scale fusion block; Knowledge alignment multi-scale distillation can align multi-scale features and predictions between teacher and student models, significantly improving model segmentation performance.

Images