Image segmentation method based on cross-scale interactive fusion and uncertainty perception dynamic distillation
Through the image segmentation method of cross-scale interactive fusion and uncertain perception dynamic distillation, the computational complexity and boundary blur problems of existing medical image segmentation methods are solved, and efficient and accurate medical image segmentation is achieved.
Patent Information
- Application Number
- CN202510555372.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-29
- Publication Date
- 2025-08-12
AI Technical Summary
Existing medical image segmentation methods rely on huge computational complexity and massive storage, are impractical, and are prone to problems of area disappearance and blurred boundaries when dealing with medical imaging.
The image segmentation method of cross-scale interactive fusion and uncertain perception dynamic distillation is adopted, and the segmentation performance is improved through the teacher and student networks.
While maintaining low computing costs and high operating speed, the accuracy and robustness of medical image segmentation are significantly improved and are suitable for actual clinical scenarios.
Smart Images
Figure CN120472161A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of knowledge distillation and image segmentation, and specifically relates to a knowledge transfer and image segmentation method integrating a multi-level distillation mechanism. Background Art
[0002] Recent progress has been made in applying convolutional neural networks to achieve more accurate predictions for the medical image segmentation problem. However, the success of existing methods relies heavily on enormous computational complexity and massive storage, which is impractical in the real world. To address this issue, we propose an efficient architecture that trains a lightweight network by extracting knowledge from a well-trained medical image segmentation network. This architecture enables the lightweight network to significantly improve segmentation capabilities while maintaining operational efficiency. We also design a novel distillation module for medical image segmentation to transfer semantic region information from the teacher network to the student network.
[0003] It forces the student network to simulate the degree of difference in representations computed from different tissue regions. This module avoids the problems of vanishing regions and blurred boundaries encountered in medical imaging, instead encoding the internal and structural information of each semantic region for transmission. We demonstrate that the lightweight network extracted by our method is of considerable value in scenarios requiring relatively high operation speed and low memory usage.
[0004] The knowledge distillation framework proposed in this paper provides a practical and effective solution for medical image segmentation tasks, significantly reducing network complexity and computational cost, and enhancing the model's applicability in real-world clinical scenarios. In the future, this method is expected to be further extended to more medical image analysis tasks, providing strong technical support for improving the efficiency and accuracy of medical-assisted diagnosis. Summary of the Invention
[0005] The purpose of this invention is to provide an image segmentation method based on cross-scale interactive fusion and uncertainty-aware dynamic distillation, which improves the performance of medical image segmentation by integrating multiple distillation methods. Specifically, the method inputs raw CT medical slice images and passes the same batch of slices into the teacher and student networks to generate hierarchical feature maps. The generated feature maps are then transmitted to various distillation modules, including the cross-scale correlation modeling unit (CSL), the saliency guided modeling unit (AD), the local response area construction unit (RAAD), and the uncertainty-aware driven (UDG) distillation modules.
[0006] The present invention is achieved through the following technical solutions:
[0007] Step 1: Input the original CT medical slice image, pass the slice into the teacher and student networks respectively, perform multi-level feature extraction operations, and form a cross-scale feature map set;
[0008] Step 2: The multi-level feature map collection generated by the teacher and student networks is input into the Cross-Scale Interaction Unit (CSI). In this module, complementary correlation extraction and interactive mapping learning are performed between feature maps of different scales. Based on this, a fused feature representation at a unified scale is refined to guide subsequent distillation.
[0009] Step 3: The comprehensive fusion features are input into the saliency-guided modeling unit (AD module) and the local response-aware alignment module (RAAD module). The AD module guides the student network to focus on salient regional information by learning the importance distribution of key feature responses in the spatial domain, while the RAAD module dynamically mines the correlation coupling structure between feature elements in the local sub-region to strengthen the consistency of the feature representation of the target region while suppressing the interference response of irrelevant background areas.
[0010] Step 4: Based on the preliminary segmentation prediction results, a diversified perturbation generation mechanism is introduced in the reasoning stage. Multiple rounds of independent forward reasoning are performed on the same input sample to generate a prediction set. Based on the internal reasoning consistency evaluation of the set, the confidence stability index is derived point by point. According to the stability index, a local weighted mapping matrix is adaptively generated. The weighted matrix is input into the uncertainty-driven guided distillation module (UDG module). Under the guidance of the position-by-position adaptive loss, the student network features are optimized to finally obtain a segmentation result map with enhanced robustness.
[0011] Furthermore, the specific step 1 is as follows: 210 annotated data publicly available in the Kits19 dataset are used for training, 168 cases are randomly selected for training, and the remaining 42 are used for testing. A variety of mainstream large-parameter medical image segmentation frameworks are selected as teachers, and several small-parameter efficient segmentation networks are used as students to perform training segmentation on the training images to verify the effectiveness of our distillation method.
[0012] Furthermore, the step 2 is specifically as follows: the feature sets generated by the teacher and the student are transferred to the multi-scale cross-fusion (CSI) module for refinement. This module proposes a distillation method with dynamic weighting mechanism of interaction and uncertainty perception across multiple scales. The teacher network and the student network extract L feature maps of different scales respectively. and Then, the teacher's features at each scale are adjusted to a uniform spatial resolution, and then all scales are spliced together to form feature fusion. The specific expression is as follows:
[0013]
[0014] The cross-scale interaction of the fused features is then modeled through the self-attention module, where the attention mechanism adopts the standard Scaled Dot-ProductAttention, and the specific expression is as follows:
[0015] F CSI =Attention(F fusion )
[0016]
[0017] Among them, Q, K, V are F fusion Derived from linear changes;
[0018] Then for each scale Using the random forward perturbation reasoning mechanism, we perform T forward reasonings in the reasoning phase and obtain: Then calculate the mean and variance of scale l. The specific expressions are as follows:
[0019]
[0020] Among them, σ l represents the characteristic uncertainty of scale l;
[0021] Then, the distillation weight is dynamically generated based on the uncertainty. The specific expression is as follows:
[0022] w l =exp(-γσ l )
[0023] Among them, γ>0 is a hyperparameter that controls the sensitivity to uncertainty;
[0024] Finally, the cross-scale interaction feature F CSI After adjusting to the corresponding scale, the mean square error Loss is calculated with the student network features. The specific expression is as follows:
[0025]
[0026] The final comprehensive dynamic weighted distillation loss is:
[0027]
[0028] Furthermore, the step three is specifically as follows: the obtained FCSI , input to the regional weight extraction unit, first the fused F CSI After the convolution mapping operation, it becomes W attn , the specific expression is as follows:
[0029] A=Sigmoid(W attn (F CSI ))
[0030] Among them, A is between 0 and 1, which is the local saliency weight map. The larger the value, the more important it is. In addition, during feature distillation, the loss of each region is weighted by the distillation error of each feature position according to the Attention weight. The distillation loss guided by local Attention is defined as:
[0031]
[0032] in, and They are the teacher fusion features and student extraction features of the i-th position on the l-th scale. At the same time, in order to improve the regional discrimination ability of optimizing the local response intensity distribution, the auxiliary supervision signal is introduced By using the existing prediction mask M to perform guided contrast training on the foreground and background areas, the specific auxiliary loss expression is as follows:
[0033]
[0034] Among them, the final overall Attention-guided distillation loss is:
[0035]
[0036] Among them, λ attn It is the fusion ratio coefficient of auxiliary supervision loss and local distillation loss;
[0037] Next, the feature map processed by the Attention Distillation module is divided into a series of local candidate regions. Each local region covers a part of the spatial range in the feature map. Within each local candidate region, a feature subset is established. Through the dynamic relationship modeling mechanism within the region, the correlation distribution between the local internal feature points is calculated. Based on the local correlation modeling, the feature distribution of the teacher network and the student network in the corresponding local regions is distilled to guide the student network to maintain consistency in the local structure. The local guided feature distillation loss is defined as:
[0038]
[0039] Furthermore, the step 4 is specifically as follows: the prediction feature graph generated by the student network in the output stage is subjected to random perturbation inference mechanism, by introducing random intervention in the inference process, performing multiple independent inferences on the same input sample to form a diversified prediction set Based on multiple inference results, perform prediction stability assessment on each prediction set position point by point and calculate the mean prediction for each position And the stability index σ, the specific formula is as follows:
[0040]
[0041] Based on the distribution results of the stability index σ, it is input into the adaptive weight generation module to generate a dynamic weighting factor matrix W for each spatial position, and its weight is defined as:
[0042] W(x,y)=exp(-γ·σ(x,y))
[0043] An uncertainty-aware consistency alignment mechanism is introduced between the teacher prediction and the student prediction, and point-wise weighted feature transfer is performed according to the dynamic weighting factor W. The specific distillation error loss is defined as:
[0044]
[0045] in, and They represent the mean prediction results of the teacher model and the student model at the corresponding positions respectively. Finally, the score of each segmented pixel in the tumor area is distilled to obtain the final prediction result segmentation map.
[0046] The beneficial effects of the present invention are as follows:
[0047] The present invention provides an image segmentation method based on cross-scale interactive fusion and uncertainty-aware dynamic distillation. By integrating multiple distillation methods, the performance of medical image segmentation is improved. Specifically, the method inputs the original CT medical slice images and passes the slices of the same batch into the teacher and student networks respectively to generate a hierarchical feature map. The generated feature map is then transmitted to various distillation modules, including the cross-scale correlation modeling unit (CSL), the saliency-guided modeling unit (AD), the local response area construction unit (RAAD), and the uncertainty-aware drive (UDG). This achieves efficient inheritance of the teacher network structure knowledge and provides an innovative solution for extracting image feature information. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0049] Figure 1 It is a schematic diagram of the process of the present invention;
[0050] Figure 2 Schematic diagram of the model structure of the present invention;
[0051] Figure 3 Schematic diagram of CSI structure;
[0052] Figure 4 It is a structural diagram of UDG; DETAILED DESCRIPTION
[0053] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.
[0054] See also Figure 1-4 As shown, the present invention is an image segmentation method based on cross-scale interactive fusion and uncertainty-aware dynamic distillation, comprising the following steps:
[0055] S101: Input medical CT slice images (Kits19), and transfer them to the teacher and student networks in batches to generate multi-level feature maps respectively;
[0056] S102: The two feature sets generated by the teacher and the student are transferred to the Cross-Scale Correlation Modeling Unit (CSI) module for refinement;
[0057] S103: Then the data is passed to the saliency guided modeling unit (AD) and the local response area construction (RAAD) unit module to focus on more important areas and perform comparative refinement on the areas;
[0058] S104: The data is finally passed to the uncertainty perception drive (UDG) module, and finally pixel-by-pixel distillation is performed to obtain the segmentation effect map.
[0059] As an optimization solution for the above embodiment, step 1 is specifically as follows: 210 annotated data publicly available in the Kits19 dataset are trained, 168 cases are randomly selected for training, and the remaining 42 are tested. A variety of mainstream large-parameter medical image segmentation frameworks are selected as teachers, and several small-parameter efficient segmentation networks are used as students to perform training segmentation on the training images to verify the effectiveness of our distillation method.
[0060] As an optimization solution for the above embodiment, the step 2 is specifically as follows: the feature sets generated by the teacher and the student are transferred to the multi-scale cross-fusion (CSI) module for refinement. This module proposes a distillation method with a dynamic weighting mechanism of interaction and uncertainty perception across multiple scales. The teacher network and the student network extract L feature maps of different scales respectively. and Then, the teacher's features at each scale are adjusted to a uniform spatial resolution, and then all scales are spliced together to form feature fusion. The specific expression is as follows:
[0061]
[0062] The cross-scale interaction of the fused features is then modeled through the self-attention module, where the attention mechanism adopts the standard Scaled Dot-ProductAttention, and the specific expression is as follows:
[0063] F CSI =Attention(F fusion )
[0064]
[0065] Among them, Q, K, V are F fusion Derived from linear changes;
[0066] Then for each scale Using the random forward perturbation reasoning mechanism, we perform T forward reasonings in the reasoning phase and obtain: Then calculate the mean and variance of scale l. The specific expressions are as follows:
[0067]
[0068] Among them, σ l represents the characteristic uncertainty of scale l;
[0069] Then, the distillation weight is dynamically generated based on the uncertainty. The specific expression is as follows:
[0070] w l=exp(-γσ l )
[0071] Among them, γ>0 is a hyperparameter that controls the sensitivity to uncertainty;
[0072] Finally, the cross-scale interaction feature F CSI After adjusting to the corresponding scale, the mean square error Loss is calculated with the student network features. The specific expression is as follows:
[0073]
[0074] The final comprehensive dynamic weighted distillation loss is:
[0075]
[0076] As an optimization solution of the above embodiment, the step three is specifically as follows: the obtained F CSI , input to the regional weight extraction unit, first the fused F CSI After the convolution mapping operation, it becomes W attn , the specific expression is as follows:
[0077] A=Sigmoid(W attn (F CSI ))
[0078] Among them, A is between 0 and 1, which is the local saliency weight map. The larger the value, the more important it is. In addition, during feature distillation, the loss of each region is weighted by the distillation error of each feature position according to the Attention weight. The distillation loss guided by local Attention is defined as:
[0079]
[0080] in, and They are the teacher fusion features and student extraction features of the i-th position on the l-th scale. At the same time, in order to improve the regional discrimination ability of optimizing the local response intensity distribution, the auxiliary supervision signal is introduced By using the existing prediction mask M to perform guided contrast training on the foreground and background areas, the specific auxiliary loss expression is as follows:
[0081]
[0082] Among them, the final overall Attention-guided distillation loss is:
[0083]
[0084] Among them, λ attnIt is the fusion ratio coefficient of auxiliary supervision loss and local distillation loss;
[0085] Next, the feature map processed by the Attention Distillation module is divided into a series of local candidate regions. Each local region covers a part of the spatial range in the feature map. Within each local candidate region, a feature subset is established. Through the dynamic relationship modeling mechanism within the region, the correlation distribution between the local internal feature points is calculated. Based on the local correlation modeling, the feature distribution of the teacher network and the student network in the corresponding local regions is distilled to guide the student network to maintain consistency in the local structure. The local guided feature distillation loss is defined as:
[0086]
[0087] As an optimization solution of the above embodiment, the step 4 is specifically as follows: the prediction feature map generated by the student network in the output stage is subjected to random perturbation inference mechanism, and random intervention is introduced in the inference process to perform multiple independent inferences on the same input sample to form a diversified prediction set. Based on multiple inference results, perform prediction stability assessment on each prediction set position point by point and calculate the mean prediction for each position And the stability index σ, the specific formula is as follows:
[0088]
[0089] Based on the distribution results of the stability index σ, it is input into the adaptive weight generation module to generate a dynamic weighting factor matrix W for each spatial position, and its weight is defined as:
[0090] W(x,y)=exp(-γ·σ(x,y))
[0091] An uncertainty-aware consistency alignment mechanism is introduced between the teacher prediction and the student prediction, and point-wise weighted feature transfer is performed according to the dynamic weighting factor W. The specific distillation error loss is defined as:
[0092]
[0093] in, and They represent the mean prediction results of the teacher model and the student model at the corresponding positions respectively. Finally, the score of each segmented pixel in the tumor area is distilled to obtain the final prediction result segmentation map.
[0094] Although the above describes the specific embodiments of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without the need for creative work are still within the scope of protection of the present invention.
Claims
1. An image segmentation method integrating multi-cross-scale self-distillation and uncertainty-guided optimization, characterized in that: The image segmentation method integrating multi-cross-scale self-distillation and uncertainty-guided optimization comprises the following steps: Step 1: Input the original CT medical slice image, pass the slice into the teacher and student networks respectively, perform multi-level feature extraction operations, and form a cross-scale feature map set; Step 2: The multi-level feature map collection generated by the teacher and student networks is input into the Cross-Scale Interaction Unit (CSI). In this module, complementary correlation extraction and interactive mapping learning are performed between feature maps of different scales. Based on this, a fused feature representation at a unified scale is refined to guide subsequent distillation. Step 3: The comprehensive fusion features are input into the saliency-guided modeling unit (AD module) and the local response-aware alignment module (RAAD module). The AD module guides the student network to focus on salient regional information by learning the importance distribution of key feature responses in the spatial domain, while the RAAD module dynamically mines the correlation coupling structure between feature elements in the local sub-region to strengthen the consistency of the feature representation of the target region while suppressing the interference response of irrelevant background areas. Step 4: Based on the preliminary segmentation prediction results, a diversified perturbation generation mechanism is introduced in the reasoning stage. Multiple rounds of independent forward reasoning are performed on the same input sample to generate a prediction set. Based on the internal reasoning consistency evaluation of the set, the confidence stability index is derived point by point. According to the stability index, a local weighted mapping matrix is adaptively generated. The weighted matrix is input into the uncertainty-driven guided distillation module (UDG module). Under the guidance of the position-by-position adaptive loss, the student network features are optimized to finally obtain a segmentation result map with enhanced robustness.
2. The method according to claim 1, wherein Specifically, step one involves training on 210 publicly available annotated data from the Kits19 dataset, randomly selecting 168 cases for training and the remaining 42 for testing. We then used a variety of mainstream large-parameter medical image segmentation frameworks as teachers, along with several small-parameter efficient segmentation networks as students, to perform training segmentation on the training images and verify the effectiveness of our distillation method.
3. The method according to claim 1, wherein The specific step 2 is as follows: the feature sets generated by the teacher and the student are transferred to the multi-scale cross-fusion (CSI) module for refinement. This module proposes a distillation method with dynamic weighting mechanism of interaction and uncertainty perception across multiple scales. The teacher network and the student network extract L feature maps of different scales respectively. Then, the teacher's features at each scale are adjusted to a uniform spatial resolution, and then all scales are spliced together to form feature fusion. The specific expression is as follows: The cross-scale interaction of the fused features is then modeled through the self-attention module, where the attention mechanism adopts the standard Scaled Dot-ProductAttention, and the specific expression is as follows: F CSI =Attention(F fusion ) Among them, Q, K, V are F fusion Derived from linear changes; Then for each scale Using the random forward perturbation reasoning mechanism, we perform T forward reasonings in the reasoning phase and obtain: Then calculate the mean and variance of scale l. The specific expressions are as follows: Among them, σ l represents the characteristic uncertainty of scale l; Then, the distillation weight is dynamically generated based on the uncertainty. The specific expression is as follows: w l =exp(-γσ l ) Among them, γ>0 is a hyperparameter that controls the sensitivity to uncertainty; Finally, the cross-scale interaction feature F CSI After adjusting to the corresponding scale, the mean square error Loss is calculated with the student network features. The specific expression is as follows: The final comprehensive dynamic weighted distillation loss is:
4. The method according to claim 1, wherein The step three is specifically as follows: the obtained F CSI , input to the regional weight extraction unit, first the fused F CSI After the convolution mapping operation, it becomes W attn , the specific expression is as follows: A=Sigmoid(W attn (F CSI )) Among them, A is between 0 and 1, which is the local saliency weight map. The larger the value, the more important it is. In addition, during feature distillation, the loss of each region is weighted by the distillation error of each feature position according to the Attention weight. The distillation loss guided by local Attention is defined as: in, and They are the teacher fusion features and student extraction features of the i-th position on the l-th scale. At the same time, in order to improve the regional discrimination ability of optimizing the local response intensity distribution, the auxiliary supervision signal is introduced By using the existing prediction mask M to perform guided contrast training on the foreground and background areas, the specific auxiliary loss expression is as follows: Among them, the final overall Attention-guided distillation loss is: Among them, λ attn It is the fusion ratio coefficient of auxiliary supervision loss and local distillation loss; Next, the feature map processed by the Attention Distillation module is divided into a series of local candidate regions. Each local region covers a part of the spatial range in the feature map. Within each local candidate region, a feature subset is established. Through the dynamic relationship modeling mechanism within the region, the correlation distribution between the local internal feature points is calculated. Based on the local correlation modeling, the feature distribution of the teacher network and the student network in the corresponding local regions is distilled to guide the student network to maintain consistency in the local structure. The local guided feature distillation loss is defined as:
5. The method according to claim 1, wherein The specific step 4 is as follows: the prediction feature map generated by the student network in the output stage is subjected to random perturbation inference mechanism, by introducing random intervention in the inference process, performing multiple independent inferences on the same input sample to form a diversified prediction set Based on multiple inference results, perform prediction stability assessment on each prediction set position point by point and calculate the mean prediction for each position And the stability index σ, the specific formula is as follows: Based on the distribution results of the stability index σ, it is input into the adaptive weight generation module to generate a dynamic weighting factor matrix W for each spatial position, and its weight is defined as: An uncertainty-aware consistency alignment mechanism is introduced between the teacher prediction and the student prediction, and point-wise weighted feature transfer is performed according to the dynamic weighting factor W. The specific distillation error loss is defined as: in, and They represent the mean prediction results of the teacher model and the student model at the corresponding positions respectively. Finally, the score of each segmented pixel in the tumor area is distilled to obtain the final prediction result segmentation map.