Medical image segmentation model knowledge distillation method and system

Abstract

The invention belongs to the technical field of medical image segmentation, and discloses a medical image segmentation model knowledge distillation method and system, and the method comprises the steps: introducing a distillation-dedicated branch in an intermediate decoder layer based on an original architecture of a U-shaped image segmentation model, building and forming a knowledge distillation frame, and selectively transferring task-specific knowledge to obtain a knowledge distillation-dedicated branch; selective distillation between the encoder and the decoder is realized; and constructing a residual fusion selection module, integrating the characteristics of different distillation special branches, fusing the multi-stage student characteristics based on a residual fusion selection mode, matching the dimension of the teacher characteristics, and transferring the cross-stage characteristics from the teacher network to the student network. The method aims at a novel distillation framework of a U-shaped architecture model, focuses on selective distillation of encoder or decoder features, enhances the efficiency and performance of the U-shaped model in tasks such as image segmentation and medical image analysis by selectively transferring specific knowledge of tasks, has excellent performance, and reduces calculation complexity.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image segmentation, and in particular to a medical image segmentation model knowledge distillation method and system. Background Art

[0002] In recent years, deep neural networks have played an important role in medical image segmentation. U-net, as the first model with U-shaped structure and jump connection, has become a popular choice for medical image segmentation. On this basis, U-Net++ makes better use of feature representations at different levels by adding denser structures in jump connections; based on the idea of ​​Res-Net (residual network), Res-Unet extends jump connections to residual connections to enhance segmentation. Subsequently, the success of the self-attention mechanism began with the introduction of VisionTransformer (ViT, visual transformer) and expanded to the field of medical image segmentation - Trans-Unet achieved remarkable results by fusing Transformer (deep learning model architecture) and U-net to capture global context. Subsequent developments include Swin-Unet (U-Net model based on Swin Transformer architecture, commonly used in medical image segmentation tasks), which improves the results by replacing each convolutional block in the encoder-decoder structure with SwinTransformer (a new visual transformer model with hierarchical window self-attention mechanism) blocks.

[0003] However, in the field of medical image segmentation, the application of knowledge distillation is still in its infancy. Existing distillation techniques are inefficient for full-stage distillation, and not all stages of the teacher network contribute equally to the learning process. In addition, the unique structural features of the U-shaped architecture of the popular U-Net-like model in the field of medical image segmentation are not fully utilized, especially the interaction between the encoder and decoder features, which are connected by skip connections.

[0004] Therefore, how to provide a knowledge distillation method and system for medical image segmentation models is a problem that needs to be solved urgently. Summary of the invention

[0005] The embodiments of the present invention provide a medical image segmentation model knowledge distillation method and system to solve the above-mentioned technical problems existing in the prior art.

[0006] In order to have a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended to be a general review, nor is it intended to identify key / important components or to delineate the scope of protection of these embodiments. Its only purpose is to present some concepts in a simple form as a preface to the detailed description that follows.

[0007] According to a first aspect of an embodiment of the present invention, a medical image segmentation model knowledge distillation method is provided.

[0008] In one embodiment, the medical image segmentation model knowledge distillation method includes:

[0009] Based on the original architecture of the U-shaped image segmentation model, a dedicated distillation branch is introduced in the intermediate decoder layer to build a knowledge distillation framework. By selectively transferring task-specific knowledge, selective distillation between the encoder and decoder is achieved.

[0010] A residual fusion selection module is constructed to integrate the features of different distillation-dedicated branches. Based on the residual fusion selection method, multi-stage student features are fused to match the dimensions of teacher features and transfer cross-stage features from the teacher network to the student network.

[0011] In one embodiment, the original architecture based on the U-shaped image segmentation model introduces a dedicated distillation branch in the intermediate decoder layer to build a knowledge distillation framework, and selectively transfers task-specific knowledge to achieve selective distillation between the encoder and the decoder, including:

[0012] In the original architecture of the U-shaped image segmentation model, the decoder distillation and the distillation branch in the feature are introduced to form a two-part knowledge distillation framework, and the loss function of the decoder distillation is established to extract different contextual information in medical images;

[0013] Based on the upsampling and skip connection features of the U-shaped image segmentation model, for the encoder with global vision and cross-stage features, selective distillation is performed between the encoder and decoder by selectively transferring task-specific knowledge, thereby retaining the key spatial and contextual information inherent in medical image analysis tasks.

[0014] In one embodiment, the loss function of the decoder distillation is expressed as:

[0015]

[0016] In the formula, Loss fm,Decode is the loss function of decoder distillation; m1 is the number of feature maps in the encoder; m2 is the number of feature maps in the decoder; The student model features are processed to adapt to the teacher model features; F t is the teacher model feature in the teacher model corresponding to the student model feature layer; i is the feature position corresponding to the current student and teacher.

[0017] In one embodiment, the construction of the residual fusion selection module, integrating the features of different distillation dedicated branches, fusing multi-stage student features based on the residual fusion selection method, matching the dimensions of the teacher features, and transferring the cross-stage features from the teacher network to the student network includes:

[0018] Construct a residual fusion selection module consisting of a fusion feature module, a feature selection module, and a residual gating module, integrate the features of the decoder distillation in the knowledge distillation framework and the distillation branch in the feature, fuse the student features of the two stages, and obtain preliminary fusion features;

[0019] After the preliminary processing, the fusion features are obtained, and based on the weighted selective fusion of softmax, different weights are assigned to each feature channel to achieve selective weighted fusion of features;

[0020] The feature maps output by each stage of the knowledge distillation framework are obtained and divided into 4 levels of context information through adaptive average pooling. In each stage, the knowledge is distilled between the context levels using Euclidean distance, and each stage is distilled separately using Euclidean distance. The final hierarchical context loss between the feature maps of the nth stage is calculated, and the student features are matched to the teacher feature dimensions to transfer the cross-stage features from the teacher network to the student network.

[0021] In one embodiment, the calculation formula for calculating the final hierarchical context loss between the feature maps of the nth stage is:

[0022]

[0023] In the formula, loss fm is the hierarchical context loss; l is the Euclidean distance; F out is the student model feature output after feature fusion processing; F t is the teacher model feature in the teacher model corresponding to the student model feature layer; i is the feature position corresponding to the current student and teacher.

[0024] According to a second aspect of an embodiment of the present invention, a medical image segmentation model knowledge distillation system is provided.

[0025] In one embodiment, the medical image segmentation model knowledge distillation system includes:

[0026] The knowledge distillation framework module is used to introduce a dedicated distillation branch in the intermediate decoder layer based on the original architecture of the U-shaped image segmentation model to build a knowledge distillation framework. It achieves selective distillation between the encoder and decoder by selectively transferring task-specific knowledge.

[0027] The residual fusion selection module is used to integrate the features of different distillation-dedicated branches. Based on the residual fusion selection method, it fuses the student features of multiple stages, matches the dimensions of the teacher features, and transfers the cross-stage features from the teacher network to the student network.

[0028] In one embodiment, the knowledge distillation framework module includes: a branch construction module and a selective distillation module, wherein:

[0029] The branch construction module is used to introduce the distillation branch in the decoder and the feature into the original architecture of the U-shaped image segmentation model to form a two-part knowledge distillation framework, and establish a loss function for the decoder distillation to extract different contextual information in the medical image;

[0030] The selective distillation module is used to perform selective distillation between the encoder and the decoder based on the upsampling and skip connection features of the U-shaped image segmentation model, for an encoder with global vision and cross-stage features, by selectively transferring task-specific knowledge, thereby retaining the key spatial and contextual information inherent in the medical image analysis task.

[0031] In one embodiment, the loss function of the decoder distillation is expressed as:

[0032]

[0033] In the formula, Loss fm,Decode is the loss function of decoder distillation; m1 is the number of feature maps in the encoder; m2 is the number of feature maps in the decoder; The student model features are processed to adapt to the teacher model features; F t is the teacher model feature in the teacher model corresponding to the student model feature layer; i is the feature position corresponding to the current student and teacher.

[0034] In one embodiment, the residual fusion selection module includes: a fusion feature module, a feature selection module and a residual gating module, wherein:

[0035] The fusion feature module is used to integrate the features of the decoder distillation and the distillation branch in the feature in the knowledge distillation framework, fuse the student features of the two stages, and obtain preliminary fusion features;

[0036] The feature selection module is used to obtain the fusion features after preliminary processing, and to assign different weights to each feature channel based on the weighted selective fusion of softmax to achieve selective weighted fusion of features;

[0037] The residual gating module is used to obtain the feature map output by each stage of the knowledge distillation framework, which is divided into 4 levels of context information through adaptive average pooling. In each stage, the knowledge is distilled between the context levels using the Euclidean distance, and each stage is distilled separately using the Euclidean distance. The final hierarchical context loss between the feature maps of the nth stage is calculated, and the student features are matched to the teacher feature dimensions to transfer the cross-stage features from the teacher network to the student network.

[0038] In one embodiment, the calculation formula for calculating the final hierarchical context loss between the feature maps of the nth stage is:

[0039]

[0040] In the formula, loss fm is the hierarchical context loss; l is the Euclidean distance; F out is the student model feature output after feature fusion processing; F t is the teacher model feature in the teacher model corresponding to the student model feature layer; i is the feature position corresponding to the current student and teacher.

[0041] According to a third aspect of an embodiment of the present invention, a computer device is provided.

[0042] In some embodiments, the computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0043] According to a fourth aspect of embodiments of the present invention, a computer-readable storage medium is provided.

[0044] In one embodiment, the computer-readable storage medium stores a computer program, and the computer program implements the steps of the above method when executed by a processor.

[0045] The technical solution provided by the embodiment of the present invention may include the following beneficial effects: a special distillation module is adopted at the key position of the decoder through the knowledge distillation framework, which reflects the structural importance of these features in the U-type model, not only improving the efficiency of knowledge transfer, but also retaining the key spatial and contextual information inherent in the medical image analysis task; that is, a novel distillation framework is designed specifically for the U-type architecture model, focusing on the selective distillation of encoder or decoder features. By selectively transferring task-specific knowledge, the framework of the present invention enhances the efficiency and performance of the U-type model in tasks such as image segmentation and medical image analysis, has excellent performance while reducing computational complexity, and highlights its potential in practical applications.

[0046] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0048] Figure 1 is a flow chart of a medical image segmentation model knowledge distillation method according to an exemplary embodiment;

[0049] Figure 2 is a schematic structural diagram of a medical image segmentation model knowledge distillation system according to an exemplary embodiment;

[0050] Figure 3 is a schematic diagram of a decoder distillation feature structure according to an exemplary embodiment;

[0051] Figure 4 is a schematic diagram of computing resources for various types of knowledge distillation methods according to an exemplary embodiment;

[0052] Figure 5 is the performance of the knowledge distillation module in the U-type model on the Synapse dataset according to an exemplary embodiment;

[0053] Figure 6 is the performance of the knowledge distillation module in the U-type model on the ACDC dataset according to an exemplary embodiment;

[0054] Figure 7 is a schematic diagram of the structure of a knowledge distillation framework according to an exemplary embodiment;

[0055] Figure 8 is an architecture diagram of a residual fusion selection module according to an exemplary embodiment;

[0056] Fig. 9 The figure is a schematic diagram showing the structure of a computer device according to an exemplary embodiment. DETAILED DESCRIPTION

[0057] The following description and accompanying drawings fully illustrate the specific embodiments of this article so that those skilled in the art can practice them. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. The scope of the embodiments of this article includes the entire scope of the claims, as well as all available equivalents of the claims. Herein, the terms "first", "second", etc. are only used to distinguish one element from another, without requiring or implying any actual relationship or order between these elements. In fact, the first element can also be called the second element, and vice versa. Moreover, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that the structure, device or equipment including a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also include elements inherent to such structure, device or equipment. In the absence of more restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the structure, device or equipment including the elements. Each embodiment is described in a progressive manner herein, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the embodiments can be referred to each other.

[0058] The terms "longitudinal", "lateral", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc. in this document indicate the orientation or position relationship based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing this document and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In the description of this document, unless otherwise specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a mechanical connection or an electrical connection, it can also be the internal communication of two elements, it can be a direct connection, or it can be an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0059] As used herein, the term "plurality" means two or more than two, unless otherwise specified.

[0060] In this document, the character " / " indicates that the preceding and following objects are in an "or" relationship. For example, A / B means: A or B.

[0061] In this article, the term "and / or" is a description of the association relationship between objects, indicating that three relationships may exist. For example, A and / or B means: A or B, or, A and B.

[0062] It should be understood that, although the various steps in the flow chart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the figure may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0063] Each module in the device or system of the present application can be implemented in whole or in part by software, hardware, or a combination thereof. The above modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to the above modules.

[0064] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0065] Figure 1 An embodiment of a medical image segmentation model knowledge distillation method of the present invention is shown.

[0066] In this optional embodiment, the medical image segmentation model knowledge distillation method includes:

[0067] Step S101: Based on the original architecture of the U-shaped image segmentation model, a dedicated distillation branch is introduced in the intermediate decoder layer to build a knowledge distillation framework, and selective distillation between the encoder and the decoder is achieved by selectively transferring task-specific knowledge;

[0068] Step S103: construct a residual fusion selection module to integrate the features of different distillation-dedicated branches, fuse the student features of multiple stages based on the residual fusion selection method, match the dimensions of the teacher features, and transfer the cross-stage features from the teacher network to the student network.

[0069] In this optional embodiment, in the original architecture based on the U-shaped image segmentation model, a dedicated distillation branch is introduced in the intermediate decoder layer to build a knowledge distillation framework. When selective distillation between the encoder and the decoder is achieved by selectively transferring task-specific knowledge, the decoder distillation and the distillation branch in the feature can be introduced into the original architecture of the U-shaped image segmentation model to form a two-part knowledge distillation framework, and a loss function for the decoder distillation is established to extract different contextual information in medical images; based on the upsampling and jump connection features of the U-shaped image segmentation model, for the encoder with global vision and cross-stage features, selective distillation between the encoder and the decoder is performed by selectively transferring task-specific knowledge to retain the key space and contextual information inherent in the medical image analysis task.

[0070] In this optional embodiment, the loss function of the decoder distillation is expressed as:

[0071]

[0072] In the formula, Loss fm,Decode is the loss function of decoder distillation; m1 is the number of feature maps in the encoder; m2 is the number of feature maps in the decoder; The student model features are processed to adapt to the teacher model features; F t is the teacher model feature in the teacher model corresponding to the student model feature layer; i is the feature position corresponding to the current student and teacher.

[0073] In this optional embodiment, when constructing the residual fusion selection module, integrating the features of different distillation-dedicated branches, fusing multi-stage student features based on the residual fusion selection method, matching the dimension of the teacher features, and transferring the cross-stage features from the teacher network to the student network, a residual fusion selection module composed of a fusion feature module, a feature selection module, and a residual gating module can be constructed to integrate the features of the distillation branches in the decoder distillation and features in the knowledge distillation framework, fuse the student features of the two stages, and obtain preliminary fusion features; obtain the fusion features after preliminary processing, and assign different weights to each feature channel based on weighted selective fusion of softmax (a normalized activation function commonly used in classification tasks that converts a set of numerical values ​​into a probability distribution) to achieve selective weighted fusion of features; obtain the feature map output by each stage of the knowledge distillation framework, and divide it into 4 levels of context information through adaptive average pooling. In each stage, use Euclidean distance to distill knowledge between context levels, and use Euclidean distance to distill each stage separately, calculate the final hierarchical context loss between the feature maps of the nth stage, match the student features to the teacher feature dimensions, and transfer the cross-stage features from the teacher network to the student network.

[0074] In this optional embodiment, the calculation formula for calculating the final hierarchical context loss between the feature maps of the nth stage is:

[0075]

[0076] In the formula, loss fm is the hierarchical context loss; l is the Euclidean distance; F out is the student model feature output after feature fusion processing; F t is the teacher model feature in the teacher model corresponding to the student model feature layer; i is the feature position corresponding to the current student and teacher.

[0077] Figure 2 An embodiment of a medical image segmentation model knowledge distillation system of the present invention is shown.

[0078] In this optional embodiment, the medical image segmentation model knowledge distillation system includes:

[0079] The knowledge distillation framework module 201 is used to introduce a dedicated distillation branch in the intermediate decoder layer based on the original architecture of the U-shaped image segmentation model to build a knowledge distillation framework, and realize selective distillation between the encoder and the decoder by selectively transferring task-specific knowledge;

[0080] The residual fusion selection module 203 is used to integrate the features of different distillation dedicated branches, fuse the student features of multiple stages based on the residual fusion selection method, match the dimensions of the teacher features, and transfer the cross-stage features from the teacher network to the student network.

[0081] In this optional embodiment, the knowledge distillation framework module 201 includes: a branch construction module (not shown in the figure) and a selective distillation module (not shown in the figure), wherein the branch construction module is used to introduce the distillation branches in the decoder distillation and features in the original architecture of the U-shaped image segmentation model, forming a two-part knowledge distillation framework, and establishing a loss function for the decoder distillation to extract different contextual information in the medical image; the selective distillation module is used to perform selective distillation between the encoder and the decoder based on the upsampling and jump connection features of the U-shaped image segmentation model, for the encoder with global vision and cross-stage features, by selectively transferring task-specific knowledge, thereby retaining the key space and contextual information inherent in the medical image analysis task.

[0082] In this optional embodiment, the loss function of the decoder distillation is expressed as:

[0083]

[0084] In the formula, Loss fm,Decodeis the loss function of decoder distillation; m1 is the number of feature maps in the encoder; m2 is the number of feature maps in the decoder; The student model features are processed to adapt to the teacher model features; F t is the teacher model feature in the teacher model corresponding to the student model feature layer; i is the feature position corresponding to the current student and teacher.

[0085] In this optional embodiment, the residual fusion selection module 203 includes: a fusion feature module (not shown in the figure), a feature selection module (not shown in the figure) and a residual gating module (not shown in the figure), wherein the fusion feature module is used to integrate the features of the decoder distillation and the distillation branch in the feature in the knowledge distillation framework, fuse the student features of the two stages, and obtain preliminary fusion features; the feature selection module is used to obtain the fusion features after preliminary processing, and assign different weights to each feature channel based on the weighted selective fusion of softmax to achieve selective weighted fusion of features; the residual gating module is used to obtain the feature map output by each stage of the knowledge distillation framework, and divide it into 4 levels of context information through adaptive average pooling. In each stage, the Euclidean distance is used to distill knowledge between context levels, and the Euclidean distance is used to distill each stage separately, and the final hierarchical context loss between the feature maps of the nth stage is calculated, and the student features are matched to the teacher feature dimensions to transfer the cross-stage features from the teacher network to the student network.

[0086] In this optional embodiment, the calculation formula for calculating the final hierarchical context loss between the feature maps of the nth stage is:

[0087]

[0088] In the formula, loss fm is the hierarchical context loss; l is the Euclidean distance; F out is the student model feature output after feature fusion processing; F t is the teacher model feature in the teacher model corresponding to the student model feature layer; i is the feature position corresponding to the current student and teacher.

[0089] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0090] Inspired by the dynamic feature weighting mechanism in ResNeSt, we propose a new relation-based knowledge distillation method called Residual Fusion Selection Module (ResFuSe). ResFuSe dynamically aggregates information from different branches while preserving pre-fused features. It adopts precise cross-channel softmax attention for adaptive information selection and uses gated residuals to fine-tune the scale of fused features. This approach not only serves as the input of subsequent stages but also promotes learning from the corresponding teacher features. In addition, we introduce a distillation framework specifically for U-shaped segmentation models. Our framework focuses on distilling decoder features, considering the unique characteristics of the U-shaped architecture, instead of indiscriminately distilling all feature maps.

[0091] Knowledge distillation traditionally focuses on transferring information from the output layers of the teacher model to the student model. However, the intermediate layers often contain rich feature representations that can significantly benefit the student's learning process. By extracting knowledge from these hidden layers, our invention can capture a wider range of representations learned by the teacher, including low-level features and high-level abstractions. This approach allows for a more comprehensive transfer of knowledge, potentially leading to better performing and more efficient student models.

[0092] Given an image x, a lightweight student network S, and a high-performance teacher network T (such as TransUnet), both generate predictions y of the same size. As with the normal segmentation network, each feature layer from x to the final prediction output layer y must be processed by convolution, pooling, and activation. Due to different models, different hidden layers F are obtained. The process of the normal model is as follows:

[0093] F1(x)=g1·x,

[0094] F2(x)=g2·F1(x)=g2·g1·.x,

[0095] F3(x)=g3·F2(x),

[0096] ......,

[0097] F m (x) = g m ·F m-1 (x),

[0098] y=c·(F m (x)).

[0099] Unlike applying knowledge distillation at all stages of the student feature map, the present invention believes that based on the superiority of the U-shaped structure, it is better to specifically distill more important feature maps. In addition, in the field of medical image segmentation, researchers are very interested in the U-shaped structure because of its scalability, high-precision segmentation capabilities and high training efficiency. The advantages of the U-shaped structure, such as upsampling (from the decoder), alleviate the problem of feature reduction in the U-shaped model from downsampling (from the encoder). The jump connection module ensures that the decoder's feature map retains more important information from the encoder.

[0100] The main innovative contribution of the present invention is the KD (knowledge distillation) framework for the U-shaped structure. Inspired by the recent progress in efficient knowledge distillation and mask-based distillation techniques, the present invention proposes a novel KD framework specifically for the U-shaped variant model used in medical semantic segmentation or medical object detection. Different from the traditional KD method that focuses on distilling each feature of the student model, the framework of the present invention strategically targets the decoder features, such as Figure 3 shown.

[0101] The decoder KD framework of the present invention adopts specialized distillation modules at key positions in the decoder, reflecting the structural importance of these features in the U-shaped model. This approach not only improves the efficiency of knowledge transfer, but also preserves key spatial and contextual information inherent in medical image analysis tasks. Summary The present invention proposes a novel distillation framework specifically for U-shaped architecture models, focusing on the selective distillation of encoder or decoder features. By selectively transferring task-specific knowledge, the framework of the present invention enhances the efficiency and performance of the U-shaped model in tasks such as image segmentation and medical image analysis. Experimental results demonstrate that the method of the present invention reduces computational complexity while achieving excellent performance, highlighting its potential for practical applications.

[0102] 1. Normal distillation. Like all previously proposed relation-based distillation methods, all feature layers of the student model are transferred as feature layers of the teacher. As a normal KD framework, the distillation loss penalty function of the entire U-Net network is, as shown in the following formula, where m1 is the number of feature maps in the encoder and m2 is the number of feature maps in the decoder.

[0103]

[0104] 2. Encoder distillation. Based on the downsampling of the U-type model, the customized EncoderKD only distills the features of the encoder in the U-type model, and its loss function is shown as follows:

[0105]

[0106] 3. Decoder distillation. Based on the upsampling and skip connection features of the U-shaped model, this paper focuses on the decoder with global vision and cross-stage features. The present invention only distills the features of the decoder in (c) as follows:

[0107]

[0108] In this paper, the residual fusion selection module (ResFuSe) is a key component, which consists of a fusion feature module, a feature selection module, and a residual gating module. The ResFuSe module proposed in this paper fuses cross-stage features and uses a residual gating mechanism to maintain aggregated cross-stage features for relation-based distillation. Fusion cross features: This module obtains preliminary fusion features by fusing the student features of the two stages, which are then processed by the selective fusion module. It mainly consists of a global pooling layer f gp and two pairs of fully connected layers f fc and activation layer f a composition:

[0109]

[0110] In the formula, f a is the activation layer; y fuse for Intermediate features after processing by pooling and activation layers; It is the student feature that has been convolved with the previous student feature to facilitate the fusion of the current student feature.

[0111] Feature selection module: After the module obtains the processed compact features, it is processed into two channels by softmax, and the number of channels corresponds to the channel soft attention of C2. Neutralization The weight in .

[0112]

[0113] In the formula, a c 、b c are all intermediate characters; A and B are two learnable parameter matrices; e A*yfuse and e B*yfuse All of them are matrix point multiplication y fuse The obtained value is processed by the exponential function e to calculate a c With b c The relative weight of two values ​​to facilitate matrix update parameters.

[0114] The performance of the ResFuSe method was compared with that of various previously published KD methods on the Synapse validation set. The ResFuSe method of the present invention improved by +5.75%, +1.57%, +1.71% and +0.83% on the +response (response-based knowledge distillation) module, +Feature (feature-based knowledge distillation) module, +ABF (relationship-based feature distillation) and +CSF (relationship-based feature distillation), respectively. Figure 4 As shown in Table 1, when comparing various types of KD methods, the required computational resources are negligible compared to the effect improvement brought by ResFuSe. The results show that the ResFuSe of the present invention can better capture the decoder characteristics and make the student model more learning-capable.

[0115] Table 1: Computational resources for various types of KD methods

[0116] Model(+KDModule) AverageDSC(↑) RV Myo LV Teacher(TransUnet5) 95.06 94.67 88.94 96.73 Student 94.05 93.47 86.59 96.33 +KD 94.03 93.10 87.64 95.58 +Feature 94.66 94.45 88.18 96.15 +ABF 94.61 94.51 88.10 95.97 +CSF 94.54 94.17 88.05 96.11 +ResFuSe(Ours) 94.90(+0.85) 94.60 88.60 96.53

[0117] Table 2 shows the performance of the baseline student model and the student model with the KD module on the ACDC dataset. In addition to the average Dice score, the present invention also examines the performance of the model on three different MRI scanners (magnetic resonance scanners) in the ACDC dataset, namely right ventricle (RV), myocardium (Myo), and left ventricle (LV). The ResFuSeKD module of the present invention achieved the best result with a Dice score of 94.90%.

[0118] The ResFuSe module achieved the best performance in all categories, with scores of 94.60%, 88.60% and 96.53% for RV, Myo and LV respectively. Compared with the CSF of TransKD, the KD module of the present invention improved by +0.43%, +0.55% and +0.42% in the categories respectively, and the average performance was improved by +0.36%. Although there is still a gap in segmentation accuracy compared with large-volume models such as TransUnet, this is enough to demonstrate the efficiency of the distillation method of the present invention. It can also be seen from the pre-classification accuracy analysis that the ResFuse module proposed in the present invention is sensitive to small area features.

[0119] Table 2: Effect of KD module in U-type model

[0120]

[0121]

[0122] The KD module effect in the U-type model is Figure 5As shown in the table below, the present invention provides a powerful comparison between EncodeKD, DecodeKD and NormalKD (referred to as EncodeKD+DecodeKD in the present invention), showing the performance of Dice score, average Hausdorff distance and number of model parameters. In most cases, the present invention finds that the KD method using NormalKD obtains better performance, while the performance of DecodeKD is better than EncodeKD.

[0123] However, when the present invention compares DecodeKD (decoder knowledge distillation) and NormalKD (normal knowledge distillation) by performance and number of parameters, the present invention finds that improving model performance is not worth paying more parameters. It is worth noting that the ResFuSe method used in DecodeKD by the present invention achieves the best Dice score of 78.01%, an average Hausdorff distance of 25.59 mm, and saves a large number of parameters compared to other NormalKD frameworks. Methods that consume more parameters did not achieve the expected results. All NormalKDs are in an upper limit state, and the present invention believes that simply distilling all stage features of the student model will reduce the performance of the model.

[0124] Table 3: Overall experimental results of the U-type model

[0125]

[0126] from Figure 6As shown in Table 3, the overall experimental results show that the effects are NormalKD, DecodeKD and EncodeKD from best to worst. This is also called model performance is better with more model parameters. But in fact, the present invention obtains the same results on the ACDC dataset (cardiac image dataset, usually used for cardiac MRI image segmentation tasks) and the Synapse dataset (a medical image dataset, usually used for brain structure segmentation tasks), that is, the parameters used in the NormalKD framework to improve model performance are very wasteful. Specifically, the present invention compares the U-shaped structure KD in DecodeKD and NormalKD, in which the number of parameters is reduced by 1.10M on average, and the four KD methods (+feature, +ABF, +CSF, +ResFuSe) applied in the former have performance changes of +0.01%, +0.15%, -0.12%, and +0.26% respectively compared with the latter. The research results also show that the ResFuSe method proposed in the present invention obtains the best results in the DecodeKD framework, with a Dice score of 94.90%. The experimental results of the present invention on the ACDC dataset also verify the huge waste caused by the normal structure KD method, and strongly demonstrate the effectiveness and feasibility of the U-shaped structure distillation framework provided by the present invention for decoder features.

[0127] like Figure 7 As shown in Figure 1, the overall framework of ResFuSe proposed in this invention is shown in Figure 2. It consists of two parts: decoder distillation and feature distillation branches. In each branch, ResFuSe converts F i s and the next stage of branching Fusion And this feature will learn the teacher feature The high-dimensional knowledge contained in it will be used as F i s-1 Output to the next ResFuSe module.

[0128] like Figure 8 As shown in Figure 2, the architecture of the residual fusion selection (ResFuSe) module is introduced. The branch contains the feature fusion module (ResFuSe) and the loss function (HCL).

[0129] (i) The ResFuSe module consists of a feature fusion module, a feature selection module, and a residual gating module. This module integrates the features of the two branches and is used to enhance the learning ability of the student hidden layer.

[0130] (ii) The loss function (HCL) pyramids the student and teacher features to extract different contextual information.

[0131] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig. 9 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store static information and dynamic information data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps in the above method embodiment are implemented.

[0132] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0133] In addition, the present invention also provides a computer device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiment when executing the computer program.

[0134] In addition, the present invention further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiment are implemented.

[0135] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0136] The present invention is not limited to the structures which have been described above and shown in the drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A knowledge distillation method for a medical image segmentation model, characterized in that: include: Based on the original architecture of the U-shaped image segmentation model, a dedicated distillation branch is introduced in the intermediate decoder layer to build a knowledge distillation framework. By selectively transferring task-specific knowledge, selective distillation between the encoder and decoder is achieved. A residual fusion selection module is constructed to integrate the features of different distillation-dedicated branches. Based on the residual fusion selection method, multi-stage student features are fused to match the dimensions of teacher features and transfer cross-stage features from the teacher network to the student network.

2. The medical image segmentation model knowledge distillation method according to claim 1, characterized in that: The original architecture based on the U-shaped image segmentation model introduces a dedicated distillation branch in the intermediate decoder layer to build a knowledge distillation framework. By selectively transferring task-specific knowledge, selective distillation between the encoder and the decoder is achieved, including: In the original architecture of the U-shaped image segmentation model, the decoder distillation and the distillation branch in the feature are introduced to form a two-part knowledge distillation framework, and the loss function of the decoder distillation is established to extract different contextual information in medical images; Based on the upsampling and skip connection features of the U-shaped image segmentation model, for the encoder with global vision and cross-stage features, selective distillation is performed between the encoder and decoder by selectively transferring task-specific knowledge, thereby retaining the key spatial and contextual information inherent in medical image analysis tasks.

3. The medical image segmentation model knowledge distillation method according to claim 2, characterized in that: The expression of the loss function of the decoder distillation is: In the formula, Loss fm,Decode is the loss function of decoder distillation; m1 is the number of feature maps in the encoder; m2 is the number of feature maps in the decoder; The student model features are processed to adapt to the teacher model features; F t is the teacher model feature in the teacher model corresponding to the student model feature layer; i is the feature position corresponding to the current student and teacher.

4. The medical image segmentation model knowledge distillation method according to claim 1, characterized in that: The construction of the residual fusion selection module integrates the features of different distillation dedicated branches, fuses the student features of multiple stages based on the residual fusion selection method, matches the dimensions of the teacher features, and transfers the cross-stage features from the teacher network to the student network, including: Construct a residual fusion selection module consisting of a fusion feature module, a feature selection module, and a residual gating module, integrate the features of the decoder distillation in the knowledge distillation framework and the distillation branch in the feature, fuse the student features of the two stages, and obtain preliminary fusion features; After the preliminary processing, the fusion features are obtained, and based on the weighted selective fusion of softmax, different weights are assigned to each feature channel to achieve selective weighted fusion of features; The feature maps output by each stage of the knowledge distillation framework are obtained and divided into 4 levels of context information through adaptive average pooling. In each stage, the knowledge is distilled between the context levels using Euclidean distance, and each stage is distilled separately using Euclidean distance. The final hierarchical context loss between the feature maps of the nth stage is calculated, and the student features are matched to the teacher feature dimensions to transfer the cross-stage features from the teacher network to the student network.

5. The medical image segmentation model knowledge distillation method according to claim 4, characterized in that: The calculation formula for calculating the final hierarchical context loss between the feature maps of the nth stage is: In the formula, loss fm is the hierarchical context loss; l is the Euclidean distance; F out is the student model feature output after feature fusion processing; F t is the teacher model feature in the teacher model corresponding to the student model feature layer; i is the feature position corresponding to the current student and teacher.

6. A medical image segmentation model knowledge distillation system, characterized in that: include: The knowledge distillation framework module is used to introduce a dedicated distillation branch in the intermediate decoder layer based on the original architecture of the U-shaped image segmentation model to build a knowledge distillation framework. It achieves selective distillation between the encoder and decoder by selectively transferring task-specific knowledge. The residual fusion selection module is used to integrate the features of different distillation-dedicated branches. Based on the residual fusion selection method, it fuses the student features of multiple stages, matches the dimensions of the teacher features, and transfers the cross-stage features from the teacher network to the student network.

7. The medical image segmentation model knowledge distillation system according to claim 6, characterized in that: The knowledge distillation framework module includes: a branch construction module and a selective distillation module, wherein: The branch construction module is used to introduce the distillation branch in the decoder and the feature into the original architecture of the U-shaped image segmentation model to form a two-part knowledge distillation framework, and establish a loss function for the decoder distillation to extract different contextual information in the medical image; The selective distillation module is used to perform selective distillation between the encoder and the decoder based on the upsampling and skip connection features of the U-shaped image segmentation model, for an encoder with global vision and cross-stage features, by selectively transferring task-specific knowledge, thereby retaining the key spatial and contextual information inherent in the medical image analysis task.

8. The medical image segmentation model knowledge distillation system according to claim 7, characterized in that: The expression of the loss function of the decoder distillation is: In the formula, Loss fm,Decode is the loss function of decoder distillation; m1 is the number of feature maps in the encoder; m2 is the number of feature maps in the decoder; The student model features are processed to adapt to the teacher model features; F t is the teacher model feature in the teacher model corresponding to the student model feature layer; i is the feature position corresponding to the current student and teacher.

9. The medical image segmentation model knowledge distillation system according to claim 6, characterized in that: The residual fusion selection module includes: a fusion feature module, a feature selection module and a residual gating module, wherein: The fusion feature module is used to integrate the features of the decoder distillation and the distillation branch in the feature in the knowledge distillation framework, fuse the student features of the two stages, and obtain preliminary fusion features; The feature selection module is used to obtain the fusion features after preliminary processing, and to assign different weights to each feature channel based on the weighted selective fusion of softmax to achieve selective weighted fusion of features; The residual gating module is used to obtain the feature map output by each stage of the knowledge distillation framework, which is divided into 4 levels of context information through adaptive average pooling. In each stage, the knowledge is distilled between the context levels using the Euclidean distance, and each stage is distilled separately using the Euclidean distance. The final hierarchical context loss between the feature maps of the nth stage is calculated, and the student features are matched to the teacher feature dimensions to transfer the cross-stage features from the teacher network to the student network.

10. The medical image segmentation model knowledge distillation system according to claim 9, characterized in that: The calculation formula for calculating the final hierarchical context loss between the feature maps of the nth stage is: In the formula, loss fm is the hierarchical context loss; l is the Euclidean distance; F out is the student model feature output after feature fusion processing; F t is the teacher model feature in the teacher model corresponding to the student model feature layer; i is the feature position corresponding to the current student and teacher.

Images