Self-distillation and self-learning medical image segmentation method, device and storage medium

Abstract

The present application discloses a self-distillation and self-learning medical image segmentation method, device and storage medium, the method comprising: using a MOD framework to copy based on a target network model to obtain a teacher model, and using the target network model as a student model; obtaining an image to be processed and performing mask processing on the image to be processed to obtain a mask image; inputting the image to be processed into the teacher model to obtain a first output result output by the teacher model; inputting the mask image into an encoder of a student model to obtain a second output result output by the student model; using a MOD framework to perform distillation processing on the first output result and the second output result and to perform reconstruction processing on the second output result; determining a distillation loss value and a reconstruction loss value of the MOD framework; adjusting parameters of the student model according to the distillation loss value and the reconstruction loss value to obtain a medical image analysis model, and segmenting the tumor image to be segmented based on the medical image analysis model. The present application can improve the segmentation accuracy of the medical image analysis model.

Description

Technical Field

[0001] The present application relates to the field of medical images and artificial intelligence technology, and in particular to a self-distillation and self-learning medical image segmentation method, device and storage medium. Background Art

[0002] In the field of brain tumor image segmentation, although a lot of research work has been accumulated, existing segmentation methods still face major challenges. First, due to practical issues such as data privacy protection, high costs, and ethical norms, it has become extremely difficult to obtain large-scale, high-quality annotated medical imaging data. This dilemma seriously restricts the effectiveness of brain tumor segmentation algorithms that rely on supervised learning strategies. Secondly, compared with the large number of natural image datasets with a wide variety of types, medical image datasets are not only relatively small in overall size, but also have limited sample diversity. This limitation causes the model to show poor generalization ability when facing unseen brain tumor cases. In addition, in the task of three-dimensional medical image segmentation, the effective extraction of complex high-level semantic features is crucial for the accurate segmentation of the model, but existing methods often fail to fully capture and utilize these deep-level feature information, resulting in the relatively low segmentation accuracy of existing brain tumor segmentation models. Summary of the invention

[0003] The present application provides a self-distillation and self-learning medical image segmentation method, device and storage medium, which can improve the accuracy of medical image analysis model segmentation.

[0004] In a first aspect, the present application provides a self-distillation and self-learning medical image segmentation method, wherein a target network model is provided with a MOD framework, and the method comprises:

[0005] The teacher model is obtained by copying the target network model using the MOD framework, and the target network model is used as the student model.

[0006] Acquire an image to be processed, wherein the image to be processed is a sample tumor image;

[0007] Performing mask processing on the image to be processed to obtain a mask image;

[0008] Inputting the image to be processed into the teacher model to obtain a first output result output by the teacher model;

[0009] Input the mask image to the encoder of the student model to obtain a second output result output by the student model;

[0010] Inputting the first output result and the second output result into a MOD framework, and performing distillation processing on the first output result and the second output result using the MOD framework;

[0011] and simultaneously reconstructing the second output result using the MOD framework;

[0012] Determine the distillation loss and reconstruction loss values ​​of the MOD framework;

[0013] The parameters of the student model are adjusted according to the distillation loss value and the reconstruction loss value to obtain a medical image analysis model, and the tumor image to be segmented is segmented based on the medical image analysis model.

[0014] A further technical solution is that the MOD framework includes a dense predictor and an online marker, wherein the encoder of the teacher model is inserted with the online marker, and the encoder of the student model is inserted with the online marker and the dense predictor;

[0015] The first output result and the second output result are distilled using the MOD framework, including:

[0016] Performing distillation processing on the first output result and the second output result using an online tagger;

[0017] The second output result is reconstructed using the MOD framework, including:

[0018] And reconstructing the second output result using the dense predictor.

[0019] A further technical solution is that the first output result includes a first feature map and a second feature map, and the second output result includes a first mask feature map and a second mask feature map. After obtaining the image to be processed, the method includes:

[0020] Performing Gaussian blur processing on the image to be processed to obtain a first enhanced image;

[0021] Performing contrast processing on the image to be processed to obtain a second enhanced image;

[0022] The first enhanced image and the second enhanced image are respectively subjected to mask processing to obtain a first mask image and a second mask image;

[0023] The image to be processed is input into the teacher model to obtain the first output result of the teacher model, including:

[0024] Inputting the first enhanced image and the second enhanced image into the teacher model respectively, obtaining a first feature map corresponding to the first enhanced image and a second feature map corresponding to the second enhanced image;

[0025] The mask image is input into the encoder of the student model to obtain the second output result output by the student model, including:

[0026] The first mask image and the second mask image are respectively input into the student model to obtain a first mask feature map corresponding to the first mask image and a second mask feature map corresponding to the second enhanced image.

[0027] A further technical solution is to use the MOD framework to perform distillation processing on the first output result and the second output result, including:

[0028] Obtain patch tokens and classification tokens corresponding to the first feature map, the second feature map, the first mask feature map, and the second mask feature map, respectively;

[0029] Performing a first internal learning of the patch tokens of the first feature map and the patch tokens of the first mask feature map;

[0030] Performing a second internal learning on the patch tokens of the second feature map and the patch tokens of the second mask feature map;

[0031] Performing a first cross learning on the classification tokens of the first feature map and the classification tokens of the second mask feature map;

[0032] The classification tokens of the second feature map are subjected to a second cross learning with the classification tokens of the first mask feature map.

[0033] A further technical solution is to determine the distillation loss value of the MOD framework, including:

[0034] Determine the loss values ​​corresponding to the first internal learning, the second internal learning, the first cross learning, and the second cross learning respectively;

[0035] The loss values ​​of the first internal learning and the second internal learning are averaged to obtain an internal loss value;

[0036] The loss values ​​of the first cross learning and the second cross learning are averaged to obtain the cross loss value;

[0037] The internal loss value and the cross loss value are summed to obtain the distillation loss value of the MOD framework.

[0038] A further technical solution is to determine the loss values ​​corresponding to the first internal learning, the second internal learning, the first cross learning and the second cross learning, including:

[0039] Obtain patch token feature values ​​and classification token feature values ​​corresponding to the first feature map and the second feature map respectively;

[0040] and the patch token logarithm values ​​and the classification token logarithm values ​​corresponding to the first mask feature map and the second mask feature map respectively;

[0041] Determine a first internal learning loss value based on the patch token feature value of the first feature map and the patch token logarithm value of the first mask feature map;

[0042] determining a second internal learning loss value based on the patch token feature value of the second feature map and the patch token logarithm value of the second mask feature map;

[0043] Determine a loss value of a first cross learning based on the classification token feature value of the first feature map and the classification token logarithm value of the second mask feature map;

[0044] A loss value of the second cross learning is determined based on the classification token feature value of the second feature map and the classification token logarithm value of the first mask feature map.

[0045] A further technical solution is that after determining the distillation loss value and the reconstruction loss value of the MOD framework, the method further includes:

[0046] The second output result is input into the segmentation head of the student model, and the segmentation head is used to perform segmentation processing on the second feature map to obtain a segmentation result of the image to be processed;

[0047] Based on the segmentation result, determine the segmentation loss value;

[0048] Adjust the parameters of the student model according to the distillation loss value and the reconstruction loss value, including:

[0049] Adjust the parameters of the student model according to the segmentation loss value, distillation loss value, reconstruction loss value, preset control factor, and preset gradient operator.

[0050] Its further technical solution is that after adjusting the parameters of the student model according to the distillation loss value and the reconstruction loss value, the method also includes: updating the teacher model using a preset exponential moving average.

[0051] In a second aspect, the present application provides a self-distillation and self-learning medical image segmentation device, which includes a memory and a processor, the memory is used to store program data, and the processor is used to execute the program data to implement the steps of any of the above methods.

[0052] In a third aspect, the present application provides a computer-readable storage medium, which is used to store a computer program. When the computer program is executed by a processor, it is used to implement the steps of any of the above methods.

[0053] The beneficial effect of the present application is that, different from the prior art, the present application optimizes the knowledge distillation process between the teacher model and the student model by inserting a MOD framework with a self-distillation and self-modeling mechanism into the target network model. The teacher model is used to guide the learning of the student model through the MOD framework, so that the student model can learn richer and more advanced feature representations on a limited data set, thereby achieving better brain tumor image segmentation performance under a limited resource environment. And by masking the sample tumor image, the mask image is used to stimulate the student model to perform self-learning and feature optimization, and the output result of the student model after masking is further reconstructed through the MOD framework, so that the student model can reconstruct global and detailed image features based on limited and local feature information, which helps to strengthen the student model's understanding and extraction of complex semantic features, and adjust the parameters of the student model in combination with the distillation loss value and the reconstruction loss value to obtain a medical image analysis model with relatively high segmentation accuracy, that is, the medical image analysis model obtained after adjusting the student model parameters can more accurately depict the edge and internal structure of the tumor, thereby improving the technical problem of relatively low segmentation accuracy of the medical image segmentation model in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0055] Figure 1 It is a flowchart of the first embodiment of the training method provided by the present application;

[0056] Figure 2 is a flow chart of the second embodiment of the training method provided by the present application;

[0057] Figure 3 It is a structural schematic diagram of an embodiment of a computer-readable storage medium provided by the present application. DETAILED DESCRIPTION

[0058] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It will be understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only some but not all structures related to the present application are shown in the drawings. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0059] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0060] In the field of brain tumor image segmentation, although a lot of research work has been accumulated, existing segmentation methods still face major challenges. First, due to practical issues such as data privacy protection, high costs, and ethical norms, it has become extremely difficult to obtain large-scale, high-quality annotated medical imaging data. This dilemma seriously restricts the effectiveness of brain tumor segmentation algorithms that rely on supervised learning strategies. Secondly, compared with the large number of natural image datasets with a wide variety of types, medical image datasets are not only relatively small in overall size, but also have limited sample diversity. This limitation causes the model to show poor generalization ability when facing unseen brain tumor cases. In addition, in the task of three-dimensional medical image segmentation, the effective extraction of complex high-level semantic features is crucial for the accurate segmentation of the model, but existing methods often fail to fully capture and utilize these deep-level feature information, resulting in the relatively low segmentation accuracy of existing brain tumor segmentation models.

[0061] Therefore, in order to solve the technical problem of low segmentation accuracy of medical image segmentation models in the prior art, the present application provides a training method, wherein a network model is inserted with a MOD framework, and specific details are given in the following embodiments.

[0062] See also Figure 1 , Figure 1 : is a flow chart of the first embodiment of the training method provided by the present application. The method comprises:

[0063] Step 110: Use the MOD framework to copy the target network model to obtain the teacher model, and use the target network model as the student model.

[0064] Step 120: Acquire the image to be processed.

[0065] The image to be processed is a sample tumor image, such as a 3D brain tumor image.

[0066] Step 130: performing mask processing on the image to be processed to obtain a mask image.

[0067] Step 140: Input the image to be processed into the teacher model to obtain a first output result output by the teacher model.

[0068] Among them, the first output result can be a depth feature map corresponding to the image to be processed.

[0069] Step 150: Input the mask image to the encoder of the student model to obtain a second output result output by the student model.

[0070] Among them, the teacher model in this application is trained together with the student model, and the number of parameters of the teacher model is the same as that of the student model, that is, the student model is not a lightweight model, but is of the same order of magnitude as the teacher model.

[0071] Step 160: Input the first output result and the second output result into the MOD framework, and use the MOD framework to perform distillation processing on the first output result and the second output result.

[0072] Step 170: Reconstruct the second output result using the MOD framework.

[0073] The MOD framework includes a dense predictor and an online marker, the encoder of the teacher model is plugged with an online marker, and the encoder of the student model is plugged with an online marker and a dense predictor.

[0074] The Online Tokenizer uses self-distillation technology to demonstrate significant advantages in medical image analysis scenarios where labeled data is scarce. This technology helps the model efficiently learn high-level semantic features, especially during the training phase, which can enhance the model's core network structure (such as the student model)'s ability to understand and integrate complex feature representations, thereby comprehensively improving the model's basic performance.

[0075] The Dense Predictor focuses on deepening the model's extraction and analysis of pixel-level semantic features of 3D brain tumor images to achieve the goal of accurate segmentation. It follows the principle of mask image modeling and directly enhances the accuracy of segmentation results by optimizing the restoration and reconstruction of pixel information by the student model. Integrating the self-modeling mechanism into the dense predictor part of the student model can further improve the model's expression accuracy at the micro-pixel level, thereby achieving higher accuracy in fine-grained segmentation tasks.

[0076] Among them, the MOD framework aims to accelerate the training process in a limited data environment and improve the performance of the segmentation model. The MOD framework interacts with each other at different semantic levels through the collaborative work of online self-distillation and self-modeling, enabling the model to efficiently learn high-level semantic features and detailed pixel-level information during the training phase, thereby improving the accuracy and efficiency of the model for medical image segmentation tasks.

[0077] In addition, as a plug-in component, the MOD framework can provide flexibility and scalability without affecting the original computational efficiency of the model, enabling it to be easily integrated into various existing medical image segmentation frameworks, thereby improving the quality of 3D brain tumor segmentation in data-constrained environments.

[0078] Based on this, for step 160 and step 170, specifically, the first output result and the second output result may be distilled using the online marker, and the second output result may be reconstructed using the dense predictor.

[0079] Step 180: Determine the distillation loss value and the reconstruction loss value of the MOD framework.

[0080] Step 190: Adjust the parameters of the student model according to the distillation loss value and the reconstruction loss value to obtain a medical image analysis model, and segment the tumor image to be segmented based on the medical image analysis model.

[0081] For example, if the original parameter of the student model is θ, then the adjusted parameter θ' can be Among them, lr is the preset control factor, is the preset gradient operator.

[0082] Among them, the preset exponential moving average can be used to update the teacher model, and the progress of the student model can be indirectly judged by observing the performance maintenance and improvement of the teacher model after the update.

[0083] This embodiment optimizes the knowledge distillation process between the teacher model and the student model by inserting a MOD framework with self-distillation and self-modeling mechanisms into the target network model. The teacher model is used to guide the learning of the student model through the MOD framework, so that the student model can learn richer and more advanced feature representations on a limited data set, thereby achieving better brain tumor image segmentation performance under limited resource environments. And by masking the sample tumor image, the mask image is used to stimulate the student model to perform self-learning and feature optimization, and the output results of the student model after masking are further reconstructed through the MOD framework, so that the student model can reconstruct global and detailed image features based on limited and local feature information, which helps to strengthen the student model's understanding and extraction of complex semantic features, and adjust the parameters of the student model in combination with the distillation loss value and the reconstruction loss value to obtain a medical image analysis model with relatively high segmentation accuracy, that is, the medical image analysis model obtained after adjusting the student model parameters can more accurately depict the edge and internal structure of the tumor, thereby improving the technical problem of relatively low segmentation accuracy of the existing medical image segmentation model.

[0084] In addition, during the training phase, the MOD framework is used to reconstruct the output results of the student model, so that the student model not only imitates the prediction behavior of the teacher model during the training process, but also continuously optimizes its own output results through self-feedback and correction mechanisms, thereby improving the generalization performance of the model, accelerating the model convergence speed, reducing the time required for training, and improving training efficiency.

[0085] In addition, the present application adjusts the parameters of the student model according to the distillation loss value and the reconstruction loss value to obtain a medical image analysis model, that is, the medical image analysis model does not carry the MOD framework when it is applied. Based on this, the present application can not only solve the problems of high hardware requirements and excessive latency for deployed terminal devices caused by large memory usage and long inference time of the network model during the application process, but also avoid the problem of additional processing costs and computing resource usage caused by adding the MOD framework during application.

[0086] Refer to the flowchart of the second embodiment of the training method provided in this application. The method includes:

[0087] Step 210: Use the MOD framework to copy the target network model to obtain a teacher model, and use the target network model as a student model.

[0088] Step 220: Acquire the image to be processed.

[0089] The image to be processed is a tumor image.

[0090] Step 230: Perform Gaussian blur processing on the image to be processed to obtain a first enhanced image.

[0091] Step 240: performing contrast processing on the image to be processed to obtain a second enhanced image.

[0092] Step 250: performing mask processing on the first enhanced image and the second enhanced image respectively to obtain a first mask image and a second mask image.

[0093] For steps 220 to 250, please refer to Figure 2 , where Gaussian blur processing can be Figure 2 AUGv in the image, the first enhanced image is ν1, and the contrast processing can be Figure 2 AUGμ in the image, the second enhanced image is μ1, and the first mask image is Figure 2 ν2 in, the second mask image is Figure 2 μ2 in.

[0094] Step 260: Input the first enhanced image and the second enhanced image into the teacher model respectively to obtain a first feature map corresponding to the first enhanced image and a second feature map corresponding to the second enhanced image.

[0095] See also Figure 2 , input the first enhanced image ν1 and the second enhanced image μ1 into the teacher model respectively, and obtain the feature map Z t .

[0096] Among them, Z t The first feature map Z corresponding to the first enhanced image ν1 is included t1 , and the second feature map Z corresponding to the second enhanced image μ1 t2 .

[0097] Step 270: Input the first mask image and the second mask image into the student model respectively to obtain a first mask feature map corresponding to the first mask image and a second mask feature map corresponding to the second enhanced image.

[0098] Likewise, see Figure 2 , the first mask image ν2 and the second mask image μ2 are respectively input into the encoder of the student model to obtain the feature map Z s .

[0099] Among them, Z s The first feature map Z corresponding to the first mask image ν2 is included s1 , and the second feature map Z corresponding to the second mask image μ2 s2 .

[0100] Among them, the initial network parameters of the teacher model and the student model are obtained through full empty initialization.

[0101] Step 280: Input the first output result and the second output result into the MOD framework, and use the MOD framework to perform distillation processing on the first output result and the second output result.

[0102] The first output result includes the first feature map and the second feature map, and the second output result includes the first mask feature map and the second mask feature map. Step 280 may specifically include the following steps:

[0103] Step 281: Obtain patch tokens and classification tokens corresponding to the first feature map, the second feature map, the first mask feature map, and the second mask feature map, respectively.

[0104] Among them, Figure 2 As shown, the first feature map Z s1 The corresponding patch token is patch v1 , the classification token is cls v1 ;

[0105] The second feature map Z s2 The corresponding patch token is patch μ1 , the classification token is cls μ1 ;

[0106] The first mask feature map Z s1 The corresponding patch token is patch v2 , the classification token is cls v2 ;

[0107] The second mask feature map Z s2 The corresponding patch token is patch μ2 , the classification token is cls μ2 .

[0108] Step 282: Perform a first internal learning on the patch tokens between the first feature map and the first mask feature map.

[0109] like Figure 2 Medium patch v1 With patch v2 Internal learning between.

[0110] Step 283: Perform a second internal learning on the patch tokens between the second feature map and the second mask feature map.

[0111] like Figure 2 Medium patch μ1 With patch μ2 Internal learning between.

[0112] Step 284: Perform a first cross-learning on the classification tokens between the first feature map and the second mask feature map.

[0113] like Figure 2 Middlecls v1 With cls μ2 Cross-learning between them.

[0114] Step 285: Perform a second cross-learning on the classification tokens between the second feature map and the first mask feature map.

[0115] like Figure 2 Middlecls μ1 With cls v2 Cross-learning between them.

[0116] Step 290: Simultaneously, the MOD framework is used to reconstruct the second output result.

[0117] Step 300: Determine the distillation loss value and the reconstruction loss value of the MOD framework.

[0118] Among them, the distillation loss value of the MOD framework can be determined by the following process, please refer to the following for details:

[0119] 1) Determine the loss values ​​corresponding to the first internal learning, the second internal learning, the first cross learning, and the second cross learning.

[0120] Specifically, the loss values ​​corresponding to the first internal learning, the second internal learning, the first cross learning, and the second cross learning may be determined in the following manner.

[0121] 1-1) Obtain the patch token feature values ​​and the classification token feature values ​​corresponding to the first feature map and the second feature map respectively.

[0122] 1-2) as well as the patch token logarithm values ​​and classification token logarithm values ​​corresponding to the first mask feature map and the second mask feature map respectively.

[0123] 1-3) Determine the loss value of the first internal learning based on the patch token feature value of the first feature map and the patch token logarithm of the first mask feature map.

[0124] 1-4) Determine the loss value of the second internal learning based on the patch token feature value of the second feature map and the patch token logarithm of the second mask feature map.

[0125] For 1-3) and 1-4), the loss value of internal learning can refer to the following formula 1:

[0126]

[0127] Where N represents the total number of training data, i represents the i-th sample, and m i Indicates the mth i eigenvectors, T is the transpose, Patch token feature values ​​representing feature maps, Patch token pair values ​​representing the masked feature map.

[0128] Let μ i For patch v1 , Patch v1 and patch v2 Substitute into formula 1 for the first calculation to get the loss value of the first internal learning.

[0129] And let μ i For patch μ1 , Patch μ1 and patch μ2 Substitute into formula 1 for a second calculation to obtain the loss value of the second internal learning.

[0130] 1-5) Determine the loss value of the first cross-learning based on the classification token feature value of the first feature map and the classification token logarithm value of the second mask feature map.

[0131] 1-6) Determine the loss value of the second cross-learning based on the classification token feature value of the second feature map and the classification token logarithm value of the first mask feature map.

[0132] For 1-5) and 1-6), the loss value of cross-learning can refer to the following formula 2:

[0133]

[0134] Where N represents the total number of training data, i represents the i-th sample, and m i Indicates the mth i eigenvectors, T is the transpose, represents the classification token feature value of the feature map, Represents the classification token pair values ​​of the masked feature map.

[0135] Let v i for cls v1 , for cls μ2 , cls v1 and cls μ2 Substitute into formula 2 for the first calculation to obtain the loss value of the first cross-learning.

[0136] And let v i for cls μ1 , cls μ1 and cls v2 Substitute into formula 2 for a second calculation to obtain the loss value of the second cross-learning.

[0137] 2) The loss values ​​of the first internal learning and the second internal learning are averaged to obtain an internal loss value.

[0138] 3) The loss values ​​of the first cross-learning and the second cross-learning are averaged to obtain the cross-loss value.

[0139] For 2)-3), since the present application adopts two data enhancement methods to obtain two sets of enhanced images, namely μ and ν, it is necessary to use formula 1 to calculate the internal learning loss values ​​corresponding to the two sets of enhanced images μ and ν respectively and to find the average of the internal learning loss values ​​of the two sets of enhanced images as the final internal learning loss value.

[0140] Similarly, it is necessary to use Formula 2 to calculate the cross-learning loss values ​​corresponding to the two groups of enhanced images μ and ν respectively, and calculate the average of the cross-learning loss values ​​of the two groups of enhanced images as the final cross-learning loss value.

[0141] 4) Sum the internal loss value and the cross loss value to get the distillation loss value of the MOD framework.

[0142] Among them, the distillation loss value of the MOD framework can be expressed as follows:

[0143]

[0144] in, is the distillation loss value of the MOD framework, is the internal loss value, is the cross loss value, α and β are weighting parameters.

[0145] Among them, the reconstruction loss value can be calculated using the following formula 4:

[0146]

[0147] Among them, L in Formula 4 DP represents the reconstruction loss value, N represents the total number of training data, μ i represents the original image of the i-th sample, Represents the reconstructed image obtained by the student model.

[0148] By minimizing the loss function of Formula 4, the pixel-level semantic feature extraction capability of the dense predictor to refine the student model can be improved, thereby improving the accuracy of the medical image analysis model, i.e., the student model segmentation.

[0149] Step 310: Adjust the parameters of the student model according to the distillation loss value and the reconstruction loss value to obtain a medical image analysis model, and segment the tumor image to be segmented based on the medical image analysis model.

[0150] In some embodiments, the segmentation head loss of the student model can be further calculated based on the above embodiments, and then the parameters of the student model can be adjusted according to the loss value of the MOD framework and the segmentation head loss. For details, refer to the following:

[0151] Step 320: Input the second output result into the segmentation head of the student model, use the segmentation head to perform segmentation processing on the second feature map, and obtain the segmentation result of the image to be processed.

[0152] like Figure 2 As shown in s Input to the segmentation head of the student model.

[0153] Step 330: Determine a segmentation loss value based on the segmentation result.

[0154] Step 340: Adjust the parameters of the student model according to the segmentation loss value, the distillation loss value, the reconstruction loss value, the preset control factor, and the preset gradient operator.

[0155] For example, if the original parameter of the student model is θ, then the adjusted parameters θ',θ' can be Among them, lr is the control factor, is the gradient operator, is the loss value of the MOD framework, is the segmentation loss value.

[0156] Among them, the loss value of the MOD framework Including distillation loss value and reconstruction loss L DP .

[0157] Since the segmentation loss value is an important indicator to measure the performance of the model in the actual segmentation task, it directly reflects the accuracy of the model in the pixel-level classification task. In the field of medical image segmentation, the segmentation loss value can ensure that the model can focus on improving the task goal that is most concerned in practical applications while learning the knowledge of the teacher model - accurately segmenting specific areas in the image (such as brain tumor areas). Therefore, this embodiment adjusts the parameters of the student model by combining the segmentation loss value, distillation loss value, and reconstruction loss value of the student model, so that the student model can obtain more comprehensive feedback signals during the training process, which helps the model converge faster to a state that can simulate the behavior of the teacher model and complete the actual segmentation task well, thereby reducing training time and improving training efficiency.

[0158] In addition, the present application adjusts the parameters of the student model according to the distillation loss value and the reconstruction loss value to obtain a medical image analysis model, that is, the medical image analysis model does not carry the MOD framework when it is applied. Based on this, the present application can not only solve the problems of high hardware requirements and excessive latency for deployed terminal devices caused by large memory usage and long inference time of the network model during the application process, but also avoid the problem of additional processing costs and computing resource usage caused by adding the MOD framework during application.

[0159] The present application also provides a self-distillation and self-learning medical image segmentation device, which includes a memory and a processor, the memory is used to store program data, and the processor is used to execute the program data to implement the following method:

[0160] The teacher model is obtained by copying the target network model using the MOD framework, and the target network model is used as the student model.

[0161] Acquiring an image to be processed, wherein the image to be processed is a tumor image;

[0162] Performing mask processing on the image to be processed to obtain a mask image;

[0163] Inputting the image to be processed into the teacher model to obtain a first output result output by the teacher model;

[0164] Input the mask image to the encoder of the student model to obtain a second output result output by the student model;

[0165] Inputting the first output result and the second output result into a MOD framework, and performing distillation processing on the first output result and the second output result using the MOD framework;

[0166] and simultaneously reconstructing the second output result using the MOD framework;

[0167] Determine the distillation loss and reconstruction loss values ​​of the MOD framework;

[0168] The parameters of the student model are adjusted according to the distillation loss value and the reconstruction loss value to obtain a medical image analysis model, and the tumor image to be segmented is segmented based on the medical image analysis model.

[0169] It can be understood that the processor is also used to execute program data to implement the method of any of the above embodiments.

[0170] See also Figure 3 , Figure 3 The structure of an embodiment of a computer-readable storage medium provided by the present application is shown in FIG. 7 . The computer-readable storage medium 70 is used to store a computer program 71. When the computer program 71 is executed by a processor, it is used to implement the following method steps:

[0171] The teacher model is obtained by copying the target network model using the MOD framework, and the target network model is used as the student model.

[0172] Acquiring an image to be processed, wherein the image to be processed is a tumor image;

[0173] Performing mask processing on the image to be processed to obtain a mask image;

[0174] Inputting the image to be processed into the teacher model to obtain a first output result output by the teacher model;

[0175] Input the mask image to the encoder of the student model to obtain a second output result output by the student model;

[0176] Inputting the first output result and the second output result into a MOD framework, and performing distillation processing on the first output result and the second output result using the MOD framework;

[0177] and simultaneously reconstructing the second output result using the MOD framework;

[0178] Determine the distillation loss and reconstruction loss values ​​of the MOD framework;

[0179] The parameters of the student model are adjusted according to the distillation loss value and the reconstruction loss value to obtain a medical image analysis model, and the tumor image to be segmented is segmented based on the medical image analysis model.

[0180] It can be understood that when the computer program 71 is executed by the processor, it is also used to implement the technical solution of any embodiment of the present application.

[0181] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only illustrative, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as two units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0182] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on two network units. Some or all of the units may be selected according to actual needs to achieve the purpose of this embodiment.

[0183] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0184] If the integrated units in the above other embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0185] The above are only implementation methods of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A self-distillation and self-learning medical image segmentation method, characterized in that: The target network model is plugged with a MOD framework, and the method includes: The MOD framework is used to copy the target network model to obtain a teacher model, and the target network model is used as a student model; Acquiring an image to be processed, wherein the image to be processed is a sample tumor image; Performing mask processing on the image to be processed to obtain a mask image; Inputting the image to be processed into the teacher model to obtain a first output result output by the teacher model; Inputting the mask image into the encoder of the student model to obtain a second output result output by the student model; Inputting the first output result and the second output result into the MOD framework, and performing distillation processing on the first output result and the second output result by using the MOD framework; and simultaneously reconstructing the second output result using the MOD framework; Determining a distillation loss value and a reconstruction loss value of the MOD framework; Adjusting parameters of the student model according to the distillation loss value and the reconstruction loss value to obtain a medical image analysis model, and segmenting the tumor image to be segmented based on the medical image analysis model; The MOD framework includes a dense predictor and an online marker, wherein the encoder of the teacher model is inserted with the online marker, and the encoder of the student model is inserted with the online marker and the dense predictor; The using the MOD framework to perform distillation processing on the first output result and the second output result includes: Using the online marker to perform distillation processing on the first output result and the second output result; The reconstructing the second output result by using the MOD framework includes: And reconstructing the second output result using the dense predictor.

2. The self-distillation and self-learning medical image segmentation method according to claim 1, characterized in that: The first output result includes a first feature map and a second feature map, the second output result includes a first mask feature map and a second mask feature map, and after acquiring the image to be processed, the method includes: Performing Gaussian blur processing on the image to be processed to obtain a first enhanced image; Performing contrast processing on the image to be processed to obtain a second enhanced image; Performing mask processing on the first enhanced image and the second enhanced image respectively to obtain a first mask image and a second mask image; The step of inputting the image to be processed into the teacher model to obtain a first output result output by the teacher model includes: Inputting the first enhanced image and the second enhanced image into a teacher model respectively to obtain a first feature map corresponding to the first enhanced image and a second feature map corresponding to the second enhanced image; The step of inputting the mask image to an encoder of a student model to obtain a second output result output by the student model comprises: The first mask image and the second mask image are respectively input into the student model to obtain a first mask feature map corresponding to the first mask image and a second mask feature map corresponding to the second enhanced image.

3. The self-distillation and self-learning medical image segmentation method according to claim 2, characterized in that: The using the MOD framework to perform distillation processing on the first output result and the second output result includes: Obtain patch tokens and classification tokens corresponding to the first feature map, the second feature map, the first mask feature map, and the second mask feature map, respectively; Performing a first internal learning on the patch tokens of the first feature map and the patch tokens of the first mask feature map; Performing a second internal learning on the patch tokens of the second feature map and the patch tokens of the second mask feature map; Performing a first cross-learning on the classification tokens of the first feature map and the classification tokens of the second mask feature map; The classification tokens of the second feature map and the classification tokens of the first mask feature map are subjected to a second cross learning.

4. The self-distillation and self-learning medical image segmentation method according to claim 3, characterized in that: Determining the distillation loss value of the MOD framework includes: Determine loss values ​​corresponding to the first internal learning, the second internal learning, the first cross learning, and the second cross learning respectively; Averaging the loss values ​​of the first internal learning and the second internal learning to obtain an internal loss value; Averaging the loss values ​​of the first cross learning and the second cross learning to obtain a cross loss value; The internal loss value and the cross loss value are summed to obtain the distillation loss value of the MOD framework.

5. The self-distillation and self-learning medical image segmentation method according to claim 4, characterized in that: The determining the loss values ​​corresponding to the first internal learning, the second internal learning, the first cross learning, and the second cross learning respectively includes: Obtaining patch token feature values ​​and classification token feature values ​​corresponding to the first feature map and the second feature map respectively; and the patch token logarithm values ​​and the classification token logarithm values ​​corresponding to the first mask feature map and the second mask feature map respectively; Determine a loss value of the first internal learning based on the patch token feature value of the first feature map and the patch token logarithm value of the first mask feature map; Determine a loss value of the second internal learning based on the patch token feature value of the second feature map and the patch token logarithm value of the second mask feature map; Determine a loss value of the first cross learning based on the classification token feature value of the first feature map and the classification token logarithm value of the second mask feature map; The loss value of the second cross learning is determined based on the classification token feature value of the second feature map and the classification token logarithm value of the first mask feature map.

6. The self-distillation and self-learning medical image segmentation method according to claim 1, characterized in that: After adjusting the parameters of the student model according to the distillation loss value and the reconstruction loss value, the method further includes: updating the teacher model using a preset exponential moving average.

7. A self-distillation and self-learning medical image segmentation device, characterized in that: The self-distillation and self-learning medical image segmentation device includes a memory and a processor, the memory is used to store program data, and the processor is used to execute the program data to implement the self-distillation and self-learning medical image segmentation method as described in any one of claims 1-6.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program data, which, when executed by a processor, is used to implement the self-distillation and self-learning medical image segmentation method as described in any one of claims 1-6.

Images