A medical image segmentation method, system, device and computer readable medium
By introducing self-distillation learning and multi-scale feature extraction modules into the encoder-decoder network, the problems of insufficient network depth and variable target scale in medical image segmentation are solved, thereby achieving improved accuracy and parameter control.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SOUTH CHINA UNIV OF TECH
- Filing Date
- 2022-03-31
- Publication Date
- 2026-05-12
AI Technical Summary
Existing medical image segmentation methods suffer from shallow network depth and variable target scale, making it difficult to improve segmentation accuracy and increasing the number of additional parameters and training difficulty.
A self-distillation learning mechanism is adopted to perform self-distillation learning in the encoder-decoder network structure. Combined with a multi-scale feature extraction module, the dense imitation loss and target perception loss are calculated through self-distillation learning to optimize the network parameters.
Without adding additional parameters, the performance of the medical image segmentation network was improved, and the segmentation accuracy and multi-scale feature extraction capabilities were enhanced.
Smart Images
Figure CN115239950B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of medical image processing technology, specifically relating to a medical image segmentation method, system, device, and computer-readable medium. Background Technology
[0002] Medical images are crucial for doctors to assess patient conditions and determine treatment plans. When observing medical images, lesions and medical targets are the primary focus. However, medical image observation is challenging, time-consuming, and labor-intensive, relying heavily on specialized medical knowledge, and doctors' professional experience varies from person to person. Automated medical image segmentation can objectively and quickly extract the desired targets from medical images, alleviating the diagnostic burden on doctors to some extent and reducing missed diagnoses and misdiagnoses caused by subjective factors. Traditional medical image segmentation methods have poor performance and low generalization ability. In recent years, the application of deep learning models has significantly improved the accuracy of medical image segmentation. Among them, the encoder-decoder structure represented by U-Net has become the mainstream solution for medical image segmentation. However, although methods like U-Net have great universality, their accuracy is limited by the inherent low precision, low contrast, large image differences, low network depth, and variable target scale of medical images, making further improvements difficult.
[0003] To address the challenges of shallow network depth and variable target scales in deep learning-based medical image segmentation methods, existing methods have attempted various approaches. However, these often introduce a massive amount of additional parameters, and performance improvement remains limited. For instance, using deeper, more powerful base networks can improve segmentation performance to some extent, but these networks typically have enormous parameters, making training difficult and placing a heavy burden on equipment. Many existing multi-scale methods also extract multi-scale features through additional modules, which usually have multiple branches and introduce a large number of additional parameters. For example, the pyramid pooling module in PSPNet pools input features to multiple scales, extracts features separately, and then fuses them to improve adaptability to multi-scale targets. The DeepLab series of networks extracts information at multiple scales from a fixed input scale using dilated convolutions. Dilated convolutions expand the spacing between convolutional kernel parameters, thus increasing the kernel size and covering a larger data range. However, all of these operations are performed on complete input features, and such structures typically have multiple branches, meaning the number of parameters in a single multi-scale module is several times that of a typical convolutional layer. Summary of the Invention
[0004] To address the shortcomings of existing technologies, this invention provides a medical image segmentation method, system, device, and computer-readable medium that can effectively improve the performance of medical image segmentation networks without increasing or significantly increasing parameters.
[0005] Firstly, a medical image segmentation method includes:
[0006] Preprocess the raw medical images;
[0007] The preprocessed medical images are input into an encoder-decoder network structure to extract image features;
[0008] The image features output by each decoder stage in the encoder-decoder network structure are processed so that the image features of any decoder stage other than the deepest decoder stage are self-distilled and learned from the image features of all deeper decoder stages, and the dense imitation loss is calculated.
[0009] The prediction result is obtained by combining the image features of the output layer in the encoder-decoder network structure with the corresponding activation function;
[0010] The loss is calculated based on the prediction results and the target mask.
[0011] The network parameters of the encoder-decoder network structure are updated using the backpropagation algorithm.
[0012] As a preferred embodiment, the image features output by each decoder stage in the encoder-decoder network structure are processed so that the image features of any decoder stage other than the deepest decoder stage undergo self-distillation learning with the image features of all deeper decoder stages, and the dense imitation loss is calculated, including the following steps:
[0013] Obtain the image features output at each decoder stage;
[0014] Based on the image features output by each decoder stage, a corresponding feature map is obtained;
[0015] Between any two feature maps in all decoder stages, the feature map corresponding to the deeper decoder stage is used as the label, and the imitation loss is calculated according to the loss function; all imitation losses are added together to obtain the dense imitation loss.
[0016] As a preferred approach, feature maps corresponding to at least a portion of the decoder stages are selected, and loss calculations are performed on them separately with the real labels of the input medical images. All results are then summed to obtain the target perception loss.
[0017] As a preferred embodiment, the feature map used for the target perception loss adopts the feature map corresponding to a deeper partial decoder stage.
[0018] As a preferred embodiment, the preprocessing includes at least one of data augmentation, controlling the input range, and controlling the distribution of input data.
[0019] As a preferred embodiment, the decoder stage includes a multi-scale feature extraction process consisting of multiple consecutively performed multi-scale feature extraction methods, wherein the multi-scale feature extraction method includes the following steps:
[0020] Channel splitting: The input image features are divided into multiple groups, with each group of image features being input into a different branch;
[0021] Branch processing: The image features input into each branch are processed by convolutional feature extraction at different scales while keeping the number of feature channels constant. Different branches have different pooling sizes.
[0022] Channel stitching: Image features output from different branches are stitched together according to channels to obtain stitched image features, wherein the stitched image features have the same number of channels as the input image features;
[0023] The stitched image features are added to the input image features channel by channel and pixel by pixel to obtain the output features, which are then passed on to the next stage.
[0024] As a preferred embodiment, after passing it backward, the method further includes:
[0025] Channel obfuscation: The output features are rearranged in a way that shuffles the channel order.
[0026] Secondly, a medical image segmentation device includes:
[0027] The data preprocessing module is used to preprocess the raw medical images;
[0028] The feature extraction module is used to input the preprocessed medical image into the encoder-decoder network structure to extract image features;
[0029] The self-distillation learning module is used to process the image features output by each decoder stage in the encoder-decoder network structure, so that the image features of any decoder stage other than the deepest decoder stage are self-distilled to the image features of all deeper decoder stages, and the dense imitation loss is calculated.
[0030] The result prediction module is used to obtain the prediction result by combining the image features of the output layer in the encoder-decoder network structure with the corresponding activation function;
[0031] The loss calculation module is used to calculate the loss based on the prediction results and the target mask.
[0032] The backpropagation module is used to update the network parameters of the encoder-decoder network structure using the backpropagation algorithm.
[0033] As a preferred embodiment, the self-distillation learning module includes:
[0034] The feature acquisition module is used to acquire the image features output by each decoder stage;
[0035] The feature map acquisition module is used to obtain the corresponding feature map based on the image features output by each decoder stage;
[0036] The dense imitation loss module is used to calculate the imitation loss between any two feature maps in all decoder stages, using the feature map corresponding to the deeper decoder stage as the label, according to the loss function. All imitation losses are then added together to obtain the dense imitation loss.
[0037] As a preferred embodiment, the self-distillation learning module further includes:
[0038] The target perception loss module is used to select feature maps corresponding to at least some of the decoder stages, and perform loss calculations with the real labels of the input medical images respectively, and add all the results to obtain the target perception loss.
[0039] As a preferred embodiment, the decoder includes a multi-scale feature extraction module, which comprises a plurality of sequentially connected multi-scale feature extraction units; the multi-scale feature extraction unit includes:
[0040] The channel splitting module is used to divide the input image features into multiple groups, with each group of image features being input into a different branch;
[0041] The branch processing module is used to extract image features from each branch through convolutional features at different scales while keeping the number of feature channels constant during the process. Different branches have different pooling sizes.
[0042] The channel stitching module is used to stitch together image features from different branches according to channels to obtain stitched image features, wherein the stitched image features have the same number of channels as the input image features;
[0043] The output module is used to add the stitched image features to the input image features channel by channel and pixel by pixel to obtain the output features, and then pass them on.
[0044] As a preferred embodiment, the decoder further includes a channel obfuscation module located between adjacent multi-scale feature extraction units;
[0045] The channel obfuscation module is used to rearrange the output features in a way that shuffles the channel order.
[0046] Thirdly, a medical image segmentation device includes:
[0047] One or more processors;
[0048] A storage device on which one or more programs are stored;
[0049] When the one or more programs are executed by the one or more processors, the one or more processors implement a medical image segmentation method as described in any of the first aspects.
[0050] Fourthly, a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements a medical image segmentation method as described in any of the first aspects.
[0051] Compared with the prior art, the present invention has the following beneficial effects:
[0052] 1. By adding a self-distillation learning module to the encoder-decoder network structure, the segmentation performance of the network can be improved without adding additional parameters;
[0053] 2. By adding a multi-scale feature extraction module based on channel splitting to the decoder, effective and diverse multi-scale features can be extracted without significant increase in parameters, which can effectively improve the performance of medical image segmentation networks. Attached Figure Description
[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0055] Figure 1 This is a flowchart illustrating a medical image segmentation method according to the present invention;
[0056] Figure 2 This is a schematic diagram of the encoder-decoder network structure in this invention;
[0057] Figure 3 This is a schematic diagram of the self-distillation learning process in a medical image segmentation method of the present invention;
[0058] Figure 4 This is a flowchart illustrating the multi-scale feature extraction method in a medical image segmentation method according to the present invention.
[0059] Figure 5 This is a schematic diagram of the decoder structure in this invention;
[0060] Figure 6 This is a schematic diagram of the structure of the multi-scale feature extraction unit in this invention;
[0061] Figure 7 This is a schematic diagram of the structure of a medical image segmentation device according to the present invention;
[0062] Figure 8 This is a schematic diagram of the self-distillation learning module in a medical image segmentation device of the present invention. Detailed Implementation
[0063] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0064] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0065] Firstly, such as Figure 1 , Figure 2 As shown, a medical image segmentation method includes:
[0066] S101: Preprocess the raw medical images;
[0067] S102: Input the preprocessed medical image into the encoder-decoder network structure to extract image features;
[0068] S103: Process the image features output by each decoder stage in the encoder-decoder network structure so that the image features of any decoder stage other than the deepest decoder stage are self-distilled to the image features of all deeper decoder stages, and calculate the dense imitation loss.
[0069] S104: Based on the image features of the output layer in the encoder-decoder network structure and the corresponding activation function, the prediction result is obtained;
[0070] S105: Calculate the loss based on the prediction results and the target mask;
[0071] S106: Update the network parameters of the encoder-decoder network structure using the backpropagation algorithm.
[0072] In this embodiment, the medical image segmentation method mainly consists of three stages: data preprocessing, feature extraction, and result prediction. The feature extraction stage is based on the existing encoder-decoder network structure and is divided into encoder feature extraction and decoder feature extraction. The encoder feature extraction can be consistent with the basic structure in existing technologies; the main improvement lies in the decoder feature extraction part, which adds the calculation of dense imitation loss through self-distillation learning to improve the effectiveness of network parameters. The result prediction stage can be consistent with common deep learning models. After obtaining the image features output by the feature extraction stage, the corresponding activation function is used according to the number of categories of the segmentation target. For single-class prediction, the Sigmoid function can be used; for multi-class prediction, the Softmax function can be used, thereby obtaining the prediction result.
[0073] The preprocessing is used to transform the original medical image into an image suitable for deep learning. This preprocessing may include data augmentation, controlling the input range, and controlling the input data distribution. Specifically, data augmentation may use operations such as rotation, cropping, and translation; controlling the input range may use minimax normalization; and controlling the input data distribution may use standardization. The encoder-decoder network structure may be a network based on encoder and decoder structures, such as U-net, V-net, or SegNet, or its derivatives. This embodiment uses the classic U-net structure as an example.
[0074] like Figure 2 As shown, the encoder-decoder network structure in medical image segmentation methods typically consists of a multi-stage encoder and a multi-stage decoder. After inputting a medical image, each stage of the encoder uses convolutional layers to extract features from the input. At the end of each stage (except the last one), pooling is used to reduce the image size before inputting it into the next stage. The decoder works similarly, using convolutional layers to extract features from the input at each stage, and then upsampling is used at the end of each stage (except the last one) to enlarge the image size. The image sizes of each stage in the encoder and decoder are corresponding. U-Net fuses features of the same size into the corresponding stage of the decoder through skip connections, typically using channel concatenation to supplement the decoder with detailed image features. In this invention, when choosing U-Net as the base scheme, the fusion method can be element-wise addition, that is, adding the feature values at each channel and pixel position, instead of using channel concatenation. This reduces the number of channels and parameters. When choosing other encoder-decoder structures without skip connections, the fusion method is not considered; the structure remains consistent with the original structure.
[0075] Self-distillation is a special form of knowledge distillation. Knowledge distillation typically occurs between a large model and a small model, aiming to transfer knowledge from the large model to the small model. Self-distillation occurs within a model, where one part of the network learns from another. In deep learning models, this usually involves learning features from shallow layers to deeper layers. In this embodiment, it refers to learning image features output from a shallower decoder stage to image features output from a deeper decoder stage. The self-distillation learning process yields a mimicry loss, and the sum of all mimicry losses obtained from self-distillation learning constitutes the dense mimicry loss. In existing encoder-decoder network structures, after predicting the segmentation target, the loss is calculated using the prediction result and the target mask according to a given loss function. This loss can then be used to update network parameters via backpropagation. In the encoder-decoder network structure of this embodiment, after predicting the segmentation target, the loss and the dense mimicry loss are used together to update network parameters. Specifically, the loss and the dense mimicry loss can be added together before backpropagation. Furthermore, since the image features generated by different decoder stages have different sizes, when performing dense feature mimicry, the mimicked features are transformed to the size of the mimicked features through bilinear interpolation.
[0076] This embodiment employs a self-distillation learning mechanism to enhance the feature representation of shallow layers in a deep neural network. Simultaneously, it strengthens this learning process through dense connections and enhances the accuracy of the shallow feature learning target through objective supervision. This structure introduces no additional parameters; only the additional network loss needs to be calculated.
[0077] A preferred embodiment, such as Figure 3 As shown, the image features output by each decoder stage in the encoder-decoder network structure are processed so that the image features of any decoder stage other than the deepest decoder stage undergo self-distillation learning with the image features of all deeper decoder stages, and the dense imitation loss is calculated. This includes the following steps:
[0078] S201: Obtain the image features output by each decoder stage.
[0079] The image features are derived from the end of each decoder stage.
[0080] S202: Obtain the corresponding feature map based on the image features output by each decoder stage.
[0081] In this step, the feature map with 1 channel is obtained by summing the squared feature values of all channels by utilizing the features of each pixel position in the image features of each stage.
[0082] S203: Between any two feature maps in all decoder stages, using the feature map corresponding to the deeper decoder stage as the label, calculate the imitation loss according to the loss function; sum all the imitation losses to obtain the dense imitation loss.
[0083] The loss function can be selected from various types of distance loss functions, such as mean square loss function, cosine distance loss function, etc.
[0084] A preferred embodiment, such as Figure 3 As shown, it also includes:
[0085] S204: Select feature maps corresponding to at least a portion of the decoder stages, and calculate the loss for each with the ground truth label of the input medical image. Sum all results to obtain the target perception loss. Specifically, feature maps corresponding to deeper decoder stages may be preferred.
[0086] In this embodiment, the loss calculation can employ the binary cross-entropy loss function. Specifically, when the true label has more than two categories, all positive classes of the true label are treated as one class, and then the loss is calculated using the binary cross-entropy loss function with the feature map. The target perception loss is used to update the network parameters of the encoder-decoder network structure. The feature map used to calculate the target perception loss can correspond to all decoder stages or some decoder stages. In this embodiment, combining the target perception loss with the dense imitation loss from the previous embodiment to update network parameters yields better results.
[0087] A preferred embodiment, such as Figures 4-6 As shown, the decoder stage includes a multi-scale feature extraction process consisting of multiple consecutively performed multi-scale feature extraction methods. The multi-scale feature extraction methods include the following steps:
[0088] S301: Channel splitting: Divide the input image features into multiple groups, with each group of image features input into a different branch;
[0089] S302: Branch processing: The image features input into each branch are processed by convolutional feature extraction at different scales while keeping the number of feature channels unchanged. Different branches have different pooling sizes.
[0090] S303: Channel stitching: Image features output from different branches are stitched together according to channels to obtain stitched image features, wherein the stitched image features have the same number of channels as the input image features;
[0091] S304: The stitched image features are added to the input image features channel by channel and pixel by pixel to obtain the output features, and then passed on to the next stage.
[0092] In this embodiment, the branching process is as follows: Figure 6 As shown, a four-branch approach can be adopted. The first branch can directly perform convolutional feature extraction; the second branch can first undergo a 2x downsampling operation, then perform convolutional feature extraction, and then perform a 2x upsampling operation; the third branch can first undergo a 4x downsampling operation, then perform convolutional feature extraction, and then perform a 4x upsampling operation; the fourth branch can first undergo an 8x downsampling operation, then perform convolutional feature extraction, and then perform an 8x upsampling operation. The number of branches and the sampling multipliers can be flexibly set according to specific needs. Considering that each branch only uses a portion of the input channels, its feature extraction is incomplete. Therefore, the multi-scale feature extraction method is repeated multiple times in the multi-scale feature extraction process. The output feature in this multi-scale feature extraction method corresponds to the input image feature in the next multi-scale feature extraction method. This embodiment utilizes the principle of grouped convolution. By splitting the input image features into multiple feature groups, pooling downsampling, ordinary convolutional feature extraction, and upsampling are performed on each feature group to extract features. The results of multiple feature groups are then combined to obtain the multi-scale feature extraction result. Although this method of feature extraction adds an extra branch, the number of parameters in a 4-branch module is only 1 / 4 of that in a single ordinary convolutional layer, consistent with the number of parameters in a grouped convolution.
[0093] A preferred embodiment, such as Figure 4 , Figure 6 As shown, after step S304, the method further includes:
[0094] S305: Channel obfuscation: The output features are rearranged in a way that shuffles the channel order.
[0095] In this embodiment, a channel scrambling operation is used between every two adjacent multi-scale feature extraction methods. This involves rearranging the channels of the output features of the preceding multi-scale feature extraction method and the input features of the following method. During rearrangement, the channel dimension can be split into two dimensions, keeping the total number of channels the same. These two dimensions are then swapped, and finally merged into one dimension, thus shuffling the channel order. Other methods can also be used, as long as the order is shuffled while maintaining the property that deep neural networks can perform backpropagation. After rearrangement, the input features of corresponding branches in the preceding and following multi-scale feature extraction methods will differ. Repeated scrambling allows a particular branch to perceive more input channels, thereby maintaining the effectiveness of the extracted multi-scale features.
[0096] Secondly, such as Figure 7 As shown, a medical image segmentation device includes:
[0097] The data preprocessing module is used to preprocess the raw medical images;
[0098] The feature extraction module is used to input the preprocessed medical image into the encoder-decoder network structure to extract image features;
[0099] The self-distillation learning module is used to process the image features output by each decoder stage in the encoder-decoder network structure, so that the image features of any decoder stage other than the deepest decoder stage are self-distilled to the image features of all deeper decoder stages, and the dense imitation loss is calculated.
[0100] The result prediction module is used to obtain the prediction result by combining the image features of the output layer in the encoder-decoder network structure with the corresponding activation function;
[0101] The loss calculation module is used to calculate the loss based on the prediction results and the target mask.
[0102] The backpropagation module is used to update the network parameters of the encoder-decoder network structure using the backpropagation algorithm.
[0103] In this embodiment, the dense imitation loss obtained in the self-distillation learning module is the sum of the imitation losses obtained from all self-distillation learning processes in that module. The backpropagation module in this embodiment uses both the loss and the dense imitation loss to update the network parameters; specifically, the loss can be added to the dense imitation loss before backpropagation. This embodiment employs a self-distillation learning mechanism to enhance the feature representation of shallow layers in a deep neural network, while simultaneously strengthening this learning process through dense connections, thereby enhancing the accuracy of the shallow feature learning target through target supervision. This structure does not introduce any additional parameters; only additional network loss needs to be calculated. This embodiment, by adding a self-distillation learning module to the encoder-decoder network structure, can improve the network's segmentation performance without increasing the number of additional parameters.
[0104] A preferred embodiment, such as Figure 8 As shown, the self-distillation learning module includes:
[0105] The feature acquisition module is used to acquire the image features output by each decoder stage;
[0106] The feature map acquisition module is used to obtain the corresponding feature map based on the image features output by each decoder stage;
[0107] The dense imitation loss module is used to calculate the imitation loss between any two feature maps in all decoder stages, using the feature map corresponding to the deeper decoder stage as the label, according to the loss function. All imitation losses are then added together to obtain the dense imitation loss.
[0108] The loss function can be selected from various types of distance loss functions, such as mean square loss function, cosine distance loss function, etc.
[0109] A preferred embodiment, such as Figure 8 As shown, the self-distillation learning module further includes:
[0110] The target perception loss module is used to select feature maps corresponding to at least some of the decoder stages, and perform loss calculations with the real labels of the input medical images respectively, and add all the results to obtain the target perception loss.
[0111] In this embodiment, the loss calculation can employ the binary cross-entropy loss function. Specifically, when the true label has more than two categories, all positive classes of the true label are considered as one class, and then the loss is calculated using the binary cross-entropy loss function with the feature map. The target perception loss is used to update the network parameters of the encoder-decoder network structure. The feature map used to calculate the target perception loss can correspond to all decoder stages or some decoder stages. In this embodiment, the target perception loss and the dense imitation loss from the previous embodiment are used together to update the network parameters, resulting in better performance.
[0112] A preferred embodiment, such as Figure 5 , Figure 6 As shown, the decoder includes a multi-scale feature extraction module, which comprises multiple multi-scale feature extraction units connected in sequence; each multi-scale feature extraction unit includes:
[0113] The channel splitting module is used to divide the input image features into multiple groups, with each group of image features being input into a different branch;
[0114] The branch processing module is used to extract image features from each branch through convolutional features at different scales while keeping the number of feature channels constant during the process. Different branches have different pooling sizes.
[0115] The channel stitching module is used to stitch together image features from different branches according to channels to obtain stitched image features, wherein the stitched image features have the same number of channels as the input image features;
[0116] The output module is used to add the stitched image features to the input image features channel by channel and pixel by pixel to obtain the output features, and then pass them on.
[0117] In this embodiment, the output features of the multi-scale feature extraction unit correspond to the input image features in the next multi-scale feature extraction unit. This embodiment utilizes the principle of grouped convolution, splitting the input image features into multiple feature groups. Pooling downsampling, ordinary convolution feature extraction, and upsampling are performed on each feature group to extract features. The results of multiple feature groups are then combined to obtain the multi-scale feature extraction result. This embodiment, by adding a channel-splitting-based multi-scale feature extraction module to the decoder, extracts effective and diverse multi-scale features without a significant increase in parameters, effectively improving the performance of the medical image segmentation network.
[0118] A preferred embodiment, such as Figure 6 As shown, the decoder also includes a channel obfuscation module located between adjacent multi-scale feature extraction units;
[0119] The channel obfuscation module is used to rearrange the output features in a way that shuffles the channel order.
[0120] In this embodiment, a channel scrambling operation is used between every two adjacent multi-scale feature extraction units. This involves rearranging the channels of the output features of the preceding multi-scale feature extraction unit and the input features of the following multi-scale feature extraction unit. During rearrangement, the channel dimension can be split into two dimensions, keeping the total number of channels the same. These two dimensions are then swapped, and finally merged into one dimension, thus shuffling the channel order. Other methods can also be used, as long as the scrambling ensures that the deep neural network can perform backpropagation. After rearrangement, the input features of corresponding branches of the preceding and following multi-scale feature extraction units will become different. Repeated scrambling and rearranging allows a particular branch to perceive more input channels, thereby maintaining the effectiveness of the extracted multi-scale features.
[0121] Thirdly, a medical image segmentation device includes:
[0122] One or more processors;
[0123] A storage device on which one or more programs are stored;
[0124] When the one or more programs are executed by the one or more processors, the one or more processors implement a medical image segmentation method as described in any of the first aspects.
[0125] Fourthly, a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements a medical image segmentation method as described in any of the first aspects.
[0126] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.
Claims
1. A medical image segmentation method, characterized in that, include: Preprocess the raw medical images; The preprocessed medical images are input into an encoder-decoder network structure to extract image features; The image features output from each decoder stage in the encoder-decoder network structure are processed such that the image features from any decoder stage other than the deepest decoder stage undergo self-distillation learning with the image features from all deeper decoder stages, and a dense imitation loss is calculated. Specifically, this includes the following steps: obtaining the image features output from each decoder stage; obtaining the corresponding feature map based on the image features output from each decoder stage; calculating the imitation loss based on the loss function between any two feature maps from all decoder stages, using the feature map corresponding to the deeper decoder stage as a label; and summing all imitation losses to obtain the dense imitation loss. The decoder stage includes multiple consecutive multi-scale... A multi-scale feature extraction process composed of a multi-scale feature extraction method includes the following steps: Channel splitting: The input image features are divided into multiple groups, and each group of image features is input into a different branch; Branch processing: The image features input into each branch are processed by convolutional feature extraction at different scales while keeping the number of feature channels unchanged. Different branches have different pooling sizes; Channel concatenation: The image features output from different branches are concatenated according to channels to obtain concatenated image features. The concatenated image features have the same number of channels as the input image features; The concatenated image features are added to the input image features channel by channel and pixel by pixel to obtain output features, which are then passed forward. The prediction result is obtained by combining the image features of the output layer in the encoder-decoder network structure with the corresponding activation function; The loss is calculated based on the prediction results and the target mask. The network parameters of the encoder-decoder network structure are updated using the backpropagation algorithm.
2. The medical image segmentation method according to claim 1, characterized in that, Also includes: Select feature maps corresponding to at least some of the decoder stages, and calculate the loss with the real labels of the input medical images respectively. Add all the results to obtain the target perception loss.
3. The medical image segmentation method according to claim 2, characterized in that: The feature map used for the target perception loss is a feature map corresponding to a deeper partial decoder stage.
4. The medical image segmentation method according to claim 1, characterized in that: The preprocessing includes at least one of data augmentation, controlling the input range, and controlling the distribution of input data.
5. The medical image segmentation method according to claim 1, characterized in that: After that and passing it backward, it also includes: Channel obfuscation: The output features are rearranged in a way that shuffles the channel order.
6. A medical image segmentation device, characterized in that, include: The data preprocessing module is used to preprocess the raw medical images; The feature extraction module is used to input the preprocessed medical image into the encoder-decoder network structure to extract image features; The self-distillation learning module processes the image features output by each decoder stage in the encoder-decoder network structure, enabling image features from any decoder stage other than the deepest decoder stage to undergo self-distillation learning towards the image features of all deeper decoder stages, and calculates the dense imitation loss. Specifically, it includes the following steps: obtaining the image features output by each decoder stage; obtaining the corresponding feature map based on the image features output by each decoder stage; calculating the imitation loss based on the loss function between any two feature maps from all decoder stages, using the feature map corresponding to the deeper decoder stage as a label; and summing all imitation losses to obtain the dense imitation loss. The decoder stage comprises multiple consecutive... The multi-scale feature extraction process comprises the following steps: Channel splitting: The input image features are divided into multiple groups, and each group of image features is input into a different branch; Branch processing: The image features input into each branch are processed by convolutional feature extraction at different scales while maintaining the number of feature channels unchanged. Different branches have different pooling sizes; Channel concatenation: The image features output from different branches are concatenated according to channels to obtain concatenated image features. The concatenated image features have the same number of channels as the input image features; The concatenated image features are added to the input image features channel by channel and pixel by pixel to obtain output features, which are then passed forward. The result prediction module is used to obtain the prediction result by combining the image features of the output layer in the encoder-decoder network structure with the corresponding activation function; The loss calculation module is used to calculate the loss based on the prediction results and the target mask. The backpropagation module is used to update the network parameters of the encoder-decoder network structure using the backpropagation algorithm.
7. A medical image segmentation device according to claim 6, characterized in that: The self-distillation learning module includes: The feature acquisition module is used to acquire the image features output by each decoder stage; The feature map acquisition module is used to obtain the corresponding feature map based on the image features output by each decoder stage; The dense imitation loss module is used to calculate the imitation loss between any two feature maps in all decoder stages, using the feature map corresponding to the deeper decoder stage as the label, according to the loss function. All imitation losses are then added together to obtain the dense imitation loss.
8. The medical image segmentation device according to claim 7, characterized in that: The self-distillation learning module also includes: The target perception loss module is used to select feature maps corresponding to at least some of the decoder stages, and perform loss calculations with the real labels of the input medical images respectively, and add all the results to obtain the target perception loss.
9. A medical image segmentation device according to claim 6, characterized in that: The decoder includes a multi-scale feature extraction module, which comprises multiple sequentially connected multi-scale feature extraction units; each multi-scale feature extraction unit includes: The channel splitting module is used to divide the input image features into multiple groups, with each group of image features being input into a different branch; The branch processing module is used to extract image features from each branch through convolutional features at different scales while keeping the number of feature channels constant during the process. Different branches have different pooling sizes. The channel stitching module is used to stitch together image features from different branches according to channels to obtain stitched image features, wherein the stitched image features have the same number of channels as the input image features; The output module is used to add the stitched image features to the input image features channel by channel and pixel by pixel to obtain the output features, and then pass them on.
10. A medical image segmentation device according to claim 9, characterized in that: The decoder further includes a channel obfuscation module located between adjacent multi-scale feature extraction units; The channel obfuscation module is used to rearrange the output features in a way that shuffles the channel order.
11. A medical image segmentation device, comprising: One or more processors; A storage device on which one or more programs are stored; When the one or more programs are executed by the one or more processors, the one or more processors implement a medical image segmentation method as described in any one of claims 1-5.
12. A computer-readable medium having a computer program stored thereon, wherein, When the program is executed by the processor, it implements a medical image segmentation method as described in any one of claims 1-5.