Medical image segmentation system with mixed structure
By combining the hybrid structure of convolutional neural network, Transformer and efficient channel attention module, the shortcomings of existing medical image segmentation models in understanding the global context and handling long-distance dependency tasks are solved, and more efficient and accurate medical image segmentation effects are achieved.
Patent Information
- Application Number
- CN202411869376.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-17
- Publication Date
- 2025-04-11
AI Technical Summary
The existing medical image segmentation model has insufficient understanding of global context and handling long-distance dependency tasks, and has high computational complexity, resulting in memory and time consumption problems during large-scale data processing.
A mixed-structure medical image segmentation system is adopted, combining convolutional neural network, Transformer and high-efficiency channel attention module, and the model's performance in processing complex structures and details is improved through shallow feature extraction, deep feature processing and upsampling operations.
By introducing an efficient channel attention module, we can enhance the selectivity of the model for key features, improve segmentation accuracy and efficiency, reduce computational complexity, and achieve more efficient medical image segmentation.
Smart Images
Figure CN119810115A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the field of medical image segmentation, and in particular to a medical image segmentation system with a hybrid structure. Background Art
[0002] Medical image segmentation technology plays a vital role in the field of medical image analysis. It can separate specific areas in the image (such as tumors, organs, etc.) from the background. This technology is crucial for doctors to make accurate diagnoses and formulate treatment plans, and lays the foundation for subsequent image processing. With the rapid development of medical imaging technologies such as CT, MRI, and ultrasound, the number and complexity of images are increasing, which also drives the demand for efficient automatic segmentation algorithms.
[0003] Advances in deep learning, especially the use of convolutional neural networks (CNNs), have greatly improved the accuracy and efficiency of medical image segmentation. Models based on deep learning, such as U-Net, FCN, and TransUNet, have shown wide application value in a variety of medical image segmentation tasks. These models learn features by analyzing a large amount of labeled data to achieve accurate recognition and segmentation of target areas. In order to improve the segmentation effect, researchers have also developed a variety of enhancement techniques and post-processing methods. Despite certain progress, medical image segmentation still faces challenges in practical applications, such as differences in image quality between patients, the diversity of tumors, and the needs of different application scenarios. Therefore, developing a higher performance and more robust segmentation model has become a focus of medical image segmentation research, which is of great significance for improving the practical application effect of the model and supporting personalized medicine and precision treatment.
[0004] Currently, the U-architecture medical image segmentation framework, especially the CNN-based U-Net, has become mainstream due to its effectiveness in capturing local features. However, CNN has shortcomings in understanding the global context, especially when dealing with tasks that require long-distance dependencies. Visual Transformer is able to handle long-distance dependency problems through the self-attention mechanism, but its computational complexity is high, especially when dealing with large-scale data, which may lead to significant memory and time consumption. To address these challenges, more efficient architectures and hybrid models are explored to combine the advantages of convolution and Transformer and overcome their respective limitations. The medical image segmentation model composed of convolutional neural network (CNN) and Transformer is a representative example, but it is insufficient in utilizing channel features.
[0005] Therefore, a medical image segmentation system with a hybrid structure is desired, which combines the advantages of convolutional neural networks, Transformer, and efficient channel attention modules to form a hybrid model. Summary of the invention
[0006] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a medical image segmentation system with a hybrid structure, which maximizes the segmentation accuracy and provides an efficient method for the field of medical image segmentation.
[0007] According to one aspect of the present application, a medical image segmentation system with a hybrid structure is provided, which includes: a data processing module for preparing a medical image dataset Synapse to be segmented, wherein the dataset is divided into two parts: a training set and a test set; a shallow feature extraction module for inputting the preprocessed training set into the three-layer convolutional neural network part of the encoder for shallow feature extraction; an efficient channel attention module for inputting the extracted feature map into the efficient channel attention module to obtain a high-resolution feature map; a deep feature processing module for inputting the high-resolution feature map into the Transformer to process the deep features, and the convolutional neural network of the decoder part performs upsampling operations on the deep features; a prediction segmentation module for obtaining a trained weight file after multiple rounds of training, and performing inference tests on the trained weight file and the test set to obtain a final prediction segmentation result.
[0008] The medical image segmentation system with a hybrid structure provided in the embodiment of the present application introduces an efficient channel attention module, which can more accurately screen and enhance key features and improve the performance of the model in processing complex structures and details. By combining channel attention in jump connections, the model can better retain the feature information of the encoder and improve the reconstruction capability of the decoder, thereby enhancing the accuracy of the segmentation results.
[0009] Preferably, the data processing module preprocesses the images in the training set, including: resizing the images in the training set to 224×224 pixels, and performing data enhancement, including random flipping, rotation, scaling, and adding Gaussian noise.
[0010] Preferably, the shallow feature extraction module comprises: inputting the image into the convolutional neural network part of the model encoder, which is composed of three convolution blocks of ResNet-50, and the convolution block includes a convolution operation, an activation function, and a batch normalization; wherein the activation function is ReLu, and the formula is: f(x)=max(0,x), when x>0, the output is x; when x≤0, the output is 0; the input shape is (B, C, H, W), wherein B is the batch size, C is the number of channels, and H and W are the height and width of the image, respectively.
[0011] Preferably, the efficient channel attention module includes: the input feature first enters a two-dimensional convolution block 1 to reduce the number of channels to 1 / 16 of the original, and then enters an adaptive average pooling layer to reduce the spatial dimension of the input feature map to 1x1. Then, a one-dimensional convolution layer and a Sigmoid activation function are used to generate channel attention weights. During the forward propagation process, the input feature is pooled, transposed, convolved, and activated, and then multiplied element-by-element with the original input feature to obtain an output with an attention mechanism. Finally, a two-dimensional convolution block 2 with a convolution kernel of 1×1 is used to restore the channel to the channel size before input.
[0012] Preferably, the deep feature processing module includes: in the Transformer, the feature map is first subjected to a linear transformation to adjust its dimension to adapt to the requirements of the self-attention mechanism. Then, the model evaluates the relationship between features through the self-attention mechanism, and calculates the importance of each position using the query, key, and value. The query at each position is dot-producted with all keys to obtain the attention weight, which is normalized by softmax. The enhanced features are sent to the feedforward neural network for further nonlinear transformation and feature expression.
[0013] Preferably, the deep feature processing module further includes: after the deep features are processed by Transformer, the convolutional neural network (CNN) of the decoder part is responsible for upsampling these enhanced features. The decoder uses a series of transposed convolutions and upsampling layers, and also fuses the information of surrounding pixels through convolution operations to improve the details and clarity of the image. The decoder uses jump connections to combine the features of the corresponding layer in the encoder with the feature map of the decoder.
[0014] Preferably, the prediction segmentation module includes: S51. Continuously optimizing the loss function and adjusting the model parameters to minimize the difference between the prediction result and the true label; S52. By loading the weights and inputting the test image into the model for forward propagation, the model classifies each pixel according to its learned features, and the obtained prediction segmentation result will be used to evaluate the performance of the model, using the Dice similarity coefficient (DSC) and 95% Hausdorff distance (HD95) as evaluation criteria.
[0015] Preferably, in the optimization loss function unit, the loss function of the model consists of two main parts, including: a cross entropy loss function and a Dice loss function; the formula of the cross entropy loss function is as follows:
[0016]
[0017] Where N is the number of samples, c is the number of categories, and yi,c is the true label of sample i in category c, p i,c is the predicted probability of sample i in category c output by the model;
[0018] In loss calculation, the Dice loss function is expressed as:
[0019]
[0020] X represents the predicted segmentation result of the model, Y represents the true label, and ∈ represents a very small constant to prevent the denominator from being zero;
[0021] The final loss function is a weighted combination of cross entropy loss and Dice loss, and the formula is as follows:
[0022] loss = 0.5 × loss dice +0.5×loss ce
[0023] Among them, 0.5 is used as a coefficient to balance the influence of different loss terms.
[0024] Preferably, the Dice similarity coefficient formula of the performance evaluation unit is as follows:
[0025]
[0026] Where X represents the predicted segmentation result of the model, Y represents the true label, |X∩Y| represents the number of pixels in the overlapped part between the predicted result and the true label, |X| and |Y| are the number of pixels of all positive samples in the predicted result and the true label respectively, and the value of the Dice coefficient ranges from 0 to 1. The closer the value is to 1, the higher the overlap between the predicted result and the true label, which means the better the segmentation performance of the model.
[0027] The Hausdorff distance calculation formula is:
[0028] HD(A,B)=max{h(A,B),h(B,A)}
[0029] Among them, h(A,B) is the one-sided Hausdorff distance from set A to set B, and the calculation formula is:
[0030]
[0031] Among them, A and B are the boundary point sets of the predicted segmentation result and the true label respectively; d(a,b) is the Euclidean distance between point a and point b. The smaller the Hausdorff distance, the closer the predicted segmentation boundary is to the true boundary. Otherwise, it indicates that there is a large deviation in the prediction result.
[0032] The present invention introduces an efficient channel attention module to enhance the model's selectivity for key features and improve segmentation accuracy. In the jump connection between the decoder and the encoder, channel attention is combined to better retain details and semantic information. A medical image segmentation model with a U-shaped hybrid structure is designed, which consists of an encoder composed of a convolutional neural network, an efficient channel attention module, and a Transformer; an encoder composed of a convolutional neural network. An efficient channel attention module is innovatively added to the jump connection part to enhance the feature fusion capability and thus improve the segmentation accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.
[0034] Figure 1 It is a schematic block diagram of a medical image segmentation system with a hybrid structure according to an embodiment of the present application.
[0035] Figure 2 Schematic diagram of the model structure of a medical image segmentation system with a hybrid structure according to an embodiment of the present application.
[0036] Figure 3 This is a flowchart of the training part of the medical image segmentation system with a hybrid structure according to an embodiment of the present application.
[0037] Figure 4 Schematic diagram of the structure of an efficient channel attention module of a medical image segmentation system with a hybrid structure according to an embodiment of the present application.
[0038] Figure 5 Schematic diagram of the reasoning process of the medical image segmentation system with a hybrid structure according to an embodiment of the present application.
[0039] Figure 6 It is a comparison diagram of the segmentation effects of the medical image segmentation system with a hybrid structure according to an embodiment of the present application. DETAILED DESCRIPTION
[0040] Various exemplary embodiments, features and aspects of the present application will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.
[0041] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.
[0042] In addition, in order to better illustrate the present application, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present application can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present application.
[0043] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.
[0044] The current mainstream medical image segmentation framework is the U-type architecture, among which the representative one is the UNet based on the convolutional neural network (CNN). Although the convolutional layer can effectively capture local features, its understanding of the global context is relatively weak. This makes CNN perform poorly when dealing with tasks with long-distance dependencies.
[0045] To overcome this difficulty, the visual Transformer can effectively capture the relationship between each position in the sequence through the self-attention mechanism, especially when dealing with long-distance dependencies. However, due to its computational complexity of O(), this may lead to significant memory and time overhead when processing large-scale data. This problem is particularly evident when processing long texts or high-resolution images. The above challenges have prompted researchers to explore more efficient architectures and hybrid models to combine the advantages of convolution and Transformer to overcome their respective limitations. The most representative of these is TransUNet, but it also lacks the ability to utilize channel features. In recent years, the attention mechanism has played an important role in deep learning. One of its main advantages is that it can effectively capture global contextual information in the input data. By dynamically weighting different input parts, the model can adaptively focus on the features most relevant to the output according to the needs of the current task. In this application, the advantages of convolutional neural networks, Transformers, and efficient channel attention modules are innovatively combined to form a hybrid model. The purpose is to maximize the segmentation accuracy without consuming too much computing resources, and to provide an efficient method for the field of medical image segmentation.
[0046] The present invention is further described in detail below with reference to the accompanying drawings and specific implementation methods:
[0047] like Figure 1 As shown, according to the hybrid structure medical image segmentation system 100 of the embodiment of the present application, it includes: a data processing module 110, which is used to prepare a medical image data set Synapse to be segmented, and the data set is divided into two parts: a training set and a test set; a shallow feature extraction module 120, which is used to input the preprocessed training set into the three-layer convolutional neural network part of the encoder for shallow feature extraction; an efficient channel attention module 130, which is used to input the extracted feature map into the efficient channel attention module to obtain a high-resolution feature map; a deep feature processing module 140, which is used to input the high-resolution feature map into the Transformer to process the deep features, and the convolutional neural network of the decoder part upsamples the deep features; a prediction segmentation module 150, which is used to obtain a trained weight file after multiple rounds of training, and perform inference test on the trained weight file and the test set to obtain the final prediction segmentation result.
[0048] In the embodiment of the present application, the data processing module prepares a medical image dataset Synapse to be segmented, and the dataset is divided into two parts: a training set and a test set. In order to evaluate the dataset, the Dice similarity coefficient (DSC) and the 95% Hausdorff distance (HD95) are reported.
[0049] Specifically, the images in the training set were resized to 224×224 pixels and data augmentation was performed, including random flipping, rotation, scaling, and adding Gaussian noise. The dataset includes 30 scans of 8 abdominal organs: left kidney, right kidney, aorta, spleen, gallbladder, liver, stomach, and pancreas. A total of 3779 axial enhanced clinical CT images of the abdomen are included.
[0050] In the embodiment of the present application, the shallow feature extraction module inputs the preprocessed training set into the medical image segmentation model for training, and first inputs it into the three-layer convolutional neural network part of the encoder for shallow feature extraction. These convolutional layers usually include multiple important operations, namely convolution operation, activation function, and pooling layer.
[0051] Specifically, Figure 2The Encoder part shown in the figure inputs the image into the convolutional neural network part of the model encoder, which consists of three convolutional blocks of ResNet-50. The convolutional block includes convolution operations, activation functions, and batch normalization. The activation function is ReLu, and the formula is: f(x) = max(0,x). When x>0, the output is x. When x≤0, the output is 0. The input shape is (B, C, H, W), where B is the batch size, C is the number of channels, and H and W are the height and width of the image, respectively. After the three convolutional block downsampling operations, the original input shape of (16, 3, 224, 224) becomes (16, 768, 14, 14).
[0052] In the embodiment of the present application, after the high-efficiency channel attention module completes the shallow feature extraction, the extracted feature map will be further input into the high-efficiency channel attention module. The design of this module aims to improve the extraction accuracy of local features by enhancing the expression ability of important features.
[0053] Accordingly, if Figure 4 In the efficient channel attention module shown in the figure, the input features first enter the two-dimensional convolution block 1 to reduce the number of channels to 1 / 16 of the original. The purpose of this is to significantly reduce the amount of calculation while maintaining the feature information, thereby improving the training efficiency and inference speed of the model. In addition, reducing the number of channels can also help reduce the memory usage of subsequent layers, reduce the risk of overfitting, and create more favorable conditions for subsequent feature extraction and processing. Then enter an adaptive average pooling layer to reduce the spatial dimension of the input feature map to 1x1. Next, a one-dimensional convolution layer and a Sigmoid activation function are used to generate channel attention weights. During the forward propagation process, the input features are pooled, transposed, convolved, and activated, and then multiplied element-by-element with the original input features to obtain an output with an attention mechanism. This structure is designed to enhance the feature representation capability and improve the model performance. Finally, a two-dimensional convolution block 2 with a convolution kernel of 1×1 is used to restore the channel to the channel size before input (16, 768, 14, 14). Not only is the efficient channel attention module introduced in the decoder part to improve the selectivity and representation capability of the features, but this mechanism is also used in the skip connection part. This design allows the model to combine feature information from the encoder when upsampling, ensuring that details are preserved and enhancing the transfer of semantic information. By combining channel attention with skip connections, the model is able to better integrate features from different levels, thereby improving segmentation accuracy and overall performance. This approach is particularly important in tasks such as medical image segmentation because it can effectively handle complex structures and detail information.
[0054] In the embodiment of the present application, the deep feature processing module extracts more detailed feature maps through the efficient channel attention module, which are then input into the Transformer module to further enhance the model's global context understanding and feature expression capabilities. After the deep features are processed by the Transformer module, the convolutional neural network (CNN) in the decoder part will be responsible for upsampling these deep features, aiming to restore the low-resolution feature maps to the same high-resolution output as the original input image.
[0055] Specifically, Figure 2 In the Bridge part shown, the feature map is first linearly transformed to adjust its dimension to meet the requirements of the self-attention mechanism. Then, the model evaluates the relationship between features through the self-attention mechanism, and calculates the importance of each position using the query, key, and value. Specifically, the query at each position is dot-producted with all keys to obtain the attention weight, which is normalized by softmax. This enables the model to perform weighted summation of features to capture long-distance dependencies and global contextual information. After this processing, the enhanced features are fed into the feedforward neural network for further nonlinear transformation and feature expression, ultimately providing the decoder with richer information, thereby improving the performance of the overall model. This design is particularly suitable for processing complex image segmentation tasks, enabling the model to understand the input data more comprehensively.
[0056] Accordingly, after the deep features are processed by the Transformer module, the Convolutional Neural Network (CNN) in the decoder is responsible for upsampling these enhanced features. The purpose of this process is to gradually restore the low-resolution feature map to the same high-resolution output as the original input image. The decoder uses a series of transposed convolutions and upsampling layers, which not only expand the spatial dimensions of the feature map, but also fuse the information of surrounding pixels through convolution operations, thereby improving the details and clarity of the image. In addition, during the upsampling process, the decoder also uses skip connections to combine the features of the corresponding layer in the encoder with the feature map of the decoder, which can effectively retain the detail information and enhance the accuracy of the segmentation results. In this way, the decoder can better capture complex structures and edge information, ensuring that the final output high-resolution image can accurately reflect the features of the original input.
[0057] In the embodiment of the present application, the prediction segmentation module obtains a trained weight file after the model is trained for 150 rounds. The trained weight file and the test set are subjected to inference test to obtain the final prediction segmentation result.
[0058] Furthermore, the model gradually learns the features and patterns in the input data and finally generates a trained weight file. This process minimizes the difference between the predicted results and the true labels by continuously optimizing the loss function and adjusting the model parameters. The loss function of the model consists of two main parts: Cross Entropy Loss (abbreviated as loss_ce) and Dice Loss (abbreviated as loss_dice). Cross Entropy Loss is a commonly used classification loss function that measures the difference between the probability distribution of the model output and the true label distribution.
[0059] The cross entropy loss function formula is as follows:
[0060]
[0061] Where N is the number of samples, c is the number of categories, and y i,c is the true label of sample i in category c, p i,c is the predicted probability of sample i in category c output by the model; the cross entropy loss encourages the model to improve the prediction accuracy of each category by penalizing the model's high probability output for misclassification.
[0062] In loss calculation, the Dice loss function is expressed as:
[0063]
[0064] X represents the predicted segmentation result of the model, Y represents the true label, and ∈ represents a very small constant to prevent the denominator from being zero;
[0065] The final loss function is a weighted combination of cross entropy loss and Dice loss, and the formula is as follows:
[0066] loss = 0.5 × loss dice +0.5×loss ce
[0067] Among them, 0.5 is used as a coefficient to balance the influence of different loss terms. Ensure that their contributions during training are relatively balanced. This setting helps to maintain scale consistency and prevent a certain loss term from being too dominant. It also makes it easy to adjust these coefficients through experiments to optimize model performance. This simple coefficient makes the form of the loss function clearer and easier to understand. By combining the cross entropy loss and Dice loss, the model can improve its ability to handle small targets and unbalanced data while ensuring category accuracy. So far, Figure 3 The complete process of the training part shown is completed.
[0068] Correspondingly, cross entropy loss and dice loss are the two commonly used loss functions in medical image segmentation models. Combining cross entropy loss and Dice loss in medical image segmentation can significantly improve the accuracy and robustness of the model. Cross entropy loss optimizes the classification ability of the model, while Dice loss focuses on enhancing the overlap between the prediction and the true label, and is particularly effective in dealing with class imbalance problems. Through this combination, the model is able to obtain feedback from multiple angles, more comprehensively optimize the segmentation performance, and adapt to complex medical image data.
[0069] Furthermore, after the model training is completed, the weight file obtained from the training is used to perform inference test on the test set to generate the final predicted segmentation result, such as Figure 5 As shown. By loading the weights and inputting the test image into the model for forward propagation, the model classifies each pixel according to its learned features. Ultimately, the predicted segmentation results obtained will be used to evaluate the performance of the model. In this application, in order to comprehensively evaluate the performance of the model, two key evaluation criteria are adopted: the Dice similarity coefficient (DSC) and the Hausdorff distance (HD). The Dice coefficient measures the degree of overlap between two sets, which is particularly suitable for dealing with class imbalance problems and can effectively reflect the model's segmentation performance for small targets; while the Hausdorff distance focuses on the accuracy of the boundary, measures the maximum distance between the predicted boundary and the true boundary, and is suitable for applications that require high positioning accuracy. Combining these two indicators, the performance of the segmentation model can be comprehensively evaluated, thereby optimizing its application effect in actual medical scenarios.
[0070] The Dice similarity coefficient formula is as follows:
[0071]
[0072] Where X represents the predicted segmentation result of the model, Y represents the true label, |X∩Y| represents the number of pixels in the overlapped part between the predicted result and the true label, |X| and |Y| are the number of pixels of all positive samples in the predicted result and the true label respectively, and the value of the Dice coefficient ranges from 0 to 1. The closer the value is to 1, the higher the overlap between the predicted result and the true label, which means the better the segmentation performance of the model.
[0073] The Hausdorff distance calculation formula is:
[0074] HD(A,B)=max{h(A,B),h(B,A)}
[0075] Among them, h(A,B) is the one-sided Hausdorff distance from set A to set B, and the calculation formula is:
[0076]
[0077] Among them, A and B are the boundary point sets of the predicted segmentation result and the true label respectively; d(a,b) is the Euclidean distance between point a and point b. The smaller the Hausdorff distance, the closer the predicted segmentation boundary is to the true boundary. Otherwise, it indicates that there is a large deviation in the predicted result. Figure 6 The figure shows the actual segmentation effect. Compared with TransUNet, the proposed method is closer to the true value, which reflects the advantages of the method.
[0078] Specifically, the experimental results on the Synapse dataset are as follows, with the best results in bold.
[0079] Model DSC(%)↑ HD(mm)↓ UNet 76.85 39.70 MT-UNet 78.59 26.59 TransUNet 77.48 31.69 Ours 79.92 26.21
[0080] Improvement: The DSC evaluation standard is that the higher the better, the better, which is 2.44% higher than TransUNet, and the HD evaluation standard is that the lower the better, the better, which is 5.48mm lower than TransUNet. By designing efficient channel attention blocks (such as Figure 3 ), and introduce it into the encoder and jump connection of the TransUNet model (as shown in Figure 2 The ECA-Block part), the model performance has been greatly improved.
[0081] In summary, this application discloses a medical image segmentation system with a hybrid structure. By combining convolutional neural networks, efficient channel attention modules and Transformers to improve segmentation accuracy and efficiency, the medical image dataset Synapse is preprocessed, and then shallow feature extraction is performed using a three-layer convolutional neural network. The features are then enhanced through an efficient channel attention module, and these high-resolution feature maps are input into the Transformer for deep feature processing. Finally, upsampling operations are performed through the convolutional neural network of the decoder part to obtain the final predicted segmentation results. By adding channel attention to the jump connection, the feature fusion capability is enhanced, thereby improving the accuracy of the segmentation results, providing an efficient method for the field of medical image segmentation.
[0082] Furthermore, according to the hybrid structure of the medical image segmentation system of the embodiment of the present application, it includes: a data processing module, which is used to prepare a medical image data set Synapse to be segmented, and the data set is divided into two parts: a training set and a test set; a shallow feature extraction module, which is used to input the preprocessed training set into the three-layer convolutional neural network part of the encoder for shallow feature extraction; an efficient channel attention module, which is used to input the extracted feature map into the efficient channel attention block to obtain a high-resolution feature map; a deep feature processing module, which is used to input the high-resolution feature map into the Transformer to process the deep features, and the convolutional neural network of the decoder part upsamples the deep features; a prediction segmentation module, which is used to obtain a trained weight file after multiple rounds of training, and perform inference tests on the trained weight file and the test set to obtain the final prediction segmentation result.
[0083] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.
Claims
1. A medical image segmentation system with a hybrid structure, characterized in that: include: The data processing module is used to prepare the medical image dataset Synapse to be segmented. The dataset is divided into two parts: a training set and a test set. The shallow feature extraction module is used to input the preprocessed training set into the three-layer convolutional neural network part of the encoder for shallow feature extraction; An efficient channel attention module, used to input the extracted feature map into the efficient channel attention module to obtain a high-resolution feature map; A deep feature processing module, used to input the high-resolution feature map into the Transformer to process the deep features, and the convolutional neural network of the decoder part performs upsampling operations on the deep features; The prediction segmentation module is used to obtain the trained weight file after multiple rounds of training, and to perform inference tests on the trained weight file and the test set to obtain the final prediction segmentation result.
2. The medical image segmentation system of the hybrid structure according to claim 1, characterized in that: The data processing module preprocesses the images in the training set, including: resizing the images in the training set to 224×224 pixels, and performing data enhancement, including random flipping, rotation, scaling, and adding Gaussian noise.
3. The medical image segmentation system of the hybrid structure according to claim 2, characterized in that: The shallow feature extraction module includes: inputting the image into the convolutional neural network part of the model encoder, which is composed of three convolution blocks of ResNet-50, and the convolution block includes a convolution operation, an activation function, and batch normalization; Among them, the activation function is ReLu, the formula is: f(x) = max(0,x), When x>0, the output is x; when x≤0, the output is 0; The shape of the input is (B, C, H, W), where B is the batch size, C is the number of channels, and H and W are the height and width of the image, respectively.
4. The medical image segmentation system of the hybrid structure according to claim 3, characterized in that: The efficient channel attention module includes: the input feature first enters a two-dimensional convolution block 1 to reduce the number of channels to 1 / 16 of the original, and then enters an adaptive average pooling layer to reduce the spatial dimension of the input feature map to 1x1. Then, a one-dimensional convolution layer and a Sigmoid activation function are used to generate channel attention weights. In the forward propagation process, the input feature is pooled, transposed, convolved and activated, and then multiplied element-by-element with the original input feature to obtain an output with an attention mechanism. Finally, a two-dimensional convolution block 2 with a convolution kernel of 1×1 is used to restore the channel to the channel size before input.
5. The medical image segmentation system of the hybrid structure according to claim 4, characterized in that: The deep feature processing module includes: in the Transformer, the feature map is first subjected to a linear transformation to adjust its dimension to adapt to the requirements of the self-attention mechanism. Then, the model evaluates the relationship between features through the self-attention mechanism, and calculates the importance of each position using the three parts of query, key and value. The query of each position is dot-producted with all keys to obtain the attention weight, which is normalized by softmax. The enhanced features are sent to the feedforward neural network for further nonlinear transformation and feature expression.
6. The medical image segmentation system of the hybrid structure according to claim 5, characterized in that: The deep feature processing module further includes: after the deep features are processed by Transformer, the convolutional neural network (CNN) of the decoder part is responsible for upsampling these enhanced features. The decoder uses a series of transposed convolutions and upsampling layers, and also fuses the information of surrounding pixels through convolution operations to improve the details and clarity of the image. The decoder uses jump connections to combine the features of the corresponding layer in the encoder with the feature map of the decoder.
7. The medical image segmentation system of the hybrid structure according to claim 6, characterized in that: The prediction segmentation module comprises: The loss function optimization unit is used to continuously optimize the loss function and adjust the model parameters to minimize the difference between the predicted results and the true labels; The performance evaluation unit is used to perform forward propagation by loading weights and inputting the test image into the model. The model classifies each pixel according to its learned features. The predicted segmentation results will be used to evaluate the performance of the model, using the Dice similarity coefficient (DSC) and 95% Hausdorff distance (HD95) as evaluation criteria.
8. The medical image segmentation system of the hybrid structure according to claim 7, characterized in that: The loss function of the optimization loss function unit is composed of two main parts: a cross entropy loss function and a Dice loss function; The cross entropy loss function formula is as follows: Where N is the number of samples, c is the number of categories, and y i,c is the true label of sample i in category c, p i,c is the predicted probability of sample i in category c output by the model; In loss calculation, the Dice loss function is expressed as: X represents the predicted segmentation result of the model, Y represents the true label, and ∈ represents a very small constant to prevent the denominator from being zero; The final loss function is a weighted combination of cross entropy loss and Dice loss, and the formula is as follows: loss=0.5×loss dice +0.5×loss ce Among them, 0.5 is used as a coefficient to balance the influence of different loss terms.
9. The medical image segmentation system of the hybrid structure according to claim 8, characterized in that: The performance evaluation unit, the Dice similarity coefficient formula is as follows: Where X represents the predicted segmentation result of the model, Y represents the true label, |X∩Y| represents the number of pixels in the overlapped part between the predicted result and the true label, |X| and |Y| are the number of pixels of all positive samples in the predicted result and the true label respectively, and the value of the Dice coefficient ranges from 0 to 1. The closer the value is to 1, the higher the overlap between the predicted result and the true label, which means the better the segmentation performance of the model. The Hausdorff distance calculation formula is: HD(A,B)=max{h(A,B),h(B,A)} Among them, h(A,B) is the one-sided Hausdorff distance from set A to set B, and the calculation formula is: Among them, A and B are the boundary point sets of the predicted segmentation result and the true label respectively; d(a,b) is the Euclidean distance between point a and point b. The smaller the Hausdorff distance, the closer the predicted segmentation boundary is to the true boundary. Otherwise, it indicates that there is a large deviation in the prediction result.