Brain tumor segmentation method based on combination of attention mechanism and semi-supervised learning
By introducing attention mechanism and semi-supervised learning in the UNet model, the problems of insufficient feature representation and global information loss in complex medical imaging processing are solved, which significantly improves the accuracy and robustness of brain tumor segmentation.
Patent Information
- Application Number
- CN202411189195.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-27
- Publication Date
- 2024-12-20
AI Technical Summary
When the existing UNet model processes complex medical images, the feature representation ability and global information are insufficient, resulting in inaccurate brain tumor segmentation results.
Introduce attention mechanisms, including channel attention, spatial attention and self-attention mechanisms, and combine semi-supervised learning to optimize the feature extraction and global information capture capabilities of the UNet model.
It significantly improves the accuracy and robustness of brain tumor segmentation, enhances the model's ability to pay attention to important features and capture global information, and improves the consistency and accuracy of segmentation results.
Smart Images
Figure CN119169287A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of medical image processing, and in particular, relates to a brain tumor segmentation method based on an attention mechanism combined with semi-supervised learning. Background Art
[0002] Brain tumor is one of the most deadly and difficult to treat diseases, and its diagnosis and treatment require precise medical image analysis. Magnetic resonance imaging (MRI) is a commonly used tool for brain tumor detection, which can provide high-resolution soft tissue contrast images. However, manual annotation of tumor areas by radiologists alone is time-consuming and prone to subjective differences. Therefore, automated brain tumor segmentation technology is of great significance in clinical practice.
[0003] In the field of medical image processing, automated image segmentation is a crucial task, which has wide applications in clinical diagnosis and treatment planning. As a classic convolutional neural network architecture, the UNet model has shown excellent performance in medical image segmentation. The symmetrical structure of UNet consists of an encoder and a decoder, which can effectively capture local and global features in the image. However, with the increasing complexity of medical imaging data, the UNet model still has the following shortcomings when processing certain details and boundary information.
[0004] Insufficient feature representation: Although the UNet model can capture multi-scale features in images, its feature representation capability is still insufficient when dealing with complex backgrounds and areas with rich details, which can easily lead to inaccurate segmentation results.
[0005] Lack of global information: The UNet model may lose some global information during the upsampling and downsampling process layer by layer. Especially when processing larger images or complex lesion areas, it is difficult for the model to fully utilize the global context information.
[0006] In order to solve the above problems, researchers have proposed a variety of improvement methods. For example, using multi-scale feature fusion, introducing residual connections, and adopting deeper network structures to enhance the feature extraction ability of the UNet model. However, these methods have improved the performance of the model to a certain extent, but still have obvious limitations when processing complex medical images.
[0007] In recent years, attention mechanisms have made significant progress in natural language processing and computer vision. By simulating the attention selectivity of the human visual system, attention mechanisms can automatically focus on key areas in large-scale data, improving the model's feature representation ability and global information capture ability. Therefore, introducing the attention mechanism into the UNet model and optimizing its performance in medical image segmentation has become an important research direction. Summary of the invention
[0008] Based on the above background, the present invention proposes a brain tumor segmentation method based on attention mechanism combined with semi-supervised learning for medical image segmentation, especially automatic segmentation of brain tumors. The brain tumor segmentation task has extremely high requirements on the accuracy and robustness of the model, because the morphology and position of brain tumors are highly variable, and the segmentation results directly affect clinical diagnosis and treatment decisions. By integrating multiple attention mechanisms in the encoder and decoder of UNet, such as channel attention, spatial attention, and self-attention, the method of the present invention can effectively enhance the model's ability to focus on important features and improve segmentation accuracy and robustness.
[0009] To achieve the above object, the present invention provides the following technical solutions:
[0010] A brain tumor segmentation method based on attention mechanism combined with semi-supervised learning includes the following steps:
[0011] 1. Data preprocessing
[0012] Standardization processing: The input brain MRI images are standardized, including noise removal, normalization and other steps to ensure data consistency and model stability.
[0013] Data enhancement: During the training process, data enhancement techniques such as rotation, flipping, and translation are used to improve the generalization ability of the model.
[0014] 2. Model training
[0015] Loss function: A combination of the cross entropy loss function and the Dice loss function is used to balance the segmentation effect of the model on large and small tumor areas. The cross entropy loss function is used to measure the classification error of each pixel, while the Dice loss function is used to measure the overlap of the overall segmentation results. Dice can be expressed as follows:
[0016]
[0017] Training strategy: A phased training strategy is adopted. First, the basic UNet structure is trained, and then the attention mechanism is gradually introduced and trained to ensure the stability and gradual optimization of the model.
[0018] 3. Prediction and post-processing
[0019] Prediction: In the model prediction stage, the input MRI image is fed into the optimized UNet model to obtain the probability map of tumor segmentation.
[0020] Post-processing: Binarize the probability map and remove noise and small pseudo-detection areas through morphological operations (such as closing and opening operations) to obtain the final tumor segmentation result.
[0021] Compared with the prior art, the present invention has the following beneficial effects:
[0022] (1) Enhanced feature representation capability. By introducing the channel attention mechanism, the present invention can adaptively adjust the weights of each feature channel, so that the model can better capture and represent important features when processing complex backgrounds and areas with rich details. Compared with the traditional UNet model, the present invention shows higher accuracy and reliability in feature extraction.
[0023] (2) Improved global information capture capability. The introduction of the self-attention mechanism enables the model to interact with feature information on a global scale, thereby capturing more global contextual information. This mechanism enables the model to better utilize global information when processing larger images or complex lesion areas, thereby improving the overall consistency and accuracy of segmentation. Compared with the prior art, the present invention significantly improves the model's ability to capture global information.
[0024] (3) Improved boundary segmentation accuracy. Through the spatial attention mechanism, the present invention can focus on specific spatial locations and enhance the model's ability to segment the target area, especially when segmenting structures with clear boundaries. Experimental results show that the present invention significantly reduces boundary blurring and significantly improves segmentation accuracy in processing brain tumor boundary segmentation tasks. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1 Schematic diagram of data mode of an embodiment of the present invention;
[0026] Figure 2 A flow chart of a scaling dot product attention mechanism according to an embodiment of the present invention;
[0027] Figure 3 This is a flow chart of the channel attention mechanism according to an embodiment of the present invention;
[0028] Figure 4 This is a flow chart of the spatial attention mechanism of an embodiment of the present invention;
[0029] Figure 5 This is a flow chart of the self-attention mechanism of an embodiment of the present invention. DETAILED DESCRIPTION
[0030] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0031] In an embodiment of the present invention, a brain tumor segmentation method based on an attention mechanism combined with semi-supervised learning is provided, and the specific technical solution includes the following aspects:
[0032] 1. Optimization of UNet model structure
[0033] 1. Encoder part
[0034] Structure: The encoder part consists of multiple convolutional layers and pooling layers to gradually extract image features. Each convolutional layer is followed by a Batch Normalization layer and a ReLU activation function to improve the training speed and stability of the model.
[0035] Channel Attention Mechanism: A channel attention mechanism is introduced after each convolutional layer to enhance the model's attention to important features by adaptively adjusting the weights of each channel. The specific implementation method is to use global average pooling and global maximum pooling to process the feature map, and then generate channel attention weights through two fully connected layers to weight the original features.
[0036] 2. Decoder part
[0037] Structure: The decoder part gradually upsamples the features extracted by the encoder, and fuses the features in the encoder to the corresponding decoding layer through skip connections to restore the spatial resolution of the image.
[0038] Spatial Attention Mechanism: A spatial attention mechanism is introduced after each upsampling layer to enhance the model's ability to segment the target area by focusing on specific spatial locations. The specific implementation method is to use global average pooling and global maximum pooling to process the feature map, concatenate the results, and then generate a spatial attention map through a convolution layer to weight the original features.
[0039] 3. Self-Attention Mechanism
[0040] In the middle of the encoder and decoder, a self-attention mechanism is integrated to capture global context information. This mechanism enables the model to interact with feature information on a global scale by calculating the correlation within the feature map, thereby improving the overall consistency of the segmentation. The specific implementation method is to calculate the attention weight by multiplying the query, key, and value matrices, and perform a weighted summation on the original features.
[0041] 2. Specific Implementation of Brain Tumor Segmentation
[0042] 1. Data preprocessing
[0043] Standardization processing: The input brain MRI images are standardized, including noise removal, normalization and other steps to ensure data consistency and model stability.
[0044] Data enhancement: During the training process, data enhancement techniques such as rotation, flipping, and translation are used to improve the generalization ability of the model.
[0045] 2. Model training
[0046] Loss function: A combination of the cross entropy loss function and the Dice loss function is used to balance the segmentation effect of the model on large and small tumor areas. The cross entropy loss function is used to measure the classification error of each pixel, while the Dice loss function is used to measure the overlap of the overall segmentation results. Dice can be expressed as follows:
[0047]
[0048] Training strategy: A phased training strategy is adopted. First, the basic UNet structure is trained, and then the attention mechanism is gradually introduced and trained to ensure the stability and gradual optimization of the model.
[0049] 3. Prediction and post-processing
[0050] Prediction: In the model prediction stage, the input MRI image is fed into the optimized UNet model to obtain the probability map of tumor segmentation.
[0051] Post-processing: Binarize the probability map and remove noise and small pseudo-detection areas through morphological operations (such as closing and opening operations) to obtain the final tumor segmentation result.
[0052] 3. Technical features and their functions
[0053] 1. Channel Attention Mechanism
[0054] Function: By adaptively adjusting the weights of each channel of the feature map, the model's attention to important features is enhanced and the feature representation capability is improved.
[0055] Implementation: Global average pooling and global maximum pooling are combined to generate channel weights and weight the original features.
[0056] 2. Spatial Attention Mechanism
[0057] Function: By focusing on specific spatial locations, the model can segment the target area more accurately, especially when dealing with complex backgrounds and boundary areas.
[0058] Implementation: Global average pooling and global maximum pooling are combined to generate a spatial attention map and weight the original features.
[0059] 3. Self-Attention Mechanism
[0060] Function: Capture global context information, so that the model can better utilize global information when processing larger images or complex lesion areas, and improve the overall consistency and accuracy of segmentation.
[0061] Implementation: Attention weights are calculated by multiplying the query, key, and value matrices, and performing a weighted sum over the original features.
[0062] 4. Skip Connections
[0063] Function: By fusing features between the encoder and decoder, the detailed information of the image is retained and the resolution and accuracy of the segmentation results are improved.
[0064] By introducing multiple attention mechanisms, the present invention effectively enhances the UNet model's ability to focus on important features and capture global information, thereby improving the accuracy and robustness of brain tumor segmentation and providing strong technical support for clinical diagnosis and treatment.
[0065] The following examples illustrate the specific implementation of the brain tumor segmentation method based on the attention mechanism combined with semi-supervised learning in the present invention.
[0066] 1. Data preprocessing and enhancement
[0067] The best implementation of the present invention first performs data preprocessing and enhancement on the input MRI brain images, including noise removal and normalization. Subsequently, data enhancement techniques such as rotation, flipping and translation are used to increase the diversity of training data to improve the generalization ability of the model.
[0068] 2. Encoder part
[0069] The encoder part of the UNet model is used to extract features of the input image. The encoder consists of a series of convolutional layers and pooling layers, which extract image features at different levels by convolution and downsampling layer by layer. In a specific implementation, each convolutional layer is followed by a ReLU activation function to introduce nonlinear features.
[0070] 3. Channel Attention Mechanism
[0071] Based on the encoder part, the channel attention mechanism is used to further optimize the feature extraction process. The channel attention mechanism first performs global average pooling and global maximum pooling on the input feature map, and then generates a channel attention map through a fully connected layer. The channel attention map is used to weight each channel in the input feature map to highlight important features and suppress unimportant features. The specific process is as follows:
[0072] Perform global average pooling and global maximum pooling on the input feature map to generate two different feature maps.
[0073] The two feature maps are input into the fully connected layer respectively to generate two channel weight maps.
[0074] The two channel weight maps are added element by element, and then the Sigmoid activation function is used to generate the final channel attention map.
[0075] Multiply the channel attention map by the input feature map element-wise to obtain the optimized feature map.
[0076] 4. Spatial Attention Mechanism
[0077] After the encoder extracts features, the spatial attention mechanism is introduced to further optimize the feature extraction process. The spatial attention mechanism first performs global average pooling and global maximum pooling on the input feature map, and then generates a spatial attention map through a convolutional layer. The spatial attention map is used to weight each spatial position in the input feature map to highlight important positions and suppress unimportant positions. The specific process is as follows:
[0078] Perform global average pooling and global maximum pooling on the input feature map to generate two different feature maps.
[0079] The two feature maps are concatenated along the channel dimension to generate a joint feature map.
[0080] The joint feature map is input into the convolutional layer to generate a spatial weight map.
[0081] The spatial weight map is element-wise multiplied to generate the final spatial attention map.
[0082] The spatial attention map is element-wise multiplied with the input feature map to obtain the optimized feature map.
[0083] 5. Decoder part
[0084] The optimized feature map is input to the decoder, which consists of a series of deconvolution layers and upsampling layers. The decoder reconstructs the resolution of the original image layer by layer, and transmits the features of each layer of the encoder through skip connections, retaining high-resolution feature information to improve the accuracy of the segmentation result.
[0085] 6. Skip connections and detail preservation
[0086] The skip connection is used to directly transfer the features of each layer of the encoder to the corresponding layer of the decoder to preserve the high-resolution detail information. In a specific implementation, the features of each layer of the encoder are fused with the features of the corresponding layer of the decoder through an element-by-element addition operation to ensure that the detail information of the image is preserved during the segmentation process.
[0087] 7. Final segmentation result
[0088] The feature map optimized by the channel attention mechanism and the spatial attention mechanism is used to reconstruct the segmented image through the decoder. The final segmentation result retains the detailed information of the original image and improves the accuracy and precision of the segmentation.
[0089] 8. Training and Optimization
[0090] The present invention uses the cross entropy loss function to jointly optimize the entire network. The loss function is defined as follows:
[0091]
[0092] Among them, N is the number of samples, C is the number of categories, and y ic is the true category label of the sample, p ic The class probabilities predicted by the model.
[0093] By performing the above optimization on the UNet model, the present invention can significantly improve the accuracy and efficiency of brain tumor segmentation, and provide more accurate auxiliary support for clinical diagnosis and treatment.
[0094] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.
[0095] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.
Claims
1. A brain tumor segmentation method based on attention mechanism combined with semi-supervised learning, characterized in that: The steps include:
1. Data Preprocessing Standardization processing: Standardize the input brain MRI images to ensure data consistency and model stability; Data enhancement: During the training process, data enhancement technology is used to improve the generalization ability of the model; 2. Model Training Loss function: A combination of the cross entropy loss function and the Dice loss function is used to balance the segmentation effect of the model on large and small tumor areas. The cross entropy loss function is used to measure the classification error of each pixel, and the Dice loss function is used to measure the overlap of the overall segmentation results. Dice is expressed as follows: Training strategy: A phased training strategy is adopted. First, the basic UNet structure is trained, and then the attention mechanism is gradually introduced and trained to ensure the stability and gradual optimization of the model.
3. Prediction and Post-processing Prediction: In the model prediction stage, the input MRI image is fed into the optimized UNet model to obtain the probability map of tumor segmentation; Post-processing: Binarize the probability map and remove noise and small pseudo-detection areas through morphological operations to obtain the final tumor segmentation result.
2. The brain tumor segmentation method based on attention mechanism combined with semi-supervised learning according to claim 1 is characterized in that: In the standardization process, the input brain MRI image is standardized, including noise removal and normalization steps to ensure the consistency of data and the stability of the model.
3. The brain tumor segmentation method based on attention mechanism combined with semi-supervised learning according to claim 1 is characterized in that: The data enhancement techniques used in the data enhancement training process include rotation, flipping, and translation.