Multi-scale residual brain tumor image segmentation method based on attention mechanism

Abstract

The present application belongs to the technical field of image segmentation, and particularly relates to a multi-scale residual brain tumor image segmentation method based on an attention mechanism. Firstly, a double residual feature fusion module is used to enhance the fusion of feature information between different levels. Secondly, a hollow spatial pyramid pooling is introduced as a bridging module of the network to capture the features of different scales of the image, thereby improving the extraction ability of the network for detailed features. Finally, a reverse residual coordinate attention module is designed to replace the direct splicing operation of Unet to fuse large feature information at different levels and scales, thereby enhancing the recognition ability of the model for spatial position information of brain tumors. Through the double residual feature fusion module DRFF, the present application effectively fuses feature information at different levels, retains local details and combines global semantic features, and improves the accuracy of the model in processing complex tumor structures.

Description

Technical Field

[0001] This invention belongs to the field of image segmentation technology, specifically relating to a multi-scale residual brain tumor image segmentation method based on attention mechanisms. Background Technology

[0002] In the field of medical image processing, segmentation in magnetic resonance imaging (MRI) of brain tumors is a crucial research topic. Its main purpose is to segment the tumor structure within brain tumor MRI images. This process is essential for helping physicians assess tumor growth, activity, and treatment response, thereby supporting more accurate diagnostic and treatment decisions. Brain tumor MRI images consist of information from three core tumor regions: the whole tumor (WT), the core tumor (TC), and the tumor enhancement (ET). The WT region, representing the whole tumor, provides information on the overall extent of the tumor, helping physicians accurately locate its position. The TC region, the core tumor, contains proliferating tumor cells as well as dead or necrotic portions of the tumor, representing the malignant tumor that requires resection. The ET region is generally considered the most active tumor region, composed of necrotic cells.

[0003] To improve the accuracy of semantic segmentation, Google launched the DeepLab network based on FCN, which combines deep convolutional neural networks and fully connected conditional random fields, effectively addressing the shortcomings of deep learning models in capturing details. Subsequently, the UNet network, due to its simple structure and excellent segmentation performance, became the preferred method for brain tumor segmentation tasks. For example, Aghalari et al. designed dual-channel path residual blocks for UNet to detect irregularly shaped brain tumors; Huang et al. designed a cross-channel attention residual UNet parallel network to achieve multimodal brain tumor segmentation; and Han Yang et al. introduced triple attention into the UNet network to fully extract feature information from brain tumor images.

[0004] Although these methods have shown excellent results in the field of medical image segmentation, they still have limitations in the segmentation accuracy of some complex and poorly defined tumor structures in brain tumor MRI image segmentation. Summary of the Invention

[0005] To address the technical problems in the prior art, this invention proposes a multi-scale residual brain tumor image segmentation method based on an attention mechanism.

[0006] To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0007] A multi-scale residual brain tumor image segmentation method based on attention mechanisms includes the following steps:

[0008] Step 1: Adopt the U-net encoder-decoder architecture;

[0009] Step 2: In order to fuse shallow detail feature information and deep semantic feature information in the encoder, the feature maps of each level of the encoder are upsampled to the maximum resolution and the features are stitched together. The feature is then sent to the dual residual feature fusion module DRFF through feature copying operation, so as to fuse local detail discrimination feature information and global high-level semantic discrimination feature information.

[0010] Step 3: To compensate for the semantic differences between the encoder and decoder, Inverted Residual Coordinate Attention (IRCA) is introduced in the feature layer connection stage. The feature information obtained by DRFF downsampling is fed into this module, which can effectively encode channel relationships and long-range dependencies, and improve the accuracy of the model in locating and identifying the location information of the tumor region.

[0011] Step 4: The bridging module introduces the Hollow Space Pyramid Pooling (ASPP) to fuse contextual information and mine feature information.

[0012] Step 5: In the decoder section, upsampling is performed by transposed convolution, and 1×1 convolution is used to restore the segmentation result to the input image size, finally generating segmentation maps of each region of the brain tumor.

[0013] As a further preferred technical solution of the present invention, in step 2, a novel dual residual feature fusion module is constructed by introducing depthwise separable convolution (DConv) and efficient multi-scale feature fusion module (EMA). While encoding global information to highlight the feature information of small targets, the number of model parameters is effectively reduced. The output characteristics of parallel branches are aggregated, and cross-space processing of short-term and long-term dependencies and information fusion between different channels are enhanced through cross-dimensional interaction, thereby preserving a wider range of contextual information. Among them, the EMA module uses parallel substructures to reduce network depth and optimizes pixel-level attention of high-level feature maps without reducing channel dimensions.

[0014] As a further preferred embodiment of the present invention, the DRFF module includes a shallow feature processing branch and a deep feature processing branch, and adopts a dual residual structure to extract feature information at different levels.

[0015] The shallow feature processing branch takes a shallow feature map as input and processes it using depthwise separable convolution (DConv). This involves using a k×k convolution kernel to perform individual convolution operations on each channel of the input feature map, resulting in a depthwise convolution output feature map. Then, pointwise convolution is used to perform a weighted summation of the features of all input channels at each position of the depthwise convolution output feature map to obtain the final output feature. Finally, a 1×1 convolution is performed to reduce the dimensionality so that the number of channels is consistent with the input, and then the data is connected to the EMA module.

[0016] The deep feature processing branch takes a deep feature map as input, first adjusts the number of channels through a 1×1 convolution, then normalizes the feature map using the softmax function, and then processes it through a 1×1 convolution and the Leaky ReLU activation function to output the transformed feature map.

[0017] In the feature fusion stage, the low-level and high-level features output from the two branches are added together to fuse feature information at different levels. The fused feature map is then passed through a final 1×1 convolutional layer to adjust the number of channels. Finally, the feature maps from the two branches are multiplied by the fused feature map using a residual structure to obtain the final fused feature map F. fusion .

[0018] As a further preferred technical solution of the present invention, in step 3, a coordinate attention mechanism (CA) is used, and an inverted residual structure is designed for brain tumor MRI images with complex feature information, namely, inverted residual coordinate attention (IRCA). The CA module includes the steps of coordinate information embedding and coordinate attention generation. This module aims to capture long-distance spatial dependencies with precise location information.

[0019] Coordinate information embedding is achieved by decomposing global pooling into feature encoding in two dimensions. For an input feature map x with height H and width W, pooling kernels with dimensions (H, 1) and (1, W) are used to encode along the horizontal and vertical directions, respectively, thereby obtaining the position information of the input feature map on the x-axis and y-axis. Through feature transformation and aggregation in the two directions shown in formulas (1) and (2), output feature maps along the two dimensions are generated.

[0020]

[0021] in, This represents the output of the c-th channel at height h. This represents the output of the c-th channel with width w; after the above transformation, it will be... and The concatenation is performed, and the intermediate feature map f is generated by processing it with a 1x1 convolution function F and a nonlinear activation function δ, as shown in formula (3):

[0022] f=δ(F1([Z h Z w ])) (3)

[0023] The coordinate attention mechanism decomposes f into two separate tensors along the spatial dimension. h and f w And adjust f through the convolution function F h and f w The number of channels is used to match the input feature map, and then the data is processed through a sigmoid activation function σ to generate attention weights g in the horizontal and vertical directions, respectively. h and g w Finally, g h and g w Further, the output y is used as the attention weight and coordinate attention respectively, as shown in formulas (4), (5), and (6):

[0024] g h =σ(F h (f h (4)

[0025] g w =σ(F w (f w (5)

[0026]

[0027] Among them, f h ∈R c / r×H and f w ∈R c / r×W , where r represents the reduction rate.

[0028] As a further preferred technical solution of the present invention, in step 4, the traditional bridging module of the U-Net network is replaced with Spatial Pyramid Pooling (ASPP) with dilated spatial pyramid pooling. The ASPP module has a total of five branch structures, including one 1×1 convolution, three 3×3 dilated convolutions with dilation rates of 6, 12 and 18 respectively, and one global average pooling.

[0029] The multi-scale residual brain tumor image segmentation method based on the attention mechanism described in this invention has the following technical advantages compared with existing technologies:

[0030] 1. By using the Dual Residual Feature Fusion (DRFF) module, feature information at different levels is effectively fused, preserving local details while incorporating global semantic features, thus improving the model's accuracy in processing complex tumor structures.

[0031] 2. The introduced Spatial Pyramid Pooling (ASPP) module serves as a bridging module for the network. By using parallel convolutions with different dilation rates, it captures multi-scale features of the image, effectively improving the model's ability to extract detailed features and enhancing its ability to fuse contextual information.

[0032] 3. The designed Inverse Residual Coordinate Attention (IRCA) module replaces the traditional direct stitching operation. By encoding channel relationships and long-distance dependencies, it significantly enhances the model's ability to identify the spatial location information of brain tumors and improves segmentation accuracy.

[0033] 4. On the four evaluation metrics (Dice coefficient, positive predictive value (PPV), sensitivity, and Hausdorff distance (HD)), the attention-based multi-scale residual brain tumor image segmentation method described in this invention achieved 84.54%, 87.43%, 88.37%, and 2.248, respectively, which are improvements of 1.85%, 2.11%, 2.88%, and 6.0% compared with the existing Unet model technology. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the system structure according to an embodiment of the present invention;

[0035] Figure 2 This is a schematic diagram of the EMA structure according to an embodiment of the present invention;

[0036] Figure 3 This is a schematic diagram of the DRFF structure according to an embodiment of the present invention;

[0037] Figure 4 This is a schematic diagram of the IRCA structure according to an embodiment of the present invention;

[0038] Figure 5 This is a schematic diagram of the ASPP structure according to an embodiment of the present invention;

[0039] Figure 6 These are example diagrams illustrating different segmentation methods according to embodiments of the present invention. Detailed Implementation

[0040] The present invention will be further explained in detail below with reference to the accompanying drawings, so that those skilled in the art can better understand and implement the present invention. However, the following examples are only used to explain the present invention and are not intended to limit the present invention.

[0041] like Figure 1 As shown, the multi-scale residual brain tumor image segmentation method based on the attention mechanism includes the following steps:

[0042] Step 1: Adopt the U-net encoder-decoder architecture;

[0043] Step 2: In order to fuse shallow detail feature information and deep semantic feature information in the encoder, the feature maps of each level of the encoder are upsampled to the maximum resolution and the features are stitched together. The feature is then sent to the dual residual feature fusion module DRFF through feature copying operation, so as to fuse local detail discrimination feature information and global high-level semantic discrimination feature information.

[0044] Step 3: To compensate for the semantic differences between the encoder and decoder, Inverted Residual Coordinate Attention (IRCA) is introduced in the feature layer connection stage. The feature information obtained by DRFF downsampling is fed into this module, which can effectively encode channel relationships and long-range dependencies, and improve the accuracy of the model in locating and identifying the location information of the tumor region.

[0045] Step 4: The bridging module introduces the Hollow Space Pyramid Pooling (ASPP) to fuse contextual information and mine feature information.

[0046] Step 5: In the decoder section, upsampling is performed by transposed convolution, and 1×1 convolution is used to restore the segmentation result to the input image size, finally generating segmentation maps of each region of the brain tumor.

[0047] As deep convolutional neural networks become increasingly layered, semantic information from feature maps is gradually extracted and aggregated across multiple convolutional layers, often resulting in deep feature maps containing a great deal of similar information. Furthermore, small target feature information in brain tumor MRI images is easily lost, and scale capture capability is weak. To address this, a novel Double Residual Feature Fusion (DRFF) module is constructed by introducing Depthwise Separable Convolutions (DConv) and Efficient Multi-Scale Attention (EMA). This module encodes global information to highlight the feature information of small targets while effectively reducing the number of model parameters. Then, the output characteristics of parallel branches are further aggregated, enhancing cross-spatial processing of short-term and long-term dependencies and information fusion between different channels through cross-dimensional interactions, thereby preserving a broader range of contextual information.

[0048] The EMA module utilizes a parallel substructure to reduce network depth, optimizing pixel-level attention to high-level feature maps without reducing channel dimensions. For example... Figure 2As shown, the branches containing XAvg Pool, YAvg Pool, and 3×3 convolution are denoted as 1×1 branch, 1×1 branch, and 3×3 branch, respectively. EMA uses these three branches to extract attention weights from grouped feature maps. In the 1×1 branch, global average pooling is first performed on the x and y directions of each group of feature maps to extract global contextual information. Then, the encoded features from the two average pooling directions are concatenated and share a 1×1 convolution. After the 1×1 convolution outputs two vectors, a non-linear transformation is performed using the Sigmoid activation function. Finally, the weighted channel features are obtained by multiplying the group normalization and the output of the Sigmoid function. In the 3×3 branch, a 3×3 convolution kernel is used to capture multi-scale feature representations. Similar to the 1×1 branch, global pooling is used to output global information, and spatial attention weights are calculated using the softmax activation function. Finally, the channel features of the 1×1 branch and the spatial attention weights of the 3×3 branch are multiplied by matrix multiplication to produce the final attention weights. The sigmoid activation function is applied to these attention weights, and the dot product operation is used to output them to the final feature map.

[0049] The DRFF module contains two feature extraction branches, such as Figure 3 As shown, a dual residual structure is used to extract feature information at different levels. The shallow feature processing branch takes a shallow feature map as input and processes it using depthwise separable convolution (Dwise Conv). This involves using a k×k convolution kernel to perform individual convolution operations on each channel of the input feature map, resulting in a depthwise convolution output feature map. Then, pointwise convolution is used to weight and sum the features of all input channels at each position of the depthwise convolution output feature map to obtain the final output feature. Finally, a 1×1 convolution is used to reduce the dimensionality to match the input channel count before connecting to the EMA module. The deep feature processing branch takes a deep feature map as input. First, a 1×1 convolution is used to adjust the channel count. Then, a softmax function is used to normalize the feature map. Finally, a 1×1 convolution and the Leaky ReLU activation function are used to process the transformed feature map. In the feature fusion stage, the low-level and high-level features from the two branches are added to fuse feature information at different levels. The fused feature map is then processed by a final 1×1 convolution layer to adjust the channel count. The feature maps of the two branches are multiplied with the fused feature map using a residual structure to obtain the final fused feature map F. fusion This module successfully integrates global contextual information and local texture details through two branches at different levels to obtain multi-scale feature representations, providing powerful pixel-level attention for brain tumor MRI image segmentation tasks, thereby improving the model's accuracy in identifying and segmenting tumor regions in complex brain tumor images.

[0050] In computer vision, attention mechanisms are applied to enhance a model's feature representation capabilities, thereby improving its performance on specific tasks. The SE module can dynamically adjust the weights of different feature channels, allowing the network to focus more on important features. However, SE primarily focuses on the relationships between channels, neglecting the importance of spatial location. Furthermore, CBAM further introduces a spatial attention mechanism on top of SE, enhancing feature representation by sequentially focusing on channel and spatial information, but it lacks the ability to extract long-distance relationships.

[0051] To address the aforementioned issues, this invention introduces a coordinate attention (CA) mechanism and designs an inverted residual structure, namely Inverted Residual Coordinate Attention (IRCA), specifically for brain tumor MRI images with complex feature information. Figure 4 As shown, compared to the initial coordinate attention mechanism module, the improved module first increases the dimensionality of the input feature map by using a 1x1 convolution, then applies a 3x3 depthwise separable convolution to process these features, incorporating positional information into channel attention, and finally reduces the dimensionality by using a 1x1 convolution. This reduces the number of model parameters, improves the module's ability to learn rich feature information, and enhances the ability to capture cross-channel information and perceive direction and position information, thereby better learning the global positional information of brain tumors.

[0052] The CA module includes coordinate information embedding and coordinate attention generation. This module aims to capture long-range spatial dependencies with precise location information. Specifically, coordinate information embedding is achieved by decomposing global pooling into feature encoding in two dimensions. For an input feature map x with height H and width W, pooling kernels of size (H, 1) and (1, W) are used to encode along the horizontal and vertical directions, respectively, to obtain the location information of the input feature map on the x-axis and y-axis. Through feature transformation and aggregation in the two directions shown in formulas (1) and (2), output feature maps along the two dimensions are generated.

[0053]

[0054] in, This represents the output of the c-th channel at height h. This represents the output of the c-th channel with width w; after the above transformation, it will be... and The concatenation is performed, and the intermediate feature map f is generated by processing it with a 1x1 convolution function F and a nonlinear activation function δ, as shown in formula (3):

[0055] f=δ(F1([Z h Z w ])) (3)

[0056] The coordinate attention mechanism decomposes f into two separate tensors along the spatial dimension. h and f w And adjust f through the convolution function F h and f w The number of channels is used to match the input feature map, and then the data is processed through a sigmoid activation function σ to generate attention weights g in the horizontal and vertical directions, respectively. h and g w Finally, g h and g w Further, the output y is used as the attention weight and coordinate attention respectively, as shown in formulas (4), (5), and (6):

[0057] g h =σ(F h (f h (4)

[0058] g w =σ(F w (f w (5)

[0059]

[0060] Among them, f h ∈R c / r×H and f w ∈R c / r×W , where r represents the reduction rate.

[0061] Due to the high variability in the size, shape, and appearance of brain tumors in images, the U-Net network is prone to losing important semantic features during the process of downsampling to expand the receptive field and upsampling to restore the original image size. To address this issue, this paper replaces the traditional bridging module of the U-Net network with Atrous Spatial Pyramid Pooling (ASPP), such as... Figure 5 As shown, it can use dilated convolutions with different dilation rates to capture image features at different scales in parallel, thereby obtaining different receptive fields and enhancing the network's ability to fuse contextual information at different scales.

[0062] The ASPP module has five branches: a 1×1 convolution, three 3×3 dilated convolutions with dilation rates of 6, 12, and 18, and a global average pooling. This design, combining spatial pyramid pooling and dilated convolution, enhances the model's sensitivity to small-scale tumors and improves its ability to identify the edges of large-scale tumors through its parallel structure. By integrating ASPP into the model's bridging module, the model can effectively capture contextual information at different scales, from local to global, using parallel convolutional operations with varying dilation rates, thus maintaining high segmentation accuracy under conditions of different tumor sizes and complex backgrounds.

[0063] The experimental platform used in the experiment was as follows: Windows 10 Professional Edition operating system; 12th Gen Intel(R) Core(TM) i9-12900K CPU@3.20GHz processor; NVIDIA GeForce RTX 3080 graphics card; PyTorch 1.10.0 deep learning framework; and Python version 3.6.3.

[0064] This invention utilizes the publicly available datasets BraTs18 and BraTs19. The BraTs18 dataset comprises 285 cases acquired by 19 different institutions using various MRI scanners.

[15] The BraTs19 dataset, containing 210 high-grade gliomas (HGG) and 75 low-grade gliomas (LGG), was used as the training set for the model. Compared to BraTs18, the BraTs19 dataset includes more cases; to prevent overfitting during training, these additional cases were selected as the validation set. Since MRI images are stored in NIFTI format with significant contrast differences between modalities, z-score normalization was used to standardize the images across all modalities.

[0065] Since the MRS-Net network is a 2D network, while most medical images are 3D data, to ensure strong generalization ability on a limited dataset, the 240x240x155 3D images need to be converted to 2D data before training. To alleviate class imbalance, slices without lesions were discarded in the experiment. To improve network performance, each 240x240mm slice was cropped to 160x160mm while maintaining the corresponding distribution of the images.

[16] After preprocessing, the images are divided into four channels, and the dataset slices are divided into training and validation sets in an 8:2 ratio.

[0066] Accurate segmentation of brain tumor MRI images is crucial for automated radiological diagnosis and treatment planning. However, this task is challenging due to the inherent class imbalance in the data.

[17] Therefore, this paper uses a hybrid loss function combining the binary cross-entropy loss function and the Dice loss function to solve the class imbalance problem. As shown in equations (7), (8), and (9):

[0067]

[0068] L total =α·L Dice +(1-α)·L BCE (9)

[0069] Among them, L Dice L represents the Dice loss function; BCE Represents the binary cross-entropy loss function; y i Indicates a real label; p i Let L represent the predicted label of the i-th sample; N represent the total number of samples; α represents the hyperparameter, set to 0.5; L total This represents the mixed loss function.

[0070] To comprehensively evaluate the performance of the MRS-Net network proposed in this invention on the brain tumor segmentation task, this study used the following four quantitative indicators for multi-dimensional performance verification: Dice coefficient, Positive Predictive Value (PPV), Sensitivity, and Hausdorff Distance (HD). These measures three sub-regions of the tumor, as shown in equations (10), (11), (12), and (13).

[0071]

[0072] H(P, T) = max{sup} p∈P inf t∈T d(p,t),sup t∈T inf p∈P d(t, p)} (13)

[0073] Here, Dice quantifies the overlap between the predicted tumor region and the actual tumor region; PPV represents the proportion of actual tumors among all pixels predicted as tumors; Sensitivity represents the proportion of correctly detected actual tumor pixels among all real tumor pixels; and HD indicates the maximum boundary distance deviation between the predicted tumor region and the actual tumor region. TP, TN, FP, and FN represent the number of true positive, true negative, false positive, and false negative pixels, respectively. The symbols sup and inf represent the supremum and inf of the elements in the set, respectively. T represents the set of tumor regions manually labeled by experts, and P represents the set of regions predicted as tumors by the model. The function d is used to calculate the distance between two points.

[0074] To gain a deeper understanding of the contributions of each module in the model proposed in this paper, a series of ablation experiments were conducted.

[0075] The model contains several key modules, including: Dual Residual Feature Fusion (DRFF), Inverted Residual Coordinate Attention (IRCA), and Hollow Spatial Pyramid Pooling (ASPP). To systematically evaluate the effectiveness of each module, the following experimental groups were constructed: (1) baseline: a simplified model without DRFF, IRCA, and ASPP, using only the basic Unet network structure for segmentation; (2) baseline+DRFF: adding a multi-scale residual feature fusion module to the baseline model; (3) baseline+DRFF+IRCA: adding an inverted residual coordinate attention module to the baseline model+DRFF; (4) baseline+DRFF+IRCA+ASPP: including all modules. As shown in Tables 1 and 2:

[0076] Table 1: Analysis Results of Ablation Experiments

[0077]

[0078] Table 2: Analysis Results of Ablation Experiments

[0079]

[0080] The results of the ablation experiments show that each component positively contributes to improving model performance. In particular, the addition of the IRCA module further enhances the model's ability to capture detailed local features and contextual relevance; the DRFF module effectively eliminates small-region errors in segmentation by fusing features across different levels, thereby improving overall accuracy; and ASPP significantly enhances the model's ability to capture complex brain tumor structural information. These ablation experiments not only validate the effectiveness of the proposed modules but also provide a basis for future research and optimization of model structures.

[0081] This invention compares and analyzes MRS-Net with other classic image segmentation models, and evaluates the segmentation effect of brain tumor sub-regions using four indicators: Dice coefficient, positive predictive value (PPV), sensitivity, and Hausdorff distance.

[0082] Since some of the selected models do not utilize the multimodal input feature, the following constraints were set: (1) All models were trained and tested using the BraTs18 and BraTs19 datasets; (2) MRS-Net and the comparison models used the same loss function; (3) All models were optimized using stochastic gradient descent and the Adam optimizer. Furthermore, the initial learning rate was set to 0.0003, momentum to 0.1, batch size to 32, and the maximum training epochs to 400. The experimental results are shown in Table 3. It can be seen that compared to Unet, MRS-Net improved the Dice coefficient by an average of 1.85% in the whole tumor (WT), tumor core (TC), and enhanced tumor (ET) regions; PPV by 2.11%; Sensitivity by an average of 2.88%; and HD by 6.0%. The experimental results demonstrate the effectiveness of MRS-Net in brain tumor segmentation tasks.

[0083] Table 3: Performance Comparison of Six Methods

[0084]

[0085] Table 4: Performance Comparison of Six Methods

[0086]

[0087] Figure 6 The results show the segmentation results for four patients using different methods. In the first row, MRS-Net accurately segmented the target region while capturing its detailed features. In the second and third rows, when faced with target regions of complex shapes, MRS-Net effectively reduced errors compared to other models, ensuring the accuracy of the segmentation results. In the last row, when the region of interest exhibited subtle continuity, MRS-Net was able to comprehensively identify the entire tumor region. The results demonstrate that MRS-Net is suitable for complex brain tumor segmentation tasks.

[0088] Ablation experiments demonstrate that, in the encoder stage, the dual residual feature fusion module enhances information fusion between different levels; in the skip connection stage, the inverted residual coordinate attention effectively captures long-range spatial dependencies with precise location information; and in the bridging module, the use of dilated spatial convolutional pooling pyramids expands the receptive field, allowing the network to acquire richer contextual information. Comparative experiments show that MRS-Net effectively addresses the problems of complex tumor structures and insufficient utilization of cross-channel information in brain tumor MRI segmentation tasks, and its results on brain tumor segmentation tasks outperform other advanced networks such as TransUnet and DenseNet.

[0089] The specific implementation schemes described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific implementation schemes of the present invention and are not intended to limit the scope of the present invention. Any equivalent changes and modifications made by those skilled in the art without departing from the concept and principles of the present invention should fall within the scope of protection of the present invention.

Claims

1. A multi-scale residual brain tumor image segmentation method based on attention mechanism, characterized in that, Includes the following steps: Step 1: Adopt the U-net encoder-decoder architecture; Step 2: In order to fuse shallow detail feature information and deep semantic feature information in the encoder, the feature maps of each level of the encoder are upsampled to the maximum resolution and the features are stitched together. The feature is then sent to the dual residual feature fusion module DRFF through feature copying operation, so as to fuse local detail discrimination feature information and global high-level semantic discrimination feature information. The DRFF module includes shallow feature processing branches and deep feature processing branches, and uses a dual residual structure to extract feature information at different levels. The shallow feature processing branch takes a shallow feature map as input and processes it using depthwise separable convolution (DConv). This involves using a k×k convolution kernel to perform individual convolution operations on each channel of the input feature map, resulting in a depthwise convolution output feature map. Then, pointwise convolution is used to perform a weighted summation of the features of all input channels at each position of the depthwise convolution output feature map to obtain the final output feature. Finally, a 1×1 convolution is performed to reduce the dimensionality so that the number of channels is consistent with the input, and then the data is connected to the EMA module. The deep feature processing branch takes a deep feature map as input, first adjusts the number of channels using a 1×1 convolution, then normalizes the feature map using the softmax function, and then processes it using a 1×1 convolution and the Leaky ReLU activation function to output the transformed feature map. In the feature fusion stage, the low-level and high-level features output from the two branches are added together to fuse feature information at different levels. The fused feature map is then passed through a final 1×1 convolutional layer to adjust the number of channels. Finally, the feature maps from the two branches are multiplied by the fused feature map using a residual structure to obtain the final fused feature map F. fusion ; Step 3: To compensate for the semantic differences between the encoder and decoder, Inverted Residual Coordinate Attention (IRCA) is introduced in the feature layer connection stage. The feature information obtained by DRFF downsampling is fed into this module, which can effectively encode channel relationships and long-range dependencies, and improve the accuracy of the model in locating and identifying the location information of the tumor region. In step 3, the coordinate attention mechanism CA is used, and an inverted residual structure, namely inverted residual coordinate attention IRCA, is designed for brain tumor MRI images with complex feature information. The CA module includes coordinate information embedding and coordinate attention generation, and this module aims to capture long-distance spatial dependencies with precise location information. Coordinate information embedding is achieved by decomposing global pooling into feature encoding in two dimensions. For an input feature map x with height H and width W, pooling kernels with dimensions (H, 1) and (1, W) are used to encode along the horizontal and vertical directions, respectively, so as to obtain the position information of the input feature map on the x-axis and y-axis. Through feature transformation and aggregation in two directions as shown in formulas (1) and (2), output feature maps along the two dimensions are generated. （1） （2） in, This represents the output of the c-th channel at height h. This represents the output of the c-th channel with width w; after the above transformation, it will be... and The concatenation is performed, and the intermediate feature map ƒ is generated by processing it with a 1x1 convolution function F and a nonlinear activation function δ, as shown in formula (3): （3） The coordinate attention mechanism decomposes ƒ into two separate tensors along the spatial dimension. and And adjusted by the convolution function F and The number of channels is used to match the input feature map, and then the data is processed through a sigmoid activation function σ to generate attention weights in the horizontal and vertical directions, respectively. and ,at last and Further, the output y is used as the attention weight and coordinate attention respectively, as shown in formulas (4), (5), and (6): (4) (5) (6) in, and r represents the reduction rate; Step 4: The bridging module introduces the Hollow Space Pyramid Pooling (ASPP) to fuse contextual information and mine feature information. Step 5: In the decoder section, upsampling is performed by transposed convolution, and 1×1 convolution is used to restore the segmentation result to the input image size, finally generating segmentation maps of each region of the brain tumor.

2. The multi-scale residual brain tumor image segmentation method based on attention mechanism according to claim 1, characterized in that, In step 2, a novel dual residual feature fusion module is constructed by introducing depthwise separable convolution (DConv) and efficient multi-scale feature fusion module (EMA). While encoding global information to highlight the feature information of small targets, the number of model parameters is effectively reduced. The output characteristics of parallel branches are aggregated, and cross-dimensional interaction is used to enhance the cross-space processing of short-term and long-term dependencies and the information fusion between different channels, thereby preserving a wider range of contextual information. Among them, the EMA module uses parallel substructures to reduce network depth and optimizes pixel-level attention of high-level feature maps without reducing channel dimensions.

3. The multi-scale residual brain tumor image segmentation method based on attention mechanism according to claim 2, characterized in that, In step 4, the traditional bridging module of the U-Net network is replaced with Spatial Pyramid Pooling (ASPP). The ASPP module has a total of five branches, including a 1×1 convolution, three 3×3 dilated convolutions with dilation rates of 6, 12 and 18, and a global average pooling.

Images