Brain segmentation map acquisition method based on dual-feature attention and residual pyramid

Abstract

The present invention discloses a method for obtaining a cranial brain segmentation map based on dual-feature attention and residual pyramid. The method comprises the following steps: adding a multi-scale residual pyramid and a dual-feature attention mechanism to an original U-Net network to form a dual-feature attention residual pyramid model; inputting an original cranial brain image into the dual-feature attention residual pyramid model to obtain a cranial brain segmentation map; adding a multi-scale residual pyramid module after the convolution module of each layer of the original U-Net network; the multi-scale residual pyramid module can enhance the feature extraction capability; adding a dual-feature attention mechanism to the image channel connection part of the same layer of the original U-Net network; increasing the image size during upsampling, which causes some features to be lost; using the upper layer downsampling information and the upsampling result to perform feature fusion, thereby retaining the information of cranial brain soft tissue in the original image; and combining channel attention and spatial attention to enhance feature information and position information, thereby improving the segmentation accuracy of the model.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method for acquiring a brain segmentation map based on dual-feature attention and residual pyramid. Background Art

[0002] For brain MRI segmentation, the U-Net network is currently the most commonly used. However, the U-Net network uses the CNN model as its backbone network, which cannot establish a global model of contextual connections and is insensitive to image segmentation with larger lesion areas. Common U-Net network variants include: combining the residual module ResNet network, which combines the U-Net network with residual connections. This method can segment small, fuzzy features in terms of image segmentation performance, but the increased network depth leads to an excessive number of parameters and complex calculations; combining the hollow spatial pyramid FECC-Net network, which adds the ASPP module to the U-Net network to fully capture subtle contextual information and solve the problem of feature loss during the U-Net network downsampling process. However, this method is considered to have an insufficient number of neurons, resulting in inaccurate segmentation results when performing subtle target segmentation on small batch datasets.

[0003] At the same time, existing image processing technologies such as U-Net segmentation cannot establish a global model of contextual connections and are insensitive to image segmentation with larger lesion areas; residual network segmentation, combining the U-Net network with the residual module, can segment small, fuzzy features, but the increased network depth will lead to too many parameters and complex calculations; residual and attention network segmentation, combining the U-Net network with the residual module and the attention module, this method is mainly used for category-imbalanced datasets, and its performance on general datasets needs to be improved; atrous spatial pyramid (ASPP) network segmentation, adding the ASPP module to the U-Net network, fully captures subtle contextual information, and can solve the problem of feature loss during the U-Net network downsampling process, but the number of network neurons is insufficient, which will lead to inaccurate segmentation results when performing subtle target segmentation on small batch datasets. Summary of the Invention

[0004] Purpose of the invention: In order to overcome the shortcomings of the prior art, the present invention provides a method for acquiring brain segmentation maps based on dual-feature attention and residual pyramid, which adopts a multi-scale residual pyramid module to extract image features, and utilizes a dual-feature attention mechanism formed by a dual-feature fusion module and a dual-attention mechanism module to fuse the upper-layer downsampling results with the upsampling results, and enhance the feature information and position information, thereby improving the segmentation accuracy of the model.

[0005] Technical solution: To achieve the above-mentioned purpose, the present invention provides a method for acquiring a brain segmentation map based on dual-feature attention and residual pyramid, which is characterized by comprising a dual-feature attention residual pyramid model; the dual-feature attention residual pyramid model is composed of multiple layers of feature processing units, each layer of the feature processing units comprising a multi-scale residual pyramid module, a dual-feature fusion module, a dual-attention mechanism module and a convolution module; the first layer of feature processing units in the multi-layer feature processing units extracts multi-level features of the original image by combining the convolution module with the multi-scale residual pyramid module, the second layer of feature processing units in the multi-layer feature processing units extracts features of the previous layer to obtain a multi-level feature image by combining the convolution module with the multi-scale residual pyramid module, and so on, the nth layer of feature processing units in the multi-layer feature processing units extracts features of the n-1th layer to obtain a multi-level feature image by combining the convolution module with the multi-scale residual pyramid module;

[0006] The feature results obtained by the multi-scale residual pyramid module of each n-1 layer and the results obtained by upsampling of the n layer are fused through the dual feature fusion module in the n-1 layer feature processing unit; the dual attention mechanism module in each n-1 layer feature processing unit calibrates the features of the feature results fused by the dual feature fusion module in each n-1 layer feature processing unit; the dual attention mechanism module in each n layer feature processing unit transmits the feature results of the calibrated features to the dual feature fusion module in the n-1 layer feature processing unit through the convolution operation and upsampling operation of the convolution module, and so on;

[0007] The dual attention mechanism module of the second-layer feature processing unit transfers the feature results of the calibrated features to the dual feature fusion module of the first-layer feature processing unit through the convolution operation and upsampling operation of the convolution module, and in the dual feature fusion module, it extracts multi-level features of the original image for fusion with the first-layer feature processing unit through the combination of the convolution module and the multi-scale residual pyramid module; the dual attention mechanism module of the first-layer feature processing unit calibrates the features of the feature results fused by the dual feature fusion module in the first-layer feature processing unit, and transfers them to the output convolution layer through the convolution operation and upsampling operation of the convolution module, and finally outputs the image segmentation map of the convolution layer.

[0008] Furthermore, the residual network is combined with the dilated spatial pyramid pooling to construct a multi-scale residual pyramid module. The multi-scale residual pyramid module is based on the dilated spatial convolution, and the results of different scales obtained by each convolution are superimposed with the upper layer results to obtain multi-level features on a layer of dilated convolution. The multi-scale residual pyramid performs batch normalization before each layer of dilated convolution. The image feature information of the upper and lower layers is pixel-added to fuse the feature information. The AVG 2×2 average pooling operation is used to retain the global information of the feature image. Upsampling is used to restore the size of the feature image, and bilinear interpolation is used to obtain a result size that is consistent with the original input, and the number of channels is 5 times the original. A feature map R with C channels and a width and height of W and H are defined. H×W×C :

[0009]

[0010] Where, F Feature ∈R H×W×C , BN(x) is the batch normalization function, Indicates the first calculation output result of the nth layer;

[0011]

[0012] Where, output k ∈R H×W , represents the k-th channel feature of the feature map, AVG(x) represents the average pooling operation;

[0013]

[0014] Where conv(x) represents the convolution operation;

[0015]

[0016] Where output is the final output result, and concat(x) is the channel concatenation operation.

[0017] Furthermore, in the dual-feature attention residual pyramid model, a dual-feature fusion mechanism is added on the basis of copy connection, and the feature results obtained by the n-1 layer through the multi-scale residual pyramid module are fused with the results obtained by upsampling the n layer; in the dual-feature fusion, the feature results of the n-1 layer and the n layer are first convolved, and then the average pooling and maximum pooling operations are used respectively to extract features. After the average pooling operation, each pixel of the image is added, and after the maximum pooling operation, each pixel of the image is multiplied.

[0018] Furthermore, the number of channels of the n-1 layer feature map after the multi-scale residual pyramid operation is 5 times that of the same layer, and the number of channels of the n layer feature map after upsampling is 2 times that of the same layer. After 1×1 convolution, the number of channels is changed;

[0019] After n-1 layers of operation, the resulting feature map obtained by the multi-scale residual pyramid module is

[0020]

[0021] Where conv(x) represents the convolution operation, Represents the first output of the n-1 layer operation;

[0022]

[0023] Where AVG(x) represents the average pooling operation and MAX(x) represents the maximum pooling operation.

[0024] Furthermore, for n-layer operation, the image input into the n-layer operation is defined as

[0025]

[0026] Where upconv(x) is the upsampling operation, Represents the first output of the n-th layer operation;

[0027]

[0028] Where AVG(x) represents the average pooling operation and MAX(x) represents the maximum pooling operation.

[0029] Furthermore, after the above n-layer and n-1-layer operations, the AVG average pooling operation is performed and the pixels in the up- and down-sampled feature maps are added one by one. The MAX maximum pooling operation is performed and the up- and down-sampled results are multiplied pixel by pixel. Upsampling is used to restore the image size after the convolution and pooling operations, so that the image size is consistent with the original image, and the number of channels becomes twice the original.

[0030]

[0031] Where, output k Represents the kth channel feature of the feature map, output k (x,y) represents the eigenvalue corresponding to the coordinate (x,y) on the kth channel;

[0032]

[0033] The input image and Perform channel superposition, and the number of channels obtained is twice that of the input image;

[0034] output4 W×H×4C =upconv(output3 W×H×4C ).

[0035] Furthermore, the spatial attention mechanism in the dual attention mechanism module focuses the model's attention on the location of useful feature information. The spatial attention module uses 3×3 and 1×1 convolution to extract spatial features. The input image is the final output value of the dual feature fusion output4 W×H×4C , redefine it as image W×H×4C

[0036] In spatial attention:

[0037]

[0038] Where conv3(x) represents a convolution with a kernel size of 3, a padding value of 2, and doubles the number of channels.

[0039]

[0040] Where conv1(x) represents a convolution with a kernel size of 1, and Sigmoid(x) represents the activation function Sigmoid, which is used to normalize the spatial weight value to between 0 and 1.

[0041] Furthermore, the channel attention mechanism in the dual attention mechanism module can focus the model's attention on useful feature information.

[0042] Channel attention:

[0043]

[0044] In the formula, FC(x) represents the fully connected operation, which converts the feature map into a scalar, ReLu(x) represents the ReLu activation function,

[0045]

[0046] Where Sigmoid(x) represents the Sigmoid activation function, which normalizes the calculated result to between 0 and 1;

[0047]

[0048] In the formula, i represents a channel of the image, and the formula The calculated scalar result is multiplied by each channel of the original input to form the final feature image with channel attention;

[0049] The feature results with channel attention and the feature results with spatial attention are added channel by channel and point by point, and the final output is:

[0050]

[0051] Where, output i W×H (x, y) represents the pixel value of the feature point with coordinates (x, y) in the feature map of the i-th channel. The results obtained by spatial attention and channel attention are added channel by channel and pixel by pixel to form the final feature map with dual attention.

[0052] Further, the following steps are included:

[0053] Step 1: Obtain brain MRI images and mask images;

[0054] Step 2: Combine the residual and void space pyramid pooling to construct a multi-scale residual pyramid module, and use convolution operation and multi-scale residual pyramid module to extract the features of cranial soft tissue in the image;

[0055] Step 3: Use the maximum pooling operation to reduce the feature dimension;

[0056] Step 4: Perform dual-feature fusion on the feature results obtained by the multi-scale residual pyramid module of the n-1 layer and the results obtained by upsampling the n layer. Based on the dual-feature fusion, a dual-attention mechanism combining spatial attention and channel attention is introduced to obtain a dual-feature attention mechanism. In the channel connection part of the same layer image of the U-Net network, the dual-feature attention mechanism is introduced to calibrate the obtained features.

[0057] Step 5: Use upsampling operation to increase the scale of the feature image;

[0058] Step 6: Output the convolutional layer to output the brain soft tissue segmentation map.

[0059] Beneficial effects: The method for acquiring brain segmentation maps based on dual-feature attention and residual pyramid of the present invention, in the downsampling part, adds a multi-scale residual pyramid pooling module after convolution of each downsampling layer, for obtaining more image feature information and enhancing the connection between feature information; the multi-scale residual pyramid module can effectively expand the receptive field while retaining more image features and obtaining spatial correlation information during the downsampling process; the idea of ​​dual-feature fusion is adopted in the middle cropping and splicing part, and the maximum pooling and average pooling are fused to enhance the background image information while enhancing the texture characteristics of the tumor soft tissue, laying the foundation for foreground feature extraction in upsampling; the dual-feature attention mechanism can help restore all features of the image and reduce the probability of loss of detail features during the upsampling process, and integrate the characteristics of channel attention and spatial attention to further improve the detail features of the brain soft tissue and enhance the segmentation performance; by using the method of the present invention, the accuracy of brain soft tissue segmentation can be improved, and the exact position and boundary information of the brain soft tissue that needs to be segmented can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Attachment Figure 1 Schematic diagram of the dual-feature attention residual pyramid model;

[0061] Attachment Figure 2 Schematic diagram of the multi-scale residual pyramid module;

[0062] Attachment Figure 3 Schematic diagram of dual feature fusion module;

[0063] Attachment Figure 4 Schematic diagram of the dual attention mechanism module;

[0064] Attachment Figure 5 Comparison of image segmentation using the dual-feature attention residual pyramid model. DETAILED DESCRIPTION

[0065] The present invention will be further described below with reference to the accompanying drawings.

[0066] As attached Figure 1As shown, a method for acquiring a brain segmentation map based on dual-feature attention and residual pyramid is characterized by: including a dual-feature attention residual pyramid model; the dual-feature attention residual pyramid model is composed of multiple layers of feature processing units, each layer of the feature processing unit includes a multi-scale residual pyramid module, a dual-feature fusion module, a dual-attention mechanism module and a convolution module; the first layer of feature processing units in the multi-layer feature processing units extracts multi-level features of the original image by combining the convolution module with the multi-scale residual pyramid module, the second layer of feature processing units in the multi-layer feature processing units extracts features of the previous layer to obtain a multi-level feature image by combining the convolution module with the multi-scale residual pyramid module, and so on, the nth layer of feature processing units in the multi-layer feature processing units extracts features of the n-1th layer to obtain a multi-level feature image by combining the convolution module with the multi-scale residual pyramid module; the multi-level feature image obtained by the combination of the convolution module and the multi-scale residual pyramid module in the n-1th layer feature processing unit is down-sampled during the process of transmitting it to the convolution module of the nth layer feature processing unit, and the down-sampling operation is performed to reduce the number of channels; at the same time, a maximum pooling operation is performed on the multi-level feature image during the down-sampling operation.

[0067] The feature results obtained by the multi-scale residual pyramid module of each n-1 layer and the results obtained by upsampling of the n layer are fused through the dual feature fusion module in the n-1 layer feature processing unit; the dual attention mechanism module in each n-1 layer feature processing unit calibrates the features of the feature results fused by the dual feature fusion module in each n-1 layer feature processing unit; the above-mentioned n-1 layers are any layer in the multi-layer feature processing unit; the dual attention mechanism module in each n-layer feature processing unit transmits the feature results of the calibrated features to the dual feature fusion module in the n-1 layer feature processing unit through the convolution operation and upsampling operation of the convolution module, and the above-mentioned n layers are any layer in the multi-layer feature processing unit, and so on;

[0068] The dual attention mechanism module of the second-layer feature processing unit transfers the feature results of the calibration features to the dual feature fusion module of the first-layer feature processing unit through the convolution operation and upsampling operation of the convolution module. In the dual feature fusion module, the features of the original image multi-level features extracted by the first-layer feature processing unit through the combination of the convolution module and the multi-scale residual pyramid module are fused. The dual attention mechanism module of the first-layer feature processing unit calibrates the features of the feature results of the dual feature fusion module in the first-layer feature processing unit, and transfers them to the output convolution layer through the convolution operation and upsampling operation of the convolution module. Finally, the convolution layer outputs the image segmentation map. The convolution kernel of the output convolution layer is 1×1, which changes the number of channels of the output image to two.

[0069] The last layer of feature processing units of the multi-layer feature processing unit is different from the feature processing units of other layers. It only includes two convolution modules and a multi-scale residual pyramid module. The feature processing unit of the last layer extracts the features of the multi-level feature image obtained in the previous layer by combining the convolution module and the multi-scale residual pyramid module to obtain a feature result; the obtained feature result is directly transmitted to the dual feature fusion module of the feature processing unit of the previous layer through the convolution operation and upsampling operation of the convolution module; the feature result obtained by combining the convolution module and the multi-scale residual pyramid module in the feature processing unit of the last layer, since the last layer does not have a dual feature fusion module and a dual attention mechanism module, the feature result is not transmitted to the dual feature fusion module, and therefore the fused feature result is not transmitted to the dual attention mechanism module, but is directly transmitted to the convolution module for convolution operation, and then transmitted to the dual feature fusion module of the feature processing unit of the previous layer after upsampling operation. All of the above convolution modules are convolution layers with a convolution kernel of 3×3.

[0070] As attached Figure 2 As shown in the figure, the residual network is combined with the void space pyramid pooling to construct a multi-scale residual pyramid module; the multi-scale residual pyramid module is based on the void space convolution, and the results of different scales obtained by each convolution are superimposed with the upper layer results to obtain multi-level features on a layer of void convolution; compared with the original structure, the feature extraction is enhanced, the connection between different spatial features is improved, and the information structure of the original image is more retained, which solves the problem that simple void convolution easily loses local information and has no connection with long-distance information.

[0071] As attached Figure 2 As shown, the multi-scale residual pyramid performs batch normalization before each layer of hole convolution to prevent the problem of gradient disappearance in subsequent training and speed up network training and model convergence; 3×3 convolution is used to replace the 1×1 convolution in the original structure, and the hole rate is changed to 3. In the hole convolution, when the hole rate is 1, it is a normal convolution. In order to obtain feature information of more scales, this part is changed and the hole rates are set to 3, 6, 12, and 18 respectively, so that multi-scale receptive field information can be obtained; the image feature information of the upper and lower layers is pixel-added to fuse the feature information and enhance the connection between features; the AVG 2×2 average pooling operation is used to retain the global information of the feature image; since the image size becomes half of the original after the pooling operation, all upsampling is used to restore the feature image size, and bilinear interpolation is used. The resulting size is consistent with the original input, and the number of channels becomes 5 times the original; a feature map R with C channels and width and height of W and H are defined. H×W×C :

[0072]

[0073] Where, F Feature ∈R H×W×C , BN(x) is the batch normalization function, Indicates the first calculation output result of the nth layer;

[0074]

[0075] Where, output k ∈R H×W , represents the k-th channel feature of the feature map, AVG(x) represents the average pooling operation;

[0076]

[0077] Where conv(x) represents the convolution operation;

[0078]

[0079] Where output is the final output result, and concat(x) is the channel concatenation operation.

[0080] As attached Figure 3 As shown in the figure, in the U-Net network, the increase in image size during the sampling process can easily lead to a decrease in feature accuracy. In order to solve this problem, the downsampling results of the same layer are resized and connected through channels, but the feature image is only thickened through channels, and the connection between the features of each part is not obtained. Therefore, in the dual-feature attention residual pyramid model, a dual-feature fusion mechanism is added on the basis of copy connection, and the feature results obtained by the n-1 layer through the multi-scale residual pyramid module are fused with the results obtained by upsampling the n layer as the input for the next step of processing. In the dual-feature fusion, the feature results of the n-1 layer and the n layer are first convolved, and then the average pooling and maximum pooling operations are used respectively to perform fine feature extraction. After the average pooling operation, each pixel of the image is added, and after the maximum pooling operation, each pixel of the image is multiplied. Average pooling can retain more background feature information of the image, and maximum pooling can better extract texture information. After dual-feature fusion, the features of the background and foreground textures can be strengthened, thereby enhancing the model segmentation performance.

[0081] As attached Figure 3 As shown in the figure, after the average pooling and maximum pooling operations, each pixel of the image needs to be added or multiplied, so the image input into the pooling operation needs to ensure the uniformity of image size and number of channels; the number of channels obtained by the multi-scale residual pyramid operation of the n-1 layer feature map is 5 times that of the same layer, and the number of channels obtained by the upsampling of the n layer feature map is 2 times that of the same layer. After 1×1 convolution, the number of channels is changed;

[0082] After n-1 layers of operation, the resulting feature map obtained by the multi-scale residual pyramid module is

[0083]

[0084] Where conv(x) represents the convolution operation, Represents the first output of the n-1 layer operation;

[0085]

[0086] Where AVG(x) represents the average pooling operation and MAX(x) represents the maximum pooling operation.

[0087] n-layer operation, the image input into the n-layer operation is defined as

[0088]

[0089] Where upconv(x) is the upsampling operation, Represents the first output of the n-th layer operation;

[0090]

[0091] Where AVG(x) represents the average pooling operation and MAX(x) represents the maximum pooling operation.

[0092] As attached Figure 3 As shown in the figure, after the above n-layer and n-1-layer operations, the AVG average pooling operation is performed and the pixels in the upper and lower sampled feature maps are added one by one to enhance the background information in the brain image, which is beneficial for segmentation; after the MAX maximum pooling operation, the results obtained by upsampling and downsampling are multiplied pixel by pixel to enhance the texture features; after the convolution and pooling operations, the feature image is reduced to half of its original size, and upsampling is used to restore the image size after the convolution and pooling operations, so that the image size is consistent with the original image, that is, the final result is consistent with the image size obtained in the original U-Net network, and the channel becomes twice the original; this method can further extract the background and foreground images, expand the contrast, and improve the network performance.

[0093]

[0094] Where, output k Represents the kth channel feature of the feature map, output k (x,y) represents the eigenvalue corresponding to the coordinate (x,y) on the kth channel;

[0095]

[0096] The input image and Perform channel superposition, and the number of channels obtained is twice that of the input image;

[0097] output4 W×H×4C =upconv(output3 W×H×4C )

[0098] Since the image needs to be pooled in the feature fusion step, which reduces the image size, an up-convolution operation is required when the final image is output, and bilinear interpolation is used to increase the image to its original size.

[0099] As attached Figure 4 As shown in the figure, the channel attention mechanism in the dual attention mechanism module can focus the model's attention on useful feature information, use global average pooling for feature compression, turn the feature map into a scalar, and let the model learn the feature information of the image; while the maximum pooling operation can enhance the texture information, allowing the model to learn the final discriminative characteristics; unlike the channel attention that focuses on feature information, the spatial attention mechanism in the dual attention mechanism module focuses the model's attention on the location of useful feature information. Based on the dual feature fusion, the channel attention and spatial attention are combined to obtain a more effective feature discrimination and positioning effect;

[0100] The spatial attention module uses 3×3 and 1×1 convolutions to extract spatial features. In the 3×3 convolution, padding = 2 is used to ensure that the image size does not change; the input image is the final output value output4 of the dual feature fusion. W×H×4C , redefine it as image W×H×4C

[0101] In spatial attention:

[0102]

[0103] Where conv3(x) represents a convolution with a kernel size of 3. In order not to change the image size, the padding value is 2, and the number of channels is doubled to obtain more spatial features.

[0104]

[0105] Where conv1(x) represents a convolution with a kernel size of 1, which is used to change the number of channels of the feature image to make it consistent with the original input; Sigmoid(x) represents the activation function Sigmoid, which is used to normalize the spatial weight value to between 0 and 1 to improve the subsequent processing speed;

[0106] As attached Figure 4 As shown in the figure, in the channel attention module FC represents the fully connected layer, where the hyperparameter R is set to 16, which is used to convert the feature image into a scalar. The difference from the original structure is that this part uses two full connections and activation functions. The ReLu function can optimize the parameters, improve the calculation rate, reduce the mutual dependence between parameters, and reduce the disadvantage of using the Sigmoid function to discard information during the propagation process; finally, the Sigmoid function normalizes the weight to between 0 and 1, scales each channel, and obtains the final channel weight feature parameter, which is multiplied by the original image to obtain a feature result with channel attention;

[0107] Channel attention:

[0108]

[0109] Where FC(x) represents the fully connected operation, which converts the feature map into a scalar. ReLu(x) represents the ReLu activation function, which can set the scalar value less than or equal to 0 to 0, that is, the background feature is 0, and the part greater than 0 remains the original value, that is, the foreground remains unchanged, which can expand the weight of the effective feature.

[0110]

[0111] Where Sigmoid(x) represents the Sigmoid activation function, which normalizes the calculated result to between 0 and 1;

[0112]

[0113] In the formula, i represents a channel of the image, and the formula The calculated scalar result is multiplied by each channel of the original input to form the final feature image with channel attention;

[0114] The feature results with channel attention and the feature results with spatial attention are added channel by channel and point by point to obtain the final dual attention mechanism module, and the final output is:

[0115]

[0116] Where, output i W×H (x,y) represents the pixel value of the feature point with coordinates (x,y) in the feature map of the i-th channel. The results of spatial attention and channel attention are added channel by channel and pixel by pixel to form the final feature map with dual attention. The dual attention mechanism module is combined with the dual feature fusion module to form the dual feature attention mechanism.

[0117] A multi-scale residual pyramid and a dual-feature attention mechanism are added to the original U-Net network to form a dual-feature attention residual pyramid model. The structure of the dual-feature attention residual pyramid model includes an input module, five encoding modules, four decoding modules and an output module. The structure of the dual-feature attention residual pyramid model is shown in Table 1.

[0118] Table 1 Dual-feature attention residual pyramid model structure

[0119]

[0120] In the dual-feature attention residual pyramid model, a multi-scale residual pyramid module is added after the convolution module of each layer of the original U-Net network. The multi-scale residual pyramid module can extract features in different receptive fields, enhance feature extraction capabilities, and solve the lack of connection between long-distance features. It can retain global information while extracting detailed features at different scales, providing foreground information for subsequent brain segmentation. The downsampling process is still retained in the U-Net network to extract feature map information of different sizes to enhance the feature recognition ability of the model.

[0121] A dual-feature attention mechanism is added to the same-layer image channel connection part of the original U-Net network. During the upsampling process, some features will be lost due to the need to increase the image size. The upper-layer downsampling information is used with the upsampling result for feature fusion, which can retain more information about the cranial soft tissue in the original image. The feature information and position information are enhanced by combining channel attention and spatial attention, which can improve the segmentation accuracy of the model.

[0122] The method for obtaining a brain segmentation map based on dual-feature attention and residual pyramid includes the following steps:

[0123] Step 1: Obtain brain MRI images and mask images;

[0124] Step 2: Combine the residual and void space pyramid pooling to construct a multi-scale residual pyramid module, and use convolution operation and multi-scale residual pyramid module to extract the features of cranial soft tissue in the image;

[0125] Step 3: Use the maximum pooling operation to reduce the feature dimension;

[0126] Step 4: Perform dual-feature fusion on the feature results obtained by the multi-scale residual pyramid module of the n-1 layer and the results obtained by upsampling the n layer. Based on the dual-feature fusion, a dual-attention mechanism combining spatial attention and channel attention is introduced to obtain a dual-feature attention mechanism. In the channel connection part of the same layer image of the U-Net network, the dual-feature attention mechanism is introduced to calibrate the obtained features.

[0127] Step 5: Use upsampling operation to increase the scale of the feature image;

[0128] Step 6: Output the convolutional layer to output the brain soft tissue segmentation map.

[0129] As attached Figure 5 As shown, based on the original images in the public data set Kaggle_3M, the segmentation map of abnormal cranial soft tissue in the brain segmented by the method of the present invention is compared with the segmentation map manually segmented by experts. It is found that the segmentation map obtained by the method of the present invention has higher accuracy and more obvious features of the segmentation map; it provides a more accurate segmentation map for subsequent work and ensures the normal operation of subsequent work.

[0130] The above is only a description of the preferred embodiment of the present invention. Ordinary technicians in this technical field can make several modifications and optimizations based on the above disclosure without departing from the above basic principles. These improvements and optimizations should be regarded as the scope of protection understood by the present invention.

Claims

1. A method for obtaining brain segmentation maps based on dual-feature attention and residual pyramid, characterized by: The invention comprises a dual-feature attention residual pyramid model; the dual-feature attention residual pyramid model is composed of multiple layers of feature processing units, each layer of the feature processing unit includes a multi-scale residual pyramid module, a dual-feature fusion module, a dual-attention mechanism module and a convolution module; The first layer of feature processing units in the multi-layer feature processing unit extracts multi-level features of the original image by combining a convolution module with a multi-scale residual pyramid module. The second layer of feature processing units in the multi-layer feature processing unit extracts features of the previous layer to obtain a multi-level feature image by combining a convolution module with a multi-scale residual pyramid module. The nth layer of feature processing units in the multi-layer feature processing unit extracts features of the n-1th layer to obtain a multi-level feature image by combining a convolution module with a multi-scale residual pyramid module. The feature results obtained by the multi-scale residual pyramid module of each n-1 layer and the results obtained by upsampling of the n layer are fused through the dual feature fusion module in the n-1 layer feature processing unit; the dual attention mechanism module in each n-1 layer feature processing unit calibrates the features of the feature results fused by the dual feature fusion module in each n-1 layer feature processing unit; The dual attention mechanism module in each n-layer feature processing unit transmits the feature results of the calibration features to the dual feature fusion module in the n-1-layer feature processing unit through the convolution operation and upsampling operation of the convolution module; The dual attention mechanism module of the second-layer feature processing unit transmits the feature results of the calibration features to the dual feature fusion module of the first-layer feature processing unit through the convolution operation and upsampling operation of the convolution module. In the dual feature fusion module, the multi-level features of the original image are extracted and fused with the first-layer feature processing unit through the combination of the convolution module and the multi-scale residual pyramid module. The dual attention mechanism module of the first-layer feature processing unit calibrates the features of the feature results fused by the dual feature fusion module in the first-layer feature processing unit, and transmits them to the output convolution layer through the convolution module for convolution and upsampling operations, and finally outputs the image segmentation map of the output convolution layer; In dual feature fusion, the feature results of the n-1 layer and the n layer are first convolved, and then the average pooling and maximum pooling operations are used to extract features. After the average pooling operation, each pixel of the image is added, and after the maximum pooling operation, each pixel of the image is multiplied; The number of channels of the n-1 layer feature map after multi-scale residual pyramid operation is 5 times that of the same layer, and the number of channels of the n layer feature map after upsampling is twice that of the same layer; After n-1 layers of operation, the resulting feature map obtained by the multi-scale residual pyramid module is Where conv(x) represents the convolution operation, Represents the first output of the n-1 layer operation; Where AVG(x) represents the average pooling operation, and MAX(x) represents the maximum pooling operation; n-layer operation, the image input into the n-layer operation is defined as Where upconv(x) is the upsampling operation, Represents the first output of the n-th layer operation; Where AVG(x) represents the average pooling operation, and MAX(x) represents the maximum pooling operation; After the above n-layer and n-1-layer operations, the AVG average pooling operation is performed, and the pixels in the two feature maps are added one by one; after the MAX maximum pooling operation, the results are multiplied pixel by pixel; Upsampling is used to restore the image size after convolution and pooling operations, so that the image size is consistent with the original image and the number of channels is doubled; Where, output k Represents the kth channel feature of the feature map, output k (x,y) represents the eigenvalue corresponding to the coordinate (x,y) on the kth channel; The input image and Perform channel superposition, and the number of channels obtained is twice that of the input image; output4 W×H×4C =upconv(output3 W×H×4C )。 2. The method for acquiring a brain segmentation map based on dual-feature attention and residual pyramid according to claim 1, characterized in that: The residual network is combined with the dilated spatial pyramid pooling to construct a multi-scale residual pyramid module. The multi-scale residual pyramid module is based on the dilated spatial convolution, and the results of different scales obtained by each convolution are superimposed with the upper layer results to obtain multi-level features on a layer of dilated convolution. The multi-scale residual pyramid performs batch normalization before each layer of dilated convolution. The image feature information of the upper and lower layers is pixel-added to fuse the feature information. The AVG 2×2 average pooling operation is used to retain the global information of the feature image. Upsampling is used to restore the size of the feature image, and bilinear interpolation is used. The resulting size is consistent with the original input, and the number of channels is 5 times the original. A feature map R with C channels and a width and height of W and H are defined. H×W×C : Where, F Feature ∈R H×W×C , BN(x) is the batch normalization function, Indicates the first calculation output result of the nth layer; Where, output k ∈R H×W , represents the k-th channel feature of the feature map, AVG(x) represents the average pooling operation; Where conv(x) represents the convolution operation; Where output is the final output result, and concat(x) is the channel concatenation operation.

3. The method for acquiring a brain segmentation map based on dual-feature attention and residual pyramid according to claim 1, wherein: The spatial attention mechanism in the dual attention mechanism module focuses the model's attention on the location of useful feature information. The spatial attention module uses 3×3 and 1×1 convolution to extract spatial features. The input image is the final output value of the dual feature fusion output4 W×H×4C , redefine it as image W×H×4C In spatial attention: Where conv3(x) represents a convolution with a kernel size of 3, a padding value of 2, and doubles the number of channels. Where conv1(x) represents a convolution with a kernel size of 1, and Sigmoid(x) represents the activation function Sigmoid, which is used to normalize the spatial weight value to between 0 and 1.

4. The method for acquiring a brain segmentation map based on dual-feature attention and residual pyramid according to claim 3, characterized in that: The channel attention mechanism in the dual attention mechanism module can focus the model's attention on useful feature information. Channel attention: In the formula, FC(x) represents the fully connected operation, which converts the feature map into a scalar, ReLu(x) represents the ReLu activation function, Where Sigmoid(x) represents the Sigmoid activation function, which normalizes the calculated result to between 0 and 1; In the formula, i represents a channel of the image, and the formula The calculated scalar result is multiplied by each channel of the original input to form the final feature image with channel attention; The feature results with channel attention and the feature results with spatial attention are added channel by channel and point by point, and the final output is: Where, output i W×H (x, y) represents the pixel value of the feature point with coordinates (x, y) in the feature map of the i-th channel. The results obtained by spatial attention and channel attention are added channel by channel and pixel by pixel to form the final feature map with dual attention.

5. The method for acquiring a brain segmentation map based on dual-feature attention and residual pyramid according to claim 1, wherein: The following steps are involved: Step 1: Obtain brain MRI images and mask images; Step 2: Combine the residual and void space pyramid pooling to construct a multi-scale residual pyramid module, and use convolution operation and multi-scale residual pyramid module to extract the features of cranial soft tissue in the image; Step 3: Use the maximum pooling operation to reduce the feature dimension; Step 4: Perform dual-feature fusion on the feature results obtained by the multi-scale residual pyramid module of the n-1 layer and the results obtained by upsampling the n layer. Based on the dual-feature fusion, a dual-attention mechanism combining spatial attention and channel attention is introduced to obtain a dual-feature attention mechanism. In the channel connection part of the same layer image of the U-Net network, the dual-feature attention mechanism is introduced to calibrate the obtained features. Step 5: Use upsampling operation to increase the scale of the feature image; Step 6: Output the convolutional layer to output the brain soft tissue segmentation map.

Images