Brain tumor MRI image segmentation method and system based on multiple attention mechanisms

Abstract

The invention discloses a brain tumor MR I image segmentation method and system based on a multi-attention mechanism, and the method comprises the steps: obtaining a brain tumor MRI source image data set, and carrying out the data preprocessing, and obtaining a preprocessed brain tumor MR I source image data set; based on a UNet neural network structure, a multi-scale feature fusion module and a channel space multi-attention module are introduced, and a brain tumor MRI segmentation network model is constructed; and performing image segmentation processing on the preprocessed brain tumor MRI source image data set based on the brain tumor MRI segmentation network model to obtain a segmented brain tumor MRI image. According to the embodiment of the invention, the importance weight of the scale feature can be adaptively learned, and the channel and space dimension information can be fully utilized, so that the segmentation precision of each tumor region and boundary of the brain tumor MRI image is improved. The method can be widely applied to the technical field of medical image segmentation.

Description

Technical Field

[0001] The present application relates to the field of medical image segmentation technology, and in particular to a brain tumor MRI image segmentation method and system based on a multi-attention mechanism. Background Art

[0002] Brain tumors are the most common malignant tumors of the central nervous system in neurosurgery, which can affect peripheral nerve tissues and endanger life, health and safety. At present, magnetic resonance imaging (MRI) technology has been widely used in the diagnosis and treatment of brain tumors. Automatic and accurate segmentation of brain tumor MRI images is of great significance for the formulation of surgical plans and radiotherapy plans. According to the imaging relaxation time, it can be divided into T1 sequence, T1ce sequence, T2 sequence and Flair sequence. Currently, the commonly used brain tumor MRI datasets include the MSD (Medical Segmentation Decathlon) Challenge dataset. According to the different morphological characteristics of the tumor, it can be divided into edema, non-enhanced tumors, and enhanced tumor areas. As the most widely used network in the field of medical image segmentation, UNet has excellent segmentation performance by virtue of its symmetrical encoder-decoder structure and jump connection to restore detailed features. However, the brain tumor segmentation task is different from other medical image segmentation tasks. The position and size of the lesion area of ​​the brain tumor in the MRI image often show irregular changes, and the tumor position in different cases will also have obvious size changes. Previous improvements to the UNet network were usually on one path, the encoder and the decoder. This did not fully learn features of different scales, and insufficiently utilized information in the two dimensions of channel and space, resulting in unclear recognition of tumor area boundaries in MRI images.

[0003] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to propose a brain tumor MRI image segmentation method and system based on a multi-attention mechanism, which can adaptively learn the importance weights of scale features and make full use of channel and spatial two-dimensional information, thereby improving the segmentation accuracy of each tumor area and boundary of brain tumor MRI images.

[0005] To achieve the above objectives, one aspect of an embodiment of the present application proposes a brain tumor MRI image segmentation method based on a multi-attention mechanism, the method comprising:

[0006] Acquire a brain tumor MRI source image dataset and perform data preprocessing to obtain a preprocessed brain tumor MRI source image dataset;

[0007] Based on the UNet neural network structure, a multi-scale feature fusion module and a channel space multi-attention module were introduced to build a brain tumor MRI segmentation network model;

[0008] The preprocessed brain tumor MRI source image data set is subjected to image segmentation processing based on the brain tumor MRI segmentation network model to obtain a segmented brain tumor MRI image.

[0009] In some embodiments, the step of acquiring a brain tumor MRI source image dataset and performing data preprocessing to obtain a preprocessed brain tumor MRI source image dataset includes:

[0010] Obtain brain tumor MRI source image dataset;

[0011] Normalizing the brain tumor MRI source image data set to obtain a normalized brain tumor MRI source image data set;

[0012] Performing data cleaning processing on the normalized brain tumor MRI source image data set to obtain a cleaned brain tumor MRI source image data set;

[0013] The cleaned brain tumor MRI source image dataset is subjected to sample augmentation processing to obtain the preprocessed brain tumor MRI source image dataset.

[0014] In some embodiments, the brain tumor MRI segmentation network model includes an encoder module, a multi-scale feature fusion module, a channel space multiple attention module and a decoder module, the output end of the encoder module is connected to the input end of the multi-scale feature fusion module, the output end of the multi-scale feature fusion module is connected to the first input end of the channel space multiple attention module, the output end of the channel space multiple attention module is connected to the input end of the decoder module, and the output end of the decoder module is connected to the second input end of the channel space multiple attention module, wherein:

[0015] The encoder module has a plurality of encoders, and the plurality of encoders are connected by jumps;

[0016] The decoder module has a plurality of decoders, and the plurality of decoders are connected via jumps.

[0017] In some embodiments, the multi-scale feature fusion module includes a concatenated convolution compression layer, a first channel, a second channel, a first activation function layer and a scale attention module, the input end of the concatenated convolution compression layer is respectively connected to the input end of the first channel and the input end of the second channel, the output end of the first channel and the output end of the second channel are both connected to the input end of the first activation function layer, and the output end of the first activation function layer is connected to the input end of the scale attention module, wherein:

[0018] The first channel includes an average pooling layer and a first multi-layer perceptron layer;

[0019] The second channel includes a maximum pooling layer and a second multi-layer perceptron layer;

[0020] The scale attention module includes a first convolutional layer, a second convolutional layer, and a second activation function layer.

[0021] In some embodiments, the channel space multiple attention module includes a first coordinate attention module, a first spatial attention module, a second coordinate attention module, a second spatial attention module, a third convolutional layer and a third activation function layer, the output end of the first coordinate attention module is connected to the input end of the first spatial attention module, the output end of the second spatial attention module is connected to the input end of the second coordinate attention module, the output end of the first spatial attention module and the output end of the second coordinate attention module are both connected to the input end of the third convolutional layer, and the output end of the third convolutional layer is connected to the input end of the third activation function layer, wherein:

[0022] The first coordinate attention module and the second coordinate attention module have the same structure, both including a first average pooling layer, a second average pooling layer, a third average pooling layer, a three-dimensional convolutional splicing layer, a normalization layer, a first three-dimensional convolutional layer, a second three-dimensional convolutional layer and a third three-dimensional convolutional layer;

[0023] The first spatial attention module and the second spatial attention module have the same structure, both including a fourth convolutional layer, a fifth convolutional layer and a sixth convolutional layer.

[0024] In some embodiments, the loss function of the brain tumor MRI segmentation network model includes a weighted cross entropy loss function and a Dice loss function. The expression of the loss function of the brain tumor MRI segmentation network model is specifically as follows:

[0025] L=L w +L d

[0026]

[0027] In the above formula, L represents the loss function of the brain tumor MRI segmentation network model, L w represents the weighted cross entropy loss function, L d represents the Dice loss function, i represents the label number, I represents the total number of labels, and w i represents the weight of the i-th label, l i represents the true label value of the i-th label, f i Represents the model prediction value of the i-th label.

[0028] In some embodiments, performing image segmentation processing on the preprocessed brain tumor MRI source image dataset based on the brain tumor MRI segmentation network model to obtain a segmented brain tumor MRI image includes:

[0029] Inputting the preprocessed brain tumor MRI source image data set into the brain tumor MRI segmentation network model;

[0030] Based on the encoder module of the brain tumor MRI segmentation network model, feature extraction and fusion processing are performed on the preprocessed brain tumor MRI source image data set, and a brain tumor MRI cross-layer feature map and a brain tumor MRI corresponding layer feature map are output;

[0031] Based on the multi-scale feature fusion module of the brain tumor MRI segmentation network model, adaptive learning of different scales is performed on the cross-layer feature map of the brain tumor MRI and the corresponding layer feature map of the brain tumor MRI to obtain a multi-scale feature weight coefficient;

[0032] Based on the channel space multi-attention module of the brain tumor MRI segmentation network model, multi-channel feature extraction is performed on the corresponding layer feature map of the brain tumor MRI and the decoder feature according to the multi-scale feature weight coefficient to obtain a multi-channel feature map of the brain tumor MRI;

[0033] Based on the decoder module of the brain tumor MRI segmentation network model, the feature map of the corresponding layer of the brain tumor MRI is decoded to obtain the decoder features, which are then mapped in combination with the brain tumor MRI multi-channel feature map to obtain the segmented brain tumor MRI image.

[0034] In some embodiments, the multi-scale feature fusion module based on the brain tumor MRI segmentation network model performs adaptive learning of different scales on the brain tumor MRI cross-layer feature map and the brain tumor MRI corresponding layer feature map to obtain a multi-scale feature weight coefficient, including:

[0035] Inputting the brain tumor MRI cross-layer feature map and the brain tumor MRI corresponding layer feature map into the multi-scale feature fusion module of the brain tumor MRI segmentation network model;

[0036] Based on the splicing convolution compression layer of the multi-scale feature fusion module, the brain tumor MRI cross-layer feature map and the brain tumor MRI corresponding layer feature map are spliced ​​and compressed to obtain a brain tumor MRI mixed feature map;

[0037] Based on the first channel of the multi-scale feature fusion module, performing coefficient feature extraction processing on the brain tumor MRI mixed feature map to obtain a first coefficient;

[0038] Based on the second channel of the multi-scale feature fusion module, performing coefficient feature extraction processing on the brain tumor MRI mixed feature map to obtain a second coefficient;

[0039] Based on the first activation function layer of the multi-scale feature fusion module, the first coefficient and the second coefficient are combined and mapped to obtain a channel coefficient;

[0040] The channel coefficient is multiplied by the brain tumor MRI mixed feature map and input into the scale attention module of the multi-scale feature fusion module for convolution processing to obtain a feature scale coefficient;

[0041] The feature scale coefficient is combined with the brain tumor MRI mixed feature map to obtain the multi-scale feature weight coefficient.

[0042] In some embodiments, the channel space multi-attention module based on the brain tumor MRI segmentation network model performs multi-channel feature extraction on the brain tumor MRI corresponding layer feature map and decoder features according to the multi-scale feature weight coefficient to obtain a brain tumor MRI multi-channel feature map, including:

[0043] Inputting the brain tumor MRI corresponding layer feature map and the decoder features into the channel space multi-attention module of the brain tumor MRI segmentation network model;

[0044] Based on the first coordinate attention module of the channel space multiple attention module, coordinate attention feature extraction processing is performed on the corresponding layer feature map of the brain tumor MRI to obtain encoder coordinate features;

[0045] The encoder coordinate feature is combined with the decoder feature and input into the first spatial attention module of the channel space multi-attention module for spatial feature extraction to obtain a first multi-channel feature map;

[0046] A second spatial attention module based on the channel spatial multi-attention module performs spatial feature extraction on the brain tumor MRI corresponding layer feature map and the decoder feature to obtain a decoder spatial feature;

[0047] The decoder spatial features are combined with the brain tumor MRI corresponding layer feature map and input into the second coordinate attention module of the channel space multi-attention module for coordinate attention feature extraction processing to obtain a second multi-channel feature map;

[0048] The first multi-channel feature map and the second multi-channel feature map are combined and input into the third convolution layer and the third activation function layer of the channel space multi-attention module for convolution mapping to obtain the brain tumor MRI multi-channel feature map.

[0049] To achieve the above objectives, another aspect of the embodiment of the present application proposes a brain tumor MRI image segmentation system based on a multi-attention mechanism, the system comprising:

[0050] The first module is used to obtain a brain tumor MRI source image dataset and perform data preprocessing to obtain a preprocessed brain tumor MRI source image dataset;

[0051] The second module is used to build a brain tumor MRI segmentation network model based on the UNet neural network structure, introducing a multi-scale feature fusion module and a channel space multi-attention module;

[0052] The third module is used to perform image segmentation processing on the preprocessed brain tumor MRI source image data set based on the brain tumor MRI segmentation network model to obtain a segmented brain tumor MRI image.

[0053] The embodiments of the present application include at least the following beneficial effects: The present application provides a brain tumor MRI image segmentation method and system based on a multi-attention mechanism. The scheme obtains a brain tumor MRI source image data set and performs data preprocessing to obtain a preprocessed brain tumor MRI source image data set, and then introduces a multi-scale feature fusion module and a channel space multi-attention module based on the UNet neural network structure to construct a brain tumor MRI segmentation network model. The multi-scale feature fusion module learns features of different levels and scales, and the scale attention is used to adaptively learn the importance weights of scale features, so that the model can perceive the correlation between different scale features and tumor areas of different sizes, thereby improving the network's learning ability for tumor features. The channel space multi-attention module learns the importance coefficients of features in two dimensions, namely, channel and space, to suppress the expression of irrelevant features and highlight important features. Finally, based on the brain tumor MRI segmentation network model, the preprocessed brain tumor MRI source image data set is subjected to image segmentation processing to obtain a segmented brain tumor MRI image, thereby improving the segmentation accuracy of each tumor area and boundary of the brain tumor MRI image. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 is a flowchart of a brain tumor MRI image segmentation method based on a multi-attention mechanism provided in an embodiment of the present application;

[0055] Figure 2 It is a structural schematic diagram of a brain tumor MRI image segmentation system based on a multi-attention mechanism provided in an embodiment of the present application;

[0056] Figure 3 It is a schematic diagram of the structure of the brain tumor MRI segmentation network model provided in the embodiment of the present application;

[0057] Figure 4is a schematic diagram of the structure of a multi-scale feature fusion module provided in an embodiment of the present application;

[0058] Figure 5 It is a schematic diagram of the structure of the channel space multi-attention module provided in an embodiment of the present application;

[0059] Figure 6 is a schematic diagram of the structure of the coordinate attention module provided in an embodiment of the present application;

[0060] Figure 7 is a schematic diagram of the structure of the spatial attention module provided in an embodiment of the present application;

[0061] Figure 8 3 is a schematic diagram comparing the segmentation results of the MSD data set provided in the embodiment of the present application. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are only examples of systems and methods consistent with some aspects of the embodiments of the present application as detailed in the attached claims.

[0063] It is understood that the terms "first", "second", etc. used in this application can be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiment of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination".

[0064] The terms "at least one", "multiple", "each", "any", etc. used in this application, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0065] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0066] Reference Figure 1 , Figure 1 A flowchart of a brain tumor MRI image segmentation method based on a multi-attention mechanism provided by an embodiment of the present invention, referring to Figure 1 , the method comprises the following steps:

[0067] S100, acquiring a brain tumor MRI source image dataset and performing data preprocessing to obtain a preprocessed brain tumor MRI source image dataset;

[0068] It should be noted that, in some embodiments, step S100 may include: S110, acquiring a brain tumor MRI source image dataset; S120, normalizing the brain tumor MRI source image dataset to obtain a normalized brain tumor MRI source image dataset; S130, performing data cleaning processing on the normalized brain tumor MRI source image dataset to obtain a cleaned brain tumor MRI source image dataset; S140, performing sample augmentation processing on the cleaned brain tumor MRI source image dataset to obtain a preprocessed brain tumor MRI source image dataset.

[0069] In some specific embodiments, the brain tumor segmentation dataset of the MSD (Medical Segmentation Decathlon) challenge is an open source dataset. The brain tumor MRI source image dataset is obtained by downloading the open source dataset, and then normalized. In order to make the intensity value of the MRI image more uniform and improve the accuracy of tumor boundary recognition, the MRI image is Z-scoring normalized, the mean of the image is set to 0 and the variance is set to 1 to eliminate anisotropy, and further data cleaning is performed. In view of the class imbalance in the dataset, all blank slices with zero values ​​are deleted, and each slice is cropped to the non-zero value area. Finally, sample augmentation is performed. During the training process, random rotation, scaling, radial transformation and cropping are used to enhance the data to improve the generalization ability of the model.

[0070] S200, based on the UNet neural network structure, introduces a multi-scale feature fusion module and a channel space multi-attention module to build a brain tumor MRI segmentation network model;

[0071] It should be noted that in some embodiments, Figure 3As shown, the brain tumor MRI segmentation network model includes an encoder module En, a multi-scale feature fusion module MSFF, a channel space multiple attention module CSA and a decoder module De, the output end of the encoder module is connected to the input end of the multi-scale feature fusion module, the output end of the multi-scale feature fusion module is connected to the first input end of the channel space multiple attention module, the output end of the channel space multiple attention module is connected to the input end of the decoder module, and the output end of the decoder module is connected to the second input end of the channel space multiple attention module, wherein the encoder module has a plurality of encoders, and the plurality of encoders are connected by jumps; the decoder module has a plurality of decoders, and the plurality of decoders are connected by jumps.

[0072] Further, it should be noted that in some specific embodiments, such as Figure 4 As shown, the multi-scale feature fusion module includes a splicing convolution compression layer, a first channel, a second channel, a first activation function layer and a scale attention module, the input end of the splicing convolution compression layer is respectively connected to the input end of the first channel and the input end of the second channel, the output end of the first channel and the output end of the second channel are both connected to the input end of the first activation function layer, and the output end of the first activation function layer is connected to the input end of the scale attention module, wherein the first channel includes an average pooling layer Pavg and a first multi-layer perceptron layer MLP; the second channel includes a maximum pooling layer Pmax and a second multi-layer perceptron layer MLP; the scale attention module includes a first convolution layer Conv1, a second convolution layer Conv2 and a second activation function layer.

[0073] Among them, Conv1 is a convolution operation of size 3×3, Conv2 is a convolution operation of size 1×1, the step sizes of these two convolutions are both 1 and neither changes the number of feature map channels, and ε is the ReLU activation function.

[0074] In this embodiment, during the network training process, UNet connects the shallow features in the encoder and the deep features in the decoder through jump connections, so that information at each level can be retained. However, only using features of the same scale for connection will ignore the semantic gap between the encoder and the decoder, and will not be able to fully utilize the shallow low-level detail features and deep high-level semantic features. In addition, the scale differences in the position and shape of brain tumors in MRI images will further aggravate this semantic gap. To this end, we design a multi-scale feature fusion module. After fusing the features of encoders at different layers, the multi-scale feature fusion module explicitly models the most significant feature maps at multiple scales by adaptively learning the specific scale weights of MRI images, and learns the relationship between feature maps of different scales and the size of the target area, so that the network is sensitive to the size of the tumor and improves the ability to segment the target area.

[0075] Further, it should be noted that in some specific embodiments, such as Figure 5 As shown, the channel space multi-attention module includes a first coordinate attention module CA1, a first spatial attention module SA1, a second coordinate attention module CA2, a second spatial attention module SA2, a third convolutional layer and a third activation function layer. The output end of the first coordinate attention module is connected to the input end of the first spatial attention module, the output end of the second spatial attention module is connected to the input end of the second coordinate attention module, the output end of the first spatial attention module and the output end of the second coordinate attention module are both connected to the input end of the third convolutional layer, and the output end of the third convolutional layer is connected to the input end of the third activation function layer. The first coordinate attention module and the second coordinate attention module have the same structure, as shown in FIG. Figure 6 As shown, both include the first average pooling layer P avg,h , the second average pooling layer P avg,w , the third average pooling layer P avg,d , 3D convolution concatenation layer, normalization layer, first 3D convolution layer, second 3D convolution layer and third 3D convolution layer; the first spatial attention module and the second spatial attention module have the same structure. Figure 7 As shown, they all include the fourth convolution layer, the fifth convolution layer and the sixth convolution layer.

[0076] In this embodiment, in view of the irregular shape of brain gliomas in MRI images, the large differences in tumors between different patients, and the blurred boundaries, we borrowed from the CBAM module and designed a channel spatial attention module to infer attention coefficients along the two dimensions of channel and space, learn the importance weights of features, and thus improve the segmentation accuracy of the target area. The channel spatial multi-attention module consists of two branches, namely channel attention CA1 and CA2 and spatial attention SA1 and SA2.

[0077] Furthermore, the loss function of the brain tumor MRI segmentation network model in the embodiment of the present invention includes a weighted cross entropy loss function and a Dice loss function. The expression of the loss function of the brain tumor MRI segmentation network model is specifically as follows:

[0078] L=L w +L d

[0079]

[0080] In the above formula, L represents the loss function of the brain tumor MRI segmentation network model, L w represents the weighted cross entropy loss function, L d represents the Dice loss function, i represents the label number, I represents the total number of labels, and w i represents the weight of the i-th label, l i represents the true label value of the i-th label, fi Represents the model prediction value of the i-th label.

[0081] S300, performing image segmentation processing on the preprocessed brain tumor MRI source image dataset based on the brain tumor MRI segmentation network model to obtain a segmented brain tumor MRI image;

[0082] It should be noted that, in some embodiments, step S300 may include:

[0083] S310 , inputting the preprocessed brain tumor MRI source image dataset into the brain tumor MRI segmentation network model.

[0084] S320, an encoder module based on the brain tumor MRI segmentation network model, performing feature extraction and fusion processing on the preprocessed brain tumor MRI source image data set, and outputting a brain tumor MRI cross-layer feature map and a brain tumor MRI corresponding layer feature map;

[0085] S330, a multi-scale feature fusion module based on a brain tumor MRI segmentation network model, performing adaptive learning of different scales on the brain tumor MRI cross-layer feature map and the brain tumor MRI corresponding layer feature map to obtain a multi-scale feature weight coefficient;

[0086] Specifically, it should be noted that step S330 also includes: S331, inputting the brain tumor MRI cross-layer feature map and the brain tumor MRI corresponding layer feature map into the multi-scale feature fusion module of the brain tumor MRI segmentation network model; S332, based on the splicing convolution compression layer of the multi-scale feature fusion module, splicing and compressing the brain tumor MRI cross-layer feature map and the brain tumor MRI corresponding layer feature map to obtain a brain tumor MRI mixed feature map; S333, based on the first channel of the multi-scale feature fusion module, performing coefficient feature extraction processing on the brain tumor MRI mixed feature map to obtain a first coefficient; S334, based on the multi-scale feature fusion module, performing coefficient feature extraction processing on the brain tumor MRI mixed feature map to obtain a first coefficient; 334. Based on the second channel of the multi-scale feature fusion module, the coefficient feature extraction process is performed on the brain tumor MRI mixed feature map to obtain the second coefficient; S335. Based on the first activation function layer of the multi-scale feature fusion module, the first coefficient and the second coefficient are combined and mapped to obtain the channel coefficient; S336. The channel coefficient is multiplied by the brain tumor MRI mixed feature map and input into the scale attention module of the multi-scale feature fusion module for convolution processing to obtain the feature scale coefficient; S337. The feature scale coefficient is combined with the brain tumor MRI mixed feature map to obtain the multi-scale feature weight coefficient.

[0087] In this embodiment, the multi-scale feature fusion module first transforms the cross-layer feature map X j And the corresponding layer feature map X iSplicing, a multi-scale feature map with low-level local features and high-level semantic features is obtained. To reduce the amount of calculation, the multi-scale feature map is compressed into a 3-channel mixed feature map X using 1×1 convolution. Then the mixed feature map X is sent to the average pooling Pavg and multi-layer perceptron MLP and the maximum pooling Pmax and MLP combination respectively. The obtained coefficients are added and then passed through the Sigmoid activation function to obtain the channel coefficient γ. Multiply γ by X and input the scale attention module to obtain the feature scale coefficient, which is expressed as:

[0088] γ * =Pavg(ε(Conv2(σ(Conv1(γX)))))

[0089] In the above formula, γ * represents the feature scale coefficient, γ represents the channel coefficient, Pavg(·) represents the average pooling operation, ε(·) and σ(·) represent the activation operation, Conv2(·) represents the second convolution operation, Conv1(·) represents the first convolution operation, and X represents the mixed feature map.

[0090] Finally, the output of the multi-scale feature fusion module is γ * γX+γX+X.

[0091] S340, a channel space multi-attention module based on the brain tumor MRI segmentation network model, performing multi-channel feature extraction on the brain tumor MRI corresponding layer feature map and decoder features according to the multi-scale feature weight coefficient, to obtain a brain tumor MRI multi-channel feature map;

[0092] Specifically, it should be noted that step S340 also includes: S341, inputting the brain tumor MRI corresponding layer feature map and the decoder feature into the channel space multi-attention module of the brain tumor MRI segmentation network model; S342, based on the first coordinate attention module of the channel space multi-attention module, performing coordinate attention feature extraction processing on the brain tumor MRI corresponding layer feature map to obtain the encoder coordinate feature; S343, combining the encoder coordinate feature with the decoder feature and inputting them into the first spatial attention module of the channel space multi-attention module to extract spatial features to obtain the first multi-channel feature map; S344, based on the channel The second spatial attention module of the spatial multiple attention module performs spatial feature extraction on the feature map of the corresponding layer of brain tumor MRI and the decoder features to obtain the decoder spatial features; S345, combines the decoder spatial features with the feature map of the corresponding layer of brain tumor MRI and inputs them into the second coordinate attention module of the channel spatial multiple attention module for coordinate attention feature extraction processing to obtain a second multi-channel feature map; S346, combines the first multi-channel feature map with the second multi-channel feature map and inputs them into the third convolution layer and the third activation function layer of the channel spatial multiple attention module for convolution mapping to obtain a multi-channel feature map of brain tumor MRI.

[0093] In this embodiment, the first branch is composed of CA1 and SA1. The former first extracts the encoder feature X i The important channels related to the target area are filtered out, and then the SA1 module analyzes X i and the next layer decoder feature Y i , focusing on the spatial location of the tumor area. To further explicitly model the correlation dependency between channel and spatial features, the second branch first analyzes the spatial position and then locks the important channel features, effectively enhancing feature correlation and suppressing redundant feature expression. Finally, the feature maps obtained by the first and second branches are restored to the original dimension through 1×1 convolution, and then activated by the ReLU function to obtain the output of the channel space multi-attention module.

[0094] S350, a decoder module based on the brain tumor MRI segmentation network model decodes the feature map of the corresponding layer of the brain tumor MRI to obtain decoder features, and then maps them in combination with the brain tumor MRI multi-channel feature map to obtain a segmented brain tumor MRI image.

[0095] In summary, the brain tumor MRI segmentation network model of the embodiment of the present invention adopts the UNet encoder-decoder structure. The left side is the encoder part, which consists of convolution, batch normalization and activation function, and performs downsampling operation through maximum pooling. The middle part upsamples the feature maps of each layer of the encoder to the maximum resolution, performs feature splicing, and then sends it to the multi-scale feature fusion module through downsampling. The multi-scale feature fusion module enables the network to adaptively learn specific weights of different scales by learning the correlation between feature maps of different scales and target areas of different sizes; and designs a channel space attention module to suppress the expression of irrelevant areas and highlight the features that are more critical to the segmentation task, helping the model to more accurately locate and identify the location information of the tumor area. In the decoder part, transposed convolution is used for upsampling operation, and finally the Softmax activation function is used to map the multi-channel features to the corresponding tumor sub-regions to obtain the segmentation result.

[0096] like Figure 8 As shown in the figure, the segmentation results of edema, non-enhanced tumor, enhanced tumor and whole tumor area are 80.35, 62.79, 80.60 and 74.56 on the Dice coefficient index of the MSD dataset. The segmentation results of each area are 7.31, 19.33, 6.54 and 11.08 on the Hausdorff 95 distance index.

[0097] Therefore, the embodiments of the present invention have the following beneficial effects:

[0098] 1) Design a brain tumor MRI image segmentation algorithm based on a multi-attention mechanism, which can more accurately segment the edema, non-enhanced tumor, and enhanced tumor areas of brain tumors in multi-sequence MRI images.

[0099] 2) In order to fully and effectively utilize shallow low-level detail features and deep high-level semantic features, a multi-scale feature fusion module is designed to adaptively learn the relationship between scale feature weights and the sizes of various brain tumor regions.

[0100] 3) Design a channel-space multi-attention module to learn attention coefficients in both channel and space dimensions, effectively improving the network's segmentation accuracy of the target area and more accurately segmenting tumor boundaries.

[0101] See also Figure 2 The embodiment of the present application also provides a brain tumor MRI image segmentation system based on a multi-attention mechanism, which can implement the above-mentioned brain tumor MRI image segmentation method based on a multi-attention mechanism. The system includes:

[0102] The first module 201 is used to acquire a brain tumor MRI source image dataset and perform data preprocessing to obtain a preprocessed brain tumor MRI source image dataset;

[0103] The second module 202 is used to construct a brain tumor MRI segmentation network model based on the UNet neural network structure by introducing a multi-scale feature fusion module and a channel space multi-attention module;

[0104] The third module 203 is used to perform image segmentation processing on the preprocessed brain tumor MRI source image data set based on the brain tumor MRI segmentation network model to obtain a segmented brain tumor MRI image.

[0105] It can be understood that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0106] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A brain tumor MRI image segmentation method based on multi-attention mechanism, characterized in that: The method comprises the following steps: Acquire a brain tumor MRI source image dataset and perform data preprocessing to obtain a preprocessed brain tumor MRI source image dataset; Based on the UNet neural network structure, a multi-scale feature fusion module and a channel space multi-attention module were introduced to build a brain tumor MRI segmentation network model; The preprocessed brain tumor MRI source image data set is subjected to image segmentation processing based on the brain tumor MRI segmentation network model to obtain a segmented brain tumor MRI image.

2. The method according to claim 1, characterized in that The step of acquiring a brain tumor MRI source image dataset and performing data preprocessing to obtain a preprocessed brain tumor MRI source image dataset includes: Obtain brain tumor MRI source image dataset; Normalizing the brain tumor MRI source image data set to obtain a normalized brain tumor MRI source image data set; Performing data cleaning processing on the normalized brain tumor MRI source image data set to obtain a cleaned brain tumor MRI source image data set; The cleaned brain tumor MRI source image dataset is subjected to sample augmentation processing to obtain the preprocessed brain tumor MRI source image dataset.

3. The method according to claim 1, characterized in that The brain tumor MRI segmentation network model includes an encoder module, a multi-scale feature fusion module, a channel space multiple attention module and a decoder module, wherein the output end of the encoder module is connected to the input end of the multi-scale feature fusion module, the output end of the multi-scale feature fusion module is connected to the first input end of the channel space multiple attention module, the output end of the channel space multiple attention module is connected to the input end of the decoder module, and the output end of the decoder module is connected to the second input end of the channel space multiple attention module, wherein: The encoder module has a plurality of encoders, and the plurality of encoders are connected by jumps; The decoder module has a plurality of decoders, and the plurality of decoders are connected via jumps.

4. The method according to claim 3, characterized in that The multi-scale feature fusion module includes a concatenated convolution compression layer, a first channel, a second channel, a first activation function layer and a scale attention module, wherein the input end of the concatenated convolution compression layer is respectively connected to the input end of the first channel and the input end of the second channel, the output end of the first channel and the output end of the second channel are both connected to the input end of the first activation function layer, and the output end of the first activation function layer is connected to the input end of the scale attention module, wherein: The first channel includes an average pooling layer and a first multi-layer perceptron layer; The second channel includes a maximum pooling layer and a second multi-layer perceptron layer; The scale attention module includes a first convolutional layer, a second convolutional layer, and a second activation function layer.

5. The method according to claim 3, characterized in that: The channel space multiple attention module includes a first coordinate attention module, a first spatial attention module, a second coordinate attention module, a second spatial attention module, a third convolutional layer and a third activation function layer, the output end of the first coordinate attention module is connected to the input end of the first spatial attention module, the output end of the second spatial attention module is connected to the input end of the second coordinate attention module, the output end of the first spatial attention module and the output end of the second coordinate attention module are both connected to the input end of the third convolutional layer, and the output end of the third convolutional layer is connected to the input end of the third activation function layer, wherein: The first coordinate attention module and the second coordinate attention module have the same structure, both including a first average pooling layer, a second average pooling layer, a third average pooling layer, a three-dimensional convolutional splicing layer, a normalization layer, a first three-dimensional convolutional layer, a second three-dimensional convolutional layer and a third three-dimensional convolutional layer; The first spatial attention module and the second spatial attention module have the same structure, both including a fourth convolutional layer, a fifth convolutional layer and a sixth convolutional layer.

6. The method according to claim 1, characterized in that The loss function of the brain tumor MRI segmentation network model includes a weighted cross entropy loss function and a Dice loss function. The expression of the loss function of the brain tumor MRI segmentation network model is specifically as follows: L=L w +L d In the above formula, L represents the loss function of the brain tumor MRI segmentation network model, L w represents the weighted cross entropy loss function, L d represents the Dice loss function, i represents the label number, I represents the total number of labels, and w i represents the weight of the i-th label, l i represents the true label value of the i-th label, f i Represents the model prediction value of the i-th label.

7. The method according to claim 3, characterized in that The performing image segmentation processing on the preprocessed brain tumor MRI source image dataset based on the brain tumor MRI segmentation network model to obtain a segmented brain tumor MRI image includes: Inputting the preprocessed brain tumor MRI source image data set into the brain tumor MRI segmentation network model; Based on the encoder module of the brain tumor MRI segmentation network model, feature extraction and fusion processing are performed on the preprocessed brain tumor MRI source image data set, and a brain tumor MRI cross-layer feature map and a brain tumor MRI corresponding layer feature map are output; Based on the multi-scale feature fusion module of the brain tumor MRI segmentation network model, adaptive learning of different scales is performed on the cross-layer feature map of the brain tumor MRI and the corresponding layer feature map of the brain tumor MRI to obtain a multi-scale feature weight coefficient; Based on the channel space multi-attention module of the brain tumor MRI segmentation network model, multi-channel feature extraction is performed on the corresponding layer feature map of the brain tumor MRI and the decoder feature according to the multi-scale feature weight coefficient to obtain a multi-channel feature map of the brain tumor MRI; Based on the decoder module of the brain tumor MRI segmentation network model, the feature map of the corresponding layer of the brain tumor MRI is decoded to obtain the decoder features, which are then mapped in combination with the brain tumor MRI multi-channel feature map to obtain the segmented brain tumor MRI image.

8. The method according to claim 5, characterized in that The multi-scale feature fusion module based on the brain tumor MRI segmentation network model performs adaptive learning of different scales on the brain tumor MRI cross-layer feature map and the brain tumor MRI corresponding layer feature map to obtain a multi-scale feature weight coefficient, including: Inputting the brain tumor MRI cross-layer feature map and the brain tumor MRI corresponding layer feature map into the multi-scale feature fusion module of the brain tumor MRI segmentation network model; Based on the splicing convolution compression layer of the multi-scale feature fusion module, the brain tumor MRI cross-layer feature map and the brain tumor MRI corresponding layer feature map are spliced ​​and compressed to obtain a brain tumor MRI mixed feature map; Based on the first channel of the multi-scale feature fusion module, performing coefficient feature extraction processing on the brain tumor MRI mixed feature map to obtain a first coefficient; Based on the second channel of the multi-scale feature fusion module, performing coefficient feature extraction processing on the brain tumor MRI mixed feature map to obtain a second coefficient; Based on the first activation function layer of the multi-scale feature fusion module, the first coefficient and the second coefficient are combined and mapped to obtain a channel coefficient; The channel coefficient is multiplied by the brain tumor MRI mixed feature map and input into the scale attention module of the multi-scale feature fusion module for convolution processing to obtain a feature scale coefficient; The feature scale coefficient is combined with the brain tumor MRI mixed feature map to obtain the multi-scale feature weight coefficient.

9. The method according to claim 5, characterized in that The channel space multi-attention module based on the brain tumor MRI segmentation network model performs multi-channel feature extraction on the brain tumor MRI corresponding layer feature map and decoder features according to the multi-scale feature weight coefficient to obtain a brain tumor MRI multi-channel feature map, including: Inputting the brain tumor MRI corresponding layer feature map and the decoder features into the channel space multi-attention module of the brain tumor MRI segmentation network model; Based on the first coordinate attention module of the channel space multiple attention module, coordinate attention feature extraction processing is performed on the corresponding layer feature map of the brain tumor MRI to obtain encoder coordinate features; The encoder coordinate feature is combined with the decoder feature and input into the first spatial attention module of the channel space multi-attention module for spatial feature extraction to obtain a first multi-channel feature map; A second spatial attention module based on the channel spatial multi-attention module performs spatial feature extraction on the brain tumor MRI corresponding layer feature map and the decoder feature to obtain a decoder spatial feature; The decoder spatial features are combined with the brain tumor MRI corresponding layer feature map and input into the second coordinate attention module of the channel space multi-attention module for coordinate attention feature extraction processing to obtain a second multi-channel feature map; The first multi-channel feature map and the second multi-channel feature map are combined and input into the third convolution layer and the third activation function layer of the channel space multi-attention module for convolution mapping to obtain the brain tumor MRI multi-channel feature map.

10. A brain tumor MRI image segmentation system based on multi-attention mechanism, characterized in that: The system comprises: The first module is used to obtain a brain tumor MRI source image dataset and perform data preprocessing to obtain a preprocessed brain tumor MRI source image dataset; The second module is used to build a brain tumor MRI segmentation network model based on the UNet neural network structure, introducing a multi-scale feature fusion module and a channel space multi-attention module; The third module is used to perform image segmentation processing on the preprocessed brain tumor MRI source image data set based on the brain tumor MRI segmentation network model to obtain a segmented brain tumor MRI image.

Images