Brain tumor segmentation method based on space attention and edge recognition enhancement

Abstract

The invention discloses a brain tumor segmentation method based on space attention and edge recognition enhancement, and the method comprises the steps: inputting an MRI brain tumor image into a trained brain tumor segmentation model, and outputting a corresponding prediction brain tumor segmentation mask; the brain tumor segmentation model is constructed based on a U-net network and comprises an encoder composed of a plurality of lower convolution modules, a decoder composed of a plurality of upper convolution modules in one-to-one correspondence with the lower convolution modules of the encoder, and an edge attention module arranged at the bottommost layer of the U-net network. The output of each lower convolution module in the encoder is in jump connection with the input of the corresponding upper convolution module in the decoder, and a space attention module is arranged in each jump connection. According to the method, the capability of capturing long-distance dependence and context information of the whole image of the network is improved through the space attention module, and complete edge feature information is provided for the feature map through the edge attention module, so that the accuracy of brain tumor segmentation is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of brain tumor segmentation, and in particular to a brain tumor segmentation method based on spatial attention and edge recognition enhancement. Background Art

[0002] With the continuous growth and aging of the population, the incidence of brain diseases is increasing year by year, posing an increasingly serious threat to people's health. Brain diseases include cerebrovascular disease (CVD), brain injury, neurological disease (NSD) and brain tumor (BT). Compared with other diseases, brain diseases have the characteristics of long incubation period, many complications, complex condition and difficult detection. Among these brain diseases, brain tumors are particularly worrying because of their high mortality rate and unsatisfactory conditions after cure. Therefore, accurate diagnosis and treatment of brain tumors are the key to alleviating patients' pain and improving survival rate.

[0003] Magnetic resonance imaging (MRI) technology uses the phenomenon of magnetic resonance to acquire and reconstruct structures in the body. Compared with computed tomography (CT), MRI can provide brain images with higher contrast. Due to its high resolution and non-invasiveness, MRI has been widely used in the diagnosis and treatment of clinical medicine. Accurate segmentation of brain MRI tumor images is crucial for disease diagnosis, surgical planning, and postoperative evaluation. Initially, medical image segmentation mainly used thresholding, region growing, edge detection, and traditional machine learning techniques. However, the accuracy and robustness of these methods are affected by the shape and size of the tumor, as well as artifacts and noise present in the image. The complex and fuzzy contours of brain tumors often lead to subjective errors by doctors, and manual expert annotation is expensive and time-consuming, making it impractical in practical applications. Automatic brain tumor segmentation based on machine learning has gradually become the mainstream of current research. However, machine learning methods based on manual feature extraction mainly focus on the pixel-level features of brain images and lack the ability to establish associations between global image features, resulting in insufficient segmentation accuracy in practical applications.

[0004] In the field of medical image segmentation, the U-net-based brain tumor segmentation model has shown excellent performance. Although the U-net network-based method has achieved certain success in medical image segmentation tasks, it still faces the following problems: 1) The network's ability to capture long-distance dependencies and contextual information of the entire image is insufficient, resulting in poor accuracy in brain tumor segmentation. 2) The network cannot effectively extract the complete edge features of the brain tumor, making it impossible to effectively identify the tumor boundary during segmentation, which also leads to poor accuracy in brain tumor segmentation. Therefore, how to design a brain tumor segmentation method that can improve the accuracy of brain tumor segmentation is a technical problem that needs to be solved urgently. Summary of the invention

[0005] In view of the above-mentioned deficiencies of the prior art, the technical problem to be solved by the present invention is: how to provide a brain tumor segmentation method based on spatial attention and edge recognition enhancement, improve the network's ability to capture long-distance dependencies and contextual information of the entire image through the spatial attention module, and provide complete edge feature information for the feature map through the edge attention module, thereby improving the accuracy of brain tumor segmentation.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] A brain tumor segmentation method based on spatial attention and edge recognition enhancement, comprising:

[0008] S1: Acquire the MRI brain tumor image to be segmented;

[0009] S2: Input the MRI brain tumor image to be segmented into the trained brain tumor segmentation model, and output the corresponding predicted brain tumor segmentation mask;

[0010] The brain tumor segmentation model is built based on the U-net network, including an encoder composed of several down-convolution modules, a decoder composed of several up-convolution modules corresponding to the down-convolution modules of the encoder, and an edge attention module set at the bottom layer of the U-net network; the output of each down-convolution module in the encoder is jump-connected to the input of the corresponding up-convolution module in the decoder, and a spatial attention module is set in each jump connection;

[0011] The processing steps when training a brain tumor segmentation model are as follows:

[0012] S201: Acquire an MRI brain tumor image as training data and a corresponding real brain tumor segmentation result as input of a brain tumor segmentation model;

[0013] S202: performing multi-scale feature extraction on the MRI brain tumor image through each down-convolution module of the encoder, and outputting an original feature map;

[0014] S203: extracting spatial features and performing weighted calculation on the feature maps output by each down-convolution module in the encoder through the spatial attention module to generate a spatial attention weighted feature map; concatenating the spatial attention weighted feature map with the feature map output by the previous up-convolution module of the decoder and inputting the concatenated feature map into the next up-convolution module;

[0015] S204: after edge features of the MRI brain tumor image are extracted by the edge attention module to generate an edge enhancement map, the edge enhancement map is multiplied by the original feature map output by the encoder to generate an edge attention map;

[0016] S205: Rescaling the edge attention map through each up-convolution module of the decoder, wherein the input of each up-convolution module is the result of splicing the spatial attention weighted feature map generated by the corresponding down-convolution module and the feature map output by the previous up-convolution module; the decoder outputs the final edge space fusion feature map;

[0017] S206: performing convolution and sigmoid activation function operations on the edge space fusion feature map output by the decoder to generate a predicted brain tumor segmentation mask;

[0018] S207: calculating a loss function by predicting a brain tumor segmentation mask and a corresponding true brain tumor segmentation result, and reversely optimizing parameters of the brain tumor segmentation model;

[0019] S208: repeating steps S201 to S207 until the brain tumor segmentation model converges or reaches a preset number of iterations;

[0020] S3: Output the brain tumor segmentation mask as the segmentation result of the image to be segmented.

[0021] Preferably, in step S202, the down-convolution module in the encoder and the up-convolution module in the decoder both include a pooling layer and a convolution layer.

[0022] Preferably, in step S203, the spatial attention module generates a spatial attention weighted feature map through the following steps:

[0023] S2031: Perform a 1×1 convolution operation and a 3×3 convolution operation on the input feature map to generate a Query matrix;

[0024] S2032: Perform a 1×1 convolution operation and a 3×3 hole convolution operation on the input feature map to generate a Key matrix;

[0025] S2033: Perform a 1×1 convolution operation and a 3×3 transposed convolution operation on the input feature map to generate a Value matrix;

[0026] S2034: After performing a transposition operation on the Key matrix, perform a dot multiplication operation with the Query matrix to calculate the attention score; perform a Softmax operation on the attention score to obtain an attention weight matrix;

[0027] S2036: Perform weighted summation on the attention weight matrix and the Value matrix to generate a spatial attention weighted feature map.

[0028] Preferably, in step S204, the edge attention module extracts edge features of the MRI brain tumor image by using a Sobel operator to generate an edge enhancement map;

[0029] The specific steps are as follows:

[0030] S2041: converting the input MRI brain tumor image into a grayscale MRI brain tumor image;

[0031] S2042: Use two 3x3 convolution kernels of the Sobel operator to perform convolution operations on the horizontal gradient and vertical gradient of the grayscale MRI brain tumor image to generate two gradient images;

[0032] S2043: Calculate the gradient amplitude of each pixel point in the two gradient images to generate a gradient amplitude image; the value of each pixel in the gradient amplitude image represents the strength of the edge at the point;

[0033] S2044: performing non-maximum suppression processing on the gradient amplitude image;

[0034] S2045: Perform thresholding processing on the gradient amplitude image after the non-maximum suppression processing to generate a binary edge enhancement image.

[0035] Preferably, in step S204, the multiplication of the edge enhancement map and the original feature map is achieved by the following steps:

[0036] S2141: performing a convolution operation on the edge enhancement map through one or more convolution layers to generate a series of feature maps;

[0037] S2142: Perform Sigmoid activation function processing on the feature map to generate a corresponding attention weight map;

[0038] S2143: Multiply the attention weight map by the original feature map element by element to generate an edge attention map.

[0039] Preferably, in step S206, the predicted brain tumor segmentation mask is edge-refined by using a fully connected conditional random field;

[0040] The specific steps are as follows:

[0041] S2061: Constructing a fully connected conditional random field model by predicting brain tumor segmentation masks;

[0042] S2062: define energy function;

[0043] S2063: Optimizing the assignment of pixel labels in a fully connected conditional random field model by minimizing an energy function;

[0044] S2064: After minimizing the energy function, the predicted brain tumor segmentation mask with edge refinement is output through the fully connected conditional random field model.

[0045] Preferably, in step S2062, the energy function E(L) is expressed by the following formula:

[0046] E(L)=-∑ i logP(l i )+∑ i,j θ ij (l i ,l j );

[0047]

[0048] Where: L represents the binary label assignment of all pixels; θ ij (l i ,l j ) represents the pair potential; μ(l i ,l j ) represents an indicator function, l i = l j , then μ(l i ,l j )=1;P i , P j Indicates the position of the pixel; I i ,I j Indicates the intensity of the pixel; p i 、p j Represents the color characteristics of the pixel; σ α Represents ||P i -P j ||The degree of influence on the pair potential; σ β Indicates ||I i -I j || has an impact on the pairwise potential; ω1 and ω2 are weight parameters.

[0049] Preferably, in step S207, the calculation formula of the loss function is as follows:

[0050] Loss BAD =Loss BCE +Loss Dice ;

[0051] Loss BCE =-W[y n *logx n +(1-x n )*log(1-x n )];

[0052]

[0053] Where: Loss BAD Represents the loss function; Loss BCE Indicates BCE loss; Loss Dicerepresents Dice loss; x n represents the predicted brain tumor segmentation mask; y n represents the actual brain tumor segmentation result; x i represents the predicted value of pixel i in the brain tumor segmentation mask; y i represents the label value of pixel i in the true brain tumor segmentation result; N represents the total number of pixels in the predicted brain tumor segmentation mask and the true brain tumor segmentation result.

[0054] Preferably, in step S1, Gaussian distribution fitting is performed on the pixel values ​​in the segmented MRI brain tumor image to remove low-intensity noise points; and the intensity values ​​of the MRI brain tumor image are internally normalized to enhance the overall brightness and readability of the MRI brain tumor image.

[0055] Preferably, in step S1, the intensity value of each pixel in the MRI brain tumor image to be segmented is transformed using the cumulative distribution function as a transformation function, and the discrete form is:

[0056]

[0057] Where: S k Represents the intensity value r of the kth pixel in the MRI brain tumor image k The intensity value after intensity transformation; L represents the total intensity value of all pixels in the MRI brain tumor image; represents the cumulative distribution function, calculated from intensity values ​​0 to r k The cumulative probability of all intensity values ​​of j Indicates the intensity value is r j The number of pixels in the MRI brain tumor image.

[0058] Compared with the prior art, the brain tumor segmentation method based on spatial attention and edge recognition enhancement in the present invention has the following advantages:

[0059] Beneficial effects:

[0060] After the brain tumor segmentation model of the present invention extracts the original feature map through the encoder, it generates a spatial attention weighted feature map through the spatial attention module and jumps to the upper convolution module of the decoder. First, the spatial attention module can highlight the key areas related to the target (brain tumor) in the image and suppress irrelevant or noise information by performing spatial feature extraction and weighted calculation on the feature map, thereby improving the network's ability to capture long-distance dependencies and contextual information of the entire image, so that the model can focus more on these important features in the decoding stage, thereby improving the accuracy of brain tumor segmentation. Secondly, the spatial attention weighted feature map in the jump connection contains information of different scales, which is fused with the feature map of the upper convolution module during the decoding process, which helps the model capture richer contextual information, thereby improving the precision and robustness of brain tumor segmentation.

[0061] After the brain tumor segmentation model of the present invention extracts the original feature map through the encoder, the edge feature extraction of the MRI brain tumor image is performed through the edge attention module to generate an edge enhancement map, and then the edge enhancement map is multiplied with the original feature map to generate an edge attention map. First, the edge features in the MRI brain tumor image are extracted by the edge attention module to generate an edge enhancement map. The edge enhancement map defines the boundary of the brain tumor area. Multiplying it with the original feature map can provide complete edge feature information for the original feature map, so that the model can more accurately identify the tumor boundary during segmentation, thereby improving the accuracy of brain tumor segmentation. Secondly, by strengthening the edge information, the edge attention module helps to reduce the "leakage" phenomenon in the segmentation process (i.e., mistakenly dividing the non-tumor area into the tumor area), thereby significantly improving the accuracy of brain tumor segmentation, especially when the boundary between the tumor and normal tissue is blurred. Finally, the introduction of the edge attention module makes the model more robust when processing MRI images with complex backgrounds and noise. By strengthening the edge information, the model can better deal with the uncertainty factors in the image and improve the stability and reliability of the segmentation results.

[0062] After the present invention generates a spatial attention weighted feature map and an edge attention map through a spatial attention module and an edge attention module, the spatial attention weighted feature map and the edge attention map are spliced ​​in the decoder. First, by splicing the spatial attention weighted feature map and the edge attention map, the key information and edge information in the two can be integrated to form a richer and more comprehensive feature representation, which helps the model to understand the image content more accurately and improve the accuracy of brain tumor segmentation. Secondly, the spatial attention module focuses on the key areas and contextual information in the image, and the edge attention module focuses on the edge features of the image. Splicing the two together can make full use of these two complementary information, so that the model can take into account both global information and local details during the decoding process, thereby generating a more refined segmentation result. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to make the purpose, technical solution and advantages of the invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings, in which:

[0064] Figure 1 This is the network architecture diagram of the brain tumor segmentation model (BSAU-net model);

[0065] Figure 2 This is the working principle diagram of the spatial attention module (SPA);

[0066] Figure 3 This is the network architecture diagram of the edge attention module (EA);

[0067] Figure 4 A comparative example of preprocessing MRI images;

[0068] Figure 5 The segmentation results of U-net, U-net++, MDU-net, AMSU-net, and BSAU-net on the BraTS2018 dataset;

[0069] Figure 6 The segmentation results of U-net, U-net++, MDU-net, AMSU-net, and BSAU-net on the BraTS2021 dataset;

[0070] Figure 7 Detailed area comparison for U-net, U-net++, MDU-net, AMSU-net, and BSAU-net on the BraTS2018 dataset

[0071] Figure 8 Detailed area comparison for U-net, U-net++, MDU-net, AMSU-net, and BSAU-net on the BraTS2021 dataset

[0072] Fig. 9 The segmentation mask is refined by CRF. DETAILED DESCRIPTION

[0073] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but only represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0074] The following is a further detailed description through specific implementation methods:

[0075] Example:

[0076] This embodiment discloses a brain tumor segmentation method based on spatial attention and edge recognition enhancement.

[0077] like Figure 1 As shown, the brain tumor segmentation method based on spatial attention and edge recognition enhancement includes:

[0078] S1: Acquire the MRI brain tumor image to be segmented;

[0079] S2: Input the MRI brain tumor image to be segmented into the trained brain tumor segmentation model (hereinafter also referred to as BSAU-net model), and output the corresponding predicted brain tumor segmentation mask;

[0080] The brain tumor segmentation model is built based on the U-net network, including an encoder composed of several down-convolution modules connected end to end in sequence, a decoder composed of several up-convolution modules corresponding to the down-convolution modules of the encoder one by one and connected end to end in sequence, and an edge attention module set at the bottom layer of the U-net network; the output of each down-convolution module in the encoder is jump-connected to the input of the corresponding up-convolution module in the decoder, and a spatial attention module is set in each jump connection;

[0081] In this embodiment, the down-convolution module in the encoder and the up-convolution module in the decoder both include a pooling layer and a convolution layer.

[0082] The processing steps when training a brain tumor segmentation model are as follows:

[0083] S201: Acquire an MRI brain tumor image as training data and a corresponding real brain tumor segmentation result as input of a brain tumor segmentation model;

[0084] S202: performing multi-scale feature extraction on the MRI brain tumor image through each down-convolution module of the encoder, and outputting an original feature map;

[0085] In this embodiment, the encoder compresses the spatial dimensions of the MRI brain tumor image through three down-convolution modules to retain the most significant features.

[0086] S203: extracting spatial features and performing weighted calculation on the feature maps output by each down-convolution module in the encoder through the spatial attention module to generate a spatial attention weighted feature map; concatenating the spatial attention weighted feature map with the feature map output by the previous up-convolution module of the decoder and inputting the concatenated feature map into the next up-convolution module;

[0087] S204: after edge features of the MRI brain tumor image are extracted by the edge attention module to generate an edge enhancement map, the edge enhancement map is multiplied by the original feature map output by the encoder to generate an edge attention map;

[0088] S205: Scale the edge attention map through each up-convolution module of the decoder, wherein the input of each up-convolution module is the result of splicing the spatial attention weighted feature map generated by the corresponding down-convolution module and the feature map output by the previous up-convolution module; the final edge space fusion feature map is output through the decoder; it should be noted that the input of the first up-convolution module in the decoder is the result of splicing the corresponding spatial attention weighted feature map and the edge attention map.

[0089] S206: performing 1×1 convolution and sigmoid activation function operations on the edge space fusion feature map output by the decoder to generate a predicted brain tumor segmentation mask;

[0090] S207: calculating a loss function by predicting a brain tumor segmentation mask and a corresponding true brain tumor segmentation result, and reversely optimizing parameters of the brain tumor segmentation model;

[0091] S208: repeating steps S201 to S207 until the brain tumor segmentation model converges or reaches a preset number of iterations;

[0092] S3: Output the brain tumor segmentation mask as the segmentation result of the image to be segmented.

[0093] After the brain tumor segmentation model of the present invention extracts the original feature map through the encoder, it generates a spatial attention weighted feature map through the spatial attention module and jumps to the upper convolution module of the decoder. First, the spatial attention module can highlight the key areas related to the target (brain tumor) in the image and suppress irrelevant or noise information by performing spatial feature extraction and weighted calculation on the feature map, thereby improving the network's ability to capture long-distance dependencies and contextual information of the entire image, so that the model can focus more on these important features in the decoding stage, thereby improving the accuracy of brain tumor segmentation. Secondly, the spatial attention weighted feature map in the jump connection contains information of different scales, which is fused with the feature map of the upper convolution module during the decoding process, which helps the model capture richer contextual information, thereby improving the precision and robustness of brain tumor segmentation.

[0094] After the brain tumor segmentation model of the present invention extracts the original feature map through the encoder, the edge feature extraction of the MRI brain tumor image is performed through the edge attention module to generate an edge enhancement map, and then the edge enhancement map is multiplied with the original feature map to generate an edge attention map. First, the edge features in the MRI brain tumor image are extracted by the edge attention module to generate an edge enhancement map. The edge enhancement map defines the boundary of the brain tumor area. Multiplying it with the original feature map can provide complete edge feature information for the original feature map, so that the model can more accurately identify the tumor boundary during segmentation, thereby improving the accuracy of brain tumor segmentation. Secondly, by strengthening the edge information, the edge attention module helps to reduce the "leakage" phenomenon in the segmentation process (i.e., mistakenly dividing the non-tumor area into the tumor area), thereby significantly improving the accuracy of brain tumor segmentation, especially when the boundary between the tumor and normal tissue is blurred. Finally, the introduction of the edge attention module makes the model more robust when processing MRI images with complex backgrounds and noise. By strengthening the edge information, the model can better deal with the uncertainty factors in the image and improve the stability and reliability of the segmentation results.

[0095] After the present invention generates a spatial attention weighted feature map and an edge attention map through a spatial attention module and an edge attention module, the spatial attention weighted feature map and the edge attention map are spliced ​​in the decoder. First, by splicing the spatial attention weighted feature map and the edge attention map, the key information and edge information in the two can be integrated to form a richer and more comprehensive feature representation, which helps the model to understand the image content more accurately and improve the accuracy of brain tumor segmentation. Secondly, the spatial attention module focuses on the key areas and contextual information in the image, and the edge attention module focuses on the edge features of the image. Splicing the two together can make full use of these two complementary information, so that the model can take into account both global information and local details during the decoding process, thereby generating a more refined segmentation result.

[0096] In order to better introduce the technical solution of the present invention, this embodiment is described through the following parts.

[0097] 1. Spatial Attention Module (SPA)

[0098] In order to better integrate global features, this embodiment proposes a spatial attention module that can capture long-distance dependencies, which can extract richer feature maps by using the spatial attention mechanism.

[0099] Combination Figure 2 As shown in Figure 2, the spatial attention module generates a spatial attention weighted feature map through the following steps:

[0100] S2031: Perform a 1×1 convolution operation and a 3×3 convolution operation on the input feature map to generate a Query matrix;

[0101] S2032: Perform a 1×1 convolution operation and a 3×3 hole convolution operation on the input feature map to generate a Key matrix;

[0102] S2033: Perform a 1×1 convolution operation and a 3×3 transposed convolution operation on the input feature map to generate a Value matrix;

[0103] S2034: After performing a transposition operation on the Key matrix, perform a dot multiplication operation with the Query matrix to calculate the attention score; perform a Softmax operation on the attention score to obtain an attention weight matrix;

[0104] S2036: Perform weighted summation on the attention weight matrix and the Value matrix to generate a spatial attention weighted feature map.

[0105] The formula is:

[0106]

[0107] Where: Y s Represents the spatial feature map; X ′ Represents the input feature map; W Q and W k Represents the weight matrix of Query and Key (the weight matrix is ​​obtained during the training process); Represents the transposed matrix of Value (a transposed matrix is ​​a matrix that is transposed).

[0108] 2. Edge Attention Module (EA)

[0109] In this embodiment, the edge attention module extracts edge features of the MRI brain tumor image through the Sobel operator to generate an edge enhancement map;

[0110] Combination Figure 3 As shown, the processing steps of the edge attention module are as follows:

[0111] S2041: converting the input MRI brain tumor image into a grayscale MRI brain tumor image;

[0112] In this embodiment, the MRI brain tumor image is usually multi-channel (such as T1, T2, etc.), but the Sobel operator needs to operate on a grayscale image. The image can be converted to grayscale using functions in an image processing library (such as OpenCV, PIL, matplotlib, etc.). In order to reduce the impact of image noise on edge detection, the grayscale image can be denoised. Commonly used denoising methods include Gaussian filtering, median filtering, etc. This step is optional and depends on the quality and noise level of the image.

[0113] S2042: Use two 3x3 convolution kernels of the Sobel operator to perform convolution operations on the horizontal gradient and vertical gradient of the grayscale MRI brain tumor image to generate two gradient images;

[0114] The convolution operation formula of the horizontal gradient is:

[0115]

[0116] The convolution operation formula for the vertical gradient is:

[0117]

[0118] S2043: Calculate the gradient amplitude of each pixel point in the two gradient images to generate a gradient amplitude image; the value of each pixel in the gradient amplitude image represents the strength of the edge at the point;

[0119] The formula for calculating the gradient magnitude is:

[0120]

[0121] S2044: performing non-maximum suppression processing on the gradient amplitude image to refine the edge;

[0122] In this embodiment, the local maximum point in the gradient direction is retained through non-maximum suppression processing, while other points are suppressed to zero or a lower value, thereby removing false edge points.

[0123] S2045: performing thresholding processing on the gradient amplitude image after non-maximum suppression processing to generate a binary edge enhancement image; thresholding processing means setting a gradient amplitude threshold, taking pixels in the gradient amplitude image that are greater than or equal to the gradient amplitude threshold as edges, and discarding pixels that are less than the gradient amplitude threshold.

[0124] In the edge attention module, the multiplication of the edge enhancement map and the original feature map is achieved through the following steps:

[0125] S2141: performing a convolution operation on the edge enhancement map through one or more convolution layers to generate a series of feature maps;

[0126] In this embodiment, the convolution kernel (also called filter or weight) in the convolution layer slides on the edge enhancement map, performs weighted summation on each local area, and may add a bias term to generate a new feature map. This process captures important information of different scales and directions in the edge enhancement map.

[0127] S2142: Perform Sigmoid activation function processing on the feature map to generate a corresponding attention weight map; each pixel value on the attention weight map represents the attention intensity that should be given to the corresponding position on the edge enhancement map;

[0128] In this embodiment, each pixel value in the feature map obtained by the convolution operation is a real number, indicating the intensity or existence of the feature at that position. However, in order to convert these feature values ​​into attention weights, they need to be normalized to the range of (0,1). This embodiment is implemented by applying the Sigmoid activation function, which maps each pixel value of the feature map to the (0,1) interval, where 0 represents the lowest importance or activation level, and 1 represents the highest importance or activation level. After being processed by the Sigmoid activation function, the feature map output by the convolution layer becomes an attention weight map.

[0129] S2143: Multiply the attention weight map by the original feature map element by element to generate an edge attention map.

[0130] In this embodiment, element-by-element multiplication is to multiply each pixel value on the attention weight map with the pixel value at the corresponding position on the original feature map. This process will emphasize the important areas in the edge enhancement map (i.e., areas with high attention weights) and suppress unimportant areas (i.e., areas with low attention weights). The edge attention map not only retains the information of the edge enhancement map, but also highlights important features through the attention mechanism, which helps to improve the performance of subsequent brain tumor segmentation tasks.

[0131] 3. Fully Connected Conditional Random Fields

[0132] In this embodiment, the predicted brain tumor segmentation mask is edge-refined by using a fully connected conditional random field;

[0133] The specific steps are as follows:

[0134] S2061: Construct a fully connected conditional random field model by predicting brain tumor segmentation masks; each pixel in the model is regarded as a node, and the relationship between pixels is regarded as an edge, thus forming a fully connected graph structure.

[0135] In this embodiment, the goal of the fully connected conditional random field model is to find the optimal pixel label assignment scheme by optimizing the energy function. This full connectivity enables the model to capture the global context information in the image, thereby better preserving edge details during the segmentation process.

[0136] S2062: define energy function;

[0137] In this embodiment, the energy function is the core of the fully connected conditional random field model, which defines the goal of model optimization. The energy function usually includes two parts: a unary potential function and a binary potential function: a unary potential function: reflects the initial probability that each pixel belongs to a certain category, and this part of information usually comes from the initial segmentation mask. A binary potential function: describes the relationship between pixels, encouraging pixels with similar features to be assigned the same label, while pixels with different features to be assigned different labels. This helps to maintain edge continuity during the segmentation process.

[0138] S2063: Optimizing the assignment of pixel labels in a fully connected conditional random field model by minimizing an energy function;

[0139] In this embodiment, during the minimization process, the model considers the dependencies between all pixels to find a globally optimal label assignment solution.

[0140] S2064: After minimizing the energy function, the predicted brain tumor segmentation mask with edge refinement is output through the fully connected conditional random field model.

[0141] In this embodiment, compared with the initial segmentation mask, the predicted brain tumor segmentation mask after edge refinement is more refined and accurate in the edge part, and can better reflect the actual boundary of the brain tumor.

[0142] Specifically, the energy function E(L) is expressed by the following formula:

[0143] E(L)=-∑ i logP(l i )+∑ i,j θ ij (l i ,l j );

[0144]

[0145] Where: L represents the binary label assignment of all pixels; θ ij(l i ,l j ) represents the pair potential; μ(l i ,l j ) represents an indicator function, l i = l j , then μ(l i ,l j )=1;P i , P j Indicates the position of the pixel; I i ,I j Indicates the intensity of the pixel; p i 、p j Represents the color characteristics of the pixel; σ α Represents ||P i -P j ||The degree of influence on the pair potential; σ β Indicates ||I i -I j || has an impact on the pairwise potential; ω1 and ω2 are weight parameters.

[0146] The present invention refines the edge of the predicted brain tumor segmentation mask through a fully connected conditional random field. On the one hand, it can effectively process the context information in the image to make the edge smoother and more natural, thereby improving the accuracy of the segmentation edge; on the other hand, it can improve the spatial consistency of the image and improve the accuracy of the segmentation.

[0147] 4. Loss Function

[0148] In medical segmentation tasks, due to the difference in the number of samples of different categories, the problem of class imbalance is common. This class imbalance will affect the accuracy of segmentation. The overlap-based method can better overcome this problem. Therefore, this embodiment proposes to use BCELoss in combination with DiceLoss.

[0149] In this embodiment, the calculation formula of the loss function is as follows:

[0150] Loss BAD =Loss BCE +Loss Dice ;

[0151] Loss BCE =-W[y n *logx n +(1-x n )*log(1-x n )];

[0152]

[0153] Where: LossBAD Represents the loss function; Loss BCE Indicates BCE loss; Loss Dice represents Dice loss; x n represents the predicted brain tumor segmentation mask; y n represents the actual brain tumor segmentation result; x i represents the predicted value of pixel i in the brain tumor segmentation mask; y i represents the label value of pixel i in the true brain tumor segmentation result; N represents the total number of pixels in the predicted brain tumor segmentation mask and the true brain tumor segmentation result.

[0154] BCE (Binary Cross-Entropy) loss, or binary cross entropy loss, is a loss function commonly used for binary classification problems. It measures the difference between the actual output (probability) and the expected output (usually 0 or 1). In image segmentation, for each pixel, the model outputs a probability value, indicating the probability that the pixel belongs to a certain category (brain tumor or non-brain tumor in this scenario). The BCE loss function calculates the loss by comparing these probability values ​​with the true label (0 or 1), thereby guiding the training of the model.

[0155] Dice loss is a loss function based on the Dice similarity coefficient, which is used to evaluate the similarity between two samples. In image segmentation tasks, Dice loss is used to compare the similarity between the predicted segmentation mask and the true segmentation mask. The Dice similarity coefficient is a value between 0 and 1, and the higher the value, the more similar the two masks are. Dice loss is defined by 1 minus the Dice similarity coefficient, so it is a value that needs to be minimized.

[0156] 5. Experimental Description

[0157] 1. Experimental environment and dataset

[0158] This experiment was conducted on a system equipped with an Intel(R) Xeon(R) Silver 4214R CPU and an NVIDIA GeForce GTX3080ti GPU, running Windows 11. The PyTorch framework was used in conjunction with CUDA version 12.1 for training and evaluation of deep learning models. The experimental parameters were set as follows: learning rate was 1e-4, batch size was 32, and momentum was 0.9.

[0159] To evaluate the performance of the proposed method, we used the widely recognized Brain Tumor Segmentation Challenge (BraTS) benchmark datasets, specifically the BraTS2018 and BraTS2021 datasets. The training set of BraTS2018 contains MRI images of 285 glioma patients, while the training set of BraTS2021 includes 1251 cases. Both datasets are stored in .nii.gz format. The datasets include images of T1, T2, T1ce, and Flair modalities, as well as corresponding unified labels.

[0160] 2. Image preprocessing

[0161] In this experiment, we first preliminarily processed the T1, T1ce, T2, and Flair slices in the BraTS2018 and BraTS2021 datasets according to the method described in the literature (Li Nan, Zhang Hongli. Research on 2D MR brain tumor image segmentation algorithm based on multimodal fusion [J]. Optoelectronics Laser, 2023, 34(08): 890-896. DOI: 10.16136 / j.joel.2023.08.0379.). The specific steps include adjusting the resolution of each slice to 240x240 pixels to ensure data consistency.

[0162] Since the tumor regions of different patients differ in position, shape, and contrast, these differences may cause the signal intensity of tumors and healthy brain tissues to overlap in MRI images, thereby increasing the complexity of image segmentation. To this end, this experiment performs Gaussian distribution fitting on the pixel values ​​in the MRI brain tumor images to be segmented to remove low-intensity noise points; the intensity values ​​of the MRI brain tumor images are internally normalized to enhance the overall brightness and readability of the MRI brain tumor images;

[0163] The calculation formula for internal normalization is:

[0164]

[0165] Wherein: A′ represents the normalized intensity value; A represents the original intensity value; A_min and A_max represent the minimum and maximum values ​​in the original intensity values.

[0166] The cumulative distribution function is used as the transformation function to transform the intensity value of each pixel in the MRI brain tumor image to be segmented. The discrete form is:

[0167]

[0168] Where: S k Represents the intensity value r of the kth pixel in the MRI brain tumor image kThe intensity value after intensity transformation; L represents the total intensity value of all pixels in the MRI brain tumor image; represents the cumulative distribution function (CDF), calculated from intensity values ​​0 to r k The cumulative probability of all intensity values ​​of j Indicates the intensity value is r j The number of pixels in the MRI brain tumor image.

[0169] Figure 4 The figure is a comparative example of preprocessing MRI images, where (a) is the original image, (b) is the preliminary processed image, and (c) is the preprocessed image.

[0170] 3. Evaluation indicators

[0171] In this experiment, we use the Dice coefficient to evaluate the similarity between the segmentation results generated by the algorithm and the true label. In addition, the positive prediction value (PPV) is used to measure the proportion of true positive samples (TP) among all predicted positive samples. Sensitivity is used to evaluate the ability of the model to identify true positive samples. During the network testing phase, the test images are sequentially input into the trained network and the results are saved. Based on these results, key variables such as true positive samples (TP), true negative samples (TN), false positive samples (FP), and false negative samples (FN) are calculated. The specific formulas for these indicators are as follows:

[0172]

[0173]

[0174] 4. Experimental results

[0175] In order to comprehensively evaluate the performance of the BSAU-net model proposed in this paper, we conducted a comparative analysis with the baseline U-net network and the methods proposed in the literature.

[0176] in:

[0177] U-net++, from Zhou, Zongwei, et al. "Unet++: A nested u-net architecture for medical image segmentation." Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 20, 2018, Proceedings 4. Springer International Publishing, 2018.

[0178] MDU-net, from Zhang, Jiawei, et al. "Mdu-net: Multi-scale densely connected u-net for biomedical image segmentation." Health Information Science and Systems 11.1(2023):13.

[0179] SCU-net++, from Cui, Hu, Haiwei Pan, and Kejia Zhang. "SCU-Net++: A Nested U-Net Based on Sharpening Filter and Channel Attention Mechanism." Wireless Communications and Mobile Computing 2022(2022):1-8.

[0180] AMSU-net, from Huang Li, He Meiling. Brain Tumor Segmentation Method Based on U-Net Improved Model[J]. Computer Technology and Development, 2022, 32(11):58-63

[0181] Attention U-net, from Oktay, O., Schlemper, J., Folgoc, LL, Lee, M., Heinrich, M., Misawa, K., et al.: Attention U-Net: Learning where to look for thepancreas.arXiv:1804.03999(2018).

[0182] All comparison methods were trained for 200 epochs on the BraTS2018 and BraTS2021 datasets.

[0183] Table 1 Quantitative analysis results of BraTS2018 dataset

[0184]

[0185] Table 2 Quantitative analysis results of BraTS2021 dataset

[0186]

[0187]

[0188] Table 1 shows the segmentation results of different methods on the BraTS2018 dataset. We can see that the method proposed in this invention has achieved better performance evaluation indicators in all aspects. Table 2 shows the different methods on the BraTS2021 dataset. We can see that except for the method with sensitivity in T1ce mode, the index is lower than AttentionU-net, and our BSAU network achieves better segmentation performance. Our segmentation method integrates the global attention mechanism to introduce the EA module to enhance the edge area features, reduce segmentation errors and blur, and improve segmentation accuracy. The introduction of the SPA module promotes the interaction of contextual information, and utilizes the correlation and contextual information between different regions in the image to expand the network's field of view, thereby achieving a more comprehensive feature representation and enhanced segmentation effect. In addition, even in the single-modal segmentation comparison, our method demonstrates excellent segmentation results, proving its robustness.

[0189] Figure 5 and Figure 6The segmentation results of our model and other models on the BraTS2018 and BraTS2021 datasets are shown. It can be observed that compared with the true label, the baseline U-net has a significant error in the overall tumor region segmentation, with low accuracy and insufficient representation of image details. In contrast, methods such as U-net++, MDU-net and AMSU-net significantly improve the feature extraction capability by introducing skip connections, dense connections and channel attention mechanisms on the U-net skeleton, thereby improving the segmentation effect of the overall tumor region.

[0190] In addition, we also compare the performance of our model with other models in detail on the segmentation of detailed edge region features. Figure 7 and Figure 8 The segmentation results on the BraTS2018 and BraTS2021 datasets are shown. The basic U-net model performs poorly in capturing the detailed features of these areas. In contrast, U-net++ and MDU-net enhance the capture of detailed area features by improving the feature connection mechanism. AMSU-net further introduces a channel attention mechanism to strengthen the interaction of feature information between different channels, thereby improving the integrity of the features. Our BSAU-net uses the Sobel operator to extract features from the edge areas of brain tumor images, and combines them with the features extracted by the network to achieve a more comprehensive feature representation, significantly improving the segmentation accuracy of the edge details of the tumor area. Finally, Fig. 9 We show the refinement of the obtained segmentation masks using the CRF random field method.

[0191] 5. Ablation Experiment Results

[0192] To further verify the effectiveness of each module in the network, we quantitatively evaluated the segmentation performance of four models on the BraTS2018 and 2021 datasets, including U-net without attention mechanism, SPA only, EA only, and the complete BSAU-net integrating SPA and EA modules. The evaluation indicators are Dice coefficient and MIoU, where the Dice coefficient is obtained by averaging the segmentation results of the complete tumor area of ​​the four modalities. The experimental results are shown in Table 3.

[0193] As can be seen from Table 3, when the SPA module is added, the entire network can obtain the connection between the previous and next features, so that the entire network can obtain more complete context information. The segmentation result is more accurate through the concatenation between features. Compared with U-net, adding SPA improves Dice by 7% and MIoU by 3%.

[0194] When the EA module is added to the network, the network enhances the perception of the tumor edge area, effectively captures the edge information of the tumor, and improves the segmentation accuracy of the network. On the dataset, Dice is improved by 5% and MIoU is improved by 2%.

[0195] The complete network BSAU-net adds both SAP and EA modules to obtain important global and local features of the image, making feature acquisition more complete and improving the ability to analyze and understand brain tumor images. On the datasets BraTS2018 and BraTS2021, Dice is improved by 10% and MIoU is improved by nearly 4%. The above series of experiments demonstrate the effectiveness of improving segmentation accuracy based on spatial attention and edge attention.

[0196] Table 3 Ablation experiment results

[0197]

[0198] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit the technical solution. Those skilled in the art should understand that those modifications or equivalent substitutions of the technical solution of the present invention that do not depart from the purpose and scope of the technical solution should be included in the scope of the claims of the present invention.

Claims

1. A brain tumor segmentation method based on spatial attention and edge recognition enhancement, characterized in that: include: S1: Acquire the MRI brain tumor image to be segmented; S2: Input the MRI brain tumor image to be segmented into the trained brain tumor segmentation model, and output the corresponding predicted brain tumor segmentation mask; The brain tumor segmentation model is built based on the U-net network, including an encoder composed of several down-convolution modules, a decoder composed of several up-convolution modules corresponding to the down-convolution modules of the encoder, and an edge attention module set at the bottom layer of the U-net network; the output of each down-convolution module in the encoder is jump-connected to the input of the corresponding up-convolution module in the decoder, and a spatial attention module is set in each jump connection; The processing steps when training a brain tumor segmentation model are as follows: S201: Acquire an MRI brain tumor image as training data and a corresponding real brain tumor segmentation result as input of a brain tumor segmentation model; S202: performing multi-scale feature extraction on the MRI brain tumor image through each down-convolution module of the encoder, and outputting an original feature map; S203: extracting spatial features and performing weighted calculation on the feature maps output by each down-convolution module in the encoder through the spatial attention module to generate a spatial attention weighted feature map; concatenating the spatial attention weighted feature map with the feature map output by the previous up-convolution module of the decoder and inputting the concatenated feature map into the next up-convolution module; S204: after edge features of the MRI brain tumor image are extracted by the edge attention module to generate an edge enhancement map, the edge enhancement map is multiplied by the original feature map output by the encoder to generate an edge attention map; S205: Rescaling the edge attention map through each up-convolution module of the decoder, wherein the input of each up-convolution module is the result of splicing the spatial attention weighted feature map generated by the corresponding down-convolution module and the feature map output by the previous up-convolution module; the decoder outputs the final edge space fusion feature map; S206: performing convolution and sigmoid activation function operations on the edge space fusion feature map output by the decoder to generate a predicted brain tumor segmentation mask; S207: calculating a loss function by predicting a brain tumor segmentation mask and a corresponding true brain tumor segmentation result, and reversely optimizing parameters of the brain tumor segmentation model; S208: repeating steps S201 to S207 until the brain tumor segmentation model converges or reaches a preset number of iterations; S3: Output the brain tumor segmentation mask as the segmentation result of the image to be segmented.

2. The brain tumor segmentation method based on spatial attention and edge recognition enhancement as claimed in claim 1, characterized in that: In step S202, the down-convolution module in the encoder and the up-convolution module in the decoder both include a pooling layer and a convolution layer.

3. The brain tumor segmentation method based on spatial attention and edge recognition enhancement as claimed in claim 1, characterized in that: In step S203, the spatial attention module generates a spatial attention weighted feature map through the following steps: S2031: Perform a 1×1 convolution operation and a 3×3 convolution operation on the input feature map to generate a Query matrix; S2032: Perform a 1×1 convolution operation and a 3×3 hole convolution operation on the input feature map to generate a Key matrix; S2033: Perform a 1×1 convolution operation and a 3×3 transposed convolution operation on the input feature map to generate a Value matrix; S2034: After transposing the Key matrix, perform a dot product operation with the Query matrix to calculate the attention score; Perform Softmax operation on the attention score to obtain the attention weight matrix; S2036: Perform weighted summation on the attention weight matrix and the Value matrix to generate a spatial attention weighted feature map.

4. The brain tumor segmentation method based on spatial attention and edge recognition enhancement as claimed in claim 1, characterized in that: In step S204, the edge attention module extracts edge features of the MRI brain tumor image using the Sobel operator to generate an edge enhancement map; The specific steps are as follows: S2041: converting the input MRI brain tumor image into a grayscale MRI brain tumor image; S2042: Use two 3x3 convolution kernels of the Sobel operator to perform convolution operations on the horizontal gradient and vertical gradient of the grayscale MRI brain tumor image to generate two gradient images; S2043: Calculate the gradient amplitude of each pixel point in the two gradient images to generate a gradient amplitude image; the value of each pixel in the gradient amplitude image represents the strength of the edge at the point; S2044: performing non-maximum suppression processing on the gradient amplitude image; S2045: Perform thresholding processing on the gradient amplitude image after the non-maximum suppression processing to generate a binary edge enhancement image.

5. The brain tumor segmentation method based on spatial attention and edge recognition enhancement as claimed in claim 4, characterized in that: In step S204, the edge enhancement map and the original feature map are multiplied by the following steps: S2141: performing a convolution operation on the edge enhancement map through one or more convolution layers to generate a series of feature maps; S2142: Perform Sigmoid activation function processing on the feature map to generate a corresponding attention weight map; S2143: Multiply the attention weight map by the original feature map element by element to generate an edge attention map.

6. The brain tumor segmentation method based on spatial attention and edge recognition enhancement as claimed in claim 1, characterized in that: In step S206, the predicted brain tumor segmentation mask is edge-refined by using a fully connected conditional random field; The specific steps are as follows: S2061: Constructing a fully connected conditional random field model by predicting brain tumor segmentation masks; S2062: define energy function; S2063: Optimizing the assignment of pixel labels in a fully connected conditional random field model by minimizing an energy function; S2064: After minimizing the energy function, the predicted brain tumor segmentation mask with edge refinement is output through the fully connected conditional random field model.

7. The brain tumor segmentation method based on spatial attention and edge recognition enhancement as claimed in claim 6, characterized in that: In step S2062, the energy function E(L) is expressed by the following formula: E(L)=−∑ i logP(l i )+∑ i,j θ ij (l i ,l j ); Where: L represents the binary label assignment of all pixels; θ ij (l i ,l j ) represents the pair potential; μ(l i ,l j ) represents an indicator function, l i = l j , then μ(l i ,l j )=1;P i , P j Indicates the position of the pixel; I i ,I j Indicates the intensity of the pixel; p u 、p j Represents the color characteristics of pixels; σ α Represents ||P i -P j ||The degree of influence on the pair potential; σ β Indicates ||I i -I j || has an impact on the pairwise potential; ω1 and ω2 are weight parameters.

8. The brain tumor segmentation method based on spatial attention and edge recognition enhancement as claimed in claim 1, characterized in that: In step S207, the calculation formula of the loss function is as follows: Loss BAD =Loss BCE +Loss Dice ; Loss BCE =-W[y n *logx n +(1-x n )*log(1-x n )]; Where: Loss BAD Represents the loss function; Loss BCE Indicates BCE loss; Loss Dice represents Dice loss; x n represents the predicted brain tumor segmentation mask; y n represents the actual brain tumor segmentation result; x i represents the predicted value of pixel i in the brain tumor segmentation mask; y i represents the label value of pixel i in the true brain tumor segmentation result; N represents the total number of pixels in the predicted brain tumor segmentation mask and the true brain tumor segmentation result.

9. The brain tumor segmentation method based on spatial attention and edge recognition enhancement as claimed in claim 1, characterized in that: In step S1, Gaussian distribution fitting is performed on the pixel values ​​in the segmented MRI brain tumor image to remove low-intensity noise points; and the intensity values ​​of the MRI brain tumor image are internally normalized to enhance the overall brightness and readability of the MRI brain tumor image.

10. The brain tumor segmentation method based on spatial attention and edge recognition enhancement as claimed in claim 1, characterized in that: In step S1, the intensity value of each pixel in the MRI brain tumor image to be segmented is transformed using the cumulative distribution function as a transformation function, and the discrete form is: Where: S k Represents the intensity value r of the kth pixel in the MRI brain tumor image k The intensity value after intensity transformation; L represents the total intensity value of all pixels in the MRI brain tumor image; represents the cumulative distribution function, calculated from intensity values ​​0 to r k The cumulative probability of all intensity values ​​of j Indicates the intensity value is r j The number of pixels in the MRI brain tumor image.

Images