A brain tumor segmentation method based on spatial attention and edge recognition enhancement

Abstract

The present invention discloses a brain tumor segmentation method based on spatial attention and edge recognition enhancement, comprising: inputting an MRI brain tumor image into a trained brain tumor segmentation model and outputting a corresponding predicted brain tumor segmentation mask; the brain tumor segmentation model is constructed based on a U-net network, comprising an encoder composed of several down-convolution modules, a decoder composed of several up-convolution modules corresponding one-to-one to the down-convolution modules of the encoder, and an edge attention module arranged at the bottom layer of the U-net network; the output of each down-convolution module in the encoder is jump-connected to the input of the corresponding up-convolution module in the decoder, and a spatial attention module is arranged in each jump connection. The present invention improves the network's ability to capture long-range dependencies and contextual information of the entire image through the spatial attention module, and provides complete edge feature information for the feature map through the edge attention module, thereby improving the accuracy of brain tumor segmentation.

Description

Technical Field

[0001] The present invention relates to the technical field of brain tumor segmentation, and in particular to a brain tumor segmentation method based on spatial attention and edge recognition enhancement. Background Art

[0002] With the continued growth and aging of the population, the incidence of brain diseases is increasing year by year, posing an increasingly serious threat to people's health. Brain diseases include cerebrovascular disease (CVD), brain injury, neurological system diseases (NSD), and brain tumors (BT). Compared with other diseases, brain diseases have long incubation periods, many complications, complex conditions, and difficulty in detection. Among these brain diseases, brain tumors are particularly worrying due to their high mortality rate and unsatisfactory outcomes after recovery. Therefore, accurate diagnosis and treatment of brain tumors are key to alleviating patient suffering and improving survival rates.

[0003] Magnetic resonance imaging (MRI) utilizes magnetic resonance phenomena to acquire and reconstruct structures within the body. Compared to computed tomography (CT), MRI provides higher-contrast brain images. Due to its high resolution and non-invasive nature, MRI has been widely used in clinical diagnosis and treatment. Accurate segmentation of brain MRI tumor images is crucial for disease diagnosis, surgical planning, and postoperative evaluation. Initially, medical image segmentation primarily employed thresholding, region growing, edge detection, and traditional machine learning techniques. However, the accuracy and robustness of these methods are affected by tumor shape and size, as well as artifacts and noise present in the images. The complex and ambiguous contours of brain tumors often lead to subjective errors by physicians, and manual expert annotation is expensive and time-consuming, making it impractical for practical applications. Automated brain tumor segmentation based on machine learning has gradually become a mainstream research topic. However, machine learning methods based on manual feature extraction primarily focus on pixel-level features of brain images and lack the ability to establish correlations between global image features, resulting in insufficient segmentation accuracy in practical applications.

[0004] In the field of medical image segmentation, U-net-based brain tumor segmentation models have demonstrated excellent performance. While these methods have achieved some success in medical image segmentation, they still face the following challenges: 1) The network's ability to capture long-range dependencies and contextual information across the entire image is insufficient, resulting in poor brain tumor segmentation accuracy. 2) The network's inability to effectively extract complete edge features of brain tumors makes it difficult to effectively identify tumor boundaries during segmentation, similarly leading to poor segmentation accuracy. Therefore, designing a brain tumor segmentation method that can improve segmentation accuracy is a pressing technical challenge. Summary of the Invention

[0005] In view of the above-mentioned deficiencies in the prior art, the technical problem to be solved by the present invention is: how to provide a brain tumor segmentation method based on spatial attention and edge recognition enhancement, improve the network's ability to capture long-distance dependencies and the contextual information of the entire image through the spatial attention module, and provide complete edge feature information for the feature map through the edge attention module, thereby improving the accuracy of brain tumor segmentation.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] A brain tumor segmentation method based on spatial attention and edge recognition enhancement, comprising:

[0008] S1: Acquire the MRI brain tumor image to be segmented;

[0009] S2: Input the MRI brain tumor image to be segmented into the trained brain tumor segmentation model and output the corresponding predicted brain tumor segmentation mask;

[0010] The brain tumor segmentation model is built on a U-net network. It includes an encoder consisting of several downconvolutional modules, a decoder consisting of several upconvolutional modules that correspond one-to-one with the encoder's downconvolutional modules, and an edge attention module at the bottom layer of the U-net network. The output of each downconvolutional module in the encoder is jump-connected to the input of the corresponding upconvolutional module in the decoder, and each jump connection is equipped with a spatial attention module.

[0011] The processing steps when training a brain tumor segmentation model are as follows:

[0012] S201: Acquire an MRI brain tumor image as training data and a corresponding real brain tumor segmentation result as input to a brain tumor segmentation model;

[0013] S202: performing multi-scale feature extraction on the MRI brain tumor image through each down-convolution module of the encoder, and outputting an original feature map;

[0014] S203: Perform spatial feature extraction and weighted calculation on the feature map output by each down-convolution module in the encoder through the spatial attention module to generate a spatial attention weighted feature map; splice the spatial attention weighted feature map with the feature map output by the previous up-convolution module of the decoder and input the result into the next up-convolution module;

[0015] S204: After edge features are extracted from the MRI brain tumor image by the edge attention module to generate an edge enhancement map, the edge enhancement map is multiplied by the original feature map output by the encoder to generate an edge attention map;

[0016] S205: Rescale the edge attention map through each upconvolution module of the decoder, where the input of each upconvolution module is the result of concatenating the spatial attention weighted feature map generated by the corresponding downconvolution module and the feature map output by the previous upconvolution module; the decoder outputs the final edge spatial fusion feature map;

[0017] S206: performing convolution and sigmoid activation function operations on the edge space fusion feature map output by the decoder to generate a predicted brain tumor segmentation mask;

[0018] S207: Calculating a loss function by predicting the brain tumor segmentation mask and the corresponding true brain tumor segmentation result, and reversely optimizing the parameters of the brain tumor segmentation model;

[0019] S208: Repeat steps S201 to S207 until the brain tumor segmentation model converges or reaches a preset number of iterations;

[0020] S3: Output the brain tumor segmentation mask as the segmentation result of the image to be segmented.

[0021] Preferably, in step S202, the down-convolution module in the encoder and the up-convolution module in the decoder both include a pooling layer and a convolution layer.

[0022] Preferably, in step S203, the spatial attention module generates a spatial attention weighted feature map through the following steps:

[0023] S2031: Perform 1×1 convolution and 3×3 convolution operations on the input feature map to generate a Query matrix;

[0024] S2032: Perform a 1×1 convolution operation and a 3×3 dilated convolution operation on the input feature map to generate a Key matrix;

[0025] S2033: Perform a 1×1 convolution operation and a 3×3 transposed convolution operation on the input feature map to generate a Value matrix;

[0026] S2034: After performing a transposition operation on the Key matrix, perform a dot product operation with the Query matrix to calculate the attention score; perform a Softmax operation on the attention score to obtain the attention weight matrix;

[0027] S2036: Perform weighted summation on the attention weight matrix and the Value matrix to generate a spatial attention weighted feature map.

[0028] Preferably, in step S204, the edge attention module extracts edge features from the MRI brain tumor image using a Sobel operator to generate an edge enhancement map;

[0029] The specific steps are as follows:

[0030] S2041: converting the input MRI brain tumor image into a grayscale MRI brain tumor image;

[0031] S2042: Use two 3x3 convolution kernels of the Sobel operator to perform convolution operations on the horizontal gradient and vertical gradient of the grayscale MRI brain tumor image, respectively, to generate two gradient images;

[0032] S2043: Calculate the gradient magnitude of each pixel in the two gradient images to generate a gradient magnitude image; the value of each pixel in the gradient magnitude image represents the strength of the edge at that point;

[0033] S2044: performing non-maximum suppression processing on the gradient magnitude image;

[0034] S2045: Performing thresholding processing on the gradient amplitude image after the non-maximum suppression processing to generate a binary edge enhancement image.

[0035] Preferably, in step S204, the edge enhancement map and the original feature map are multiplied by the following steps:

[0036] S2141: Perform convolution operations on the edge enhancement map through one or more convolutional layers to generate a series of feature maps;

[0037] S2142: Perform Sigmoid activation function processing on the feature map to generate the corresponding attention weight map;

[0038] S2143: Multiply the attention weight map by the original feature map element by element to generate an edge attention map.

[0039] Preferably, in step S206, edge refinement is performed on the predicted brain tumor segmentation mask using a fully connected conditional random field;

[0040] The specific steps are as follows:

[0041] S2061: Constructing a fully connected conditional random field model by predicting brain tumor segmentation masks;

[0042] S2062: Define energy function;

[0043] S2063: Optimizing pixel label assignment in a fully connected conditional random field model by minimizing an energy function;

[0044] S2064: After minimizing the energy function, the fully connected conditional random field model is used to output the predicted brain tumor segmentation mask with edge refinement.

[0045] Preferably, in step S2062, the energy function E(L) is expressed by the following formula:

[0046] E(L)=-∑ i logP(l i )+∑ i,j θ ij (l i ,l j );

[0047]

[0048] Where: L represents the binary label assignment of all pixels; θ ij (l i ,l j ) represents the pair potential; μ(l i ,l j ) represents an indicator function, l i =l j , then μ(l i ,l j )=1;P i 、P j Indicates the position of the pixel; I i , I j Indicates the intensity of the pixel; p i 、p j Represents the color characteristics of the pixel; σ α Represents ||P i -P j ||The degree of influence on pair potential; σ β Indicates ||I i -I j || has an impact on the pairwise potential; ω1 and ω2 are weight parameters.

[0049] Preferably, in step S207, the calculation formula of the loss function is as follows:

[0050] Loss BAD =Loss BCE +Loss Dice ;

[0051] Loss BCE =-W[y n *logx n +(1-x n )*log(1-x n )];

[0052]

[0053] Where: Loss BAD Represents the loss function; Loss BCE Indicates BCE loss; Loss Dicerepresents Dice loss; x n represents the predicted brain tumor segmentation mask; y n represents the actual brain tumor segmentation result; x i represents the predicted value of pixel i in the brain tumor segmentation mask; y i represents the label value of pixel i in the true brain tumor segmentation result; N represents the total number of pixels in the predicted brain tumor segmentation mask and the true brain tumor segmentation result.

[0054] Preferably, in step S1, Gaussian distribution fitting is performed on the pixel values ​​in the segmented MRI brain tumor image to remove low-intensity noise points; and the intensity values ​​of the MRI brain tumor image are internally normalized to enhance the overall brightness and readability of the MRI brain tumor image.

[0055] Preferably, in step S1, the intensity value of each pixel in the MRI brain tumor image to be segmented is transformed using the cumulative distribution function as a transformation function, and the discrete form is:

[0056]

[0057] Where: S k Represents the intensity value r of the kth pixel in the MRI brain tumor image k The intensity value after intensity transformation; L represents the total intensity value of all pixels in the MRI brain tumor image; represents the cumulative distribution function, calculated from the intensity value 0 to r k The cumulative probability of all intensity values ​​of n j Indicates the intensity value is r j The number of pixels in the MRI brain tumor image; n represents the total number of pixels in the MRI brain tumor image.

[0058] Compared with the prior art, the brain tumor segmentation method based on spatial attention and edge recognition enhancement in the present invention has the following advantages:

[0059] Beneficial effects:

[0060] After the brain tumor segmentation model of the present invention extracts the original feature map through the encoder, it generates a spatial attention weighted feature map through the spatial attention module and jumps to the up-convolution module of the decoder. First, the spatial attention module can highlight the key areas related to the target (brain tumor) in the image by performing spatial feature extraction and weighted calculation on the feature map, suppress irrelevant or noise information, and thus improve the network's ability to capture long-distance dependencies and contextual information of the entire image, so that the model can focus more on these important features in the decoding stage, thereby improving the accuracy of brain tumor segmentation. Secondly, the spatial attention weighted feature map in the jump connection contains information of different scales. This information is fused with the feature map of the up-convolution module during the decoding process, which helps the model capture richer contextual information, thereby improving the precision and robustness of brain tumor segmentation.

[0061] After the brain tumor segmentation model of the present invention extracts the original feature map through the encoder, it extracts the edge features of the MRI brain tumor image through the edge attention module to generate an edge enhancement map, and then multiplies the edge enhancement map with the original feature map to generate an edge attention map. First, the edge features in the MRI brain tumor image are extracted by the edge attention module to generate an edge enhancement map. The edge enhancement map defines the boundary of the brain tumor area. Multiplying it with the original feature map can provide complete edge feature information for the original feature map, so that the model can more accurately identify the tumor boundary during segmentation, thereby improving the accuracy of brain tumor segmentation. Secondly, by strengthening the edge information, the edge attention module helps to reduce the "leakage" phenomenon in the segmentation process (i.e., mistakenly dividing the non-tumor area into the tumor area), thereby significantly improving the accuracy of brain tumor segmentation, especially when the boundary between the tumor and normal tissue is blurred. Finally, the introduction of the edge attention module makes the model more robust when processing MRI images with complex background and noise. By strengthening the edge information, the model can better deal with the uncertainty factors in the image and improve the stability and reliability of the segmentation results.

[0062] After the present invention generates the spatial attention weighted feature map and the edge attention map through the spatial attention module and the edge attention module, the spatial attention weighted feature map and the edge attention map are spliced ​​in the decoder. First, by splicing the spatial attention weighted feature map and the edge attention map, the key information and edge information in the two can be integrated to form a richer and more comprehensive feature representation, which helps the model to understand the image content more accurately and improve the accuracy of brain tumor segmentation. Secondly, the spatial attention module focuses on the key areas and contextual information in the image, and the edge attention module focuses on the edge features of the image. Splicing the two together can make full use of these two complementary information, so that the model can take into account both global information and local details during the decoding process, thereby generating more refined segmentation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to make the purpose, technical solutions and advantages of the invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings, in which:

[0064] Figure 1 This is the network architecture diagram of the brain tumor segmentation model (BSAU-net model);

[0065] Figure 2 This is the working principle diagram of the spatial attention module (SPA);

[0066] Figure 3 This is the network architecture diagram of the edge attention module (EA);

[0067] Figure 4 A comparative example of preprocessing MRI images;

[0068] Figure 5 The segmentation results of U-net, U-net++, MDU-net, AMSU-net, and BSAU-net on the BraTS2018 dataset;

[0069] Figure 6 Segmentation results of U-net, U-net++, MDU-net, AMSU-net, and BSAU-net on the BraTS2021 dataset;

[0070] Figure 7 Compare detailed regions of U-net, U-net++, MDU-net, AMSU-net, and BSAU-net on the BraTS2018 dataset

[0071] Figure 8 Compare detailed regions of U-net, U-net++, MDU-net, AMSU-net, and BSAU-net on the BraTS2021 dataset

[0072] Figure 9 The segmentation mask is refined by CRF. DETAILED DESCRIPTION

[0073] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the invention claimed for protection, but only represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0074] The following is a further detailed description through specific implementation methods:

[0075] Example:

[0076] This embodiment discloses a brain tumor segmentation method based on spatial attention and edge recognition enhancement.

[0077] like Figure 1 As shown in the figure, the brain tumor segmentation method based on spatial attention and edge recognition enhancement includes:

[0078] S1: Acquire the MRI brain tumor image to be segmented;

[0079] S2: Input the MRI brain tumor image to be segmented into the trained brain tumor segmentation model (hereinafter also referred to as the BSAU-net model), and output the corresponding predicted brain tumor segmentation mask;

[0080] The brain tumor segmentation model is built on a U-net network. It includes an encoder consisting of several downconvolutional modules connected end-to-end, a decoder consisting of several upconvolutional modules that correspond one-to-one to the encoder's downconvolutional modules and are connected end-to-end, and an edge attention module located at the bottom layer of the U-net network. The output of each downconvolutional module in the encoder is jump-connected to the input of the corresponding upconvolutional module in the decoder, and each jump connection is equipped with a spatial attention module.

[0081] In this embodiment, the down-convolution module in the encoder and the up-convolution module in the decoder both include a pooling layer and a convolution layer.

[0082] The processing steps when training a brain tumor segmentation model are as follows:

[0083] S201: Acquire an MRI brain tumor image as training data and a corresponding real brain tumor segmentation result as input to a brain tumor segmentation model;

[0084] S202: performing multi-scale feature extraction on the MRI brain tumor image through each down-convolution module of the encoder, and outputting an original feature map;

[0085] In this embodiment, the encoder compresses the spatial dimensions of the MRI brain tumor image through three down-convolution modules to retain the most significant features.

[0086] S203: Perform spatial feature extraction and weighted calculation on the feature map output by each down-convolution module in the encoder through the spatial attention module to generate a spatial attention weighted feature map; splice the spatial attention weighted feature map with the feature map output by the previous up-convolution module of the decoder and input the result into the next up-convolution module;

[0087] S204: After edge features are extracted from the MRI brain tumor image by the edge attention module to generate an edge enhancement map, the edge enhancement map is multiplied by the original feature map output by the encoder to generate an edge attention map;

[0088] S205: The edge attention map is scaled through each up-convolution module of the decoder, where the input of each up-convolution module is the result of splicing the spatial attention weighted feature map generated by the corresponding down-convolution module and the feature map output by the previous up-convolution module; the final edge spatial fusion feature map is output through the decoder; it should be noted that the input of the first up-convolution module in the decoder is the result of splicing the corresponding spatial attention weighted feature map and the edge attention map.

[0089] S206: performing 1×1 convolution and sigmoid activation function operations on the edge space fusion feature map output by the decoder to generate a predicted brain tumor segmentation mask;

[0090] S207: Calculating a loss function by predicting the brain tumor segmentation mask and the corresponding true brain tumor segmentation result, and reversely optimizing the parameters of the brain tumor segmentation model;

[0091] S208: Repeat steps S201 to S207 until the brain tumor segmentation model converges or reaches a preset number of iterations;

[0092] S3: Output the brain tumor segmentation mask as the segmentation result of the image to be segmented.

[0093] After the brain tumor segmentation model of the present invention extracts the original feature map through the encoder, it generates a spatial attention weighted feature map through the spatial attention module and jumps to the up-convolution module of the decoder. First, the spatial attention module can highlight the key areas related to the target (brain tumor) in the image by performing spatial feature extraction and weighted calculation on the feature map, suppress irrelevant or noise information, and thus improve the network's ability to capture long-distance dependencies and contextual information of the entire image, so that the model can focus more on these important features in the decoding stage, thereby improving the accuracy of brain tumor segmentation. Secondly, the spatial attention weighted feature map in the jump connection contains information of different scales. This information is fused with the feature map of the up-convolution module during the decoding process, which helps the model capture richer contextual information, thereby improving the precision and robustness of brain tumor segmentation.

[0094] After the brain tumor segmentation model of the present invention extracts the original feature map through the encoder, it extracts the edge features of the MRI brain tumor image through the edge attention module to generate an edge enhancement map, and then multiplies the edge enhancement map with the original feature map to generate an edge attention map. First, the edge features in the MRI brain tumor image are extracted by the edge attention module to generate an edge enhancement map. The edge enhancement map defines the boundary of the brain tumor area. Multiplying it with the original feature map can provide complete edge feature information for the original feature map, so that the model can more accurately identify the tumor boundary during segmentation, thereby improving the accuracy of brain tumor segmentation. Secondly, by strengthening the edge information, the edge attention module helps to reduce the "leakage" phenomenon in the segmentation process (i.e., mistakenly dividing the non-tumor area into the tumor area), thereby significantly improving the accuracy of brain tumor segmentation, especially when the boundary between the tumor and normal tissue is blurred. Finally, the introduction of the edge attention module makes the model more robust when processing MRI images with complex background and noise. By strengthening the edge information, the model can better deal with the uncertainty factors in the image and improve the stability and reliability of the segmentation results.

[0095] After the present invention generates the spatial attention weighted feature map and the edge attention map through the spatial attention module and the edge attention module, the spatial attention weighted feature map and the edge attention map are spliced ​​in the decoder. First, by splicing the spatial attention weighted feature map and the edge attention map, the key information and edge information in the two can be integrated to form a richer and more comprehensive feature representation, which helps the model to understand the image content more accurately and improve the accuracy of brain tumor segmentation. Secondly, the spatial attention module focuses on the key areas and contextual information in the image, and the edge attention module focuses on the edge features of the image. Splicing the two together can make full use of these two complementary information, so that the model can take into account both global information and local details during the decoding process, thereby generating more refined segmentation results.

[0096] In order to better introduce the technical solution of the present invention, this embodiment is described through the following parts.

[0097] 1. Spatial Attention Module (SPA)

[0098] In order to better integrate global features, this embodiment proposes a spatial attention module that can capture long-distance dependencies, which can extract richer feature maps by applying the spatial attention mechanism.

[0099] Combine Figure 2 As shown in Figure 2, the spatial attention module generates a spatial attention weighted feature map through the following steps:

[0100] S2031: Perform 1×1 convolution and 3×3 convolution operations on the input feature map to generate a Query matrix;

[0101] S2032: Perform a 1×1 convolution operation and a 3×3 dilated convolution operation on the input feature map to generate a Key matrix;

[0102] S2033: Perform a 1×1 convolution operation and a 3×3 transposed convolution operation on the input feature map to generate a Value matrix;

[0103] S2034: After performing a transposition operation on the Key matrix, perform a dot product operation with the Query matrix to calculate the attention score; perform a Softmax operation on the attention score to obtain the attention weight matrix;

[0104] S2036: Perform weighted summation on the attention weight matrix and the Value matrix to generate a spatial attention weighted feature map.

[0105] The formula is:

[0106]

[0107] Where: Y s Represents the spatial feature map; X ′ Represents the input feature map; W Q and W k Represents the weight matrix of Query and Key (the weight matrix is ​​obtained during the training process); Represents the transposed matrix of Value (a transposed matrix is ​​a matrix that performs a transposed operation).

[0108] 2. Edge Attention Module (EA)

[0109] In this embodiment, the edge attention module extracts edge features from the MRI brain tumor image using the Sobel operator to generate an edge enhancement map;

[0110] Combine Figure 3 As shown in Figure 2, the processing steps of the edge attention module are as follows:

[0111] S2041: converting the input MRI brain tumor image into a grayscale MRI brain tumor image;

[0112] In this embodiment, MRI brain tumor images are typically multi-channel (e.g., T1, T2, etc.), but the Sobel operator must operate on grayscale images. Functions in image processing libraries (e.g., OpenCV, PIL, matplotlib, etc.) can be used to convert the image to grayscale. To reduce the impact of image noise on edge detection, the grayscale image can be denoised. Common denoising methods include Gaussian filtering and median filtering. This step is optional and depends on the image quality and noise level.

[0113] S2042: Use two 3x3 convolution kernels of the Sobel operator to perform convolution operations on the horizontal gradient and vertical gradient of the grayscale MRI brain tumor image, respectively, to generate two gradient images;

[0114] The convolution operation formula of the horizontal gradient is:

[0115]

[0116] The convolution operation formula of the vertical gradient is:

[0117]

[0118] S2043: Calculate the gradient magnitude of each pixel in the two gradient images to generate a gradient magnitude image; the value of each pixel in the gradient magnitude image represents the strength of the edge at that point;

[0119] The formula for calculating the gradient magnitude is:

[0120]

[0121] S2044: performing non-maximum suppression processing on the gradient magnitude image to refine the edge;

[0122] In this embodiment, the local maximum point in the gradient direction is retained by the non-maximum suppression process, while other points are suppressed to zero or lower values, thereby removing false edge points.

[0123] S2045: Thresholding processing is performed on the gradient amplitude image after non-maximum suppression processing to generate a binary edge enhancement image; thresholding processing refers to setting a gradient amplitude threshold, treating pixels in the gradient amplitude image that are greater than or equal to the gradient amplitude threshold as edges, and discarding pixels that are less than the gradient amplitude threshold.

[0124] In the edge attention module, the multiplication of the edge enhancement map and the original feature map is achieved through the following steps:

[0125] S2141: Perform convolution operations on the edge enhancement map through one or more convolutional layers to generate a series of feature maps;

[0126] In this embodiment, the convolution kernel (also called filter or weight) in the convolution layer slides over the edge enhancement map, performing a weighted summation on each local region and possibly adding a bias term to generate a new feature map. This process captures important information at different scales and directions in the edge enhancement map.

[0127] S2142: Apply Sigmoid activation function to the feature map to generate a corresponding attention weight map; each pixel value on the attention weight map represents the attention intensity that should be given to the corresponding position on the edge enhancement map;

[0128] In this embodiment, each pixel value in the feature map obtained by the convolution operation is a real number, indicating the intensity or existence of the feature at that position. However, in order to convert these feature values ​​into attention weights, they need to be normalized to the range of (0, 1). This embodiment is achieved by applying the Sigmoid activation function, which maps each pixel value of the feature map to the (0, 1) interval, where 0 represents the lowest importance or activation level, and 1 represents the highest importance or activation level. After processing by the Sigmoid activation function, the feature map output by the convolution layer becomes an attention weight map.

[0129] S2143: Multiply the attention weight map by the original feature map element by element to generate an edge attention map.

[0130] In this embodiment, element-by-element multiplication multiplies each pixel value on the attention weight map by the pixel value at the corresponding position on the original feature map. This process emphasizes important areas in the edge enhancement map (i.e., areas with high attention weights) and suppresses unimportant areas (i.e., areas with low attention weights). The edge attention map not only retains the information of the edge enhancement map but also highlights important features through the attention mechanism, which helps improve the performance of subsequent brain tumor segmentation tasks.

[0131] 3. Fully Connected Conditional Random Field

[0132] In this embodiment, the edge of the predicted brain tumor segmentation mask is refined using a fully connected conditional random field;

[0133] The specific steps are as follows:

[0134] S2061: Construct a fully connected conditional random field model by predicting brain tumor segmentation masks; each pixel in the model is regarded as a node, and the relationship between pixels is regarded as an edge, thus forming a fully connected graph structure.

[0135] In this embodiment, the goal of the fully connected conditional random field model is to find the optimal pixel label assignment scheme by optimizing the energy function. This fully connected nature enables the model to capture global contextual information in the image, thereby better preserving edge details during the segmentation process.

[0136] S2062: Define energy function;

[0137] In this embodiment, the energy function is the core of the fully connected conditional random field model and defines the goal of model optimization. The energy function typically consists of two parts: a unary potential function and a binary potential function. The unary potential function reflects the initial probability that each pixel belongs to a certain category. This information usually comes from the initial segmentation mask. The binary potential function describes the relationship between pixels, encouraging pixels with similar features to be assigned the same label and pixels with different features to be assigned different labels. This helps maintain edge continuity during the segmentation process.

[0138] S2063: Optimizing pixel label assignment in a fully connected conditional random field model by minimizing an energy function;

[0139] In this embodiment, during the minimization process, the model considers the dependencies between all pixels to find a globally optimal label assignment solution.

[0140] S2064: After minimizing the energy function, the fully connected conditional random field model is used to output the predicted brain tumor segmentation mask with edge refinement.

[0141] In this embodiment, compared with the initial segmentation mask, the predicted brain tumor segmentation mask after edge refinement is finer and more accurate in the edge part and can better reflect the actual boundary of the brain tumor.

[0142] Specifically, the energy function E(L) is expressed by the following formula:

[0143] E(L)=-∑ i logP(l i )+∑ i,j θ ij (l i ,l j );

[0144]

[0145] Where: L represents the binary label assignment of all pixels; θ ij(l i ,l j ) represents the pair potential; μ(l i ,l j ) represents an indicator function, l i =l j , then μ(l i ,l j )=1;P i 、P j Indicates the position of the pixel; I i , I j Indicates the intensity of the pixel; p i 、p j Represents the color characteristics of the pixel; σ α Represents ||P i -P j ||The degree of influence on pair potential; σ β Indicates ||I i -I j || has an impact on the pairwise potential; ω1 and ω2 are weight parameters.

[0146] The present invention uses a fully connected conditional random field to refine the edges of the predicted brain tumor segmentation mask. On the one hand, it can effectively process the contextual information in the image to make the edges smoother and more natural, thereby improving the accuracy of the segmentation edges; on the other hand, it can improve the spatial consistency of the image and improve the accuracy of segmentation.

[0147] 4. Loss Function

[0148] In medical segmentation tasks, due to the difference in the number of samples of different types, the problem of class imbalance is common. This class imbalance will affect the accuracy of segmentation. The overlap-based method can better overcome this problem. Therefore, this embodiment proposes to combine BCELoss with DiceLoss.

[0149] In this embodiment, the calculation formula of the loss function is as follows:

[0150] Loss BAD =Loss BCE +Loss Dice ;

[0151] Loss BCE =-W[y n *logx n +(1-x n )*log(1-x n )];

[0152]

[0153] Where: LossBAD Represents the loss function; Loss BCE Indicates BCE loss; Loss Dice represents Dice loss; x n represents the predicted brain tumor segmentation mask; y n represents the actual brain tumor segmentation result; x i represents the predicted value of pixel i in the brain tumor segmentation mask; y i represents the label value of pixel i in the true brain tumor segmentation result; N represents the total number of pixels in the predicted brain tumor segmentation mask and the true brain tumor segmentation result.

[0154] BCE (Binary Cross-Entropy) loss, also known as binary cross-entropy loss, is a loss function commonly used for binary classification problems. It measures the difference between the actual output (probability) and the expected output (usually 0 or 1). In image segmentation, for each pixel, the model outputs a probability value, indicating the probability that the pixel belongs to a certain category (in this case, brain tumor or non-brain tumor). The BCE loss function calculates the loss by comparing these probabilities with the true label (0 or 1), thereby guiding model training.

[0155] Dice loss is a loss function based on the Dice similarity coefficient, used to evaluate the similarity between two samples. In image segmentation tasks, Dice loss is used to compare the similarity between the predicted segmentation mask and the ground-truth segmentation mask. The Dice similarity coefficient is a value between 0 and 1, with higher values ​​indicating greater similarity between the two masks. Dice loss is defined as 1 minus the Dice similarity coefficient, and therefore is a value to be minimized.

[0156] 5. Experimental Description

[0157] 1. Experimental environment and dataset

[0158] This experiment was conducted on a system equipped with an Intel(R) Xeon(R) Silver 4214R CPU and an NVIDIA GeForce GTX3080ti GPU, running Windows 11. The deep learning model was trained and evaluated using the PyTorch framework with CUDA version 12.1. The experimental parameters were set as follows: a learning rate of 1e-4, a batch size of 32, and a momentum of 0.9.

[0159] To evaluate the performance of our proposed method, we used the widely recognized Brain Tumor Segmentation Challenge (BraTS) benchmark datasets, specifically the BraTS2018 and BraTS2021 datasets. The BraTS2018 training set contains MRI images of 285 glioma patients, while the BraTS2021 training set includes data from 1,251 cases. Both datasets are stored in the .nii.gz format. The datasets include images in the T1, T2, T1ce, and Flair modalities, along with corresponding unified labels.

[0160] 2. Image preprocessing

[0161] In this experiment, we first performed preliminary processing on the T1, T1ce, T2, and Flair slices in the BraTS2018 and BraTS2021 datasets according to the method described in the literature (Li Nan, Zhang Hongli. Research on 2D MR brain tumor image segmentation algorithm based on multimodal fusion [J]. Optoelectronics Laser, 2023, 34(08): 890-896. DOI: 10.16136 / j.joel.2023.08.0379.). The specific steps included adjusting the resolution of each slice to 240x240 pixels to ensure data consistency.

[0162] Since tumor regions in different patients vary in position, shape, and contrast, these differences may cause signal intensity overlap between tumors and healthy brain tissues in MRI images, thereby increasing the complexity of image segmentation. To this end, this experiment performs Gaussian distribution fitting on the pixel values ​​in the segmented MRI brain tumor images to remove low-intensity noise points; internally normalizes the intensity values ​​of MRI brain tumor images to enhance the overall brightness and readability of MRI brain tumor images;

[0163] The calculation formula for internal normalization is:

[0164]

[0165] Where A′ represents the normalized intensity value; A represents the original intensity value; A_min and A_max represent the minimum and maximum values ​​in the original intensity values.

[0166] The cumulative distribution function is used as the transformation function to transform the intensity value of each pixel in the MRI brain tumor image to be segmented. The discrete form is:

[0167]

[0168] Where: S k Represents the intensity value r of the kth pixel in the MRI brain tumor image kThe intensity value after intensity transformation; L represents the total intensity value of all pixels in the MRI brain tumor image; represents the cumulative distribution function (CDF), calculated from intensity values ​​0 to r k The cumulative probability of all intensity values ​​of n j Indicates the intensity value is r j The number of pixels in the MRI brain tumor image; n represents the total number of pixels in the MRI brain tumor image.

[0169] Figure 4 A comparative example of preprocessing of MRI images, where (a) is the original image, (b) is the preprocessed image, and (c) is the preprocessed image.

[0170] 3. Evaluation indicators

[0171] In this experiment, we use the Dice coefficient to evaluate the similarity between the segmentation results generated by the algorithm and the true labels. In addition, the positive prediction value (PPV) is used to measure the proportion of true positive samples (TP) among all predicted positive samples. Sensitivity is used to evaluate the ability of the model to identify true positive samples. During the network testing phase, the test images are sequentially input into the trained network and the results are saved. Based on these results, key variables such as true positive samples (TP), true negative samples (TN), false positive samples (FP), and false negative samples (FN) are calculated. The specific formulas for these indicators are as follows:

[0172]

[0173]

[0174] 4. Experimental results

[0175] In order to comprehensively evaluate the performance of the BSAU-net model proposed in this paper, we conducted a comparative analysis with the baseline U-net network and methods proposed in the literature.

[0176] in:

[0177] U-net++, from Zhou, Zongwei, et al. "Unet++: A nested u-net architecture for medical image segmentation." Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 20, 2018, Proceedings 4. Springer International Publishing, 2018.

[0178] MDU-net, from Zhang, Jiawei, et al. "Mdu-net: Multi-scaledensely connected u-net for biomedical image segmentation." Health Information Science and Systems 11.1(2023):13.

[0179] SCU-net++, from Cui, Hu, Haiwei Pan, and Kejia Zhang. "SCU-Net++: A Nested U-Net Based on Sharpening Filter and Channel Attention Mechanism." Wireless Communications and Mobile Computing 2022(2022):1-8.

[0180] AMSU-net, from Huang Li, He Meiling. Brain Tumor Segmentation Method Based on U-Net Improved Model[J]. Computer Technology and Development, 2022, 32(11):58-63

[0181] Attention U-net, from Oktay, O., Schlemper, J., Folgoc, LL, Lee, M., Heinrich, M., Misawa, K., et al.: Attention U-Net: Learning where to look for thepancreas.arXiv:1804.03999(2018).

[0182] All comparison methods are trained for 200 epochs on the BraTS2018 and BraTS2021 datasets.

[0183] Table 1 Quantitative analysis results of BraTS2018 dataset

[0184]

[0185] Table 2 Quantitative analysis results of BraTS2021 dataset

[0186]

[0187]

[0188] Table 1 shows the segmentation results of different methods on the BraTS2018 dataset. We can see that the method proposed in this invention has achieved better performance evaluation indicators in all aspects. Table 2 shows the different methods on the BraTS2021 dataset. We can see that except for the method with sensitivity in T1ce mode, the index is lower than AttentionU-net, and our BSAU network achieves better segmentation performance. Our segmentation method integrates the global attention mechanism to introduce the EA module to enhance edge area features, reduce segmentation error and blur, and improve segmentation accuracy. The introduction of the SPA module promotes the interaction of contextual information, and utilizes the correlation and contextual information between different regions in the image to expand the network's field of view, thereby achieving a more comprehensive feature representation and enhanced segmentation effect. In addition, even in the single-modal segmentation comparison, our method demonstrates excellent segmentation results, proving its robustness.

[0189] Figure 5 and Figure 6The segmentation results of our model and other models on the BraTS2018 and BraTS2021 datasets are presented. It can be observed that the baseline U-net has significant errors in overall tumor region segmentation compared to the true labels, with low accuracy and insufficient representation of image details. In contrast, methods such as U-net++, MDU-net, and AMSU-net significantly enhance feature extraction capabilities by introducing skip connections, dense connections, and channel attention mechanisms to the U-net backbone, thereby improving overall tumor region segmentation.

[0190] In addition, we also compare the performance of our model with other models in detail on the segmentation of detailed edge region features. Figure 7 and Figure 8 The segmentation results on the BraTS2018 and BraTS2021 datasets are shown. The basic U-net model performs poorly in capturing the detailed features of these areas. In contrast, U-net++ and MDU-net enhance the capture of detailed area features by improving the feature connection mechanism. AMSU-net further introduces a channel attention mechanism to strengthen the interaction of feature information between different channels, thereby improving the integrity of the features. Our BSAU-net uses the Sobel operator to extract features from the edge areas of brain tumor images and combines them with the features extracted by the network to achieve a more comprehensive feature representation, significantly improving the segmentation accuracy of the edge details of the tumor area. Finally, Figure 9 We show the refinement of the obtained segmentation masks using the CRF random field method.

[0191] 5. Ablation Experiment Results

[0192] To further validate the effectiveness of each module in the network, we quantitatively evaluated the segmentation performance of four models on the BraTS 2018 and 2021 datasets: a U-net without the attention mechanism, a model with only SPA and EA, and a complete BSAU-net integrating both SPA and EA. The evaluation metrics used were the Dice coefficient and MIoU. The Dice coefficient was calculated by averaging the segmentation results of the complete tumor region from the four modalities. The experimental results are shown in Table 3.

[0193] As can be seen from Table 3, when the SPA module is added, the entire network can obtain the connection between the previous and next features, so that the entire network can obtain more complete context information. The segmentation result is more accurate through the concatenation between features. Compared with U-net, the addition of SPA improves Dice by 7% and MIoU by 3%.

[0194] When the EA module is added to the network, the network's perception of the tumor edge area is enhanced, effectively capturing tumor edge information and improving the network's segmentation accuracy. On the dataset, Dice improved by 5% and MIoU improved by 2%.

[0195] The complete BSAU-net network incorporates both the SAP and EA modules to simultaneously capture important global and local image features, resulting in more complete feature acquisition and improved analysis and understanding of brain tumor images. On the BraTS2018 and BraTS2021 datasets, the Dice score improved by 10% and the MIoU improved by nearly 4%. The aforementioned series of experiments demonstrate the effectiveness of spatial attention and edge attention in improving segmentation accuracy.

[0196] Table 3 Ablation experiment results

[0197]

[0198] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the technical solutions. Those skilled in the art should understand that modifications or equivalent replacements of the technical solutions of the present invention that do not depart from the purpose and scope of the technical solutions of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A brain tumor segmentation method based on spatial attention and edge recognition enhancement, characterized in that: include: S1: Acquire the MRI brain tumor image to be segmented; S2: Input the MRI brain tumor image to be segmented into the trained brain tumor segmentation model and output the corresponding predicted brain tumor segmentation mask; The brain tumor segmentation model is built on a U-net network. It includes an encoder consisting of several downconvolutional modules, a decoder consisting of several upconvolutional modules that correspond one-to-one to the encoder's downconvolutional modules, and an edge attention module at the bottom layer of the U-net network. The output of each downconvolutional module in the encoder is jump-connected to the input of the corresponding upconvolutional module in the decoder, and each jump connection is equipped with a spatial attention module. The processing steps when training a brain tumor segmentation model are as follows: S201: Acquire an MRI brain tumor image as training data and a corresponding real brain tumor segmentation result as input to a brain tumor segmentation model; S202: performing multi-scale feature extraction on the MRI brain tumor image through each down-convolution module of the encoder, and outputting an original feature map; S203: Perform spatial feature extraction and weighted calculation on the feature map output by each down-convolution module in the encoder through the spatial attention module to generate a spatial attention weighted feature map; splice the spatial attention weighted feature map with the feature map output by the previous up-convolution module of the decoder and input the result into the next up-convolution module; S204: After edge features are extracted from the MRI brain tumor image by the edge attention module to generate an edge enhancement map, the edge enhancement map is multiplied by the original feature map output by the encoder to generate an edge attention map; S205: Rescale the edge attention map through each upconvolution module of the decoder, where the input of each upconvolution module is the result of concatenating the spatial attention weighted feature map generated by the corresponding downconvolution module and the feature map output by the previous upconvolution module; the decoder outputs the final edge spatial fusion feature map; S206: performing convolution and sigmoid activation function operations on the edge space fusion feature map output by the decoder to generate a predicted brain tumor segmentation mask; S207: Calculating a loss function by predicting the brain tumor segmentation mask and the corresponding true brain tumor segmentation result, and reversely optimizing the parameters of the brain tumor segmentation model; S208: Repeat steps S201 to S207 until the brain tumor segmentation model converges or reaches a preset number of iterations; S3: Output the brain tumor segmentation mask as the segmentation result of the image to be segmented.

2. The brain tumor segmentation method based on spatial attention and edge recognition enhancement according to claim 1, characterized in that: In step S202, the down-convolution module in the encoder and the up-convolution module in the decoder both include a pooling layer and a convolution layer.

3. The brain tumor segmentation method based on spatial attention and edge recognition enhancement according to claim 1, characterized in that: In step S203, the spatial attention module generates a spatial attention weighted feature map through the following steps: S2031: Perform 1×1 convolution and 3×3 convolution operations on the input feature map to generate a Query matrix; S2032: Perform a 1×1 convolution operation and a 3×3 dilated convolution operation on the input feature map to generate a Key matrix; S2033: Perform a 1×1 convolution operation and a 3×3 transposed convolution operation on the input feature map to generate a Value matrix; S2034: After transposing the Key matrix, perform a dot product operation with the Query matrix to calculate the attention score; Perform Softmax operation on the attention score to obtain the attention weight matrix; S2036: Perform weighted summation on the attention weight matrix and the Value matrix to generate a spatial attention weighted feature map.

4. The brain tumor segmentation method based on spatial attention and edge recognition enhancement according to claim 1, wherein: In step S204, the edge attention module extracts edge features from the MRI brain tumor image using the Sobel operator to generate an edge enhancement map; The specific steps are as follows: S2041: converting the input MRI brain tumor image into a grayscale MRI brain tumor image; S2042: Use two 3x3 convolution kernels of the Sobel operator to perform convolution operations on the horizontal gradient and vertical gradient of the grayscale MRI brain tumor image, respectively, to generate two gradient images; S2043: Calculate the gradient magnitude of each pixel in the two gradient images to generate a gradient magnitude image; the value of each pixel in the gradient magnitude image represents the strength of the edge at that point; S2044: performing non-maximum suppression processing on the gradient magnitude image; S2045: Performing thresholding processing on the gradient amplitude image after the non-maximum suppression processing to generate a binary edge enhancement image.

5. The brain tumor segmentation method based on spatial attention and edge recognition enhancement according to claim 4, characterized in that: In step S204, the edge enhancement map and the original feature map are multiplied by the following steps: S2141: Perform convolution operations on the edge enhancement map through one or more convolutional layers to generate a series of feature maps; S2142: Perform Sigmoid activation function processing on the feature map to generate the corresponding attention weight map; S2143: Multiply the attention weight map by the original feature map element by element to generate an edge attention map.

6. The brain tumor segmentation method based on spatial attention and edge recognition enhancement according to claim 1, characterized in that: In step S206, the predicted brain tumor segmentation mask is refined by using a fully connected conditional random field; The specific steps are as follows: S2061: Constructing a fully connected conditional random field model by predicting brain tumor segmentation masks; S2062: Define energy function; S2063: Optimizing pixel label assignment in a fully connected conditional random field model by minimizing an energy function; S2064: After minimizing the energy function, the fully connected conditional random field model is used to output the predicted brain tumor segmentation mask with edge refinement.

7. The brain tumor segmentation method based on spatial attention and edge recognition enhancement according to claim 6, characterized in that: In step S2062, the energy function E(L) is expressed by the following formula: E(L)=−∑ i logP(l i )+∑ i,j θ ij (l i ,l j ); Where: L represents the binary label assignment of all pixels; θ ij (l i ,l j ) represents the pair potential; μ(l i ,l j ) represents an indicator function, l i =l j , then μ(l i ,l j )=1;P i 、P j Indicates the position of the pixel; I i , I j Indicates the intensity of the pixel; p u 、p j Represents the color characteristics of pixels; σ α Represents ||P i -P j ||The degree of influence on pair potential; σ β Indicates ||I i -I j || has an impact on the pairwise potential; ω1 and ω2 are weight parameters.

8. The brain tumor segmentation method based on spatial attention and edge recognition enhancement according to claim 1, characterized in that: In step S207, the calculation formula of the loss function is as follows: Loss BAD =Loss BCE +Loss Dice ; Loss BCE =-W[y n *logx n +(1-x n )*log(1-x n )]; Where: Loss BAD Represents the loss function; Loss BCE Indicates BCE loss; Loss Dice represents Dice loss; x n represents the predicted brain tumor segmentation mask; y n represents the actual brain tumor segmentation result; x i represents the predicted value of pixel i in the brain tumor segmentation mask; y i represents the label value of pixel i in the true brain tumor segmentation result; N represents the total number of pixels in the predicted brain tumor segmentation mask and the true brain tumor segmentation result.

9. The brain tumor segmentation method based on spatial attention and edge recognition enhancement according to claim 1, characterized in that: In step S1, Gaussian distribution fitting is performed on the pixel values ​​in the segmented MRI brain tumor image to remove low-intensity noise points; and the intensity values ​​of the MRI brain tumor image are internally normalized to enhance the overall brightness and readability of the MRI brain tumor image.

10. The brain tumor segmentation method based on spatial attention and edge recognition enhancement according to claim 1, characterized in that: In step S1, the intensity value of each pixel in the MRI brain tumor image to be segmented is transformed using the cumulative distribution function as a transformation function, and the discrete form is: Where: S k Represents the intensity value r of the kth pixel in the MRI brain tumor image k The intensity value after intensity transformation; L represents the total intensity value of all pixels in the MRI brain tumor image; represents the cumulative distribution function, calculated from the intensity value 0 to r k The cumulative probability of all intensity values ​​of n j Indicates the intensity value is r j The number of pixels in the MRI brain tumor image; n represents the total number of pixels in the MRI brain tumor image.

Images