Brain tumor segmentation method based on boundary perception mechanism

Abstract

The invention belongs to the technical field of medical image analysis, and relates to a brain tumor segmentation method based on a boundary perception mechanism, and the method comprises the steps: inputting T1, T1c, T2 and Flair images of a brain tumor into a trained image segmentation model, and outputting a prediction segmentation image through the trained image segmentation model, the prediction segmentation image is a brain tumor MRI image which is obtained through prediction and has a complete tumor area, a tumor core area and an enhanced tumor area; according to the brain tumor segmentation method based on the boundary perception mechanism provided by the invention, the boundary perception mechanism is introduced, and the boundary information is fused into the image segmentation model, so that the discriminability of the model to features is improved, and accurate segmentation of tumor subregions is realized; a multi-modal fusion method is adopted, different MRI sequence complementary information is integrated, and tumor features are comprehensively understood; in combination with uncertainty quantification and a loss function based on uncertainty, confidence measurement is provided for a segmentation result, the accuracy and reliability of segmentation are enhanced, and a clinician is assisted in evaluating a prediction result.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing technology, specifically relating to a brain tumor segmentation method based on boundary awareness mechanism. Background Technology

[0002] Magnetic resonance imaging (MRI) is a core technology for the diagnosis and monitoring of brain tumors. Different sequences can provide complementary tumor characterization information: T1-weighted (T1) and contrast-enhanced T1-weighted (T1c) sequences are good at delineating the core region of the tumor, while T2-weighted (T2) and fluid attenuation inversion recovery (FLAIR) sequences can clearly show edema areas. However, inconsistencies between MRI modalities, image quality fluctuations, and data gaps pose significant challenges to brain tumor segmentation.

[0003] Currently, brain tumor segmentation techniques mainly encompass traditional methods, deep learning methods, and other technologies. Among traditional segmentation methods, threshold-based methods divide regions by setting grayscale thresholds, are computationally efficient, and are often used as a preprocessing technique; region-based methods, such as region growing, rely on manually selecting seed points to expand regions, which is highly subjective, while the watershed algorithm, although based on image gradient segmentation, is prone to oversegmentation; edge-based methods use operators such as Canny and Sobel to detect boundaries and are mostly used for auxiliary purposes; model-based methods require the construction of statistical models of tumor shape and grayscale, which have high requirements for prior knowledge and are complex in modeling and parameter adjustment.

[0004] Significant progress has been made in deep learning-based segmentation methods: U-Net and its variants, through encoder-decoder structures and skip connections, effectively capture contextual information and perform excellently in segmenting complex anatomical structures; Mask R-CNN utilizes region proposal networks to generate candidate regions and combines them with attention modules to achieve tumor detection and mask generation; Transformer-based methods, such as TransBTS and SwinBTS, have unique advantages in identifying tumor regions in large fields of view of MRI images by fusing 3D convolutions with the Transformer architecture. Furthermore, atlas-based methods rely on high-quality brain atlases for registration and segmentation, making it difficult to adapt to individual differences; hybrid segmentation methods integrate multiple techniques, leveraging their respective strengths and compensating for their weaknesses.

[0005] Despite continuous technological advancements, existing brain tumor segmentation methods still have significant shortcomings. In terms of accuracy, the boundaries between brain tumors and normal brain tissue are often blurred, especially in subregions such as edema, enhanced tumors, necrotic tissue, and non-enhanced areas, where the boundaries are complex on MRI images, easily leading to segmentation errors. Furthermore, individual differences exist in patients' brain anatomy and tumor characteristics, which current models struggle to adequately account for, resulting in poor segmentation performance in certain special cases and severely impacting diagnostic accuracy.

[0006] At the level of uncertainty, existing neural network methods mostly output single deterministic predictions, failing to reflect the inherent variability in imaging data. Due to the complexity of medical images and the ambiguity of lesion boundaries, no algorithm can guarantee absolute accuracy. A single result lacking confidence information makes it difficult for clinicians to assess the reliability of segmentation, severely restricting accurate diagnosis and consequently affecting the scientific nature of treatment decisions, becoming a key bottleneck hindering the clinical application of algorithms. For example, the paper "Eoformer: Edge-oriented transformer for brain tumor segmentation[C] / / Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2023: 333-343" presents an innovative Eoformer model in terms of edge enhancement and computational efficiency, but it still has shortcomings in segmenting complex sub-region boundaries and lacks an uncertainty module and confidence quantification mechanism. These deficiencies limit the application value of this model in real clinical scenarios.

[0007] Therefore, there is an urgent need to innovate methods to overcome the limitations of existing technologies in terms of accuracy and uncertainty quantification, and to enhance the clinical value of brain tumor segmentation. Summary of the Invention

[0008] The purpose of this invention is to solve the problems in the prior art and propose a brain tumor segmentation method based on a boundary-aware mechanism.

[0009] To achieve the above objectives, the present invention adopts the following technical solution:

[0010] A brain tumor segmentation method based on boundary awareness mechanism inputs T1, FLAIR, T1c and T2 images of the same brain tumor into a trained image segmentation model, which outputs a predicted segmented image. The predicted segmented image is a brain tumor MRI image with three segmented regions, namely the complete tumor region, the tumor core region and the enhanced tumor region.

[0011] The image segmentation model includes four encoder modules. The inputs of the four encoder modules correspond to the T1 image, FLAIR image, T1c image and T2 image of the same brain tumor, respectively. The T1 image, FLAIR image, T1c image and T2 image of the same brain tumor are divided into two pairs of MRI modalities.

[0012] Each encoder module contains a BAM (Boundary Awareness Mechanism) module. The BAM module structure diagram is shown below.Figure 2 The BAM module is used to extract modal features f from the same pair of MRI modalities. m With boundary information f b Combining them, we get f out This improves the accuracy of tumor segmentation.

[0013] The method of this invention integrates multimodal data from two pairs of MRI modalities (T1 and Flair, T1c and T2). T1 and T1c images highlight the core tumor region, while Flair and T2 images focus on capturing edema regions. The image segmentation model uses this complementary information to improve the accuracy of tumor region depiction and enhance segmentation performance.

[0014] As a preferred technical solution:

[0015] The brain tumor segmentation method based on the boundary-aware mechanism described above, f out =W f f m ⊕(1-W f )⊗ f b W f =σ(W c * [f m , f b ]+ In the formula, ⊕ represents element-wise multiplication, ⊕ represents element-wise addition, W f This represents the attention map, σ(·) represents the sigmoid activation function, and W... c Represents the convolution kernel, [·] represents offset, [·] represents concatenation by channel, W c *[f m , f b ] indicates W c Feature map after channel splicing [f m , f b Perform convolution operations.

[0016] The brain tumor segmentation method based on boundary awareness mechanism described above uses four encoder modules, denoted as encoder module 1, encoder module 2, encoder module 3, and encoder module 4. The BAM module in encoder modules 1 and 2 corresponds to f... m All modal features are extracted from the T1 image. The f corresponding to the BAM module in the first encoder module and the second encoder module is... b All boundary information is extracted from Flair images; the BAM modules in the 3rd and 4th encoder modules correspond to f mAll modal features are extracted from T1c images. The f corresponding to the BAM module in the 3rd and 4th encoder modules is... b All of these are boundary information extracted from T2 images.

[0017] In the brain tumor segmentation method based on the boundary awareness mechanism described above, the first encoder module and the second encoder module exchange information through the Sobel filter module, and the third encoder module and the fourth encoder module exchange information through the Sobel filter module.

[0018] As described above, a brain tumor segmentation method based on a boundary-aware mechanism comprises the following encoder modules connected sequentially from front to back according to the data flow: a first standard convolutional normalized activation module, a first BAM module, a first max pooling module, a second standard convolutional normalized activation module, a second BAM module, a second max pooling module, a third standard convolutional normalized activation module, a third BAM module, a third max pooling module, a fourth standard convolutional normalized activation module, a fourth max pooling module, a first standard convolutional normalized activation discarding module, a fifth max pooling module, and a second standard convolutional normalized activation discarding module. The max pooling module reduces computational complexity by effectively reducing the feature map size while retaining key feature information. The BAM module utilizes high-resolution feature maps generated in the shallow stage to guide the model to focus on key features that significantly contribute to the final segmentation task, rather than all possible features. The standard convolutional normalized activation discarding module is used to prevent overfitting and improve robustness.

[0019] The standard convolutional normalized activation module consists of a standard (3×3×3) convolutional layer, an instance normalization layer, and a LeakyReLU activation function layer connected sequentially from front to back according to the data flow, in order to capture features of a specific modality;

[0020] The standard convolutional normalized activation dropout module consists of a standard convolutional layer, an instance normalization layer, a LeakyReLU activation function layer, and a dropout layer connected sequentially from front to back according to the data flow direction.

[0021] The i-th encoder module and the (i+1)-th encoder module exchange information through three Sobel filter modules, denoted as the 1st Sobel filter module, the 2nd Sobel filter module, and the 3rd Sobel filter module, i=1,3; the Sobel filter modules are used to extract boundary information to enhance the structural detail features of brain tumors.

[0022] The input of the i-th encoder module, after being processed by the first Sobel filter module, becomes the f corresponding to the first BAM module in the (i+1)-th encoder module. mThe input of the i-th encoder module is processed by the first standard convolutional normalization activation module in the i-th encoder module and then used as the f corresponding to the first BAM module in the i-th encoder module. m ;

[0023] The input of the (i+1)th encoder module, after being processed by the first Sobel filter module, becomes the f corresponding to the first BAM module in the i-th encoder module. b The input of the (i+1)th encoder module is processed by the first standard convolutional normalization activation module in the (i+1)th encoder module and used as the f corresponding to the first BAM module in the (i+1)th encoder module. b ;

[0024] The output of the first maximum pooling module in the i-th encoder module is processed by the second Sobel filter module and used as the f corresponding to the second BAM module in the (i+1)-th encoder module. m The output of the first max pooling module in the i-th encoder module is processed by the second standard convolutional normalization activation module in the i-th encoder module and used as the f corresponding to the second BAM module in the i-th encoder module. m ;

[0025] The output of the first maximum pooling module in the (i+1)th encoder module is processed by the second Sobel filter module and used as the f corresponding to the second BAM module in the i-th encoder module. b The output of the first max pooling module in the (i+1)th encoder module is processed by the second standard convolutional normalization activation module in the (i+1)th encoder module and used as the f corresponding to the second BAM module in the (i+1)th encoder module. b ;

[0026] The output of the second maximum pooling module in the i-th encoder module is processed by the third Sobel filter module and used as the f corresponding to the third BAM module in the (i+1)-th encoder module. m The output of the second max pooling module in the i-th encoder module is processed by the third standard convolutional normalization activation module in the i-th encoder module and used as the f corresponding to the third BAM module in the i-th encoder module. m ;

[0027] The output of the second maximum pooling module in the (i+1)th encoder module is processed by the third Sobel filter module and used as the f corresponding to the third BAM module in the i-th encoder module. b The output of the second max pooling module in the (i+1)th encoder module is processed by the third standard convolutional normalization activation module in the (i+1)th encoder module and used as the f corresponding to the third BAM module in the (i+1)th encoder module. b .

[0028] As described above, the brain tumor segmentation method based on a boundary-aware mechanism further includes a decoder in its image segmentation model. The decoder contains an MMF (Multimodal Fusion) module, the structure of which is shown in the diagram below. Figure 3 The input to the MMF module is denoted as f. in The output is denoted as f. out f out =w attention f in ⊕f in w attention =σ(MLP(f unified )), f unified =AvgPool(f in )⊕VarPool(f in In the formula, w attention Represents attention weight, ⊕ represents element-wise multiplication, ⊕ represents element-wise addition, σ(·) represents the sigmoid activation function, MLP(·) represents a multilayer perceptron, AvgPool(·) represents average pooling, VarPool(·) represents variance pooling, f unified The representation is a feature representation obtained by adding the two operations element by element, which integrates all and local information; by retaining the original feature information through residual connection, this fusion method can effectively integrate complementary information from multiple MRI modalities, and can simultaneously extract intensity-based global features and fine-grained local differences.

[0029] The brain tumor segmentation method based on boundary awareness mechanism described above consists of a decoder composed of five non-standard convolutional normalized activation modules, a first MMF module, a second MMF module, a first standard convolutional normalized activation discard module, a third MMF module, a second standard convolutional normalized activation discard module, a fourth MMF module, a first standard convolutional normalized activation module, a fifth MMF module, a second standard convolutional normalized activation module, a sixth MMF module, and a third standard convolutional normalized activation module.

[0030] The non-standard convolutional normalization activation module consists of a non-standard (1×1×1) convolutional layer, an instance normalization layer, and a LeakyReLU activation function layer connected sequentially from front to back according to the data flow direction;

[0031] The outputs of the second standard convolutional normalization activation drop-out module in the four encoders are concatenated and then input into the first MMF module.

[0032] The output of the first MMF module is independently upsampled, then concatenated with the output of the first standard convolution normalization activation drop module in the four encoders. After being processed by standard convolution, instance normalization and LeakyReLU activation functions in sequence, the output is then input into the second MMF module.

[0033] The second MMF module is connected to the first standard convolutional normalized activation drop module;

[0034] The output of the first standard convolutional normalization activation drop module in the decoder is independently upsampled and then concatenated with the output of the fourth standard convolutional normalization activation module in the four encoders. After being processed by standard convolution, instance normalization and LeakyReLU activation functions in sequence, they are input into the third MMF module.

[0035] The third MMF module is connected to the second standard convolutional normalized activation drop module in the decoder;

[0036] The output of the second standard convolutional normalization activation drop module in the decoder is independently upsampled and then concatenated with the output of the third standard convolutional normalization activation module in the four encoders. After being processed by the standard convolution, instance normalization and LeakyReLU activation functions in sequence, they are input into the fourth MMF module.

[0037] The fourth MMF module is connected to the first standard convolutional normalization activation module in the decoder;

[0038] The output of the first standard convolutional normalization activation module in the decoder is independently upsampled and then concatenated with the output of the second standard convolutional normalization activation module in the four encoders. After being processed by standard convolution, instance normalization and LeakyReLU activation functions in sequence, they are input into the fifth MMF module.

[0039] The fifth MMF module is connected to the second standard convolutional normalized activation module in the decoder;

[0040] The output of the second standard convolutional normalization activation module in the decoder is independently upsampled and then concatenated with the output of the first standard convolutional normalization activation module in the four encoders. After being processed by the standard convolution, instance normalization and LeakyReLU activation functions in sequence, they are input into the sixth MMF module.

[0041] The sixth MMF module is connected to the third standard convolutional normalization activation module in the decoder;

[0042] The outputs of the first standard convolutional normalized activation drop module, the second standard convolutional normalized activation drop module, the first standard convolutional normalized activation module, the second standard convolutional normalized activation module, and the third standard convolutional normalized activation module are each processed by a non-standard convolutional normalized activation module, and then added element by element. The result is the output of the decoder.

[0043] In the decoder path, spatial resolution is gradually restored through upsampling, and the MMF module further integrates multimodal features at each stage.

[0044] As described above, the brain tumor segmentation method based on boundary awareness mechanism includes an uncertainty quantification module (UQ) in its image segmentation model. The uncertainty quantification module generates T segmentation results from the decoder output through Monte Carlo (MC) sampling, and then aggregates and outputs the T segmentation results to generate a predicted segmentation image, a random uncertainty map, and a cognitive uncertainty map, where T=10. Uncertainty quantification is achieved through the random uncertainty map and the cognitive uncertainty map, providing key insights into the reliability of segmentation prediction and enhancing the clinical applicability of the model.

[0045] The loss function of the image segmentation model for the brain tumor segmentation method based on the boundary-aware mechanism described above is expressed as follows:

[0046] ;

[0047] ;

[0048] ;

[0049] ;

[0050] ;

[0051] In the formula, Used to measure the degree of overlap between predicted segmentation results and true annotations. Used to capture random uncertainties, taking into account the noise inherent in the observation data. Used to capture cognitive uncertainty, in order to quantify the uncertainty of the model. = =0.1, To divide the total number of voxels, Let i be the predicted value for voxel i. For the true value of voxel i, It is a constant, with a value of 1 × 10 -5 To avoid division by zero, Let V be the prediction variance of the voxels. Number of samples taken in Monte Carlo Let be the t-th predicted value of voxel i;

[0052] This loss function integrates three key components: dice loss, aleatoric uncertainty, and epistemic uncertainty, thereby improving the accuracy and reliability of the image segmentation model.

[0053] As described above, in a brain tumor segmentation method based on a boundary-aware mechanism, the T1, Flair, T1c, and T2 images of the same brain tumor are preprocessed before being input into the trained image segmentation model. The preprocessing involves normalization to make the mean 0 and the variance 1, and adjusting the size to 128 pixels × 128 pixels × 128 pixels.

[0054] The training steps for the image segmentation model in the brain tumor segmentation method based on the boundary awareness mechanism described above are as follows:

[0055] (a) Collection A case of brain tumor, ≥285 (data source: BraTS2018 and BraTS2019 datasets from the Brain Tumor Segmentation Challenge, which are widely used in brain tumor segmentation tasks), each brain tumor case has T1, T1c, T2 and Flair images;

[0056] (b) Perform the aforementioned preprocessing on the T1, T1c, T2, and FLAIR images; obtain the true segmentation image for each brain tumor case, which is the brain tumor MRI image with the three segmentation regions obtained through manual annotation;

[0057] (c) Adopt The training set and test set were constructed using T1 images, T1c images, T2 images, FLAIR images, and ground truth segmentation images corresponding to each brain tumor case, with a split ratio of 8:2 between the training set and the test set.

[0058] (d) The image segmentation model was trained using a training set. The model was implemented in Keras and trained on an NVIDIA RTX 4090 GPU. During training, images T1, T1c, T2, and Flair were used as inputs to the image segmentation model, and the real segmented images were used as the theoretical outputs of the image segmentation model. In addition, some key settings were made: an Adam optimizer with an initial learning rate of 0.0005 was used; if the validation loss did not improve within 10 consecutive epochs, the learning rate was halved; if the validation loss did not improve within 50 epochs, the model was stopped early to prevent overfitting, and the batch size was set to 1; the weight parameters of the image segmentation model (weights and bias parameters of convolutional layers, parameters of instance normalization layers) were continuously adjusted until the image segmentation model converged (i.e., the loss function value gradually decreased and tended to a stable value).

[0059] (e) Test the trained image segmentation model using a test set;

[0060] The segmentation accuracy of the trained image segmentation model is characterized by the Dice similarity coefficient (DSC) and Hausdorff distance (HD), and the calculation formula is as follows:

[0061] ;

[0062] In the formula, A and B represent the predicted region (intact tumor region WT, tumor core region TC, or enhanced tumor region ET) and the real region, respectively. This indicates the size of the intersection between the predicted region and the actual region. and These represent the total size of the predicted region and the actual region, respectively;

[0063] ;

[0064] In the formula, X and Y represent the predicted region boundary and the actual region boundary, respectively, and d(x,y) represents the Euclidean distance between points x and y. This represents the maximum value of the minimum distance from each point x on the predicted region boundary to the actual region boundary. This represents the maximum value of the minimum distance from each point y on the true region boundary to the predicted region boundary.

[0065] Beneficial effects:

[0066] This invention presents a brain tumor segmentation method based on a boundary-aware mechanism. By introducing this mechanism, boundary information is explicitly incorporated into the image segmentation model, enhancing the model's ability to learn more discriminative features and achieving precise segmentation of tumor subregions. Furthermore, the proposed multimodal fusion method captures complementary information from various MRI sequences, enabling a more comprehensive understanding of tumor features across different modalities. To further improve clinical reliability, uncertainty quantification is combined with a novel uncertainty-based loss function (which provides a confidence measure for each segmentation result), enhancing the accuracy and reliability of brain tumor segmentation and assisting clinicians in assessing predictions. Attached Figure Description

[0067] Figure 1 This is a model structure diagram of the brain tumor segmentation method based on boundary awareness mechanism in Embodiment 1 of the present invention;

[0068] Figure 2 This is a structural diagram of the BAM module in this invention;

[0069] Figure 3 This is a structural diagram of the MMF module in this invention;

[0070] Figure 4 The visualization results of segmenting three brain tumor cases using the brain tumor segmentation methods of Comparative Example 1, Example 2, Example 3 and Example 1 are shown respectively. In the figure, the red area is the core area of ​​the tumor, the red area + yellow area is the enhanced tumor area, the red area + yellow area + green area is the complete tumor area, the blue circle is the distinguishing part, and the value in the upper right corner is the average DSC of the segmentation results.

[0071] Figure 5 This is a model structure diagram of the brain tumor segmentation method in Comparative Example 1 of this invention;

[0072] Figure 6 This is a model structure diagram of the brain tumor segmentation method based on boundary awareness mechanism in Embodiment 2 of the present invention;

[0073] Figure 7 This is a model structure diagram of the brain tumor segmentation method based on boundary awareness mechanism in Embodiment 3 of the present invention. Detailed Implementation

[0074] The present invention will be further described below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.

[0075] Example 1

[0076] A brain tumor segmentation method based on boundary awareness mechanism, comprising the following steps:

[0077] Step 1: Establish an image segmentation model:

[0078] The image segmentation model includes a first encoder module, a second encoder module, a third encoder module, a fourth encoder module, a decoder, and an uncertainty quantization module;

[0079] The inputs to the first encoder module, the second encoder module, the third encoder module, and the fourth encoder module are T1 images, FLAIR images, T1c images, and T2 images of the same brain tumor, respectively.

[0080] Each encoder module contains a BAM module, such as Figure 2 As shown, the BAM module has two inputs and one output. The two inputs are denoted as f. m and f b One output is denoted as f. out f out =W f f m ⊕(1-W f ) f b W f =σ(W c * [f m , f b ]+ In the formula, ⊕ represents element-wise multiplication, ⊕ represents element-wise addition, W f This represents the attention map, σ(·) represents the sigmoid activation function, and W... c Represents the convolution kernel, [·] represents offset, [·] represents concatenation by channel, W c *[f m , f b ] indicates W c Feature map after channel splicing [f m , f b Perform convolution operations;

[0081] The f corresponding to the BAM module in the first encoder module and the second encoder module m All modal features are extracted from the T1 image. The f corresponding to the BAM module in the first encoder module and the second encoder module is... b All of these are boundary information extracted from Flair images;

[0082] The f corresponding to the BAM module in the third and fourth encoder modules m All modal features are extracted from T1c images. The f corresponding to the BAM module in the 3rd and 4th encoder modules is... b All of these are boundary information extracted from T2 images;

[0083] Specifically, each encoder module consists of the following modules connected sequentially from front to back according to the data flow direction: the first standard convolutional normalization activation module, the first BAM module, the first max pooling module, the second standard convolutional normalization activation module, the second BAM module, the second max pooling module, the third standard convolutional normalization activation module, the third BAM module, the third max pooling module, the fourth standard convolutional normalization activation module, the fourth max pooling module, the first standard convolutional normalization activation discarding module, the fifth max pooling module, and the second standard convolutional normalization activation discarding module.

[0084] The standard convolutional normalization activation module consists of a standard convolutional layer, an instance normalization layer, and a LeakyReLU activation function layer connected sequentially from front to back according to the data flow direction.

[0085] The standard convolutional normalized activation dropout module consists of a standard convolutional layer, an instance normalization layer, a LeakyReLU activation function layer, and a dropout layer connected sequentially from front to back according to the data flow direction.

[0086] The i-th encoder module and the (i+1)-th encoder module exchange information through three Sobel filter modules, denoted as the 1st Sobel filter module, the 2nd Sobel filter module, and the 3rd Sobel filter module, i=1,3;

[0087] The input of the i-th encoder module, after being processed by the first Sobel filter module, becomes the f corresponding to the first BAM module in the (i+1)-th encoder module. m The input of the i-th encoder module is processed by the first standard convolutional normalization activation module in the i-th encoder module and then used as the f corresponding to the first BAM module in the i-th encoder module. m ;

[0088] The input of the (i+1)th encoder module, after being processed by the first Sobel filter module, becomes the f corresponding to the first BAM module in the i-th encoder module. b The input of the (i+1)th encoder module is processed by the first standard convolutional normalization activation module in the (i+1)th encoder module and used as the f corresponding to the first BAM module in the (i+1)th encoder module. b ;

[0089] The output of the first maximum pooling module in the i-th encoder module is processed by the second Sobel filter module and used as the f corresponding to the second BAM module in the (i+1)-th encoder module. m The output of the first max pooling module in the i-th encoder module is processed by the second standard convolutional normalization activation module in the i-th encoder module and used as the f corresponding to the second BAM module in the i-th encoder module. m ;

[0090] The output of the first maximum pooling module in the (i+1)th encoder module is processed by the second Sobel filter module and used as the f corresponding to the second BAM module in the i-th encoder module. b The output of the first max pooling module in the (i+1)th encoder module is processed by the second standard convolutional normalization activation module in the (i+1)th encoder module and used as the f corresponding to the second BAM module in the (i+1)th encoder module. b ;

[0091] The output of the second maximum pooling module in the i-th encoder module is processed by the third Sobel filter module and used as the f corresponding to the third BAM module in the (i+1)-th encoder module. m The output of the second max pooling module in the i-th encoder module is processed by the third standard convolutional normalization activation module in the i-th encoder module and used as the f corresponding to the third BAM module in the i-th encoder module. m ;

[0092] The output of the second maximum pooling module in the (i+1)th encoder module is processed by the third Sobel filter module and used as the f corresponding to the third BAM module in the i-th encoder module. b The output of the second max pooling module in the (i+1)th encoder module is processed by the third standard convolutional normalization activation module in the (i+1)th encoder module and used as the f corresponding to the third BAM module in the (i+1)th encoder module. b ;

[0093] The decoder contains an MMF module, such as Figure 3 As shown, the input of the MMF module is denoted as f. in The output is denoted as f. out f out =w attention f in ⊕f in w attention =σ(MLP(f unified )), f unified =AvgPool(f in ) ⊕VarPool(f in In the formula, wattention Represents attention weight, ⊕ represents element-wise multiplication, ⊕ represents element-wise addition, σ(·) represents the sigmoid activation function, MLP(·) represents the multilayer perceptron, AvgPool(·) represents the average pooling operation, and VarPool(·) represents the variance pooling operation.

[0094] Specifically, the decoder consists of five non-standard convolutional normalized activation modules, as well as a first MMF module, a second MMF module, a first standard convolutional normalized activation drop module, a third MMF module, a second standard convolutional normalized activation drop module, a fourth MMF module, a first standard convolutional normalized activation module, a fifth MMF module, a second standard convolutional normalized activation module, a sixth MMF module, and a third standard convolutional normalized activation module.

[0095] The non-standard convolutional normalization activation module consists of a non-standard convolutional layer, an instance normalization layer, and a LeakyReLU activation function layer connected sequentially from front to back according to the data flow direction.

[0096] The outputs of the second standard convolutional normalization activation drop-out module in the four encoders are concatenated and then input into the first MMF module.

[0097] The output of the first MMF module is independently upsampled, then concatenated with the output of the first standard convolution normalization activation drop module in the four encoders. After being processed by standard convolution, instance normalization and LeakyReLU activation functions in sequence, the output is then input into the second MMF module.

[0098] The second MMF module is connected to the first standard convolutional normalized activation drop module;

[0099] The output of the first standard convolutional normalization activation drop module in the decoder is independently upsampled and then concatenated with the output of the fourth standard convolutional normalization activation module in the four encoders. After being processed by standard convolution, instance normalization and LeakyReLU activation functions in sequence, they are input into the third MMF module.

[0100] The third MMF module is connected to the second standard convolutional normalized activation drop module in the decoder;

[0101] The output of the second standard convolutional normalization activation drop module in the decoder is independently upsampled and then concatenated with the output of the third standard convolutional normalization activation module in the four encoders. After being processed by the standard convolution, instance normalization and LeakyReLU activation functions in sequence, they are input into the fourth MMF module.

[0102] The fourth MMF module is connected to the first standard convolutional normalization activation module in the decoder;

[0103] The output of the first standard convolutional normalization activation module in the decoder is independently upsampled and then concatenated with the output of the second standard convolutional normalization activation module in the four encoders. After being processed by standard convolution, instance normalization and LeakyReLU activation functions in sequence, they are input into the fifth MMF module.

[0104] The fifth MMF module is connected to the second standard convolutional normalized activation module in the decoder;

[0105] The output of the second standard convolutional normalization activation module in the decoder is independently upsampled and then concatenated with the output of the first standard convolutional normalization activation module in the four encoders. After being processed by the standard convolution, instance normalization and LeakyReLU activation functions in sequence, they are input into the sixth MMF module.

[0106] The sixth MMF module is connected to the third standard convolutional normalization activation module in the decoder;

[0107] The outputs of the first standard convolutional normalized activation drop module, the second standard convolutional normalized activation drop module, the first standard convolutional normalized activation module, the second standard convolutional normalized activation module, and the third standard convolutional normalized activation module are each processed by a non-standard convolutional normalized activation module, and then added element by element. The result is the output of the decoder.

[0108] The uncertainty quantization module generates T segmentation results from the decoder output through Monte Carlo sampling, and then aggregates and outputs the T segmentation results to generate a predicted segmentation image, a random uncertainty map, and a cognitive uncertainty map, where T=10;

[0109] Step 2: Determine the loss function for the image segmentation model:

[0110] The expression for the loss function of the image segmentation model is as follows:

[0111] ;

[0112] ;

[0113] ;

[0114] ;

[0115] ;

[0116] In the formula, Used to measure the degree of overlap between predicted segmentation results and true annotations. Used to capture random uncertainties, taking into account the noise inherent in the observation data. Used to capture cognitive uncertainty, in order to quantify the uncertainty of the model. = =0.1, To divide the total number of voxels, Let i be the predicted value for voxel i. For the true value of voxel i, It is a constant, with a value of 1 × 10 -5 , Let V be the prediction variance of the voxels. Number of samples taken in Monte Carlo Let be the t-th predicted value of voxel i;

[0117] Step 3: Train the image segmentation model:

[0118] (a) Collection There are 335 brain tumor cases (data sourced from the Brain Tumor Segmentation Challenge datasets BraTS2018 and BraTS2019, which are widely used in brain tumor segmentation tasks; BraTS2018 contains 285 brain tumor cases and BraTS2019 contains 335 brain tumor cases). Each brain tumor case has T1, T1c, T2 and FLAIR images.

[0119] (b) Preprocess the T1, T1c, T2 and FLAIR images (i.e., normalize them so that the mean is 0 and the variance is 1, and resize them to 128 pixels × 128 pixels × 128 pixels); obtain the true segmentation image for each brain tumor case, which is the brain tumor MRI image with 3 segmentation regions (complete tumor region, tumor core region and enhanced tumor region) obtained by manual annotation;

[0120] (c) Adopt The training and test sets were constructed using T1, T1c, T2, FLAIR, and ground truth segmentation images corresponding to each brain tumor case (the ratio of brain tumor cases in the training and test sets was 8:2).

[0121] (d) The image segmentation model is trained using the training set. During training, the T1 image, T1c image, T2 image, and Flair image are used as inputs to the image segmentation model, and the real segmented image is used as the theoretical output of the image segmentation model. The weight parameters of the image segmentation model are continuously adjusted until the image segmentation model converges.

[0122] (e) The trained image segmentation model is tested using a test set; the segmentation accuracy of the trained image segmentation model is characterized by the Dice similarity coefficient (DSC) and Hausdorff distance (HD), calculated as follows:

[0123] ;

[0124] In the formula, A and B represent the predicted region (intact tumor region WT, tumor core region TC, or enhanced tumor region ET) and the real region, respectively. This indicates the size of the intersection between the predicted region and the actual region. and These represent the total size of the predicted region and the actual region, respectively;

[0125] ;

[0126] In the formula, X and Y represent the predicted region boundary and the actual region boundary, respectively, and d(x,y) represents the Euclidean distance between points x and y. This represents the maximum value of the minimum distance from each point x on the predicted region boundary to the actual region boundary. This represents the maximum value of the minimum distance from each point y on the true boundary of the region to the boundary of the predicted region.

[0127] The test results are shown in Table 1 below:

[0128] Table 1

[0129] Step four: Preprocess the T1, FLAIR, T1c, and T2 images of the same brain tumor (i.e., normalize them so that the mean is 0 and the variance is 1, and adjust the size to 128 pixels × 128 pixels × 128 pixels). Input the preprocessed T1, FLAIR, T1c, and T2 images of the same brain tumor into the trained image segmentation model, which outputs a predicted segmented image. The predicted segmented image is the brain tumor MRI image with three segmented regions (intact tumor region, tumor core region, and enhanced tumor region) obtained through prediction.

[0130] Comparative Example 1

[0131] A brain tumor segmentation method, such as Figure 5 As shown, it is basically the same as Example 1, except that:

[0132] In step one, the image segmentation model does not include an uncertainty quantization module, the encoder does not include a BAM module, the i-th encoder module and the (i+1)-th encoder module do not exchange information through the Sobel filter module, i=1,3, and the decoder does not include an MMF module.

[0133] In step two, the loss function of the image segmentation model The calculation formula is as follows:

[0134] ;

[0135] ;

[0136] In the formula, N is the total number of voxels in the segmentation. Let i be the predicted value for voxel i. For the true value of voxel i, It is a constant, with a value of 1 × 10 -5 .

[0137] The test results are shown in Table 2 below:

[0138] Table 2

[0139] Example 2

[0140] A brain tumor segmentation method based on boundary-aware mechanism, such as Figure 6 As shown, it is basically the same as Example 1, except that:

[0141] In step one, the image segmentation model does not include an uncertainty quantization module, and the decoder does not include an MMF module;

[0142] In step two, the loss function of the image segmentation model The calculation formula is as follows:

[0143] ;

[0144] ;

[0145] In the formula, N is the total number of voxels in the segmentation. Let i be the predicted value for voxel i. For the true value of voxel i, It is a constant, with a value of 1 × 10 -5 .

[0146] The test results are shown in Table 3 below:

[0147] Table 3

[0148] Example 3

[0149] A brain tumor segmentation method based on boundary-aware mechanism, such as Figure 7 As shown, it is basically the same as Example 1, except that:

[0150] In step one: the image segmentation model does not include an uncertainty quantization module;

[0151] In step two, the loss function of the image segmentation model The calculation formula is as follows:

[0152] ;

[0153] ;

[0154] In the formula, N is the total number of voxels in the segmentation. Let i be the predicted value for voxel i. For the true value of voxel i, It is a constant, with a value of 1 × 10 -5 .

[0155] The test results are shown in Table 4 below:

[0156] Table 4

[0157] The tumor segmentation method in Comparative Example 1 uses the Baseline model. The tumor segmentation method in Example 2 uses the BAM module added to the model in Comparative Example 1. The tumor segmentation method in Example 3 uses the MMF module added to the model in Example 2. The brain tumor segmentation method based on boundary awareness mechanism in Example 1 uses the Uncertainty Quantization (UQ) module added to the model in Example 3.

[0158] As can be seen from the data in Tables 1 to 4, the average DSC of Comparative Example 1 was 83.2%, and the average HD was 5.1 mm. The average DSC of Example 2 was 83.9%, an improvement of 0.8% compared to Comparative Example 1, and the average HD was 4.0 mm, a reduction of 1.1 mm, representing a relative improvement of 21.6%. The most significant enhancement occurred in the tumor core region, where the HD decreased from 6.5 mm to 3.9 mm, a substantial reduction of 40.0%. These results highlight the effectiveness of the BAM module in refining boundary details. Example 3 further improved performance, achieving a relative increase of 1.1% in average DSC and a reduction of 21.6% in average HD compared to Comparative Example 1. This demonstrates the importance of utilizing complementary information from multiple modalities for improving the segmentation of complex tumor regions. Example 1 achieved the highest improvement, with an increase of 1.4% in average DSC and a reduction of 27.5% in average HD compared to Comparative Example 1. For example, in the enhanced tumor region, the HD decreased from the baseline of 3.2 mm to 2.5 mm, a relative improvement of 21.9%, further highlighting the superiority of the brain tumor segmentation method based on the boundary-aware mechanism of the present invention.

[0159] In addition, this invention also visualizes the output results of three brain tumor cases, such as... Figure 4As shown in the figure, the first column displays the FLAIR images of three brain tumor cases, the second, third, fourth, and fifth columns are the predicted segmentation images of the three brain tumor cases output by Comparative Example 1, Example 2, Example 3, and Example 1, respectively, and the sixth column is the actual region of the three brain tumor cases. The comparison in the figure shows that the brain tumor segmentation method based on the boundary awareness mechanism of the present invention can perform more accurate segmentation of tumor subregions.

Claims

1. A brain tumor segmentation method based on boundary-aware mechanism, characterized in that, T1, FLAIR, T1c, and T2 images of the same brain tumor are input into the trained image segmentation model, which outputs a predicted segmented image. The predicted segmented image is the brain tumor MRI image with three segmented regions: the intact tumor region, the tumor core region, and the enhanced tumor region. The image segmentation model includes four encoder modules. The inputs of the four encoder modules correspond to the T1 image, FLAIR image, T1c image and T2 image of the same brain tumor, respectively. The T1 image, FLAIR image, T1c image and T2 image of the same brain tumor are divided into two pairs of MRI modalities. Each encoder module contains a BAM module, which is used to extract modal features f from the same pair of MRI modalities. m With boundary information f b Combining them, we get f out .

2. The brain tumor segmentation method based on boundary awareness mechanism according to claim 1, characterized in that, f out =W f f m ⊕(1-W f ) f b W f =σ(W c * [f m , f b ]+ In the formula, ⊕ represents element-wise multiplication, ⊕ represents element-wise addition, W f This represents the attention map, σ(·) represents the sigmoid activation function, and W... c Represents the convolution kernel, [·] represents offset, [·] represents concatenation by channel, W c *[f m , f b ] indicates W c Feature map after channel splicing [f m , f b Perform convolution operations.

3. The brain tumor segmentation method based on boundary awareness mechanism according to claim 1, characterized in that, The four encoder modules are designated as encoder module 1, encoder module 2, encoder module 3, and encoder module 4; the BAM modules in encoder modules 1 and 2 correspond to f. m All modal features are extracted from the T1 image. The f corresponding to the BAM module in the first encoder module and the second encoder module is... b All boundary information is extracted from Flair images; the BAM modules in the 3rd and 4th encoder modules correspond to f m All modal features are extracted from T1c images. The f corresponding to the BAM module in the 3rd and 4th encoder modules is... b All of these are boundary information extracted from T2 images.

4. The brain tumor segmentation method based on boundary awareness mechanism according to claim 3, characterized in that, The first encoder module and the second encoder module exchange information through the Sobel filter module, and the third encoder module and the fourth encoder module exchange information through the Sobel filter module.

5. A brain tumor segmentation method based on boundary awareness mechanism according to claim 4, characterized in that, The image segmentation model also includes a decoder containing an MMF module. The input to the MMF module is denoted as f. in The output is denoted as f. out f out =w attention f in ⊕f in w attention =σ(MLP(f unified )), f unified =AvgPool(f in )⊕VarPool(f in In the formula, w attention Represents attention weight, ⊕ represents element-wise multiplication, ⊕ represents element-wise addition, σ(·) represents the sigmoid activation function, MLP(·) represents a multilayer perceptron, AvgPool(·) represents average pooling, and VarPool(·) represents variance pooling.

6. The brain tumor segmentation method based on boundary awareness mechanism according to claim 5, characterized in that, The image segmentation model also includes an uncertainty quantization module. The uncertainty quantization module generates T segmentation results from the decoder output through Monte Carlo sampling, and then aggregates the T segmentation results to generate a predicted segmented image, a random uncertainty map, and a cognitive uncertainty map, where T=10.

7. A brain tumor segmentation method based on boundary awareness mechanism according to claim 6, characterized in that, The expression for the loss function of the image segmentation model is as follows: ； ； ； ； ； In the formula, = =0.1, To divide the total number of voxels, Let i be the predicted value for voxel i. For the true value of voxel i, It is a constant, with a value of 1 × 10 -5 Let V be the prediction variance of the voxels. Number of samples taken in Monte Carlo Let t be the predicted value of voxel i.

8. The brain tumor segmentation method based on boundary awareness mechanism according to claim 1, characterized in that, Before the T1, FLAIR, T1c, and T2 images of the same brain tumor are input into the trained image segmentation model, they are preprocessed. The preprocessing involves normalization so that the mean is 0 and the variance is 1, and the size is adjusted to 128 pixels × 128 pixels × 128 pixels.

9. A brain tumor segmentation method based on boundary awareness mechanism according to claim 8, characterized in that, The training steps for the image segmentation model are as follows: (a) Collection A case of brain tumor, ≥285, each brain tumor case had T1, T1c, T2 and FLAIR images; (b) Perform the aforementioned preprocessing on the T1, T1c, T2, and FLAIR images; obtain the true segmentation image for each brain tumor case, which is the brain tumor MRI image with the three segmentation regions obtained through manual annotation; (c) Adopt The training and test sets were constructed using T1, T1c, T2, FLAIR, and real segmentation images corresponding to each brain tumor case. (d) The image segmentation model is trained using the training set. During training, the T1 image, T1c image, T2 image, and Flair image are used as inputs to the image segmentation model, and the real segmented image is used as the theoretical output of the image segmentation model. The weight parameters of the image segmentation model are continuously adjusted until the image segmentation model converges. (e) Test the trained image segmentation model using a test set.

Images