A brain tumor segmentation method based on boundary awareness mechanism

Abstract

This invention belongs to the field of medical image analysis technology and relates to a brain tumor segmentation method based on a boundary-aware mechanism. The method involves inputting T1, T1c, T2, and FLAIR images of the brain tumor into a trained image segmentation model, which outputs a predicted segmented image. The predicted segmented image is a brain tumor MRI image with a complete tumor region, a tumor core region, and an enhanced tumor region, obtained through prediction. The proposed brain tumor segmentation method based on a boundary-aware mechanism incorporates boundary information into the image segmentation model, improving the model's ability to distinguish features and achieving accurate segmentation of tumor subregions. A multimodal fusion method is used to integrate complementary information from different MRI sequences, providing a comprehensive understanding of tumor characteristics. Furthermore, uncertainty quantification and an uncertainty-based loss function are combined to provide confidence measurements for the segmentation results, enhancing the accuracy and reliability of the segmentation and assisting clinicians in evaluating the prediction results.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of medical image processing, and particularly relates to a brain tumor segmentation method based on a boundary perception mechanism. BACKGROUND

[0002] As a core technology for brain tumor diagnosis and monitoring, magnetic resonance imaging (MRI) can provide complementary tumor characterization information: T1-weighted (T1) and contrast-enhanced T1-weighted (T1c) sequences are good at outlining the core region of the tumor, and T2-weighted (T2) and fluid attenuated inversion recovery (FLAIR) sequences can clearly present the edema site. However, the inconsistency between MRI modalities, image quality fluctuations, and data missing problems bring great challenges to brain tumor segmentation tasks.

[0003] Currently, brain tumor segmentation techniques mainly include traditional methods, deep learning methods, and other techniques. In traditional segmentation methods, threshold-based methods divide regions by setting gray threshold values, are efficient in calculation, and are often used as a preprocessing method; region-based methods, such as region growing, rely on manual selection of seed points to expand regions, are highly subjective, and although watershed algorithms are based on image gradients for segmentation, they are prone to over-segmentation; edge-based methods use Canny, Sobel, and other operators to detect boundaries and are mostly used as an auxiliary tool; model-based methods require the construction of statistical models of tumor shape, gray scale, and other parameters, which have high requirements for prior knowledge and are complex in modeling and parameter adjustment.

[0004] Deep learning-based segmentation methods have made significant progress: U-Net and its variants effectively capture contextual information through encoder-decoder structures and skip connections, and perform well in complex anatomical structure segmentation; Mask RCNN uses a region proposal network to generate candidate regions and combines attention modules to achieve tumor detection and mask generation; methods based on Transformer, such as TransBTS and SwinBTS, have unique advantages in identifying tumor regions in large fields of view of MRI images by fusing 3D convolution and Transformer architecture. In addition, atlas-based methods rely on high-quality brain atlases for registration and segmentation, which are difficult to adapt to individual differences; hybrid segmentation methods integrate multiple technologies to make up for each other's weaknesses.

[0005] Despite the continuous development of technology, existing brain tumor segmentation methods still have significant defects. In terms of precision, the boundaries between brain tumors and normal brain tissue are blurred, especially the sub-regions of edema, enhanced tumors, necrotic tissue, and non-enhanced regions, which have complex boundaries in MRI images, making it easy to cause segmentation errors. At the same time, there are individual differences in patient brain anatomy and tumor characteristics, and existing models are difficult to fully consider these differences, resulting in poor segmentation results in some special cases, which seriously affects the accuracy of diagnosis.

[0006] At the level of uncertainty, existing neural network methods output a single deterministic prediction, which cannot reflect the inherent variability in imaging data. Due to the complexity of medical images and the fuzziness of lesion boundaries, no algorithm can guarantee absolute accuracy. The single result without confidence information makes it difficult for clinicians to evaluate the reliability of segmentation, seriously restricting accurate diagnosis, and thus affecting the scientificity of treatment decisions, which has become a key bottleneck hindering the clinical application of algorithms. For example, in the literature Eoformer: Edge-oriented transformer for brain tumor segmentation[C] / / Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2023: 333-343, the Eoformer model in the literature has innovation in edge enhancement and computational efficiency, but it still has deficiencies in complex sub-region boundary segmentation, and does not introduce an uncertainty module, lacking a confidence quantification mechanism, which limits the application value of the model in real clinical scenarios.

[0007] Therefore, it is urgent to develop innovative methods to break through the limitations of existing technologies in terms of accuracy and uncertainty quantification, and to improve the clinical value of brain tumor segmentation. SUMMARY

[0008] The purpose of the present application is to solve the problems in the prior art and provide a brain tumor segmentation method based on a boundary perception mechanism.

[0009] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0010] A brain tumor segmentation method based on a boundary perception mechanism, wherein the T1 image, Flair image, T1c image and T2 image of the same brain tumor are input into the trained image segmentation model, and a predicted segmentation image is output by the image segmentation model, wherein the predicted segmentation image is a brain tumor MRI image with 3 segmentation regions obtained by prediction, and the 3 segmentation regions are complete tumor region, tumor core region and enhanced tumor region, respectively.

[0011] The image segmentation model includes 4 encoder modules, and the inputs of the 4 encoder modules correspond to the T1 image, Flair image, T1c image and T2 image of the same brain tumor, respectively. The T1 image, Flair image, T1c image and T2 image of the same brain tumor are divided into two pairs of MRI modalities.

[0012] Each encoder module contains a BAM (Boundary Awareness Mechanism) module, and the structure of the BAM module is shown inFigure 2 The BAM module is used for combining modal features f m extracted from the same pair of MRI modalities respectively b to obtain f out , thereby improving the accuracy of tumor segmentation.

[0013] The method integrates multi-modal data of two pairs of MRI modalities (T1 and Flair, T1c and T2), the T1 image and the T1c image highlight the tumor core region, the Flair image and the T2 image focus on capturing the edema region, and the image segmentation model improves the accuracy of tumor region delineation and enhances the segmentation performance with the aid of the complementary information.

[0014] As a preferred technical solution:

[0015] The brain tumor segmentation method based on the boundary perception mechanism, f out =W f f m ⊕(1-W f )⊗ f b , W f =σ(W c * [f m , f b ]+ ), wherein, represents element-wise multiplication, ⊕ represents element-wise addition, W f represents an attention map, σ(·) represents a sigmoid activation function, W c represents a convolution kernel, represents a bias, [·] represents channel concatenation, W c *[f m , f b ] indicates that W c performs convolution operation on the channel-concatenated feature map [f m , f b ].

[0016] The brain tumor segmentation method based on the boundary perception mechanism, the four encoder modules are respectively a first encoder module, a second encoder module, a third encoder module and a fourth encoder module; the BAM modules in the first encoder module and the second encoder module correspond to f m are modal features extracted from the T1 image, and the BAM modules in the first encoder module and the second encoder module correspond to f b are boundary information extracted from the Flair image; the BAM modules in the third encoder module and the fourth encoder module correspond to f mare modal features extracted from the T1c image, the BAM modules in the 3rd and 4th encoder modules correspond to f b are boundary information extracted from the T2 image.

[0017] The brain tumor segmentation method based on the boundary perception mechanism as described above, the 1st and 2nd encoder modules interact information through the Sobel filter module, and the 3rd and 4th encoder modules interact information through the Sobel filter module.

[0018] The brain tumor segmentation method based on the boundary perception mechanism as described above, each encoder module is composed of a 1st standard convolution normalization activation module, a 1st BAM module, a 1st max-pooling module, a 2nd standard convolution normalization activation module, a 2nd BAM module, a 2nd max-pooling module, a 3rd standard convolution normalization activation module, a 3rd BAM module, a 3rd max-pooling module, a 4th standard convolution normalization activation module, a 4th max-pooling module, a 1st standard convolution normalization activation dropout module, a 5th max-pooling module and a 2nd standard convolution normalization activation dropout module connected in sequence from front to back according to the data flow; the max-pooling module reduces the computational complexity while retaining the key information of the features by effectively reducing the size of the feature map; the BAM module uses the high-resolution feature map generated in the shallow stage to guide the model to focus on the key features that contribute to the final segmentation task, rather than all possible features; the standard convolution normalization activation dropout module is used to prevent overfitting and improve robustness;

[0019] The standard convolution normalization activation module is composed of a standard (3x3x3) convolution layer, a instance normalization layer and a LeakyReLU activation function layer connected in sequence from front to back according to the data flow, to capture the features of a specific modality;

[0020] The standard convolution normalization activation dropout module is composed of a standard convolution layer, a instance normalization layer, a LeakyReLU activation function layer and a dropout layer connected in sequence from front to back according to the data flow;

[0021] The i-th and i+1-th encoder modules interact information through three Sobel filter modules, respectively denoted as the 1st, 2nd and 3rd Sobel filter modules, i=1,3; the Sobel filter module is used to extract boundary information to enhance the structural detail features of the brain tumor;

[0022] The input of the i-th encoder module is processed by the 1st Sobel filter module and used as f m; the input of the i encoder module is processed by the 1st standard convolution normalization activation module in the i encoder module to be the f corresponding to the 1st BAM module in the i encoder module m ;

[0023] the input of the i+1 encoder module is processed by the 1st Sobel filter module to be the f corresponding to the 2nd BAM module in the i encoder module b ; the input of the i+1 encoder module is processed by the 1st standard convolution normalization activation module in the i+1 encoder module to be the f corresponding to the 1st BAM module in the i+1 encoder module b ;

[0024] the output of the 1st max pooling module in the i encoder module is processed by the 2nd Sobel filter module to be the f corresponding to the 2nd BAM module in the i+1 encoder module m ; the output of the 1st max pooling module in the i encoder module is processed by the 2nd standard convolution normalization activation module in the i encoder module to be the f corresponding to the 2nd BAM module in the i encoder module m ;

[0025] the output of the 1st max pooling module in the i+1 encoder module is processed by the 2nd Sobel filter module to be the f corresponding to the 2nd BAM module in the i encoder module b ; the output of the 1st max pooling module in the i+1 encoder module is processed by the 2nd standard convolution normalization activation module in the i+1 encoder module to be the f corresponding to the 2nd BAM module in the i+1 encoder module b ;

[0026] the output of the 2nd max pooling module in the i encoder module is processed by the 3rd Sobel filter module to be the f corresponding to the 3rd BAM module in the i+1 encoder module m ; the output of the 2nd max pooling module in the i encoder module is processed by the 3rd standard convolution normalization activation module in the i encoder module to be the f corresponding to the 3rd BAM module in the i encoder module m ;

[0027] the output of the 2nd max pooling module in the i+1 encoder module is processed by the 3rd Sobel filter module to be the f corresponding to the 3rd BAM module in the i encoder module b ; the output of the 2nd max pooling module in the i+1 encoder module is processed by the 3rd standard convolution normalization activation module in the i+1 encoder module to be the f corresponding to the 3rd BAM module in the i+1 encoder module b .

[0028] The brain tumor segmentation method based on the boundary perception mechanism as described above, the image segmentation model further comprises a decoder, the decoder comprises an MMF (multi-modal fusion) module, a structure diagram of the MMF module is shown in Figure 3 , the input of the MMF module is denoted as f in , the output is denoted as f out , f out =w attention f in ⊕f in , w attention =σ(MLP(f unified )),f unified =AvgPool(f in )⊕VarPool(f in ), wherein w attention represents attention weight, represents element-wise multiplication, ⊕ represents element-wise addition, σ(·) represents a sigmoid activation function, MLP(·) represents a multi-layer perceptron, AvgPool(·) represents an average pooling operation, VarPool(·) represents a variance pooling operation, f unified represents a feature representation obtained by element-wise addition of the two operations, which integrates global and local information; the residual connection is used to retain the original feature information, and the fusion method can effectively integrate the complementary information of multiple MRI modalities, and can simultaneously extract global features and fine-grained local differences based on intensity.

[0029] The brain tumor segmentation method based on the boundary perception mechanism as described above, the decoder is composed of five non-standard convolution normalization activation modules, a first MMF module, a second MMF module, a first standard convolution normalization activation and dropout module, a third MMF module, a second standard convolution normalization activation and dropout module, a fourth MMF module, a first standard convolution normalization activation module, a fifth MMF module, a second standard convolution normalization activation module, a sixth MMF module, and a third standard convolution normalization activation module;

[0030] The non-standard convolution normalization activation module is composed of a non-standard (1x1x1) convolution layer, an instance normalization layer and a LeakyReLU activation function layer connected in sequence from front to back according to the data flow direction;

[0031] The outputs of the second standard convolution normalization activation and dropout modules in the four encoders are concatenated and input into the first MMF module;

[0032] The output of the first MMF module is independently processed by upsampling, concatenated with the output of the first standard convolution normalization activation and discard module in the four encoders, and then sequentially processed by standard convolution, instance normalization and LeakyReLU activation function, and then input into the second MMF module;

[0033] The second MMF module is connected with the first standard convolution normalization activation and discard module;

[0034] The output of the first standard convolution normalization activation and discard module in the decoder is independently processed by upsampling, concatenated with the output of the fourth standard convolution normalization activation module in the four encoders, and then sequentially processed by standard convolution, instance normalization and LeakyReLU activation function, and then input into the third MMF module;

[0035] The third MMF module is connected with the second standard convolution normalization activation and discard module in the decoder;

[0036] The output of the second standard convolution normalization activation and discard module in the decoder is independently processed by upsampling, concatenated with the output of the third standard convolution normalization activation module in the four encoders, and then sequentially processed by standard convolution, instance normalization and LeakyReLU activation function, and then input into the fourth MMF module;

[0037] The fourth MMF module is connected with the first standard convolution normalization activation module in the decoder;

[0038] The output of the first standard convolution normalization activation module in the decoder is independently processed by upsampling, concatenated with the output of the second standard convolution normalization activation module in the four encoders, and then sequentially processed by standard convolution, instance normalization and LeakyReLU activation function, and then input into the fifth MMF module;

[0039] The fifth MMF module is connected with the second standard convolution normalization activation module in the decoder;

[0040] The output of the second standard convolution normalization activation module in the decoder is independently processed by upsampling, concatenated with the output of the first standard convolution normalization activation module in the four encoders, and then sequentially processed by standard convolution, instance normalization and LeakyReLU activation function, and then input into the sixth MMF module;

[0041] The sixth MMF module is connected with the third standard convolution normalization activation module in the decoder;

[0042] The outputs of the first standard convolution normalization activation discard module, the second standard convolution normalization activation discard module, the first standard convolution normalization activation module, the second standard convolution normalization activation module and the third standard convolution normalization activation module are each processed by a non-standard convolution normalization activation module, then element-wise addition is performed, and the obtained result is the output of the decoder;

[0043] In the decoder path, the spatial resolution is gradually recovered through upsampling, and the MMF module further integrates the multi-modal features at each stage.

[0044] The brain tumor segmentation method based on the boundary perception mechanism as described above, the image segmentation model further comprises an uncertainty quantification module (UQ), the uncertainty quantification module generates T segmentation results through Monte Carlo (MC) sampling after the output of the decoder is generated, and then aggregates and outputs the T segmentation results to generate a predicted segmentation image, a random uncertainty map and a cognitive uncertainty map, wherein T = 10; through the random uncertainty map and the cognitive uncertainty map, uncertainty quantification is realized, key insights into the reliability of segmentation prediction are provided, and the clinical practicability of the model is enhanced.

[0045] The expression of the loss function of the image segmentation model in the brain tumor segmentation method based on the boundary perception mechanism as described above is as follows:

[0046] ;

[0047] ;

[0048] ;

[0049] ;

[0050] ;

[0051] In the formula, is used to measure the overlap between the predicted segmentation result and the true label, is used to capture accidental uncertainty to consider the inherent noise in the observed data, is used to capture cognitive uncertainty to quantify the uncertainty of the model, = = 0.1, is the total number of voxels in segmentation, is the predicted value of voxel i, is the true value of voxel i, is a constant, and the value is 1x10 -5 to avoid division by zero, is the predicted variance of the voxel, is the number of times of Monte Carlo sampling, is the t-th prediction value of the voxel i;

[0052] The loss function integrates three key parts of dice loss, aleatoric uncertainty and epistemic uncertainty, and improves the accuracy and reliability of the image segmentation model.

[0053] The brain tumor segmentation method based on the boundary perception mechanism as described above, before the same brain tumor T1 image, Flair image, T1c image and T2 image are input into the trained image segmentation model, the preprocessing is carried out, that is, the normalization processing is carried out to make the mean value 0 and the variance 1, and the size is adjusted to 128 pixels x 128 pixels x 128 pixels.

[0054] The training steps of the image segmentation model in the brain tumor segmentation method based on the boundary perception mechanism as described above are as follows:

[0055] (a) Collect brain tumor cases, ≥285 (the data sources are the data sets BraTS2018 and BraTS2019 of brain tumor segmentation challenge, which are widely used in brain tumor segmentation tasks), each brain tumor case has T1 image, T1c image, T2 image and Flair image;

[0056] (b) The T1 image, T1c image, T2 image and Flair image are preprocessed; the real segmentation image of each brain tumor case is obtained, that is, the brain tumor MRI image with the three segmentation regions obtained by artificial labeling;

[0057] (c) Using T1 images, T1c images, T2 images, Flair images and real segmentation images corresponding to the 8 brain tumor cases to construct a training set and a test set, wherein the split ratio of the training set and the test set is 8:2;

[0058] (d) training the image segmentation model using the training set, the model being implemented using Keras, the training being performed on an NVIDIA RTX 4090 GPU, during the training, the T1 image, the T1c image, the T2 image and the Flair image being used as input of the image segmentation model, the real segmentation image being used as theoretical output of the image segmentation model, in addition to which there are some key settings: an Adam optimizer with an initial learning rate of 0.0005 is used; if the validation loss does not improve for 10 consecutive epochs, the learning rate is halved; if the validation loss does not improve for 50 epochs, the training is stopped early in order to prevent overfitting, and the batch size is set to 1; the weight parameters of the image segmentation model (the weight and bias parameters of the convolutional layers, the parameters of the instance normalization layers) are constantly adjusted until the image segmentation model converges (i.e. the loss function value gradually decreases and tends to a stable value);

[0059] (e) testing the trained image segmentation model using the test set;

[0060] The Dice Similarity Coefficient (DSC) and the Hausdorff Distance (HD) are used to represent the segmentation accuracy of the trained image segmentation model, the calculation formulas being as follows:

[0061] ;

[0062] wherein A and B represent the predicted region (the complete tumor region WT, the tumor core region TC or the enhanced tumor region ET) and the real region, respectively, represents the intersection size of the predicted region and the real region, and represent the total size of the predicted region and the real region, respectively;

[0063] ;

[0064] wherein X and Y represent the predicted region boundary and the real region boundary, respectively, d(x, y) represents the Euclidean distance between point x and point y, represents the maximum value of the minimum distance from each point x of the predicted region boundary to the real region boundary, represents the maximum value of the minimum distance from each point y of the real region boundary to the predicted region boundary.

[0065] Advantageous effects:

[0066] The brain tumor segmentation method based on the boundary perception mechanism of the application introduces the boundary perception mechanism, clearly incorporates the boundary information into the image segmentation model, enhances the ability of the model to learn more discriminative features, and realizes the precise segmentation of tumor sub-regions. In addition, the proposed multi-modal fusion method can capture complementary information from various MRI sequences, so that the tumor features are more comprehensively understood between different modalities. In order to further improve the clinical reliability, the uncertainty quantification and a novel uncertainty-based loss function (which provides a confidence measure for each segmentation result) are combined, which improves the accuracy and reliability of brain tumor segmentation and assists clinicians in evaluating the prediction. BRIEF DESCRIPTION OF DRAWINGS

[0067] Figure 1 The model structure diagram of the brain tumor segmentation method based on the boundary perception mechanism of the application in embodiment 1;

[0068] Figure 2 The structure diagram of the BAM module in the application;

[0069] Figure 3 The structure diagram of the MMF module in the application;

[0070] Figure 4 The visualization results of three brain tumor cases after segmentation by using the brain tumor segmentation methods of comparative example 1, embodiment 2, embodiment 3 and embodiment 1 respectively; in the figure, the red area is the tumor core area, the red area + yellow area is the enhanced tumor area, the red area + yellow area + green area is the complete tumor area, the blue circled part is the distinguished part, and the numerical value in the upper right corner is the average DSC of the segmentation result;

[0071] Figure 5 The model structure diagram of the brain tumor segmentation method of comparative example 1 in the application;

[0072] Figure 6 The model structure diagram of the brain tumor segmentation method based on the boundary perception mechanism of the application in embodiment 2;

[0073] Figure 7 The model structure diagram of the brain tumor segmentation method based on the boundary perception mechanism of the application in embodiment 3. DETAILED DESCRIPTION

[0074] The application will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the application and not to limit the scope of the application. In addition, it should be understood that those skilled in the art can make various modifications or changes to the application after reading the content taught by the application, and these equivalent forms also fall within the scope defined by the appended claims of the application.

[0075] Embodiment 1

[0076] A brain tumor segmentation method based on a boundary perception mechanism, the steps are as follows:

[0077] Step 1: Establish an image segmentation model:

[0078] The image segmentation model includes a first encoder module, a second encoder module, a third encoder module, a fourth encoder module, a decoder, and an uncertainty quantization module;

[0079] The inputs of the first encoder module, the second encoder module, the third encoder module, and the fourth encoder module correspond to the T1 image, the Flair image, the T1c image, and the T2 image of the same brain tumor, respectively;

[0080] Each encoder module contains a BAM module, as shown in Figure 2 The BAM module has two inputs and one output, and the two inputs are denoted as f m and f b , and the output is denoted as f out , f out =W f f m ⊕(1-W f ) f b , W f =σ(W c * [f m , f b ]+ ), wherein, represents element-wise multiplication, ⊕ represents element-wise addition, W f represents an attention map, σ(·) represents a sigmoid activation function, W c represents a convolution kernel, represents a bias, [·] represents channel concatenation, and W c *[f m , f b ] indicates that W c performs convolution operation on the channel-concatenated feature map [f m , f b ];

[0081] The f m corresponding to the BAM modules in the first encoder module and the second encoder module are modal features extracted from the T1 image, and the f b corresponding to the BAM modules in the first encoder module and the second encoder module are boundary information extracted from the Flair image;

[0082] The BAM modules in the 3rd and 4th encoder modules correspond to f m , which are all boundary information extracted from the T2 image; b , which are all boundary information extracted from the T2 image;

[0083] Specifically, each encoder module is composed of, in sequence from front to back in data flow direction, a 1st standard convolution normalization activation module, a 1st BAM module, a 1st max pooling module, a 2nd standard convolution normalization activation module, a 2nd BAM module, a 2nd max pooling module, a 3rd standard convolution normalization activation module, a 3rd BAM module, a 3rd max pooling module, a 4th standard convolution normalization activation module, a 4th max pooling module, a 1st standard convolution normalization activation dropout module, a 5th max pooling module, and a 2nd standard convolution normalization activation dropout module;

[0084] The standard convolution normalization activation module is composed of, in sequence from front to back in data flow direction, a standard convolution layer, an instance normalization layer, and a LeakyReLU activation function layer;

[0085] The standard convolution normalization activation dropout module is composed of, in sequence from front to back in data flow direction, a standard convolution layer, an instance normalization layer, a LeakyReLU activation function layer, and a dropout layer;

[0086] The i-th encoder module and the i+1-th encoder module interact information through three Sobel filter modules, denoted as a 1st Sobel filter module, a 2nd Sobel filter module, and a 3rd Sobel filter module, i = 1, 3;

[0087] The input of the i-th encoder module, after being processed by the 1st Sobel filter module, serves as f m ; the input of the i-th encoder module, after being processed by the 1st standard convolution normalization activation module in the i-th encoder module, serves as f m ;

[0088] The input of the i+1-th encoder module, after being processed by the 1st Sobel filter module, serves as f b ; the input of the i+1-th encoder module, after being processed by the 1st standard convolution normalization activation module in the i+1-th encoder module, serves as f b ;

[0089] The output of the first max-pooling module in the i encoder module is processed by the second Sobel filter module and serves as the corresponding f m of the second BAM module in the i encoder module m ;

[0090] The output of the first max-pooling module in the i+1 encoder module is processed by the second Sobel filter module and serves as the corresponding f b of the second BAM module in the i encoder module b ;

[0091] The output of the second max-pooling module in the i encoder module is processed by the third Sobel filter module and serves as the corresponding f m of the third BAM module in the i encoder module m ;

[0092] The output of the second max-pooling module in the i+1 encoder module is processed by the third Sobel filter module and serves as the corresponding f b of the third BAM module in the i encoder module b ;

[0093] The decoder contains an MMF module, as shown in Figure 3 The input of the MMF module is denoted as f in , the output is denoted as f out , f out = w attention f in ⊕ f in , w attention = σ (MLP (f unified )), f unified = AvgPool (f in ) ⊕ VarPool (f in ), wherein wattention Represents attention weight, ⊕ represents element-wise multiplication, ⊕ represents element-wise addition, σ(·) represents the sigmoid activation function, MLP(·) represents the multilayer perceptron, AvgPool(·) represents the average pooling operation, and VarPool(·) represents the variance pooling operation.

[0094] Specifically, the decoder consists of five non-standard convolutional normalized activation modules, as well as a first MMF module, a second MMF module, a first standard convolutional normalized activation drop module, a third MMF module, a second standard convolutional normalized activation drop module, a fourth MMF module, a first standard convolutional normalized activation module, a fifth MMF module, a second standard convolutional normalized activation module, a sixth MMF module, and a third standard convolutional normalized activation module.

[0095] The non-standard convolutional normalization activation module consists of a non-standard convolutional layer, an instance normalization layer, and a LeakyReLU activation function layer connected sequentially from front to back according to the data flow direction.

[0096] The outputs of the second standard convolutional normalization activation drop-out module in the four encoders are concatenated and then input into the first MMF module.

[0097] The output of the first MMF module is independently upsampled, then concatenated with the output of the first standard convolution normalization activation drop module in the four encoders. After being processed by standard convolution, instance normalization and LeakyReLU activation functions in sequence, the output is then input into the second MMF module.

[0098] The second MMF module is connected to the first standard convolutional normalized activation drop module;

[0099] The output of the first standard convolutional normalization activation drop module in the decoder is independently upsampled and then concatenated with the output of the fourth standard convolutional normalization activation module in the four encoders. After being processed by standard convolution, instance normalization and LeakyReLU activation functions in sequence, they are input into the third MMF module.

[0100] The third MMF module is connected to the second standard convolutional normalized activation drop module in the decoder;

[0101] The output of the second standard convolutional normalization activation drop module in the decoder is independently upsampled and then concatenated with the output of the third standard convolutional normalization activation module in the four encoders. After being processed by the standard convolution, instance normalization and LeakyReLU activation functions in sequence, they are input into the fourth MMF module.

[0102] The fourth MMF module is connected to the first standard convolutional normalization activation module in the decoder;

[0103] The output of the first standard convolution normalization activation module in the decoder is independently up-sampled, concatenated with the output of the second standard convolution normalization activation module in the four encoders, and then processed by a standard convolution, an instance normalization, and a LeakyReLU activation function in sequence to input into the fifth MMF module;

[0104] The fifth MMF module is connected with the second standard convolution normalization activation module in the decoder;

[0105] The output of the second standard convolution normalization activation module in the decoder is independently up-sampled, concatenated with the output of the first standard convolution normalization activation module in the four encoders, and then processed by a standard convolution, an instance normalization, and a LeakyReLU activation function in sequence to input into the sixth MMF module;

[0106] The sixth MMF module is connected with the third standard convolution normalization activation module in the decoder;

[0107] The outputs of the first standard convolution normalization activation discard module, the second standard convolution normalization activation discard module, the first standard convolution normalization activation module, the second standard convolution normalization activation module, and the third standard convolution normalization activation module are each processed by a non-standard convolution normalization activation module, then element-wise addition is performed, and the obtained result is the output of the decoder;

[0108] The uncertainty quantification module generates T segmentation results by Monte Carlo sampling on the output of the decoder, aggregates the T segmentation results, and outputs a predicted segmentation image, a random uncertainty map, and a cognitive uncertainty map, where T = 10;

[0109] Step two, determine the loss function of the image segmentation model:

[0110] The expression of the loss function of the image segmentation model is as follows:

[0111] ;

[0112] ;

[0113] ;

[0114] ;

[0115] ;

[0116] In the formula, is used to measure the overlap between the predicted segmentation result and the real label, is used to capture accidental uncertainty to consider the inherent noise in the observation data, for capturing cognitive uncertainty to quantify the uncertainty of the model, = = 0.1, is the total number of voxels in the segmentation, is the predicted value of voxel i, is the true value of voxel i, is a constant, taking the value of 1 x 10 -5 , is the predicted variance of the voxel, is the number of Monte Carlo samples, is the tth predicted value of voxel i;

[0117] Step three, training the image segmentation model:

[0118] (a) Collecting brain tumor cases (the data sources are the BraTS2018 and BraTS2019 datasets of the brain tumor segmentation challenge, which are widely used in the brain tumor segmentation task, wherein the BraTS2018 contains 285 brain tumor cases and the BraTS2019 contains 335 brain tumor cases), each brain tumor case has T1 image, T1c image, T2 image and Flair image;

[0119] (b) Preprocessing the T1 image, T1c image, T2 image and Flair image (i.e. normalizing the image to make the mean value 0 and the variance 1, and adjusting the size to 128 pixels x 128 pixels x 128 pixels); obtaining the real segmentation image of each brain tumor case, which is the brain tumor MRI image with 3 segmentation regions (complete tumor region, tumor core region and enhanced tumor region) obtained by manual annotation;

[0120] (c) Using T1 images, T1c images, T2 images, Flair images and real segmentation images corresponding to 8 brain tumor cases to construct a training set and a test set (the number of brain tumor cases in the training set and the test set is in the ratio of 8:2);

[0121] (d) Training the image segmentation model using the training set, during training, taking the T1 image, T1c image, T2 image and Flair image as the input of the image segmentation model, and taking the real segmentation image as the theoretical output of the image segmentation model, and constantly adjusting the weight parameters of the image segmentation model until the image segmentation model converges;

[0122] (e) Testing the trained image segmentation model using the test set; using Dice similarity coefficient (DSC) and Hausdorff distance (HD) to represent the segmentation accuracy of the trained image segmentation model, the calculation formula is as follows:

[0123] ;

[0124] In the formula, A and B respectively represent a prediction region (a complete tumor region WT, a tumor core region TC, or an enhanced tumor region ET) and a true region, represents the intersection size of the prediction region and the true region, and respectively represent the total size of the prediction region and the true region;

[0125] ;

[0126] In the formula, X and Y respectively represent a prediction region boundary and a true region boundary, d(x, y) represents the Euclidean distance between a point x and a point y, represents the maximum value of the minimum distance from each point x of the prediction region boundary to the true region boundary, represents the maximum value of the minimum distance from each point y of the true region boundary to the prediction region boundary;

[0127] The test results are shown in Table 1 below:

[0128] Table 1

[0129]

[0130] Step four, pre-process (i.e., normalize to have a mean of 0 and a variance of 1, and adjust the size to 128 pixels x 128 pixels x 128 pixels) the T1 image, the Flair image, the T1c image, and the T2 image of the same brain tumor, and input the pre-processed T1 image, the Flair image, the T1c image, and the T2 image of the same brain tumor into the trained image segmentation model to output a prediction segmentation image, which is a brain tumor MRI image with 3 segmented regions (a complete tumor region, a tumor core region, and an enhanced tumor region) obtained by prediction.

[0131] Comparative Example 1

[0132] A brain tumor segmentation method, as shown in Figure 5 , is basically the same as that of Example 1, except that:

[0133] In step one, the image segmentation model does not include an uncertainty quantification module, the encoder does not include a BAM module, and the i-th encoder module and the i+1-th encoder module do not interact information through a Sobel filter module, i = 1, 3, and the decoder does not include an MMF module;

[0134] In step two, the calculation formula of the loss function of the image segmentation model is as follows:

[0135] ;

[0136] ;

[0137] wherein N is the total number of voxels in segmentation, is the predicted value of voxel i, is the true value of voxel i, is a constant, taking the value of 1 x 10 -5 .

[0138] The test results are shown in Table 2 as follows:

[0139] Table 2

[0140]

[0141] Example 2

[0142] A brain tumor segmentation method based on a boundary perception mechanism, as shown in Figure 6 , is basically the same as in Example 1, except that:

[0143] In step one, the image segmentation model does not include an uncertainty quantification module, and the decoder does not include an MMF module.

[0144] In step two, the loss function of the image segmentation model is calculated as follows:

[0145] ;

[0146] ;

[0147] wherein N is the total number of voxels in segmentation, is the predicted value of voxel i, is the true value of voxel i, is a constant, taking the value of 1 x 10 -5 .

[0148] The test results are shown in Table 3 as follows:

[0149] Table 3

[0150]

[0151] Example 3

[0152] A brain tumor segmentation method based on a boundary perception mechanism, as shown in Figure 7 , is basically the same as in Example 1, except that:

[0153] In step one: the image segmentation model does not include an uncertainty quantification module;

[0154] In step two, the loss function of the image segmentation model is calculated according to the following formula:

[0155]

[0156]

[0157] wherein N is the total number of voxels in segmentation, is the predicted value of voxel i, is the true value of voxel i, is a constant, and is 1x10 -5 .

[0158] The test results are shown in Table 4 below:

[0159] Table 4

[0160]

[0161] The model used in the tumor segmentation method of Comparative Example 1 is a baseline model. The model used in the tumor segmentation method of Example 2 adds a BAM module to the model of Comparative Example 1. The model used in the tumor segmentation method of Example 3 adds a MMF module to the model of Example 2. The model used in the brain tumor segmentation method based on the boundary perception mechanism of Example 1 further adds an uncertainty quantification module (UQ) to the model of Example 3.

[0162] As can be seen from the data in Tables 1-4, the average DSC of Comparative Example 1 is 83.2%, and the average HD is 5.1 mm. The average DSC of Example 2 is 83.9%, which is 0.8% higher than that of Comparative Example 1, and the average HD is 4.0 mm, which is reduced by 1.1 mm, with a relative improvement of 21.6%. The most significant improvement is in the tumor core region, where the HD is reduced from 6.5 mm to 3.9 mm, a significant reduction of 40.0%. These results highlight the effectiveness of the BAM module in refining boundary details. Example 3 further improves the performance, achieving a relative increase of 1.1% in average DSC and a reduction of 21.6% in average HD compared to Comparative Example 1, which indicates the importance of utilizing complementary information from multiple modalities for improving segmentation of complex tumor regions. Example 1 achieves the highest improvement, with an average DSC increase of 1.4% and an average HD reduction of 27.5% compared to Comparative Example 1. For example, in the enhanced tumor region, the HD is reduced from 3.2 mm in the baseline to 2.5 mm, with a relative improvement of 21.9%. This further highlights the superiority of the brain tumor segmentation method based on the boundary perception mechanism of the present application.

[0163] ​​​In addition, the output results of three brain tumor cases are visualized as shown in the following table. Figure 4 As shown in the table, the first column shows the Flair images of the three brain tumor cases, the second column, the third column, the fourth column and the fifth column are the predicted segmentation images of the three brain tumor cases of the output of Comparative Example 1, Example 2, Example 3 and Example 1 in turn, and the sixth column is the true region of the three brain tumor cases; it can be seen from the comparison in the table that the brain tumor segmentation method based on the boundary perception mechanism of the application can perform more accurate segmentation on tumor sub-regions.

Claims

1. A brain tumor segmentation method based on boundary-aware mechanism, characterized in that, The T1 image, the Flair image, the T1c image and the T2 image of the same brain tumor are input into the trained image segmentation model, and a predicted segmentation image is output, wherein the predicted segmentation image is a brain tumor MRI image with three segmentation regions obtained by prediction, and the three segmentation regions are a complete tumor region, a tumor core region and an enhanced tumor region respectively; The image segmentation model comprises four encoder modules, and the inputs of the four encoder modules correspond to the T1 image, the Flair image, the T1c image and the T2 image of the same brain tumor respectively. The BAM module is used for combining the modality features f m with the boundary information f b to obtain f out ; 4 encoder modules are respectively denoted as a first encoder module, a second encoder module, a third encoder module and a fourth encoder module; f m are all boundary information extracted from the Flair image; f b are all boundary information extracted from the Flair image; f m are all boundary information extracted from the Flair image; f b are all boundary information extracted from the Flair image.

2. The brain tumor segmentation method based on boundary-aware mechanism according to claim 1, characterized in that, f out =W f f m ⊕(1−W f ) f b W f =σ(W c * [f m , f b ]+ In the formula, ⊕ represents element-wise multiplication, ⊕ represents element-wise addition, W f This represents the attention map, σ(·) represents the sigmoid activation function, and W... c Represents the convolution kernel, [·] represents offset, [·] represents concatenation by channel, W c *[f m , f b ] indicates W c Feature map after channel splicing [f m , f b Perform convolution operations.

3. The brain tumor segmentation method based on boundary-aware mechanism according to claim 1, characterized in that, The first encoder module and the second encoder module interact information through a Sobel filter module, and the third encoder module and the fourth encoder module interact information through a Sobel filter module.

4. The brain tumor segmentation method based on boundary-aware mechanism according to claim 3, characterized in that, The image segmentation model further comprises a decoder, the decoder comprising an MMF module, an input of the MMF module being denoted as f in , and an output of the MMF module being denoted as f out , f out = w attention f in ⊕ f in , w attention = σ(MLP(f unified )), f unified = AvgPool(f in ) ⊕ VarPool(f in ), wherein w attention represents an attention weight, represents element-wise multiplication, ⊕ represents element-wise addition, σ(·) represents a sigmoid activation function, MLP(·) represents a multi-layer perceptron, AvgPool(·) represents an average pooling operation, and VarPool(·) represents a variance pooling operation.

5. The brain tumor segmentation method based on boundary-aware mechanism according to claim 4, characterized in that, The image segmentation model further comprises an uncertainty quantification module, which generates T segmentation results after the output of the decoder is generated by Monte Carlo sampling, and then aggregates the T segmentation results to output a predicted segmentation image, a random uncertainty map and a cognitive uncertainty map, wherein T = 10.

6. The brain tumor segmentation method based on boundary-aware mechanism according to claim 5, characterized in that, The expression of the loss function of the image segmentation model is as follows: ； ； ； ； ； wherein, = 0.1, = 0.1, is the total number of voxels in the segmentation, is the predicted value of voxel i, is the true value of voxel i, is a constant with value 1 x 10 -5 is the predicted variance of voxel i, is the number of Monte Carlo samples, is the t-th predicted value of voxel i.

7. The brain tumor segmentation method based on boundary-aware mechanism according to claim 1, characterized in that, Before the T1 image, the Flair image, the T1c image and the T2 image of the same brain tumor are input into the trained image segmentation model, they are preprocessed, that is, normalized to have a mean of 0 and a variance of 1, and the size is adjusted to 128 pixels x 128 pixels x 128 pixels.

8. The brain tumor segmentation method based on boundary-aware mechanism according to claim 7, characterized in that, The training steps of the image segmentation model are as follows: (a) acquiring a brain tumor case, ≥ 285, each brain tumor case has a T1 image, a T1c image, a T2 image, and a Flair image; (b) performing the preprocessing on the T1 image, the T1c image, the T2 image and the Flair image; obtaining a real segmentation image of each brain tumor case, wherein the real segmentation image is a brain tumor MRI image with the three segmentation regions obtained by manual annotation; (c) using T1 images, T1c images, T2 images, Flair images, and real segmentation images corresponding to 100 brain tumor cases are used to construct a training set and a test set. (d) training the image segmentation model using the training set, wherein during the training, the T1 image, the T1c image, the T2 image and the Flair image are input into the image segmentation model, the real segmentation image is taken as the theoretical output of the image segmentation model, the weight parameters of the image segmentation model are adjusted constantly until the image segmentation model converges; and (e) testing the trained image segmentation model using the test set.

Images