Brain tumor segmentation method based on feature decoupling and fusion

Abstract

The invention belongs to the technical field of medical image processing, and relates to a brain tumor segmentation method based on feature decoupling and fusion. The method comprises the following steps: constructing an image segmentation model comprising five encoders, three DFFM modules and one decoder, inputting four modals of an FLAIR image, a T2 image, a T1 image and a T1ce image into the trained model, and outputting a prediction segmentation image comprising a necrotic or non-enhanced tumor region, an edema region and an enhanced tumor region; the five encoders respectively extract shared features and specific features of four modes, the DFFM module takes the shared features as query, generates key value pairs for the specific features, and realizes feature decoupling and weighted fusion through an attention mechanism. According to the method, feature redundancy and modal confusion are effectively reduced, the multi-modal feature expression ability and semantic consistency are improved, modal complementary information is fully mined, the segmentation precision is guaranteed, and clinical efficient and precise segmentation requirements are met.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing technology and relates to a brain tumor segmentation method based on feature decoupling and fusion. Background Technology

[0002] Brain tumors are a highly heterogeneous and aggressive central nervous system disease. Accurate segmentation of the different components of a brain tumor is crucial for precise diagnosis and personalized treatment planning. Manual segmentation is not only time-consuming and labor-intensive but also highly dependent on the physician's professional experience, resulting in low consistency between results from different physicians and failing to meet the clinical demand for efficient, objective, and high-precision brain tumor segmentation. Multimodal magnetic resonance imaging (MRI) techniques, such as FLAIR, T2, T1, and T1ce sequences, can reveal brain structure and pathological changes from different angles, providing a rich information basis for tumor region identification and driving the development of automated brain tumor segmentation technology.

[0003] Significant progress has been made in brain tumor segmentation technology in recent years, with research generally progressing from traditional image processing and machine learning to deep learning. Early image processing methods relied on low-level features such as grayscale, texture, and edges, typically including thresholding, region growing, and watershed algorithms. While simple to implement and requiring less data, these methods were sensitive to noise and struggled to handle complex brain structures. Subsequent machine learning methods (such as support vector machines, random forests, and fuzzy C-means clustering) achieved automatic segmentation through manual feature extraction combined with classifiers, improving performance. However, they still relied on manual feature design and lacked generalization and robustness.

[0004] With the development of deep learning, segmentation methods based on convolutional neural networks (CNNs) have become mainstream. They can automatically learn multi-layer semantic features. Structures represented by U-Net and its variants effectively combine global semantic information with local details through encoder-decoder structures and skip connection mechanisms, which significantly improves segmentation accuracy and stability. Building upon this foundation, the literature (Edge u-net: Brain tumor segmentation using MRI based on deep U-Net model with boundary information[J]. Expert Systems with Applications, 2023, 213: 118833.), (Transbts: Multimodal brain tumor segmentation using Transformer[C]. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2021. Cham: Springer International Publishing, 2021: 109-119.), and (Multi-scale convolutional attentionfrequency-enhanced Transformer network for medical image segmentation[J]. Information Fusion, 2025: 103019.) further introduces multi-scale feature extraction, attention mechanisms, residual modules, and Transformer structures to enhance the network's ability to model complex lesions and its global semantic association capabilities. Simultaneously, leveraging the complementarity of multimodal MRI data, more refined tumor region identification is achieved through feature splicing, weighting, or attention fusion.

[0005] While existing CNN-based deep learning methods have significantly improved the accuracy of brain tumor segmentation, most models fail to effectively distinguish between modality-shared features and modality-specific features, leading to feature redundancy and modality confusion, which limits the full fusion of multimodal information. Furthermore, common modality fusion strategies such as feature concatenation or static weighting struggle to fully exploit the complementary characteristics between different MRI sequences, failing to achieve efficient decoupling and fusion of shared and specific features. This, in turn, affects the expressive power and semantic consistency of multimodal features, making it difficult to meet the clinical demands for higher accuracy and reliability in brain tumor segmentation. Summary of the Invention

[0006] The purpose of this invention is to address the problems existing in the prior art and provide a brain tumor segmentation method based on feature decoupling and fusion.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] A brain tumor segmentation method based on feature decoupling and fusion is proposed. Four modalities of the same brain tumor are input into a trained image segmentation model, which outputs a predicted segmented image. The predicted segmented image is a brain tumor MRI image with three segmentation regions obtained through prediction. The three segmentation regions are the necrotic or non-enhanced tumor region, the edema region, and the enhanced tumor region.

[0009] The four modalities are FLAIR image, T2 image, T1 image, and T1ce image;

[0010] The image segmentation model consists of 5 encoders, 3 DFFM modules, and 1 decoder;

[0011] The five encoders have the same structure, each consisting of six layers. The first encoder is used to extract shared features of the four modalities, while the second to fifth encoders are used to extract modality-specific features of the four modalities, respectively.

[0012] The input to the DFFM module is the feature tensor X. C represents the channel number, indicating the channel dimension or number of features of the input feature; D represents the depth, which is the dimension of the 3D feature map in the slice direction or layer direction, corresponding to the number of MRI layers; H represents the height, indicating the spatial resolution of the feature map in the vertical direction; W represents the width, indicating the spatial resolution of the feature map in the horizontal direction; X contains three-dimensional voxel features with C channels, and each voxel corresponds to a spatial location of the input MRI.

[0013] X corresponding to the j-th DFFM module is obtained by concatenating the outputs of the (j+3)-th layer of the encoders from the 1st to the 5th encoders along the channel dimension, where j = 1, ..., 3;

[0014] The output of the DFFM module is the fused feature Y;

[0015] The DFFM module uses the shared features of the four modalities as the query, generates key-value pairs (Key, Value) for the specific features of the four modalities, and achieves information interaction and weighted fusion through the attention mechanism to obtain the fused feature Y, thereby enhancing the expressive power and semantic consistency of the multimodal features.

[0016] The image segmentation model of this invention is a three-dimensional encoder-decoder network that integrates decoupled feature fusion and uncertainty modeling. It aims to achieve highly reliable brain tumor segmentation under multimodal MRI. The input data consists of four modalities. The encoder extracts multi-level features, and the decoder restores resolution. The combination of feature fusion and uncertainty analysis improves segmentation performance and reliability. The DFFM module of this invention is designed based on a shared-private attention mechanism. By explicitly decoupling queries, keys, and values, it ensures that shared features selectively gather the most informative signals from modality-specific features. While preserving modality-specific features, it fully captures cross-modal complementary information, reduces feature interference, and thus improves the discriminative power of feature representation.

[0017] As a preferred technical solution:

[0018] The workflow of the DFFM module in the brain tumor segmentation method based on feature decoupling and fusion, as described above, is as follows:

[0019] (a) Divide X into five groups along the channel dimension on an average basis:

[0020] X t =X[:,tC′:(t+1)C′];

[0021] In the formula, t = 0, 1, ..., 4.

[0022] (b) Obtain the query tensor Q (used to guide the information matching between shared and specific features) and the key tensor K respectively. i (Key tensor, used to describe the feature distribution of the i-th mode), value tensor V i (Valuetensor, used to provide the actual feature content of the i-th modality), i = 1, ..., 4, the formula is as follows:

[0023]

[0024] In the formula, Conv3D(·) represents three-dimensional convolution;

[0025] (c) Reshape Q into a matrix K i Remodeling into a matrix V i Remodeling into a matrix Where N = D × H × W;

[0026] (d) Calculate attention weights A i Then, the enhanced feature F is generated. i ;

[0027]

[0028] F i =A i ·V i ′;

[0029] In the formula, softmax(·) represents the normalization operation on each row of the matrix, so that the sum of the weights corresponding to the same query position is 1;

[0030] (e) F i Remapping to spatial dimensions yields the feature tensor

[0031] (f) Calculate Y;

[0032]

[0033] In the formula, concat(·) represents concatenating multiple feature tensors along the channel dimension, and LeakyReLU(·) represents a Leaky Rectified Linear Unit with a leakage coefficient, which is used to introduce nonlinear feature mapping and alleviate the gradient vanishing problem.

[0034] As described above, a brain tumor segmentation method based on feature decoupling and fusion has a encoder whose first to third layers are high-resolution layers, which are formed by stacking several standard convolutional layers, batch normalization layers and nonlinear activation layers, and are used to extract fine-grained texture information and local structural features from the input.

[0035] Layers 4 to 6 of the encoder are low-resolution layers. Feature extraction and spatial compression are achieved by alternately stacking standard convolutional layers and downsampling layers. Each standard convolutional layer and downsampling layer is followed by a batch normalization layer and a non-linear activation function (such as ReLU) to stabilize the training process and enhance the expressive power of high-level semantic features. The downsampling layer uses 3D convolution with a stride greater than 1 or 3D max pooling to reduce spatial resolution and expand the receptive field.

[0036] The brain tumor segmentation method based on feature decoupling and fusion, as described above, has a decoder consisting of 5 layers;

[0037] A deep supervision mechanism is introduced in the decoder stage to further obtain multi-scale feature representations and improve segmentation accuracy;

[0038] The input to the r-th layer of the decoder is obtained by concatenating the outputs of the r-th layers of the 1st to 5th encoders along the channel dimension, where r = 1, ..., 3;

[0039] The input to the fourth layer of the decoder is the output of the first DFFM module, and the input to the fifth layer of the decoder is the outputs of the second and third DFFM modules.

[0040] As described above, a brain tumor segmentation method based on feature decoupling and fusion is described. The r-th layer of the decoder performs the following operations on the input in sequence: first, spatial-channel joint features are extracted through three-dimensional convolution, then batch normalization is used to accelerate training convergence, and then nonlinearity is introduced through a nonlinear activation function (ReLU). After feature processing is completed, three-dimensional upsampling is used to expand the spatial size of the feature map to achieve progressive resolution recovery, and finally output.

[0041] The fourth layer of the decoder uses a transposed convolution or upsampled convolution structure that is symmetrical to the encoder to reconstruct the spatial dimension of its input. At the same time, it is fused with the upsampled features of the previous layer to integrate multi-scale contextual information and generate higher-level spatial semantic features before outputting.

[0042] The fifth layer of the decoder performs convolution and fusion operations on its input before outputting the result.

[0043] As described above, in a brain tumor segmentation method based on feature decoupling and fusion, the output of the first layer of the decoder is added element-wise to the fourth upsampling result after a convolution operation, and the result is the output of the decoder.

[0044] The s-th upsampling result is obtained by adding the output of the 6-s layer of the decoder element-wise to the (s-1)-th upsampling result, and then upsampling again, where s = 2, ..., 4;

[0045] The first upsampling result is obtained by upsampling the output of the 5th layer of the decoder after a convolution operation.

[0046] As described above, the brain tumor segmentation method based on feature decoupling and fusion includes an image segmentation model that further includes an uncertainty quantization module. The uncertainty quantization module generates T segmentation results from the decoder output through Monte Carlo sampling, and then aggregates and outputs the T segmentation results to generate a predicted segmentation image, a random uncertainty map, and a cognitive uncertainty map, where T = 10.

[0047] This uncertainty quantification module is based on the Monte Carlo Dropout method. Specifically, during the inference phase, a dropout rate of 0.1 is applied to the last three layers of the decoder. The prediction set is obtained through T random forward propagations with the dropout mechanism enabled, and then the average prediction map is obtained through aggregation calculation. At the same time, two types of uncertainty are estimated: random uncertainty (used to capture inherent noise in the data, obtained by calculating the average entropy of each prediction) and cognitive uncertainty (reflecting the learnability uncertainty caused by limited model knowledge or training data, obtained by the difference between the average prediction entropy and random uncertainty). This mechanism can provide a confidence metric for spatial localization, highlighting regions with ambiguous or difficult-to-determine segmentation boundaries, providing a reliable reference for clinical diagnosis and decision-making, and improving the model's segmentation performance for complex tumor sub-regions.

[0048] As described above, a brain tumor segmentation method based on feature decoupling and fusion, the loss function of the image segmentation model. The expression is as follows:

[0049]

[0050] In the formula, This represents the Dice loss function; α represents the weighting coefficient, with a value of 0.1. This represents the contrastive loss function; β represents the weighting coefficient, with a value of 0.1. Represents the consistency loss function; The y represents the predicted probability of voxel i in category c, where category c is a necrotic or non-enhancing tumor region, an edematous region, or an enhancing tumor region; ic Represents the true label of voxel i in category c; ∈ is a constant with a value of 1 × 10. -5 ;f a f represents the global average eigenvector from the positive sample mode pair (FLAIR,T2); b represents the global average eigenvector from the positive sample mode pair (T1, T1ce); sim(·,·) represents the cosine similarity; τ represents the temperature hyperparameter, with a value of 0.07; f c Representative and f a The corresponding negative sample mode pair (i.e., mode features whose semantics are irrelevant); f d Representative and f b The corresponding negative sample mode pair (i.e., mode features whose semantics are irrelevant); S m S represents the feature similarity matrix extracted from the shared encoder of the m-th modality. n This represents the feature similarity matrix extracted from the shared encoder of the nth modality, m, n∈{1,2,3,4}, m <n;||·|| F This represents the Frobenius norm.

[0051] The brain tumor segmentation method based on feature decoupling and fusion, as described above, involves the following training steps for the image segmentation model:

[0052] (a) Collect Z brain tumor cases, Z≥285, each brain tumor case has T1 image, T1ce image, T2 image and FLAIR image;

[0053] (b) Obtain the true segmentation image for each brain tumor case, which is the brain tumor MRI image with the three segmentation regions obtained through manual annotation;

[0054] (c) The training set and the test set were constructed using T1 images, T1ce images, T2 images, FLAIR images and real segmentation images corresponding to Z brain tumor cases;

[0055] (d) The image segmentation model is trained using the training set. During training, the T1 image, T1ce image, T2 image, and Flair image are used as inputs to the image segmentation model, and the real segmented image is used as the theoretical output of the image segmentation model. The weight parameters of the image segmentation model are continuously adjusted until the image segmentation model converges.

[0056] (e) Test the trained image segmentation model using a test set.

[0057] Beneficial effects:

[0058] (1) This invention extracts shared and specific features of four modalities by setting up five encoders respectively, and combines them with the DFFM module to achieve explicit decoupling and fusion of the two types of features, effectively reducing feature redundancy and modal confusion, and significantly improving the expressive power and semantic consistency of multimodal features.

[0059] (2) The DFFM module of the present invention uses shared features as queries, generates key-value pairs for specific features, and performs information interaction and weighted fusion through attention mechanism. It can fully explore the complementary characteristics between different MRI sequences, improve the pertinence and effectiveness of multimodal feature fusion, and improve the shortcomings of existing static fusion strategies that are difficult to fully utilize modal complementary information.

[0060] (3) In this invention, the encoder’s first to third layers (high-resolution layers) can extract fine-grained texture information and local structural features from the input, and the fourth to sixth layers (low-resolution layers) can extract high-level semantic features and achieve spatial compression, providing comprehensive and hierarchical feature support for feature fusion and segmentation decoding, and further ensuring segmentation accuracy. Attached Figure Description

[0061] Figure 1 This is a diagram showing the overall architecture of the image segmentation model in Example 1;

[0062] Figure 2 This is a schematic diagram of the DFFM module.

[0063] Figure 3 A visualization comparing the multi-model segmentation results and uncertainty quantification for cases with blurred tumor boundaries (row a) and cases with small enhancement foci (row b); the dashed boxes in the figure indicate the regions of brain tumors of particular interest.

[0064] Figure 4 The overall architecture diagram of the image segmentation model in Comparative Example 1 is shown.

[0065] Figure 5 This is a diagram of the overall architecture of the image segmentation model in Comparative Example 2. Detailed Implementation

[0066] The present invention will be further described below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.

[0067] Example 1

[0068] A brain tumor segmentation method based on feature decoupling and fusion, the specific steps of which are as follows:

[0069] S1. Establish an image segmentation model;

[0070] like Figure 1 As shown, the image segmentation model includes 5 encoders, 3 DFFM modules, 1 decoder, and 1 uncertainty quantization module;

[0071] The five encoders have the same structure, each consisting of six layers. The first encoder is used to extract shared features from four modalities (FLAIR image, T2 image, T1 image, and T1ce image), while the second to fifth encoders are used to extract modal-specific features. The first to third layers of the encoder are high-resolution layers, formed by stacking several standard convolutional layers, batch normalization layers, and non-linear activation layers, used to extract fine-grained texture information and local structural features from the input. The fourth to sixth layers of the encoder are low-resolution layers, which achieve feature extraction and spatial compression by alternately stacking standard convolutional layers and downsampling layers. Each standard convolutional layer and downsampling layer is followed by a batch normalization layer and a non-linear activation function. The downsampling layer uses a three-dimensional convolution or three-dimensional max pooling operation with a stride greater than 1.

[0072] like Figure 2 As shown, the input to the DFFM module is the feature tensor X. C represents the number of channels, D represents the depth, H represents the height, and W represents the width. The x corresponding to the j-th DFFM module is obtained by concatenating the outputs of the (j+3)-th layer of the 1st to 5th encoders along the channel dimension, where j = 1, ..., 3. The output of the DFFM module is the fusion feature Y. The DFFM module uses the shared features of the four modalities as the query, generates key-value pairs for the specific features of the four modalities, and achieves information interaction and weighted fusion through the attention mechanism to obtain the fusion feature Y.

[0073] The workflow of the DFFM module is as follows:

[0074] (a) Divide X into five groups along the channel dimension on an average basis:

[0075] X t =X[:,tC′:(t+1)C′];

[0076] In the formula, t = 0, 1, ..., 4.

[0077] (b) Obtain the query tensor Q and the key tensor K respectively. i Value tensor V i For i = 1, ..., 4, the formula is as follows:

[0078]

[0079] In the formula, Conv3D(·) represents three-dimensional convolution;

[0080] (c) Reshape Q into a matrix K i Remodeling into a matrix V i Remodeling into a matrix Where N = D × H × W;

[0081] (d) Calculate attention weights A i Then, the enhanced feature F is generated. i ;

[0082]

[0083] F i =A i ·V i ′;

[0084] In the formula, softmax(·) represents the normalization operation on each row of the matrix, so that the sum of the weights corresponding to the same query position is 1;

[0085] (e) F i Remapping to spatial dimensions yields the feature tensor

[0086] (f) Calculate Y;

[0087]

[0088] In the formula, concat(·) represents concatenating multiple feature tensors along the channel dimension, and LeakyReLU(·) represents a linear rectified function with a leakage coefficient;

[0089] like Figure 1 As shown, the decoder consists of 5 layers; the input of the r-th layer of the decoder is obtained by concatenating the outputs of the r-th layers of the 1st to 5th encoders sequentially along the channel dimension, r = 1, ..., 3; the input of the 4th layer of the decoder is the output of the 1st DFFM module, and the input of the 5th layer of the decoder is the outputs of the 2nd and 3rd DFFM modules; the r-th layer of the decoder performs the following operations on the input in sequence: first, it extracts spatial-channel joint features through 3D convolution, then accelerates training convergence through batch normalization, then introduces nonlinearity through a nonlinear activation function, and after completing feature processing, it uses 3D upsampling to expand the feature map spatial size to achieve progressive resolution recovery, and finally outputs the solution. The fourth layer of the encoder uses a deconvolution or upsampling convolution structure symmetrical to the encoder to reconstruct the spatial dimension of its input and then fuses it with the upsampling features of the previous layer before outputting. The fifth layer of the decoder performs convolution and fusion operations on its input before outputting. The output of the first layer of the decoder is convolved and then added element-wise with the fourth upsampling result to obtain the decoder output. The s-th upsampling result is obtained by convolving the output of the sixth-s layer of the decoder and then adding it element-wise with the s-1-th upsampling result before upsampling, where s = 2, ..., 4. The first upsampling result is obtained by convolving the output of the fifth layer of the decoder and then upsampling it.

[0090] The uncertainty quantization module generates T segmentation results from the decoder output through Monte Carlo sampling, and then aggregates and outputs the T segmentation results to generate a predicted segmentation image, a random uncertainty map, and a cognitive uncertainty map, where T = 10;

[0091] S2. Determine the loss function for the image segmentation model;

[0092] Loss function of image segmentation model The expression is as follows:

[0093]

[0094] In the formula, This represents the Dice loss function; α represents the weighting coefficient, with a value of 0.1. This represents the contrastive loss function; β represents the weighting coefficient, with a value of 0.1. Represents the consistency loss function; The y represents the predicted probability of voxel i in category c, where category c is a necrotic or non-enhancing tumor region, an edematous region, or an enhancing tumor region; ic Represents the true label of voxel i in category c; ∈ is a constant with a value of 1 × 10. -5 ;f a f represents the global average eigenvector from the positive sample mode pair (FLAIR,T2); b represents the global average eigenvector from the positive sample mode pair (T1, T1ce); sim(·,·) represents the cosine similarity; τ represents the temperature hyperparameter, with a value of 0.07; f c Representative and f a The corresponding negative sample mode pair; f d Representative and f b The corresponding negative sample mode pair; S m S represents the feature similarity matrix extracted from the shared encoder of the m-th modality. n This represents the feature similarity matrix extracted from the shared encoder of the nth modality, m, n∈{1,2,3,4}, m <n;||·|| F Represents the Frobenius norm;

[0095] S3. Train the image segmentation model;

[0096] (a) The dataset used was the BraTS2020 benchmark set, which contained 369 brain tumor cases, each with T1, T1ce, T2 and FLAIR images;

[0097] (b) All images are first subjected to Z-score intensity normalization, and then Gaussian noise is added to achieve data augmentation. Finally, they are uniformly resampled and cropped into 128×128×128 voxel cubes. Real segmentation images (manually labeled) are obtained for each brain tumor case. The real labels divide the voxels into four basic categories: background, non-enhanced tumor / necrosis area (NCR / NET), enhanced tumor area (ET), and edema area (ED). Based on the above four labels, three key segmentation regions are further combined for model training and evaluation. The specific combination rules are: complete tumor area (WT) = NCR / NET + ET + ED - background, tumor core area (TC) = NCR / NET + ET - background, enhanced tumor area (ET, key area) = ET - background.

[0098] (c) All brain tumor cases were divided into training and testing sets by using T1 images, T1ce images, T2 images, FLAIR images, and real segmentation images in a ratio of 8:2.

[0099] (d) The image segmentation model is trained using a training set. During training, images T1, T1ce, T2, and Flair are used as inputs to the image segmentation model, and the real segmented images are used as the theoretical outputs. The weight parameters of the image segmentation model are continuously adjusted. The Adam optimizer is selected, and the initial learning rate is set to 1×10⁻⁶. -4 A cosine annealing scheduling strategy is adopted, gradually decaying the learning rate to a minimum of 1×10⁻⁶ over 50 epochs. -7 If the validation set performance does not improve for 50 consecutive epochs, the early stopping mechanism is triggered to stop training and avoid overfitting. At this point, the model has reached convergence.

[0100] (e) Test the trained image segmentation model using a test set;

[0101] The segmentation accuracy of the trained image segmentation model is characterized by the Dice Similarity Coefficient (DSC) and Hausdorff Distance (HD), and the calculation formulas are as follows:

[0102]

[0103] In the formula, N represents the total number of voxels contained in the current assessment key region (intact tumor region WT, tumor core region TC, or enhanced tumor region ET); p i ∈[0,1], representing the predicted probability of voxel i; g i ∈{0,1} represents the true label of voxel i; ∈ is a constant with a value of 1×10. -5 ;

[0104]

[0105] In the formula, X and Y represent the sets of boundary points of the predicted region and the actual region, respectively, and d(·,·) represents the Euclidean distance between the two points. This represents the maximum of the minimum distances from each point on the predicted region boundary to the actual region boundary. This represents the maximum of the minimum distances from each point on the true boundary of the region to the predicted boundary of the region.

[0106] S4. Output the predicted segmentation image;

[0107] Four modalities of the same brain tumor are input into the trained image segmentation model, which outputs a predicted segmented image. The predicted segmented image is the brain tumor MRI image with three segmented regions obtained through prediction. The three segmented regions are the necrotic or non-enhanced tumor region, the edema region, and the enhanced tumor region.

[0108] Comparative Example 1

[0109] A brain tumor segmentation method, differing from Example 1 in that: the image segmentation model is as follows... Figure 4 As shown, compared to the image segmentation model in Example 1, it has only one encoder and does not include the DFFM module; the loss function of the image segmentation model is as follows:

[0110]

[0111] In the formula, p c G represents the predicted probability of category c. c Represents the true label of category c; ∈ is a constant with a value of 1×10. -5 .

[0112] Comparative Example 2

[0113] A brain tumor segmentation method, differing from Example 1 in that: the image segmentation model is as follows... Figure 5 As shown, compared to the image segmentation model of Example 1, it does not include the DFFM module; the loss function of the image segmentation model is the same as that of Example 1.

[0114] Comparative Example 3

[0115] A brain tumor segmentation method differs from Example 1 in that the loss function of the image segmentation model is the same as that in Example 1.

[0116] The test results of Examples 1 and 3 are shown in Table 1 (bold indicates the best results; * indicates a statistically significant improvement compared to the baseline method (i.e., Comparative Example 1) using the Wilcoxon signed-rank test (p<0.05):

[0117] Table 1:

[0118]

[0119] As shown in Table 1, the gradual introduction of five encoders, a DFFM module, and a loss function design including contrast loss and consistency loss can continuously improve the performance of brain tumor segmentation. Based on Comparative Example 1, introducing five encoders (Comparative Example 2) significantly improves the DSC and average DSC values ​​for WT and ET, while significantly reducing the HD and average HD values ​​for WT and ET. This indicates that the five encoder structure can effectively capture multimodal complementary information and enhance feature representation capabilities. Further introducing the DFFM module (Comparative Example 3) further improves the DSC and average DSC values ​​for TC and ET, and optimizes the average HD values ​​for ET, demonstrating the effectiveness of the DFFM module in decoupling and fusing multimodal features and enhancing the discriminative ability of tumor subregions. Finally, after introducing a loss function including contrast loss and consistency loss (Example 1), optimal results are achieved for DSC and HD values ​​in all key regions. It is worth noting that a similar trend was observed on the BraTS2021 dataset, further validating the universality and effectiveness of this model structure design.

[0120] Figure 3 This paper presents multi-model segmentation results and uncertainty quantification comparison visualizations for two typical complex brain tumor cases: those with blurred tumor boundaries and those with microenhancing lesions. For the case with blurred tumor boundaries in row a, the segmentation result of Example 1 shows a significantly better match with the true label than Comparative Examples 1-3, effectively reducing segmentation bias at the blurred boundary. Simultaneously, the randomness and cognitive uncertainty in this blurred boundary region are significantly increased, allowing for precise identification of suspicious areas in the segmentation results. For the case with microenhancing lesions in row b, these small, low-contrast lesions, which are easily missed, are accurately captured in the segmentation results of Example 1. Compared to Comparative Examples 1-3, the lesion localization is more accurate, and the uncertainty signals in the microenhancing lesions and surrounding areas are more concentrated, suggesting that these small lesions are high-risk areas for segmentation. In summary, the model segmentation scheme of Example 1 demonstrates superior segmentation accuracy in complex brain tumor cases with blurred boundaries and microenhancing lesions. Furthermore, the combination of uncertainty quantification provides intuitive identification of segmentation risk areas, offering effective support for the reliability assessment of clinical brain tumor segmentation.

Claims

1. A brain tumor segmentation method based on feature decoupling and fusion, characterized in that, Four modalities of the same brain tumor are input into the trained image segmentation model, which outputs a predicted segmented image. The predicted segmented image is the brain tumor MRI image with three segmentation regions obtained through prediction. The three segmentation regions are the necrotic or non-enhanced tumor region, the edema region, and the enhanced tumor region. The four modalities are FLAIR image, T2 image, T1 image, and T1ce image; The image segmentation model consists of 5 encoders, 3 DFFM modules, and 1 decoder; The five encoders have the same structure, each consisting of six layers. The first encoder is used to extract shared features of the four modalities, while the second to fifth encoders are used to extract specific features of the four modalities, respectively. The input to the DFFM module is the feature tensor X. C is the number of channels, D is the depth, H is the height, and W is the width; X corresponding to the j-th DFFM module is obtained by concatenating the outputs of the (j+3)-th layer of the encoders from the 1st to the 5th encoders along the channel dimension, where j = 1, ..., 3; The output of the DFFM module is the fused feature Y; The DFFM module uses the shared features of the four modalities as queries, generates key-value pairs for the specific features of the four modalities, and achieves information interaction and weighted fusion through an attention mechanism to obtain the fused feature Y.

2. The brain tumor segmentation method based on feature decoupling and fusion according to claim 1, characterized in that, The workflow of the DFFM module is as follows: (a) Divide X into five groups along the channel dimension on an average basis: X t =X[:,tC′:(t+1)C′]; In the formula, t = 0, 1, ..., 4. (b) Obtain the query tensor Q and the key tensor K respectively. i Value tensor V i For i = 1, ..., 4, the formula is as follows: In the formula, Conv3D(·) represents three-dimensional convolution; (c) Reshape Q into a matrix K i Remodeling into a matrix V i Remodeling into a matrix Where N = D × H × W; (d) Calculate attention weights A i Then, the enhanced feature F is generated. i ; F i =A i ·V i ′; In the formula, softmax(·) represents the normalization operation on each row of the matrix, so that the sum of the weights corresponding to the same query position is 1; (e) F i Remapping to spatial dimensions yields the feature tensor (f) Calculate Y; In the formula, concat(·) represents concatenating multiple feature tensors along the channel dimension, and LeakyReLU(·) represents a linear rectified function with a leakage coefficient.

3. The brain tumor segmentation method based on feature decoupling and fusion according to claim 1, characterized in that, The encoder's first to third layers are high-resolution layers, which are formed by stacking several standard convolutional layers, batch normalization layers, and nonlinear activation layers to extract fine-grained texture information and local structural features from the input. Layers 4 to 6 of the encoder are low-resolution layers. Feature extraction and spatial compression are achieved by alternately stacking standard convolutional layers and downsampling layers. Each standard convolutional layer and downsampling layer is followed by a batch normalization layer and a non-linear activation function. The downsampling layer uses a 3D convolution or 3D max pooling operation with a stride greater than 1.

4. The brain tumor segmentation method based on feature decoupling and fusion according to claim 1, characterized in that, The decoder consists of 5 layers; The input to the r-th layer of the decoder is obtained by concatenating the outputs of the r-th layers of the 1st to 5th encoders along the channel dimension, where r = 1, ..., 3; The input to the fourth layer of the decoder is the output of the first DFFM module, and the input to the fifth layer of the decoder is the outputs of the second and third DFFM modules.

5. A brain tumor segmentation method based on feature decoupling and fusion according to claim 4, characterized in that, The r-th layer of the decoder performs the following operations on the input in sequence: First, it extracts spatial-channel joint features through 3D convolution, then accelerates training convergence through batch normalization, then introduces nonlinearity through a nonlinear activation function, and after completing feature processing, it uses 3D upsampling to expand the spatial size of the feature map to achieve progressive resolution recovery, and finally outputs the result. The fourth layer of the decoder uses a deconvolution or upsampling convolution structure that is symmetrical to the encoder to reconstruct the spatial dimension of its input, and then fuses it with the upsampling features of the previous layer before outputting the result. The fifth layer of the decoder performs convolution and fusion operations on its input before outputting the result.

6. The brain tumor segmentation method based on feature decoupling and fusion according to claim 5, characterized in that, The output of the first layer of the decoder is convolved and then added element-wise to the fourth upsampled result to obtain the output of the decoder. The s-th upsampling result is obtained by adding the output of the 6-s layer of the decoder element-wise to the (s-1)-th upsampling result, and then upsampling again, where s = 2, ..., 4; The first upsampling result is obtained by upsampling the output of the 5th layer of the decoder after a convolution operation.

7. The brain tumor segmentation method based on feature decoupling and fusion according to claim 1, characterized in that, The image segmentation model also includes an uncertainty quantization module. The uncertainty quantization module generates T segmentation results from the decoder output through Monte Carlo sampling, and then aggregates the T segmentation results to generate a predicted segmented image, a random uncertainty map, and a cognitive uncertainty map, where T = 10.

8. The brain tumor segmentation method based on feature decoupling and fusion according to claim 1, characterized in that, Loss function of image segmentation model The expression is as follows: In the formula, This represents the Dice loss function; α represents the weighting coefficient, with a value of 0.

1. Represents the contrastive loss function; β represents the weighting coefficient, with a value of 0.1; Represents the consistency loss function; The y represents the predicted probability of voxel i in category c, where category c is a necrotic or non-enhancing tumor region, an edematous region, or an enhancing tumor region; ic Represents the true label of voxel i in category c; ∈ is a constant with a value of 1 × 10. -5 ; f a This represents the global average eigenvector from the positive sample mode pair (FLAIR,T2); f b represents the global average eigenvector from the positive sample mode pair (T1, T1ce); sim(·,·) represents the cosine similarity; τ represents the temperature hyperparameter, with a value of 0.07; f c Representative and f a The corresponding negative sample mode pair; f d Representative and f b The corresponding negative sample mode pair; S m S represents the feature similarity matrix extracted from the shared encoder of the m-th modality. n This represents the feature similarity matrix extracted from the shared encoder of the nth modality, m, n∈{1,2,3,4}, m <n;||·|| F This represents the Frobenius norm.

9. A brain tumor segmentation method based on feature decoupling and fusion according to claim 1, characterized in that, The training steps for the image segmentation model are as follows: (a) Collect Z brain tumor cases, Z≥285, each brain tumor case has T1 image, T1ce image, T2 image and FLAIR image; (b) Obtain the true segmentation image for each brain tumor case, which is the brain tumor MRI image with the three segmentation regions obtained through manual annotation; (c) The training set and the test set were constructed using T1 images, T1ce images, T2 images, FLAIR images and real segmentation images corresponding to Z brain tumor cases; (d) The image segmentation model is trained using the training set. During training, the T1 image, T1ce image, T2 image, and Flair image are used as inputs to the image segmentation model, and the real segmented image is used as the theoretical output of the image segmentation model. The weight parameters of the image segmentation model are continuously adjusted until the image segmentation model converges. (e) Test the trained image segmentation model using a test set.

Images