A brain tumor segmentation method based on three-axis context encoding

Abstract

The application belongs to the technical field of medical image processing, and relates to a brain tumor segmentation method based on three-axis context encoding, which comprises the following steps: inputting a FLAIR image, a T2 image, a T1 image and a T1ce image of the same glioma into a trained image segmentation model, and outputting a prediction segmentation image with a non-enhanced necrosis region, an edema region and an enhanced tumor region from the image segmentation model; the image segmentation model comprises an encoder and a decoder; the encoder comprises a TCE module, and the TCE module learns the relative importance among an axial plane branch, a coronal plane branch and a sagittal plane branch to realize dynamic weighted fusion of multi-plane features. The application solves the problems of insufficient anisotropy feature capture and poor multi-modal fusion adaptability, realizes high-precision segmentation of the non-enhanced necrosis region, the edema region and the enhanced tumor region, and provides reliable technical support for clinical diagnosis and treatment planning of brain tumors.

Description

A brain tumor segmentation method based on triaxial context coding Technical Field

[0001] This invention belongs to the field of medical image processing technology and relates to a brain tumor segmentation method based on triaxial context coding. Background Technology

[0002] Brain tumors are highly aggressive and life-threatening cancers, and accurate segmentation is crucial for clinical diagnosis, treatment planning, and prognostic assessment. Gliomas account for approximately 80% of all malignant brain tumors, with glioblastoma multiforme (GBM) being the most aggressive, boasting a 5-year survival rate of only 6.9%. Early and precise tumor segmentation is a key prerequisite for improving treatment outcomes.

[0003] Magnetic resonance imaging (MRI), with its excellent soft tissue contrast and non-ionizing radiation characteristics, has become the core imaging tool for brain tumor segmentation. Clinically, four MRI sequences are typically combined to achieve a comprehensive characterization of the tumor region: T1-weighted images provide anatomical details; T1-contrast (T1ce) images highlight areas of blood-brain barrier disruption to identify enhancing tumors; T2-weighted images are sensitive to fluid content to depict edema; and fluid attenuation inversion recovery (FLAIR) images suppress cerebrospinal fluid signals to highlight peritumoral and non-enhancing lesions. Accurately capturing the three-dimensional spatial features of the tumor and effectively fusing multimodal imaging information are crucial for improving segmentation accuracy in brain tumor segmentation.

[0004] Current brain tumor segmentation technology has many limitations.

[0005] In traditional segmentation methods, thresholding divides image pixels by setting one or more thresholds. It is simple to calculate and fast, and is often used as a preprocessing method or in combination with other methods. However, it is sensitive to noise and has difficulty handling images with uneven lighting or complex textures. Region growing starts from a seed point and includes similar neighboring pixels into regions, which can obtain connected and smooth boundary results. However, it depends on the selection of the initial seed point, is sensitive to noise, and is prone to overgrowing or undersegmentation. Edge detection uses derivatives to detect edges and combine them into closed regions. It can accurately locate boundaries and works well for objects with clear boundaries. However, it is prone to producing discontinuities or false edges and cannot handle images with blurred textures or low contrast well.

[0006] In classic machine learning methods, Support Vector Machines (SVMs) take pixel or superpixel features as input, making them suitable for high-dimensional image features. However, they are sensitive to parameters (requiring careful selection of kernel functions, C parameters, etc.), have high computational costs, especially in large-scale image or multi-class problems, and cannot directly handle multi-class problems. Random forests, as ensemble learning models, make decisions through voting from multiple decision trees, are insensitive to noise and overfitting, and have good generalization performance. However, their segmentation boundaries are not fine enough, they are highly dependent on features, and their model structure is complex and has poor interpretability.

[0007] Among deep learning-based segmentation algorithms, U-Net and its variants, with their encoder-decoder structure and skip connections, can effectively capture contextual information and perform well when data is limited. However, they are less adaptable to complex scenarios, and the large number of parameters leads to slower inference speeds. Transformer-based models (such as TransBTS and SwinUNETR) can capture long-distance dependencies and global context. Graph convolutional networks model MRI data as a graph structure to perform cross-modal inference and capture long-distance interactions. However, both suffer from similar problems to U-Net in terms of adaptability to complex scenarios and inference efficiency.

[0008] Furthermore, most existing brain tumor segmentation methods can only generate deterministic outputs, ignoring the inherent uncertainty of medical data, making it difficult for doctors to judge the reliability and confidence level of the results. Due to the complexity of medical images and the ambiguity of lesion boundaries, a single deterministic result lacks confidence level information, which can affect doctors' accurate diagnosis and patients' treatment decisions.

[0009] Patent CN119863624B (hereinafter referred to as the patent) discloses a brain tumor segmentation method based on uncertainty estimation. It processes four types of brain tumor images (T1, T1ce, T2, and FLAIR) using four encoders respectively. After feature fusion via a decoder, combined with Monte Carlo simulation and image processing modules, it outputs a predicted segmentation image containing the complete tumor, tumor core, and enhanced tumor region, as well as four types of uncertainty images: random, cognitive, entropy, and mutual information. While this patent represents a breakthrough in uncertainty estimation, it still has limitations in the targeted design of feature extraction and modality fusion.

[0010] In capturing the anisotropic features of tumors in three-dimensional space, this patent adopts a standard multi-encoder structure, with each encoder corresponding to one MRI modality for downsampling. Although its fusion module considers multimodal information, the feature extraction process is not specifically designed for the anisotropy of tumor shape. Since brain tumors often exhibit significantly different morphological features in the axial, coronal, and sagittal planes, this feature extraction method, which does not distinguish directionality, makes it difficult to accurately decouple the contextual information of different anatomical planes. This results in poor modeling of the complex geometry and boundaries of tumors, especially in areas with irregular tumor shapes or blurred boundaries, where segmentation bias is prone to occur.

[0011] Regarding multimodal feature fusion, this patent employs a "Multimodal Teacher-Student Fusion (MTSF)" strategy, which fixes the four MRI modalities into two groups: "teacher modality" and "student modality." This grouping and role allocation are fixed and lack the flexibility to dynamically adjust based on the specific situation of the tumor region. Different tumor regions exhibit varying information richness in different modalities. For example, enhanced tumor regions are more prominent in the T1ce modality, while edema regions are more easily identified in the FLAIR and T2 modalities. The fixed grouping fusion method cannot adaptively adjust the contribution of each independent modality in different regions, which may lead to the over-introduction of noise or redundant information, or the underutilization of important modal features, affecting the robustness of segmentation.

[0012] Therefore, in view of the shortcomings of existing technologies in anisotropic feature capture and dynamic modality fusion, there is an urgent need for a brain tumor segmentation method that can accurately extract multi-planar directional features and adaptively fuse multi-modal information to improve the segmentation accuracy of complex tumor structures. Summary of the Invention

[0013] The purpose of this invention is to address the problems existing in the prior art and provide a brain tumor segmentation method based on triaxial context coding.

[0014] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0015] A brain tumor segmentation method based on triaxial context coding is proposed. The FLAIR, T2, T1 and T1ce images of the same glioma are input into the trained image segmentation model, which outputs a predicted segmented image. The predicted segmented image is the glioma MRI image with three segmentation regions obtained by prediction. The three segmentation regions are the non-enhanced necrosis region, the edema region and the enhanced tumor region.

[0016] Image segmentation models include encoders and decoders;

[0017] The encoder includes a TCE (Triaxial Context Encoding) module, which dynamically weights and fuses multi-plane features by adaptively learning the relative importance of the axial, coronal, and sagittal branches.

[0018] The TCE module of this invention consists of three directional branches, which can effectively capture anisotropic contextual information in three-dimensional medical volume data. This triaxial design enables the network to decouple directional information and extract long-range contextual features from each major anatomical view. This directional encoding is particularly beneficial in brain tumor segmentation because the tumor shape exhibits strong anisotropy in different views.

[0019] As a preferred technical solution:

[0020] The TCE module of the brain tumor segmentation method described above, based on triaxial context encoding, operates as follows:

[0021] (a) Input features The data is fed into three parallel processing branches to obtain its axial plane branches. Coronal branches sagittal branch ;

[0022] (b) on , , First, perform 3D convolution, then sequentially use convolution kernels with asymmetric kernels and unique dilation rates to obtain... , , The purpose of this step is to specifically encode spatial dependencies on different anatomical planes.

[0023] (c) Calculate the feature map The formula is: In the formula, Representative instance normalization, Represents the LeakyReLU activation function. It represents Dropout;

[0024] (d) Calculate the weights The formula is as follows:

[0025] , specifically, , , ;

[0026] ;

[0027] ;

[0028] In the formula, , , These correspond to attention weights. The axial, coronal, and sagittal components, Represents the ReLU activation function. Represents the softmax function. and These are the learnable parameters of two fully connected layers. The range of values ​​for is all real numbers. The range of values ​​for is all real numbers. The global pooled feature vector representing the average branch feature. This represents a global average pooling operation;

[0029] The learned weights reflect the context-specific relevance of each directional branch, enabling the network to focus more on the most informative perspectives based on the anatomy of the input region.

[0030] (e) Calculate output characteristics The formula is as follows:

[0031] ;

[0032] ;

[0033] In the formula, This represents a 1×1×1 convolution, and its purpose is to enhance the non-linear expressive power of the model. The process of obtaining fused features aims to use a dynamically weighted fusion module (DyFusion) to adaptively learn the relative importance of each branch, automatically balancing multi-plane features through learnable attention weights, rather than using simple summation or concatenation.

[0034] Compared to traditional isotropic or symmetric convolutional modules, the TCE module of this invention has three main advantages: First, explicit directional encoding, which explicitly encodes the directional context consistent with the anatomical plane, is more effective when modeling complex tumor geometries and boundaries. Second, the dynamic fusion scheme can adaptively recalibrate the contribution of branches, enhancing robustness to modality-specific noise or missing information. Third, by integrating lightweight depthwise separable convolutions, the module significantly reduces computational overhead while maintaining strong feature representation capabilities.

[0035] In the brain tumor segmentation method based on triaxial context encoding as described above, in step (b), The corresponding convolution kernel is (1×3×3), and the expansion rate is (1,2,2); The corresponding convolution kernel is (3×1×3), and the expansion rate is (2,1,2); The corresponding convolution kernel is (3×3×1), and the expansion rate is (2,2,1).

[0036] The brain tumor segmentation method based on triaxial context coding described above employs a four-way parallel, weight-sharing structure to process FLAIR, T2, T1, and T1ce images respectively. The four branches of the encoder have the same workflow to ensure consistency in feature extraction from different modalities.

[0037] As described above, in a brain tumor segmentation method based on triaxial context coding, when any branch of the encoder is working, the image sequentially passes through the following modules: first downsampling module (input size [8, 128³], output size [16, 64³]), first 3×3×3 convolution module, second downsampling module (input size [16, 64³], output size [32, 32³]), second 3×3×3 convolution module, third downsampling module (input size [32, 32³], output size [64, 16³]), third 3×3×3 convolution module, fourth downsampling module (input size [64, 16³], output size [128, 8³]), fourth 3×3×3 convolution module, fifth downsampling module (input size [128, 8³], output size [256, 4³]), fifth 3×3×3 convolution module, and sixth downsampling module (input size [256, 4³]). 4³]);

[0038] Each downsampling module consists of a TCE module and a multi-scale dilated convolution module with residual connections. The output of the TCE module in the same downsampling module is the input of the multi-scale dilated convolution module with residual connections.

[0039] The stride of each 3×3×3 convolutional module is 2. In this invention, the stride of other convolutional operations without a defined stride is 1 by default.

[0040] The brain tumor segmentation method based on triaxial context coding described above includes a decoder comprising a GMF (Gated Modality Fusion) module. The workflow of the GMF module is as follows:

[0041] (i) Input feature tensor Divided into A mode-specific tensor , The size is , The size is , =1,2,…, ;

[0042] (ii) Calculate the initial modal characteristics The formula is: In the formula, Represents a 3×3×3 convolution. Representative instance normalization, Represents the LeakyReLU activation function. This represents Dropout; this step is a shared convolutional transform, which ensures consistent transformation across modalities while reducing parameter overhead.

[0043] (iii) Calculate the gating graph The formula is: In the formula, Represents the sigmoid activation function. This represents element-wise multiplication; the purpose of this step is to model the importance of each modality and adjust... The response of each voxel in the signal suppresses signals that are uncertain or have little information.

[0044] (iv) Calculate the fused modal features The formula is: In the formula, This indicates element-wise addition; the purpose of this step is to combine the gated features with the original input through residual addition in order to preserve low-level modality-specific cues.

[0045] (v) Calculate the feature tensor of the aggregation , In the formula, Representative feature aggregation;

[0046] (vi) Calculate output features The formula is: In the formula, This represents a 3×3×3 dilated convolution with a dilation rate r=2. This represents a 3×3×3 dilated convolution with a dilation rate of r=4; the purpose of this step is to further capture multi-scale contextual information and optimize cross-modal interactions.

[0047] The core idea of ​​the GMF module is to learn modality-specific attention gates to adaptively adjust the contribution of each modality, suppress noise or redundant information, and enhance discriminative representations.

[0048] Compared to simple modality concatenation or averaging, the GMF module of this invention offers several advantages. First, the learned gating mechanism dynamically controls the contribution of each modality, enabling the network to suppress redundant features and highlight information-rich features. This is particularly beneficial when certain modalities are impaired, missing, or weakly associated with specific regions. Second, the residual fusion strategy preserves the original modality-specific information while achieving cross-modal interaction, improving the robustness and expressiveness of the fused representation. Therefore, GMF enhances the discriminativeness and reliability of multimodal feature integration in medical image segmentation.

[0049] The brain tumor segmentation method based on triaxial context coding described above includes a decoder comprising six dilation modules, five feature refinement modules, six 1×1×1 convolutional modules, and five upsampling modules.

[0050] Each expansion module consists of a splicing module and a GMF module. The output of the splicing module in the same expansion module is the input of the GMF module.

[0051] The feature refinement module consists of a 3×3×3 convolution module, an instance normalization module, and an LReLU activation function module connected in sequence. The input of the instance normalization module is the output of the 3×3×3 convolution module, and the output of the instance normalization module is the input of the LReLU activation function module. The deepest features from all modalities are concatenated and then input into the GMF module of the expansion module for fusion, so as to achieve adaptive selection of specific features of relevant modalities.

[0052] The brain tumor segmentation method based on triaxial context coding, as described above, has the following decoder operation process:

[0053] The outputs of all the sixth downsampling modules in the encoder are simultaneously input into the first expansion module, and the outputs of the first expansion module are respectively input into the first feature refinement module and the first 1×1×1 convolution module;

[0054] The outputs of all the fifth downsampling modules in the encoder, together with the output of the first feature refinement module, are input into the second expansion module. The outputs of the second expansion module are input into the second feature refinement module and the second 1×1×1 convolution module, respectively.

[0055] The outputs of all the fourth downsampling modules in the encoder, together with the output of the second feature refinement module, are input into the third expansion module. The output of the third expansion module is input into the third feature refinement module and the third 1×1×1 convolution module, respectively.

[0056] The outputs of all the third downsampling modules in the encoder, together with the outputs of the third feature refinement module, are input into the fourth expansion module. The outputs of the fourth expansion module are input into the fourth feature refinement module and the fourth 1×1×1 convolution module, respectively.

[0057] The outputs of all the second downsampling modules in the encoder, together with the output of the fourth feature refinement module, are input into the fifth expansion module. The output of the fifth expansion module is input into the fifth feature refinement module and the fifth 1×1×1 convolution module, respectively.

[0058] The outputs of all the first downsampling modules in the encoder, together with the output of the fifth feature refining module, are input into the sixth expansion module. The output of the sixth expansion module is input into the sixth 1×1×1 convolution module.

[0059] The output of the first 1×1×1 convolutional module is added element-wise to the output of the second 1×1×1 convolutional module after passing through the first upsampling module. The result is then added element-wise to the output of the third 1×1×1 convolutional module after passing through the second upsampling module. The result is then added element-wise to the output of the fourth 1×1×1 convolutional module after passing through the third upsampling module. The result is then added element-wise to the output of the fifth 1×1×1 convolutional module after passing through the fourth upsampling module. Finally, the result is added element-wise to the output of the sixth 1×1×1 convolutional module after passing through the fifth upsampling module. This process yields the predicted segmentation image.

[0060] The decoder employs a progressive upsampling strategy. At each decoding level, features from all modalities are concatenated and then enhanced through a GMF module to strengthen cross-modal interaction and suppress redundant features, ensuring that high-level semantics and low-level details are preserved throughout the decoding process. Simultaneously, a deep supervision mechanism is used to generate intermediate segmentation outputs at multiple stages and perform progressive upsampling and aggregation, ensuring consistency from fine to coarse and improving gradient flow during training. Finally, after 1×1×1 convolution and a sigmoid activation function, a full-resolution segmentation map is obtained.

[0061] The loss function of the brain tumor segmentation method based on triaxial context encoding, as described above, is expressed as follows:

[0062] ;

[0063] ;

[0064] ;

[0065] ;

[0066] In the formula, Represents the loss function. Represents the total number of categories in the segmentation. c=1 represents a non-enhanced necrotic area, c=2 represents an edematous area, and c=3 represents an enhanced tumor area. The weight representing category c, This represents the predicted probability of category c. The true label representing category c, Represents the dot product of vectors. Represents the L1 norm (i.e., the sum of voxels). , , and These are used respectively to ensure numerical stability and to control the strength of entropy regularization. This represents the spatial average over the prediction domain Omega. Represents the binary entropy function. This represents the volume percentage of the k-th class in the actual labels. This represents the volume percentage of category c in the actual labels. The true label representing class c. The true label representing the k-th class. .

[0067] The predicted probability is the direct output of the neural network. The network output has three channels, each corresponding to the predicted probability of a region, which are called... , , .in, A 3D probability map where the value of each voxel represents the probability that the point belongs to a "non-enhanced necrosis region". A 3D probability map where the value of each voxel represents the probability that the point belongs to the "edema region". A 3D probability map where the value of each voxel represents the probability that the point belongs to an "enhancing tumor region". The ground truth labels are manually drawn "correct answers" by humans and are the learning target of the model. For these three regions, there are three corresponding ground truth labels, referred to as... , , .in, A 3D binary image where the voxel value of the "non-enhanced necrotic region" is 1, and the rest are 0. A 3D binary image where only the voxel value of the edema region is 1, and the rest are 0. A 3D binary image where the voxel value of the enhanced tumor region is 1 and the rest is 0. The loss function of the present invention differs from the prior art. It combines class-specific weighting and prediction uncertainty regularization to address two key challenges in medical image segmentation: (1) severe class imbalance; (2) model overconfidence in blurred regions.

[0068] Unlike traditional methods that rely on fixed class weights, the proposed method... The class weight w=[w1,...,w] will be dynamically calculated for each batch based on the actual label volume. C This adaptive weighting strategy reflects the actual class distribution in each mini-batch of data, enabling the model to automatically focus on underrepresented tumor subregions while avoiding overcompensation for dominant regions. In this way, It not only alleviates the problem of extreme class imbalance, but also enhances the generalization ability of the model by preventing it from relying on dataset-specific prior assumptions.

[0069] Prediction uncertainty regularization refers to introducing a regularization term into the loss function specifically to penalize the model's prediction uncertainty. Given a loss function... In this context, the regularization term is its second part: the weighted average binary cross-entropy loss (BCE), which is expressed by the formula: When the model's predicted probability When H is close to 0 or 1 (i.e., the model is very confident in its judgment and it is correct), The value of ) is small, and the penalty is also small. However, when the predicted probability... When the value approaches 0.5 (i.e., the model is highly uncertain about the judgment, hesitating between "yes" and "no"), H( The value of ) will reach its maximum, resulting in the maximum penalty. This can effectively alleviate the model's overconfidence in blurred regions.

[0070] Most existing technologies can only generate deterministic output results, ignoring the inherent uncertainty of medical data, making it difficult for doctors to judge the reliability of the results. This invention addresses this issue. The loss function, by regularizing prediction uncertainty, can address the class imbalance problem by combining class-specific weighting, and at the same time, it reflects the confidence level of the prediction results through prediction uncertainty regularization, thus making up for the shortcomings of existing technologies.

[0071] Beneficial effects:

[0072] (1) The TCE module designed in this invention uses three directional branches—axial, coronal, and sagittal—combined with asymmetric convolution kernels, to enable the network to adaptively focus on the view with the richest information according to the tumor morphology, which significantly improves the modeling accuracy of complex tumor geometry and fuzzy boundaries, and solves the segmentation deviation problem caused by the failure to distinguish directionality in traditional methods.

[0073] (2) The GMF module in the decoder of this invention is a gating map for learning each MRI modality. It generates attention weights between 0 and 1 through the sigmoid activation function, and adjusts the modal feature response element by element. It amplifies the contribution of T1ce modality in the enhanced tumor region and enhances the weights of FLAIR and T2 modality in the edema region. At the same time, it suppresses the interference of noise or low-information modality. This dynamic adjustment mechanism replaces the fixed group fusion strategy, realizes cross-regional and cross-modal adaptive fusion, avoids the introduction of redundant information or the omission of important features, and greatly improves the discriminativeness and reliability of multimodal data integration.

[0074] (3) The loss function LURW designed in this invention alleviates the class imbalance problem by dynamically calculating class weights and introduces prediction uncertainty regularization, which improves the model's generalization ability to different datasets while improving segmentation accuracy. Attached Figure Description

[0075] Figure 1 shows (a) a schematic diagram of the encoder and decoder of the image segmentation model of the present invention, (b) a schematic diagram of the TCE module, (c) a schematic diagram of the GMF module, and (d) legends of (a) to (c); where "*" is the signal connection identifier between modules.

[0076] Figure 2 is a schematic diagram of the image segmentation model in Comparative Example 1;

[0077] Figure 3 is a schematic diagram of the image segmentation model in Comparative Example 2;

[0078] Figure 4 is a schematic diagram of the image segmentation model in Comparative Example 3;

[0079] Figure 5 is a visualization comparing the segmentation performance of the modules in Example 1 and Comparative Examples 1-4. The blue circles mark the areas where the predicted segmentation images of different models differ from the ground truth images, and the red dashed boxes mark the areas where the brain tumors of particular interest are located. Detailed Implementation

[0080] The present invention will be further described below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.

[0081] Example 1

[0082] A brain tumor segmentation method based on triaxial context coding, the specific steps of which are as follows:

[0083] S1. Establish an image segmentation model;

[0084] As shown in Figures 1(a) and (d), the image segmentation model includes an encoder and a decoder;

[0085] The encoder includes a TCE module, as shown in Figures 1(b) and (d). The workflow of the TCE module is as follows:

[0086] (a) Input features The data is fed into three parallel processing branches to obtain its axial plane branches. Coronal branches Sagittal branch ;

[0087] (b) on , , First, perform 3D convolution, then sequentially use convolution kernels with asymmetric kernels and unique dilation rates to obtain... , , ; The corresponding convolution kernel is (1×3×3), and the expansion rate is (1,2,2); The corresponding convolution kernel is (3×1×3), and the expansion rate is (2,1,2); The corresponding convolution kernel is (3×3×1), and the expansion rate is (2,2,1);

[0088] (c) Calculate the feature map The formula is: In the formula, Representative instance normalization, Represents the LeakyReLU activation function. It stands for Dropout; specifically, , , ;

[0089] (d) Calculate the weights The formula is as follows:

[0090] , specifically, , , ;

[0091] ;

[0092] ;

[0093] In the formula, , , These correspond to attention weights. The axial, coronal, and sagittal components, Represents the ReLU activation function. Represents the softmax function. and These are the learnable parameters of two fully connected layers. The range of values ​​for is all real numbers. The range of values ​​for is all real numbers. The global pooled feature vector representing the average branch feature. This represents a global average pooling operation;

[0094] (e) Calculate output characteristics The formula is as follows:

[0095] ;

[0096] ;

[0097] In the formula, Represents a 1×1×1 convolution. Represents the characteristics of integration;

[0098] As shown in Figures 1(a) and 1(d), the encoder uses a four-way parallel, weight-sharing structure to process FLAIR, T2, T1, and T1ce images respectively. The four branches of the encoder have the same workflow. When any branch of the encoder is working, the image sequentially passes through the first downsampling module (input size [8, 128³], output size [16, 64³]), the first 3×3×3 convolution module, the second downsampling module (input size [16, 64³], output size [32, 32³]), the second 3×3×3 convolution module, the third downsampling module (input size [32, 32³], output size [64, 16³]), the third 3×3×3 convolution module, the fourth downsampling module (input size [64, 16³], output size [128, 8³]), the fourth 3×3×3 convolution module, and the fifth downsampling module (input size [128, 8³]). [8³], with an output size of [256, 4³]), the fifth 3×3×3 convolution module, and the sixth downsampling module (with an input size of [256, 4³]).

[0099] Each downsampling module consists of a TCE module and a multi-scale dilated convolutional module with residual connections. The output of the TCE module in the same downsampling module is the input of the multi-scale dilated convolutional module with residual connections. The stride of each 3×3×3 convolutional module is 2.

[0100] The decoder consists of six dilation modules, five feature refinement modules, six 1×1×1 convolutional modules, and five upsampling modules. Each dilation module comprises a concatenation module and a GMF module, with the output of the concatenation module serving as the input to the GMF module. The feature refinement module consists of a 3×3×3 convolutional module, an instance normalization module, and an LReLU activation function module connected in sequence. The input of the instance normalization module is the output of the 3×3×3 convolutional module, and the output of the instance normalization module serves as the input to the LReLU activation function module.

[0101] As shown in Figures 1(c) and (d), the workflow of the GMF module is as follows:

[0102] (i) Input feature tensor Divided into 4 modality-specific tensors , The size is , The size is , =1,2,…, ;

[0103] (ii) Calculate the initial modal characteristics The formula is: In the formula, Represents a 3×3×3 convolution. Representative instance normalization, Represents the LeakyReLU activation function. It represents Dropout;

[0104] (iii) Calculate the gating graph The formula is: In the formula, Represents the sigmoid activation function. This indicates element-wise multiplication;

[0105] (iv) Calculate the fused modal features The formula is: In the formula, This indicates element-wise addition;

[0106] (v) Calculate the feature tensor of the aggregation , In the formula, Representative feature aggregation;

[0107] (vi) Calculate output features The formula is: In the formula, This represents a 3×3×3 dilated convolution with a dilation rate r=2. This represents a 3×3×3 dilated convolution with a dilation rate of r=4;

[0108] As shown in Figures 1(a) and (d), the decoder's operation is as follows: The outputs of all sixth downsampling modules in the encoder are simultaneously input into the first expansion module. The outputs of the first expansion module are then input into the first feature refinement module and the first 1×1×1 convolution module, respectively. The outputs of all fifth downsampling modules in the encoder, together with the outputs of the first feature refinement module, are input into the second expansion module. The outputs of the second expansion module are then input into the second feature refinement module and the second 1×1×1 convolution module, respectively. The outputs of all fourth downsampling modules in the encoder, together with the outputs of the second feature refinement module, are input into the third expansion module. The outputs of the third expansion module are then input into the third feature refinement module and the third 1×1×1 convolution module, respectively. The outputs of all third downsampling modules in the encoder, together with the outputs of the third feature refinement module, are input into the fourth expansion module. The outputs of the fourth expansion module are then input into the fourth feature refinement module and the fourth 1×1×1 convolution module, respectively. The outputs of all second downsampling modules in the encoder... The output of the block, together with the output of the fourth feature refinement module, is input into the fifth expansion module. The output of the fifth expansion module is input into the fifth feature refinement module and the fifth 1×1×1 convolution module, respectively. The outputs of all the first downsampling modules in the encoder, together with the output of the fifth feature refinement module, are input into the sixth expansion module. The output of the sixth expansion module is input into the sixth 1×1×1 convolution module. The output of the first 1×1×1 convolution module is passed through the first upsampling module and then added element-wise with the output of the second 1×1×1 convolution module. The result is passed through the second upsampling module and then added element-wise with the output of the third 1×1×1 convolution module. The result is passed through the third upsampling module and then added element-wise with the output of the fourth 1×1×1 convolution module. The result is passed through the fourth upsampling module and then added element-wise with the output of the fifth 1×1×1 convolution module. The result is passed through the fifth upsampling module and then added element-wise with the output of the sixth 1×1×1 convolution module, thus obtaining the predicted segmentation image.

[0109] S2. Determine the loss function for the image segmentation model;

[0110] The expression for the loss function of the image segmentation model is as follows:

[0111] ;

[0112] ;

[0113] ;

[0114] ;

[0115] In the formula, Represents the loss function. Represents the total number of categories in the segmentation. c=1 represents a non-enhanced necrotic area, c=2 represents an edematous area, and c=3 represents an enhanced tumor area. The weight representing category c, This represents the predicted probability of category c. The true label representing category c, Represents the dot product of vectors. Represents the L1 norm. , , and These are used respectively to ensure numerical stability and to control the strength of entropy regularization. This represents the spatial average over the prediction domain Omega. Represents the binary entropy function. This represents the volume percentage of the k-th class in the actual labels. This represents the volume percentage of category c in the actual labels. The true label representing class c. The true label representing the k-th class. ;

[0116] S3. Train the image segmentation model;

[0117] S3.1 The dataset used is the BraTS2019 benchmark set, which contains FLAIR, T2, T1, and T1ce four-modal MRI images and expert-annotated 3D binary ground truth labels. (Non-enhanced necrotic areas) (Edema area) (Enhanced tumor region), the target region voxel value in the label is 1, and the rest are 0;

[0118] S3.2 Unified preprocessing of images in the BraTS2019 dataset: sequentially perform skull dissection, cross-modal registration to a common anatomical template, resampling to an isotropic resolution of 1 cubic millimeter, and finally complete intensity normalization.

[0119] S3.3. The preprocessed data is divided into training set and validation set in an 8:2 ratio;

[0120] S3.4 Training Parameter Settings: 3D blocks (128×128×128) are used as input units; Adam is selected as the optimizer, and the initial learning rate is set to 5×10. -4 Batch size is 1; the learning rate decays to half of its current value every 10 training epochs; an early stopping mechanism is set, and the training process is terminated if the loss on the validation set does not improve for 50 consecutive epochs.

[0121] S3.5. Input the four modal MRI images from the training set into the four parallel branches of the encoder (the four branches share weights); , , As the target output of the model, the predicted probabilities for the corresponding three regions are obtained through model forward propagation. , , ; Calculate the loss value between the predicted probability and the true label based on the loss function; Based on this loss value, update the learnable parameters of the model through backpropagation using the Adam optimizer;

[0122] S3.6. Test the trained image segmentation model using a test set; use the dice similarity coefficient (DSC) and Hausdorff distance (HD) to characterize the segmentation accuracy of the trained image segmentation model, and calculate the following formulas:

[0123] ;

[0124] In the formula, and These represent the predicted region (i.e., the whole tumor region WT, the tumor core region TC, or the enhancing tumor region ET; the whole tumor region = edema region + enhancing tumor region + non-enhancing necrosis region, and the tumor core region = enhancing tumor region + non-enhancing necrosis region) and the true region, respectively. This indicates the size of the intersection between the predicted region and the actual region. and These represent the total size of the predicted region and the actual region, respectively;

[0125] ;

[0126] In the formula, for The set of boundary points of the corresponding region for The set of boundary points of the corresponding region; Represents Euclidean distance; This represents the maximum value of the minimum distance from each point on the predicted boundary to the true boundary. This represents the maximum value of the minimum distance from each point on the true boundary to the predicted boundary;

[0127] S4. Output the predicted segmentation image;

[0128] The FLAIR, T2, T1, and T1ce images of the same glioma are input into the trained image segmentation model, which outputs a predicted segmented image. The predicted segmented image is the glioma MRI image with three segmented regions: a non-enhanced necrosis region, an edema region, and an enhanced tumor region.

[0129] Comparative Example 1

[0130] A brain tumor segmentation method differs from Example 1 in that: the image segmentation model, as shown in Figure 2, does not include a TCE module in the encoder and a GMF module in the decoder, unlike the image segmentation model in Example 1; the loss function of the image segmentation model... as follows:

[0131] ;

[0132] In the formula This represents the predicted probability of category c. The true label representing category c,

[0133] Comparative Example 2

[0134] A brain tumor segmentation method differs from Example 1 in that: the image segmentation model is shown in Figure 3, and compared with the image segmentation model of Example 1, the decoder does not contain a GMF module; the loss function of the image segmentation model is the same as that of Example 1.

[0135] Comparative Example 3

[0136] A brain tumor segmentation method differs from Example 1 in that: the image segmentation model is shown in Figure 4, and compared with the image segmentation model of Example 1, the encoder does not include a TCE module; the loss function of the image segmentation model is the same as that of Example 1.

[0137] Comparative Example 4

[0138] A brain tumor segmentation method differs from Example 1 only in that the loss function of the image segmentation model is the same as that in Example 1.

[0139] The test results of Examples 1 and 4 are shown in Table 1 (bold indicates the best results; * indicates statistically significant improvement compared to the baseline method (i.e., Comparative Example 1) using the Wilcoxon signed-rank test (p<0.05):

[0140] Table 1

[0141]

[0142] As shown in Table 1, Comparative Example 1, serving as the baseline model, has the lowest average DSC (Discretionary Segmentation) of 83.4% among all tested models, with an average HD (High Definition) of 4.2mm, indicating the worst overall segmentation performance. When only the TCE module is added to Comparative Example 1 (i.e., Comparative Example 2), the average DSC improves to 84.1%, and the DSC of TC increases from 85.1% to 86.0%, demonstrating the effectiveness of the TCE module in capturing anisotropic spatial dependencies. When only the GMF module is added (i.e., Comparative Example 3), the average DSC further improves to 84.3%, the DSC of TC reaches 86.7%, and the DSC of WT remains at a relatively high level of 86.9%. The HD of TC decreases to 4.1mm, showcasing the advantage of the GMF module in incorporating multimodal cues. When both TCE and GMF modules are added simultaneously while retaining the existing loss function (i.e., Comparative Example 4), the average DSC improves to 84.5%, and the HD of TC... The DSC reached 87.0%, and the ET's DSC also rose to 79.7%, demonstrating the synergistic effect of the two modules. In Example 1, after introducing the loss function of this invention on the basis of TCE+GMF, the average DSC was further improved to 84.8%, and the average HD was reduced to 4.0mm. Among them, the DSC of TC (87.7%) and the DSC of ET (80.2%) were the highest, and the HD of TC (3.8mm) and the HD of ET (2.8mm) were the lowest. Moreover, multiple indicators were statistically verified (p<0.05) to be significantly better than Comparative Example 1, which fully demonstrates that the sequential addition of the three key components in the framework can improve the segmentation performance, and the effect is optimal when the three work synergistically.

[0143] To further verify the above conclusions, we conducted a visualization analysis of the predicted segmentation images of two brain tumor cases based on FLAIR images from the BraTS2019 dataset (as shown in Figure 5). In the first case (row a in Figure 5), Comparative Example 1, due to the absence of a TCE module in the encoder and a GMF module in the decoder, showed blurred boundaries between the edema region and the enhanced tumor region within the blue circle, and the predicted necrotic region also exhibited "undersegmentation," showing a significant difference from the ground truth image. Comparative Example 2, with only the TCE module added, achieved more accurate spatial dependence capture in the axial and coronal planes, and the contour of the edema region within the blue circle better matched the ground truth. However, due to the lack of a GMF module, multimodal fusion was insufficient, and the enhanced tumor region still showed slight "oversegmentation." Comparative Example 3, with only the GMF module added, multimodal cue fusion helped enhance the tumor region. For tumor region prediction, the lack of TCE in the encoder resulted in weak spatial feature extraction, and the edges of the edema region within the blue circle still had "rough edges". In Comparative Example 4, TCE and GMF modules were added simultaneously, and the spatial dependence and multimodal fusion worked together to reduce the difference region within the blue circle. However, due to the lack of optimization of the loss function, the segmentation accuracy was not optimal. In Example 1, thanks to the accurate spatial encoding of the TCE module and the efficient multimodal fusion of the GMF module, combined with the self-developed loss function, the difference region within the blue circle was basically eliminated. The necrotic, enhanced tumor, and edema regions were highly consistent with the ground truth map, and the segmentation integrity and accuracy were optimal. In the second case (row b in Figure 5), Comparative Example 1, lacking both TCE and GMF modules, resulted in fragmented segmentation of the enhanced tumor region within the blue circle, with the edema region and background boundary being confused, leading to prediction results deviating from the true value. Comparative Example 2, with the addition of the TCE module, enhanced spatial anisotropy feature capture improved, and the edema region within the blue circle became closer to the true value. However, due to the lack of GMF, multimodal information fusion was insufficient, resulting in "misclassification" of the enhanced tumor region. Comparative Example 3, with the addition of the GMF module, multimodal fusion optimized the prediction of the enhanced tumor region. However, the absence of TCE in the encoder caused the loss of spatial details, resulting in incomplete segmentation of small lesions within the blue circle. Comparative Example 4, despite having both TCE and GMF modules, still showed errors in the segmentation results within the blue circle at subtle boundaries (such as the junction between the enhanced tumor and edema) due to the lack of improvement in the loss function. Example 1, relying on the accurate spatial feature extraction of the TCE module and the deep multimodal fusion of the GMF module, combined with a customized loss function, clearly segmented each region (necrosis, enhanced tumor, and edema) within the blue circle, with almost no difference from the true value map, completely restoring the tumor structure and demonstrating optimal segmentation performance. By comparing the two sets of cases, a and b, it can be clearly seen that the TCE module enhances spatial features, the GMF module optimizes multimodal fusion, and the self-developed loss function improves segmentation accuracy. The synergy of these three components enables Example 1 to achieve more accurate and more true-value-fit brain tumor segmentation in cases with different tumor morphologies and locations, fully verifying the effectiveness and synergistic advantages of the framework components.

Claims

1. A brain tumor segmentation method based on triaxial context coding, characterized in that, FLAIR, T2, T1, and T1ce images of the same glioma are input into the trained image segmentation model, which outputs a predicted segmented image. The predicted segmented image is the glioma MRI image with three segmented regions: a non-enhanced necrosis region, an edema region, and an enhanced tumor region. The image segmentation model includes an encoder and a decoder. The encoder includes a TCE module, and the workflow of the TCE module is as follows: (a) Input features The data is fed into three parallel processing branches to obtain its axial plane branches. Coronal branches sagittal branch (b) for 、 、 First, perform 3D convolution, then sequentially use convolution kernels with asymmetric kernels and unique dilation rates to obtain... 、 、 (c) Calculate the feature map The formula is: In the formula, Representative instance normalization, Represents the LeakyReLU activation function. (d) Calculate weights for Dropout The formula is as follows: ； ； In the formula, 、 、 These correspond to attention weights. The axial, coronal, and sagittal components, Represents the ReLU activation function. Represents the softmax function. and These are the learnable parameters of two fully connected layers. The range of values ​​for is all real numbers. The range of values ​​for is all real numbers. (e) Calculate the output features, representing the global average pooling operation. The formula is as follows: ； In the formula, Represents a 1×1×1 convolution. It represents the characteristics of integration.

2. The brain tumor segmentation method based on triaxial context coding according to claim 1, characterized in that, In step (b), The corresponding convolution kernel is (1×3×3), and the expansion rate is (1,2,2); The corresponding convolution kernel is (3×1×3), and the expansion rate is (2,1,2); The corresponding convolution kernel is (3×3×1), and the expansion rate is (2,2,1).

3. The brain tumor segmentation method based on triaxial context coding according to claim 1, characterized in that, The encoder uses a four-way parallel, weight-sharing structure to process FLAIR, T2, T1 and T1ce images respectively, and the four branches of the encoder have the same workflow.

4. The brain tumor segmentation method based on triaxial context coding according to claim 3, characterized in that, When any branch of the encoder is working, the image sequentially passes through the first downsampling module, the first 3×3×3 convolution module, the second downsampling module, the second 3×3×3 convolution module, the third downsampling module, the third 3×3×3 convolution module, the fourth downsampling module, the fourth 3×3×3 convolution module, the fifth downsampling module, the fifth 3×3×3 convolution module, and the sixth downsampling module. Each downsampling module consists of a TCE module and a multi-scale dilated convolution module with residual connections. The output of the TCE module in the same downsampling module is the input of the multi-scale dilated convolution module with residual connections. The stride of each 3×3×3 convolution module is 2.

5. A brain tumor segmentation method based on triaxial context coding according to claim 4, characterized in that, The decoder includes a GMF module, and the workflow of the GMF module is as follows: (i) Convert the input feature tensor... Divided into A mode-specific tensor ， The size is ， The size is ， =1,2,…, (ii) Calculate the initial modal characteristics The formula is: In the formula, Represents a 3×3×3 convolution. Representative instance normalization, Represents the LeakyReLU activation function. It represents Dropout; (iii) Calculate the gating graph The formula is: In the formula, Represents the sigmoid activation function. (iv) Calculate the fused modal features. The formula is: In the formula, (v) Calculate the feature tensor of the aggregation. ， In the formula, Representative feature aggregation; (vi) Calculate output features The formula is: In the formula, This represents a 3×3×3 dilated convolution with a dilation rate r=2. This represents a 3×3×3 dilated convolution with a dilation rate of r=4.

6. A brain tumor segmentation method based on triaxial context coding according to claim 5, characterized in that, The decoder includes six dilation modules, five feature refinement modules, six 1×1×1 convolutional modules, and five upsampling modules; each dilation module consists of a concatenation module and a GMF module, and the output of the concatenation module in the same dilation module is the input of the GMF module; The feature refinement module consists of a 3×3×3 convolution module, an instance normalization module, and an LReLU activation function module connected in sequence. The input of the instance normalization module is the output of the 3×3×3 convolution module, and the output of the instance normalization module is the input of the LReLU activation function module.

7. The brain tumor segmentation method based on triaxial context coding according to claim 1, characterized in that, The expression for the loss function of the image segmentation model is as follows: ； ； ； In the formula, Represents the loss function. Represents the total number of categories in the segmentation. c=1 represents a non-enhanced necrotic area, c=2 represents an edematous area, and c=3 represents an enhanced tumor area. The weight representing category c, This represents the predicted probability of category c. The true label representing category c, Represents the dot product of vectors. Represents the L1 norm. ， ， This represents the spatial average over the prediction domain Omega. Represents the binary entropy function. This represents the volume percentage of the k-th class in the actual labels. This represents the volume percentage of category c in the actual labels. The true label representing the k-th class. 。

Images