A brain glioma sub-region segmentation method and system based on multi-modal collaborative perception

Abstract

The application belongs to the technical field of image segmentation, and specifically discloses a brain glioma sub-region segmentation method and system based on multi-modal collaborative perception. The method builds a brain glioma sub-region segmentation model based on a multi-modal collaborative segmentation network; a modal collaborative perception module is designed in the model to perform modal contribution evaluation and spatial response modulation on the shallow layer features output by four three-dimensional convolution shallow layer feature extraction units, and to perform adaptive fusion. In addition, the application designs an HCR Block in the backbone network, extracts context information under different receptive fields, combines the modal weight prior information output by the modal collaborative perception module, and enhances the features. At the same time, the application uses a guided region screening module in the decoder to screen the region of the shallow layer jump connection features. Finally, the segmentation result output by the decoding head is processed through a region consistency correction strategy to obtain the brain glioma sub-region segmentation result.

Description

Technical Field

[0001] This invention belongs to the field of image segmentation technology, specifically relating to a method and system for segmenting glioma subregions based on multimodal collaborative perception. Background Technology

[0002] Adult gliomas are among the most common primary tumors of the central nervous system. Different subregions of gliomas have varying clinical significance in surgical planning, radiotherapy target delineation, efficacy assessment, and prognostic analysis. Adult gliomas typically include at least a necrotic / non-enhancing tumor core, a peritumoral edema area, and an enhancing tumor area, and can further develop into a tumor core and a whole tumor area. Because different subregions differ significantly in morphology, location, size, and signal intensity, and often exhibit blurred, infiltrative, and irregular boundaries, relying solely on manual layer-by-layer delineation is not only time-consuming and labor-intensive but also heavily influenced by the physician's experience, making it difficult to meet the needs of large-scale, standardized, and reproducible clinical applications.

[0003] Existing automated segmentation schemes for gliomas include traditional image segmentation methods based on single-modal images and end-to-end segmentation methods based on multimodal 3D convolutional neural networks. Compared to single-modal schemes, multimodal magnetic resonance imaging (MRI) images can provide richer tissue information. Specifically, the contrast-enhanced T1-weighted (T1c) modality is more sensitive to enhanced tumor areas, the fluid-attenuated inversion recovery T2-weighted (T2f) modality is more sensitive to the extent of peritumoral edema, and the non-enhanced T1-weighted (T1n) and T2-weighted (T2w) modalities can provide supplementary descriptions of the tumor's internal structure, boundary transition regions, and the formation of normal tissue. Therefore, utilizing multimodal information for joint segmentation has become an important direction in current research and application.

[0004] Existing solutions typically employ a 3D encoder-decoder segmentation network: first, four-modal images are directly stacked in the channel dimension as input; then, semantic features are extracted through multiple levels of convolution and downsampling; subsequently, spatial resolution is restored through upsampling and cross-layer connections, finally outputting voxel-level segmentation results. To enhance feature representation, some solutions incorporate multi-branch convolutions or convolutional units with different receptive fields in the backbone to simultaneously extract local and contextual features; some solutions add channel attention units or spatial attention units within the network to improve the saliency of lesion regions; and still others add region constraints or simple connected component post-processing at the output to reduce isolated noise.

[0005] However, most of the existing solutions are designed for general medical segmentation problems. They are still insufficient for the complementary relationship between four-modal magnetic resonance images of adult gliomas, the hierarchical relationship between different tumor subregions, and the differential modeling of small tumor targets and diffuse edema boundaries. It is difficult to simultaneously take into account the efficiency of multimodal information utilization, fine-grained subregion recognition ability, and consistency of the final output results. Summary of the Invention

[0006] The purpose of this invention is to propose a multimodal collaborative perception-based method for glioma subregion segmentation, so as to improve the accuracy, stability and clinical applicability of glioma subregion segmentation.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: A method for segmenting glioma subregions based on multimodal collaborative perception includes the following steps: Step 1. Obtain magnetic resonance images of the object to be segmented in four modalities, preprocess the magnetic resonance images of the four modalities, and construct a training dataset; Step 2. Construct a glioma subregion segmentation model based on a multimodal collaborative segmentation network; the model includes a 3D convolutional shallow feature extraction unit, a modal collaborative perception module (MCM), an encoder, a decoder, a bottleneck layer, and a decoding head; There are four 3D convolutional shallow feature extraction units, each of which is used to extract shallow features of a magnetic resonance image of a certain modality. The MCM module evaluates the modal contribution and modulates the spatial response of the shallow features output by the 3D convolutional shallow feature extraction unit, and then performs adaptive fusion to obtain fused features. The encoder uses multiple hierarchical context recalibration blocks (HCR Blocks) to extract hierarchical features from the fused features, resulting in multi-scale features, shallow skip connection features, and deep semantic features. HCR Block enhances features by extracting contextual information under different receptive fields and combining it with the modal weights output by the MCM module; The encoder and decoder are connected through a bottleneck layer, which uses HCR Block to enhance deep semantic features; The decoder uses the Guided Region Filtering (GRS) module to filter shallow skip connection features. The decoding head includes a hierarchical region decoding head (HRD Head) and an auxiliary supervision output head; the HRD Head is used to output voxel-level classification results and region-level classification results. Step 3. Train the glioma subregion segmentation model based on the training dataset from Step 1. Use the trained model to segment the input image and process the segmentation results through a region consistency correction strategy to obtain the glioma subregion segmentation results.

[0008] Furthermore, based on the aforementioned glioma subregion segmentation method based on multimodal collaborative perception, this invention also proposes a corresponding glioma subregion segmentation system based on multimodal collaborative perception, which adopts the following technical solution: The preprocessing module is used to acquire magnetic resonance images of the four modalities of the object to be segmented, preprocess the magnetic resonance images of the four modalities, and construct a training dataset. The image segmentation module is used to build a glioma subregion segmentation model based on a multimodal collaborative segmentation network. The model includes a 3D convolutional shallow feature extraction unit, a modal collaborative perception module (MCM), an encoder, a decoder, a bottleneck layer, and a decoding head. There are four 3D convolutional shallow feature extraction units, each of which is used to extract shallow features of a magnetic resonance image of a certain modality. The MCM module evaluates the modal contribution and modulates the spatial response of the shallow features output by the 3D convolutional shallow feature extraction unit, and then performs adaptive fusion to obtain fused features. The encoder uses multiple hierarchical context recalibration blocks (HCR Blocks) to extract hierarchical features from the fused features, resulting in multi-scale features, shallow skip connection features, and deep semantic features. HCR Block enhances features by extracting contextual information under different receptive fields and combining it with the modal weights output by the MCM module; The encoder and decoder are connected through a bottleneck layer, which uses HCR Block to enhance deep semantic features; The decoder uses the Guided Region Filtering (GRS) module to filter shallow skip connection features. The decoding head includes a hierarchical region decoding head (HRD Head) and an auxiliary supervision output head; the HRD Head is used to output voxel-level classification results and region-level classification results. A glioma subregion segmentation model based on a multimodal collaborative segmentation network was trained. The trained model was used to segment the input image, and the segmentation results were processed by a region consistency correction strategy to obtain the glioma subregion segmentation results.

[0009] The present invention has the following advantages: As described above, this invention proposes a glioma subregion segmentation method based on multimodal collaborative perception. Compared with existing multimodal brain tumor segmentation methods, this method designs an MCM module to explicitly model the contribution differences of different modalities. This allows for adaptive adjustment of the participation level of each modality feature based on different cases and spatial regions. Compared to simple channel stitching, the MCM module has higher multimodal information utilization efficiency and stronger discriminative ability. Furthermore, this invention designs an HCR Block in the encoding stage, simultaneously considering local details, multi-scale context, and channel-spatial joint enhancement under modal conditions. This enables better identification of smaller, irregularly shaped, and poorly defined enhanced tumor regions, and improves the overall representation of edema areas and tumor core areas, exhibiting better feature expression and boundary delineation capabilities. Simultaneously, a GRS module is designed in the decoding stage to selectively filter shallow skip connection features, reducing the interference of normal tissue texture, background noise, and irrelevant responses on the decoding and reconstruction process. This reduces the probability of false-positive segmentation regions and improves the retention of key lesion information during the decoding stage. This invention employs a dual-headed supervision approach combining basic category segmentation with region-level auxiliary prediction, and trains using a total loss function. This simultaneously constrains the network optimization process from three levels: category discrimination, region structure, and boundary quality, resulting in better model training stability and more complete and consistent output results. During the inference phase, this invention introduces a region consistency correction strategy, performing post-processing based on connected components and morphological rules on the whole tumor region, tumor core region, and enhanced tumor region. This effectively removes isolated pseudo-segmented regions, repairs local holes, and maintains a reasonable spatial hierarchy between subregions (whole tumor region, tumor core region, and enhanced tumor region), thus providing better structural rationality and stronger clinical interpretability. The invention utilizes a three-dimensional convolution-based encoder-decoder framework, ensuring multimodal collaborative modeling, context enhancement, and region consistency constraints while avoiding the high computational cost and high memory consumption associated with heavy global Transformer structures. Therefore, this invention has good feasibility, training efficiency, and deployment adaptability, making it suitable for widespread application in multicenter adult glioma MRI image segmentation tasks. Attached Figure Description

[0010] Figure 1 This is a flowchart of the glioma subregion segmentation method based on multimodal collaborative perception in an embodiment of the present invention; Figure 2 This is a structural diagram of a glioma subregion segmentation model based on a multimodal collaborative segmentation network in an embodiment of the present invention; Figure 3 This is a structural diagram of the MCM module in an embodiment of the present invention; Figure 4 This is a structural diagram of the HCR Block in an embodiment of the present invention; Figure 5 This is a structural diagram of the GRS module in an embodiment of the present invention. Detailed Implementation

[0011] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments: Example 1 This invention proposes a glioma subregion segmentation method based on multimodal collaborative perception. By constructing a glioma subregion segmentation model based on a multimodal collaborative segmentation network, the method utilizes the MCM module at the input end to perform response modeling and adaptive fusion of four modalities of magnetic resonance imaging (MRI) images (T1c, T1n, T2f, and T2w). During the backbone feature extraction process, HCRBlock is used to simultaneously enhance local details and multi-scale context. At cross-layer connection locations, the GRS module is used to filter shallow skip connection features. At the output end, multiple tumor subregion segmentation results are obtained jointly through HRD Head. In post-processing, a region consistency correction strategy is used to perform hierarchical consistency correction and pseudo-region removal on the segmentation results. This enables automatic identification and fine segmentation of the necrotic / non-enhanced tumor core region, peritumoral edema region, enhanced tumor region, and the tumor core region and whole tumor region formed by the combination of the necrotic / non-enhanced tumor core region, peritumoral edema region, and enhanced tumor region in adult gliomas. This improves the accuracy, stability, and clinical applicability of glioma subregion segmentation.

[0012] like Figure 1 As shown, the glioma subregion segmentation method based on multimodal collaborative perception includes the following steps: Step 1. Obtain magnetic resonance images of the object to be segmented in four modalities. Perform registration, resampling, non-brain region removal, intensity normalization, and region of interest cropping on the four modal images to obtain spatially aligned and scale-uniform multimodal input data, and construct the training dataset.

[0013] The four modal magnetic resonance images include T1c mode images, T1n mode images, T2f mode images, and T2w mode images.

[0014] Spatial registration was performed on the four modal magnetic resonance images to place the four modes T1c, T1n, T2f, and T2w in a unified coordinate system.

[0015] The registered images are then subjected to resolution unification to obtain three-dimensional volume data with uniform voxel spacing.

[0016] Brain region masks or non-zero voxel regions are used to remove skull, air, and irrelevant background areas.

[0017] Intensity truncation and normalization were performed on images of each modality within the brain region to reduce grayscale differences caused by different scanning devices and imaging protocols.

[0018] Based on a four-modal combined non-zero region or brain region mask, regions of interest covering lesions and surrounding tissues are cropped to reduce the impact of redundant background on the subsequent segmentation process.

[0019] To improve the ability to adapt to case differences, modal quality fluctuations, and lesion size changes during the training phase, random flipping, random rotation, random scaling, random intensity perturbation, noise perturbation, and random modal decay enhancement can also be performed on the training data.

[0020] Step 2. Construct a glioma subregion segmentation model based on a multimodal collaborative segmentation network; such as... Figure 2 As shown, the model includes a 3D convolutional shallow feature extraction unit, a modal collaborative perception module (MCM), an encoder, a decoder, a bottleneck layer, and a decoding head.

[0021] The MCM module evaluates the modal contribution and modulates the spatial response of the shallow features output by the four 3D convolutional shallow feature extraction units, and then performs adaptive fusion to obtain fused features. The encoder uses multiple hierarchical context recalibration blocks (HCR Blocks) to extract hierarchical features from the fused features, resulting in multi-scale features, shallow skip connection features, and deep semantic features. HCR Block enhances features by extracting contextual information under different receptive fields and combining it with the modal weights output by the MCM module; The encoder and decoder are connected through a bottleneck layer. The bottleneck layer uses HCR Block to enhance deep semantic features, thereby improving the global contextual expression capability and providing stable high-level semantic guidance for subsequent decoding. The decoder performs upsampling step by step and uses the Guided Region Filtering (GRS) module to filter and fuse shallow skip connection features. The filtered shallow skip connection features are then spliced ​​and fused with the upsampled features. After further refinement by the HCR Block, high-resolution segmentation features are restored. The decoding head includes the hierarchical region decoding head (HRD Head) and auxiliary supervision output heads (Aux2, Aux3, and Aux4).

[0022] HRD Head combines fine-grained category prediction with regional structure prediction to output voxel-level classification results for necrotic / non-enhanced tumor core, edema, and enhanced tumor regions, and simultaneously outputs regional classification results for enhanced tumor regions, tumor core, and the entire tumor region.

[0023] In this embodiment, the encoder includes five HCR Blocks and four downsampling modules; Five HCR Blocks are defined as the first, second, third, fourth, and fifth HCR Blocks; four downsampling modules are defined as the first, second, third, and fourth downsampling modules.

[0024] The encoder's processing flow is as follows: The input feature map is processed sequentially through the first HCR Block, the first downsampling module, the second HCR Block, the second downsampling module, the third HCR Block, the third downsampling module, the fourth HCR Block, the fourth downsampling module, and the fifth HCRBlock.

[0025] The encoder adopts a layered channel configuration, with the number of output channels for each layer being 16, 32, 64, 128, and 256 respectively. The downsampling module adopts a downsampling method with a step size of 2 to gradually reduce spatial resolution and improve semantic expression capabilities.

[0026] The decoder consists of four GRS modules, four HCR blocks, and four upsampling modules; Four GRS modules are defined as the first, second, third, and fourth GRS modules; four HCR blocks are defined as the sixth, seventh, eighth, and ninth HCR blocks; and four upsampling modules are defined as the first, second, third, and fourth upsampling modules.

[0027] The decoder's processing flow is as follows: The outputs of the first HCR Block, the second HCR Block, the third HCR Block, and the fourth HCR Block are respectively sent to the fourth GRS module, the third GRS module, the second GRS module, and the first GRS module for processing.

[0028] The bottleneck layer output features are processed by the first upsampling module, then concatenated with the output of the first GRS module, and then sent to the sixth HCR Block for processing.

[0029] The output of the sixth HCR Block is processed by the second upsampling module, then concatenated with the output of the second GRS module, and then sent to the seventh HCR Block for processing.

[0030] The output of the seventh HCR Block is processed by the third upsampling module, then concatenated with the output of the third GRS module, and then sent to the eighth HCR Block for processing.

[0031] The output of the eighth HCR Block is processed by the fourth upsampling module, then concatenated with the output of the fourth GRS module, and then sent to the ninth HCR Block for processing.

[0032] The output of the ninth HCR Block is sent to the layer region decoding head (HRD Head).

[0033] The outputs of the eighth HCR Block, the seventh HCR Block, and the sixth HCR Block are sent to the auxiliary supervisor output heads Aux2, Aux3, and Aux4, respectively.

[0034] The decoder uses a step-by-step upsampling method to restore spatial resolution and fuses the shallow skip connection features output by the encoder at each layer; the number of output channels for each layer of the decoder is 128, 64, 32, and 16 respectively.

[0035] The bottleneck layer has 256 channels.

[0036] The 3D convolutional shallow feature extraction unit performs shallow feature extraction on the four modal inputs respectively, obtaining shallow feature maps corresponding to the four modalities. Specifically, a modality-specific shallow encoding unit is set up for each modality input. Each encoding unit includes two layers of 3D convolution, normalization, and activation operations to preserve the original discriminative information of each modality. The shallow feature maps corresponding to the four modalities are denoted as follows: , , , .

[0037] Indicates the first Shallow feature maps of each modality ; Indicates batch size, Indicates the number of channels. , , These represent the depth, height, and width of the three-dimensional space, respectively.

[0038] like Figure 3 As shown, the MCM module includes a modal contribution evaluation unit, a spatial response modulation unit, and an adaptive weighted fusion unit; the processing flow of the MCM module is as follows: The MCM module receives shallow feature maps output by four parallel 3D convolutional shallow feature extraction units. , , , ; Shallow feature maps corresponding to the four modalities , , , The data are sent to the modal contribution evaluation unit and the spatial response modulation unit for processing, respectively.

[0039] The processing flow of the modal contribution evaluation unit is as follows: right , , , Global average pooling and global max pooling are performed separately to obtain statistical description vectors for the four modalities. These four modal statistical description vectors are then concatenated and input into a multilayer perceptron to obtain modal-level score vectors. Finally, the modal-level score vectors are normalized using the Softmax function to obtain modal-level weight vectors, the expression of which is as follows: ; Where GAP represents global average pooling, GMP represents global max pooling, and MLP represents a modality weight prediction network consisting of fully connected layers and activation functions; This represents the four-dimensional modal weight vector generated based on the sample.

[0040] ; Indicates the first The modal-level weights of each modality; for each sample, the four modal-level weights, after Softmax normalization, sum to 1.

[0041] The processing flow of the spatial response modulation unit is as follows: right , , , The spatial response map is obtained by performing 1×1×1 convolution, ReLU activation function, 1×1×1 convolution, and Sigmoid activation function respectively. , , , ; Indicates the first The strength of the response of each mode at different spatial locations.

[0042] The output of the modal contribution evaluation unit With the output of the spatial response modulation unit The mixture is fed into an adaptive weighted fusion unit for fusion, and the fusion formula is as follows: ; in Indicates the characteristics after fusion. Represents element-wise multiplication, modal weights After being broadcast-extended in the spatial dimension, it participates in the weighting together with the corresponding modal features and spatial response map.

[0043] The main output of the MCM module is the fused features, which are used by the subsequent encoder for hierarchical feature extraction. At the same time, the MCM module also outputs modality-level weight vectors and spatial-level response maps to characterize the global relative contribution of each modality in the current case and the strength of local response at different spatial locations.

[0044] Through the above design, the MCM module simultaneously completes sample-level modality selection and voxel-level spatial selection, so that the fused features are not simply channel splicing results, but discriminative features after explicit modality collaborative screening.

[0045] Existing multimodal input methods typically employ simple channel stacking or fixed-rule fusion, lacking explicit modeling of the differences in contributions of different modalities across different cases, spatial locations, and tumor subregions, resulting in insufficient utilization of modal complementarity information. To address the issue of inconsistent contributions of different modalities to different tumor subregions in quadrature glioma MRI images, this invention proposes a technical approach that begins differentiated collaborative modeling from the input stage. Instead of simply stitching together and processing the information from each modality, it first jointly evaluates the effectiveness of each modality at both the overall and spatial levels before performing adaptive fusion, thereby improving the targeting and effectiveness of multimodal information utilization from the source.

[0046] like Figure 4 As shown, the processing flow of the HCR Block is as follows: The HCR Block is used to perform multi-scale context extraction, modality-conditional guided channel recalibration, and spatial recalibration on the input features, and output the enhanced features.

[0047] Define input features .

[0048] First, input the features. Intermediate features are obtained by channel mapping using 1×1×1 convolution, normalization, and LeakyReLU activation function; Then, the intermediate features are input into three parallel 3×3×3 3D convolutional branches to extract multi-scale contextual features. The dilation rates d of the three 3×3×3 3D convolutions are 1, 2, and 3, respectively, to extract contextual features under different receptive fields. The outputs of the 3×3×3 3D convolutional branches are as follows: , , .

[0049] To reduce redundancy and parameter bloat caused by direct concatenation, learnable branch weights are introduced. , , The aggregation coefficients were obtained by normalizing them using the Softmax function. , , .

[0050] Then, the outputs of the three 3×3×3 three-dimensional convolution branches are weighted and aggregated to obtain the fused features. The calculation formula is as follows: ; in Indicates the first The aggregation coefficients corresponding to each 3×3×3 three-dimensional convolution branch Indicates the first Contextual features output by each 3×3×3 three-dimensional convolution branch.

[0051] Then, a 1×1×1 convolution is used to map the fused features to the number of output channels, and then... Features after identity mapping or 1×1×1 convolution are summed using residuals to achieve dimension matching when the number of input and output channels is inconsistent. Then, they are activated by LeakyReLU to obtain the basic context-enhanced features. .

[0052] Will The input is parallel to the channel recalibration unit and the spatial recalibration unit for recalibration.

[0053] The processing flow of the channel recalibration unit is as follows: right Global average pooling and global max pooling are performed separately to obtain two channel statistical description vectors for the corresponding global average pooling and global max pooling results. These two channel statistical description vectors are then combined with the modal weight vector from the MCM module. The layers are spliced ​​and input into a mapping network consisting of two fully connected layers to obtain the channel recalibration coefficients. This design ensures that the reinforcement direction of the main channel aligns with the front-end modal coordination mechanism.

[0054] The processing flow for the space recalibration unit is as follows: right Average compression and maximum compression are performed along the channel dimension respectively to obtain two single-channel spatial description maps. The two single-channel spatial description maps are then concatenated and processed by a 3×3×3 convolution and a sigmoid activation function to obtain a spatial recalibration map. The average compression and maximum compression are common spatial description methods for aggregating the channel responses at various spatial locations, used to characterize the importance of different spatial locations while preserving the spatial dimensions D, H, and W.

[0055] Will , and Perform element-wise multiplication to obtain the recalibrated features. Its expression is: ; To avoid overly strong attention enhancement leading to training instability, a learnable scaling parameter is introduced into the residual connection. And output it as follows: ; in Indicates the output characteristics of the HCR Block. This represents the learnable scaling parameter, in this embodiment... Initialized to 0.5.

[0056] Through the above design, HCR Block can simultaneously enhance local texture, small object boundaries, and multi-scale context with low computational cost, and improve the discriminativeness of feature representation.

[0057] While existing backbone feature extraction units can expand the receptive field, they lack a unified coordination mechanism between local details, small-volume enhanced tumor areas, and infiltrated regions with blurred boundaries. This leads to problems such as missed segmentation of enhanced tumors, coarse edema boundaries, and unstable identification of necrotic areas. Building upon front-end multimodal adaptive fusion, the HCRBlock designed in this invention extracts contextual information under different receptive fields and combines it with prior information on modal weights output by the MCM module to conditionally re-enhance features. This allows the network to not only preserve local details but also consider deep semantics and broad context, thereby improving the representation capabilities of complex boundaries, small target regions, and blurred transition regions.

[0058] like Figure 5 As shown, the processing flow of the GRS module is as follows: Define shallow skip connection features of the encoder The guiding features after upsampling at the current decoding layer ;in and These represent the number of channels for the shallow skip connection features of the encoder and the guided features after upsampling at the current decoding layer, respectively.

[0059] First, use 1×1×1 convolution to... and Perform channel mapping to the intermediate channel space to obtain the mapped jump features. and the mapped guiding features .

[0060] Simultaneously, a linear mapping is used to transform the modal weight vector. Mapping to the intermediate channel space yields a condition vector; then, the condition vector undergoes dimensional expansion and spatial broadcasting to align it with... and With consistent spatial dimensions, modal condition features are obtained. .

[0061] Will , and Element-wise addition is performed, followed by processing with LeakyReLU activation, 3×3×3 convolution, and Sigmoid activation to generate a region filtering map. Its expression is as follows: ; in This represents a 3×3×3 convolution operation; This represents the LeakyReLU activation function operation. This represents the Sigmoid activation function operation.

[0062] Finally, the region filtering map is used. Features of shallow skip connections Perform residual gating to obtain output features The calculation formula is as follows: ; Among them when When the value approaches 0, the output is an approximate identity mapping; when... When the size is large, the lesion-related area receives higher weight in both forward and reverse propagation.

[0063] When existing decoding stages directly use shallow skip connections, normal brain tissue texture, artifacts, or background noise are often transmitted to the decoder along with the signal, leading to an increase in false positive regions and amplification of erroneous responses near the boundaries. To address the problem of background noise and irrelevant textures being easily carried over during cross-layer information transmission between the encoding and decoding stages, this invention proposes a GRS module. This module selectively filters and enhances shallow detail information before it is transmitted to the decoding end, allowing the multimodal collaborative results from the front end to continue to influence the subsequent decoding and recovery process, forming a continuous technical chain from input fusion and feature enhancement to cross-layer filtering.

[0064] The processing flow of the HRD Head decoder is as follows: The HRD Head decoder receives the highest-resolution decoded features output from the ninth HCR Block as input; the HRD Head decoder includes a basic category segmentation branch and a region-level auxiliary prediction branch, which process the same input features in parallel.

[0065] The basic category segmentation branch performs 1×1×1 convolution processing on the input features and outputs a four-channel voxel-level classification score map, which corresponds to the background, necrotic / non-enhanced tumor core area, edema area and enhanced tumor area, respectively.

[0066] The regional auxiliary prediction branch first performs a 3×3×3 convolution on the input features for local fusion, and then outputs a three-channel regional classification score map through a 1×1×1 convolution, which corresponds to the enhanced tumor region ET, the tumor core region TC, and the whole tumor region WT, respectively.

[0067] A voxel-level labeled map G is defined, with the following category codes: 0 represents background, 1 represents necrotic / non-enhancing tumor core area, 2 represents edema area, and 3 represents enhancing tumor area. Based on the voxel-level labeled map G, the following region-level supervision targets are derived: ; ; ; in This indicates an enhanced regional surveillance objective for tumor areas. This indicates the regional surveillance target of the tumor core area. This indicates a regional-level surveillance target for the entire tumor area. Indicates an indicator function; when the condition within the parentheses is true, The value is 1; when the condition inside the parentheses is not true, The value is 0.

[0068] Through the above design, the HRD Head decoder can simultaneously output fine-grained voxel-level classification results and region-level structure prediction results, thereby simultaneously constraining local class discrimination ability and region-level structure consistency during training.

[0069] In addition, auxiliary output heads are set in several intermediate decoding layers, preferably outputting auxiliary logits at three intermediate scales. After being upsampled to the original resolution by trilinear interpolation, they are used to participate in deep supervised training to improve gradient propagation effect and multi-scale learning stability.

[0070] Existing output methods primarily rely on single voxel classification results, which lack sufficient constraints on the hierarchical relationships between enhanced tumor regions, tumor core regions, and the entire tumor region. This can easily lead to inconsistent results, such as enhanced tumor regions extending beyond the tumor core region, scattered small isolated lesions, or fragmented subregion boundaries. This invention does not rely solely on single voxel classification results for training and inference. Instead, it introduces the concept of regional hierarchical learning, jointly constraining fine-grained voxel-level category prediction with regional-level structure prediction. This allows the network to learn not only local voxel classifications but also the overall combination relationships between various tumor subregions. Consequently, the representational advantages formed by the aforementioned input, feature, and decoding ends can ultimately be translated into a more stable structured output.

[0071] Step 3. Train the glioma subregion segmentation model based on the training dataset from Step 1. Use the trained model to segment the input image and process the segmentation results through a region consistency correction strategy to obtain the glioma subregion segmentation results.

[0072] During the training phase, the main segmentation output, region-level output, and intermediate-scale auxiliary output are all fed into a joint loss function for optimization. During the inference phase, a region consistency correction strategy is used to process the initial classification results, performing small region removal, hierarchical consistency correction, and boundary repair to obtain the final glioma subregion segmentation results. The final results are then used for lesion volume statistics, region display, assisted diagnosis, treatment planning, or follow-up analysis.

[0073] The total loss function during model training includes the main segmentation loss, region-level loss, boundary loss, and deep supervision loss, and its calculation formula is as follows: ; in Main segmentation loss, This is a regional-level loss. For boundary loss, The losses were due to in-depth supervision; Main segmentation loss weights, For regional loss weights, For boundary loss weights, Weighting for losses in deep supervision.

[0074] The main segmentation loss employs a combination of multi-class Soft Dice loss and weighted cross-entropy loss. The calculation formula is: ; in For multiple types of Soft Dice loss, This is the weighted cross-entropy loss.

[0075] Multiple types of Soft Dice loss The calculation formula is: ; in Indicates the total number of basic categories. This represents the total number of voxels involved in the calculation; Indicates the first Individual elements belong to the first Predicted probability of class This indicates the corresponding one-hot encoded tag; This represents a smoothing term, used to avoid a denominator of zero.

[0076] Weighted cross-entropy loss The calculation formula is: ; in Indicates the first The category weight corresponding to the class.

[0077] The regional loss employs a combination of regional binary cross-entropy loss and Focal Tversky loss. The calculation formula is: ; in This represents the region-level binary cross-entropy loss. This indicates Focal Tversky's loss.

[0078] Regional-level binary cross-entropy loss The calculation formula is: ; in This represents the total number of voxels participating in the calculation within the current region. Represents the binary label of the corresponding region. This represents the region prediction probability obtained by performing a sigmoid transformation on the classification score map.

[0079] Focal Tversky loss The calculation formula is: ; in This represents the focusing parameter, used to enhance attention to difficult samples; The Tversky index; ; and This represents the coefficient used to balance the penalties for false positives and false negatives.

[0080] , and Let represent true positives, false positives, and false negatives, respectively, and their calculation formulas are as follows: ; ; .

[0081] Boundary loss is used to enhance the model's response to tumor boundary regions; soft boundary response maps are constructed from the predicted probability map and the label map, respectively. Its expression is: ; in Represents a category probability map or label map. MaxPool(·) and MinPool(·) represent three-dimensional max pooling and min pooling operations with a kernel size of 3×3×3, a step size of 1, and a padding size of 1, respectively.

[0082] Boundary loss The calculation formula is: ; in This represents the total number of voxels involved in the boundary loss calculation. and These represent the soft boundary responses corresponding to the predicted probability map and the label map, respectively.

[0083] The deep supervision loss applies the same form as the main segmentation loss to the intermediate-scale auxiliary output and takes the average. The calculation formula is: ; in Indicates the number of auxiliary monitoring output heads. Indicates the first The prediction results corresponding to each auxiliary output head This represents a voxel-level label map.

[0084] During training, the input data and annotations are fed into the glioma subregion segmentation model based on a multimodal collaborative segmentation network. The parameters of each module are updated through backpropagation, and the AdamW optimizer is preferred for training.

[0085] The region consistency correction strategy is set during the inference phase to regularize and correct the initial segmentation results output by the model. The processing flow of the region consistency correction strategy is as follows: First, classification is performed on the four-channel voxel-level classification score map output by the basic category segmentation branch in the HRD Head to obtain the initial label map. ; The class values ​​for each voxel include: background, necrotic / non-enhancing tumor core area, edema area, and enhancing tumor area.

[0086] Then, based on the initial label graph The three clinically commonly used areas are as follows: ; ; ; in Indicates enhancement of the tumor area, Indicates the core area of ​​the tumor. This indicates the entire tumor area.

[0087] In the enhanced tumor area Tumor core area and the entire tumor area Then, perform regional consistency correction according to the following steps: For the entire tumor area Perform connected component analysis, retain only the largest connected component or the top k largest connected components, and remove isolated pseudo-regions that are far from the main lesion.

[0088] For the tumor core area Perform small connected component removal if the number of connected components is less than a preset value. The connected components of the threshold are no longer limited to the tumor core region; in this embodiment... The threshold was set to 30 voxels.

[0089] For enhancing tumor area Perform small region detection when the volume of the enhanced tumor region connected domain is smaller than the preset value. When the threshold is reached, the region is relabeled as the necrotic / non-enhancing tumor core area; in this embodiment... The threshold is set to 20 voxels.

[0090] Perform hierarchical consistency correction to ensure that the enhanced tumor region is located within the tumor core region, and the tumor core region is located within the entire tumor region, thus satisfying the requirement... .

[0091] The entire tumor region after the above connected component screening, region relabeling, and hierarchical consistency correction (i.e., region consistency correction) and tumor core area Perform cavity filling. In this embodiment, the corrected entire tumor area is filled. Further minor three-dimensional closure operations are performed to connect local narrow cracks, repair boundary fractures, and smooth region boundaries; specifically, 3×3×3 structuring elements are used to modify the entire tumor region. Perform a slight three-dimensional closing operation.

[0092] Finally, the labels are recombined to output the glioma subregion segmentation results.

[0093] In the final result generation stage, this invention further combines the natural inclusion relationship between the subregions of glioma to perform regional consistency correction on the initial prediction results. This ensures that the information obtained from front-end adaptive fusion, mid-level hierarchical enhancement, back-end cross-layer screening, and output-end joint supervision is uniformly constrained at the result level. In this way, a complete technical solution consisting of "multimodal difference modeling - hierarchical feature enhancement - cross-layer guided screening - regional hierarchical constraint - consistency correction" is constructed.

[0094] The quality, contrast, and noise levels of MRI images vary across different hospitals and scanning protocols. Without a stable modal collaborative perception and result correction mechanism, the model's generalization ability and clinical usability are limited. This invention establishes a collaborative relationship between its various stages: the front end addresses the contribution allocation problem of multimodal inputs; the middle part addresses contextual representation and cross-layer noise propagation; and the back end addresses regional structural constraints and result consistency. This results in clear and targeted improvements across all stages of the segmentation process: input, feature extraction, decoding and recovery, and result output.

[0095] Example 2 This embodiment 2 describes a glioma subregion segmentation system based on multimodal collaborative perception, which is based on the same inventive concept as the glioma subregion segmentation method based on multimodal collaborative perception in embodiment 1 above.

[0096] The glioma subregion segmentation system based on multimodal collaborative perception includes the following modules: The preprocessing module is used to acquire magnetic resonance images of the four modalities of the object to be segmented, preprocess the magnetic resonance images of the four modalities, and construct a training dataset. The image segmentation module is used to build a glioma subregion segmentation model based on a multimodal collaborative segmentation network. The model includes a 3D convolutional shallow feature extraction unit, a modal collaborative perception module (MCM), an encoder, a decoder, a bottleneck layer, and a decoding head. There are four 3D convolutional shallow feature extraction units, each of which is used to extract shallow features of a magnetic resonance image of a certain modality. The MCM module evaluates the modal contribution and modulates the spatial response of the shallow features output by the 3D convolutional shallow feature extraction unit, and then performs adaptive fusion to obtain fused features. The encoder uses multiple hierarchical context recalibration blocks (HCR Blocks) to extract hierarchical features from the fused features, resulting in multi-scale features, shallow skip connection features, and deep semantic features. HCR Block enhances features by extracting contextual information under different receptive fields and combining it with the modal weights output by the MCM module; The encoder and decoder are connected through a bottleneck layer, which uses HCR Block to enhance deep semantic features; The decoder uses the Guided Region Filtering (GRS) module to filter shallow skip connection features. The decoding head includes a hierarchical region decoding head (HRD Head) and an auxiliary supervision output head; the HRD Head is used to output voxel-level classification results and region-level classification results. A glioma subregion segmentation model based on a multimodal collaborative segmentation network was trained. The trained model was used to segment the input image, and the segmentation results were processed by a region consistency correction strategy to obtain the glioma subregion segmentation results.

[0097] It should be noted that any content not mentioned in the above-described functional modules of the system described in Embodiment 2 can be referred to the step description of the corresponding method in Embodiment 1 above, and will not be repeated in detail here.

[0098] Of course, the above description is only a preferred embodiment of the present invention. The present invention is not limited to the above-described embodiments. It should be noted that any equivalent substitutions or obvious modifications made by those skilled in the art under the guidance of this specification fall within the scope of this specification and should be protected by the present invention.

Claims

1. A method for glioma subregion segmentation based on multimodal collaborative perception, characterized in that, Includes the following steps: Step 1. Obtain magnetic resonance images of the object to be segmented in four modalities, preprocess the magnetic resonance images of the four modalities, and construct a training dataset; Step 2. Construct a glioma subregion segmentation model based on a multimodal collaborative segmentation network; the model includes a 3D convolutional shallow feature extraction unit, a modal collaborative perception module (MCM), an encoder, a decoder, a bottleneck layer, and a decoding head; There are four 3D convolutional shallow feature extraction units, each of which is used to extract shallow features of a magnetic resonance image of a certain modality. The MCM module evaluates the modal contribution and modulates the spatial response of the shallow features output by each 3D convolutional shallow feature extraction unit, and then performs adaptive fusion to obtain fused features. The encoder uses multiple hierarchical context recalibration blocks (HCR Blocks) to extract hierarchical features from the fused features, resulting in multi-scale features, shallow skip connection features, and deep semantic features. HCR Block enhances features by extracting contextual information under different receptive fields and combining it with the modal weights output by the MCM module; The encoder and decoder are connected through a bottleneck layer, which uses HCR Block to enhance deep semantic features; The decoder uses the Guided Region Filtering (GRS) module to filter shallow skip connection features. The decoding head includes a hierarchical region decoding head (HRD Head) and an auxiliary supervision output head; the HRD Head is used to output voxel-level classification results and region-level classification results. Step 3. Train the model based on the training dataset from Step 1, use the trained model to segment the input image, and process the segmentation results through a region consistency correction strategy to obtain the glioma subregion segmentation results.

2. The method for glioma subregion segmentation based on multimodal collaborative perception according to claim 1, characterized in that, In step 2, the encoder includes five HCR blocks and four downsampling modules; Five HCR Blocks are defined as the first, second, third, fourth, and fifth HCR Blocks; four downsampling modules are defined as the first, second, third, and fourth downsampling modules; the encoder's processing flow is as follows: the input feature map is processed sequentially through the first HCR Block, the first downsampling module, the second HCR Block, the second downsampling module, the third HCR Block, the third downsampling module, the fourth HCR Block, the fourth downsampling module, and the fifth HCR Block.

3. The method for glioma subregion segmentation based on multimodal collaborative perception according to claim 1, characterized in that, In step 2, the decoder includes four GRS modules, four HCR blocks, and four upsampling modules; Define four GRS modules as the first, second, third, and fourth GRS modules; four HCR blocks as the sixth, seventh, eighth, and ninth HCR blocks; and four upsampling modules as the first, second, third, and fourth upsampling modules. The decoder's processing flow is as follows: The outputs of the first HCR Block, the second HCR Block, the third HCR Block, and the fourth HCR Block are respectively sent to the fourth GRS module, the third GRS module, the second GRS module, and the first GRS module for processing; The bottleneck layer output features are processed by the first upsampling module, then concatenated with the output of the first GRS module, and then sent to the sixth HCR Block for processing. The output of the sixth HCR Block is processed by the second upsampling module, then concatenated with the output of the second GRS module, and then sent to the seventh HCR Block for processing. The output of the seventh HCR Block is processed by the third upsampling module, then concatenated with the output of the third GRS module, and then sent to the eighth HCR Block for processing. The output of the eighth HCR Block is processed by the fourth upsampling module, then concatenated with the output of the fourth GRS module, and then sent to the ninth HCR Block for processing. The output of the ninth HCR Block is sent to the hierarchical region decoding head; the outputs of the eighth HCR Block, the seventh HCR Block, and the sixth HCRBlock are sent to the auxiliary supervision output head.

4. The method for glioma subregion segmentation based on multimodal collaborative perception according to claim 1, characterized in that, In step 2, the processing flow of the MCM module is as follows: The MCM module includes a modal contribution evaluation unit, a spatial response modulation unit, an adaptive weighted fusion unit, and a fusion feature output terminal; Shallow feature maps corresponding to the four modalities , , , The samples are respectively sent to the modal contribution evaluation unit and the spatial response modulation unit for processing; Indicates the first Shallow feature maps of each modality ; Indicates batch size, Indicates the number of channels. , , These represent the depth, height, and width of the three-dimensional space, respectively. The processing flow of the modal contribution evaluation unit is as follows: right , , , Global average pooling and global max pooling are performed separately to obtain statistical description vectors for the four modalities. These four modal statistical description vectors are then concatenated and input into a multilayer perceptron to obtain modal-level score vectors. Finally, the modal-level score vectors are normalized using the Softmax function to obtain the modal weight vectors, the expression of which is as follows: ； in Indicates global average pooling. Indicates global max pooling. This represents a modality weight prediction network consisting of fully connected layers and activation functions; This represents a four-dimensional modal weight vector generated based on the sample. ； Indicates the first Modal-level weights for each modality; The processing flow of the spatial response modulation unit is as follows: right , , , The spatial response map is obtained by performing 1×1×1 convolution, ReLU activation function, 1×1×1 convolution, and Sigmoid activation function respectively. , , , ; Indicates the first The strength of the response of each mode at different spatial locations; The output of the modal contribution evaluation unit With the output of the spatial response modulation unit The mixture is fed into an adaptive weighted fusion unit for fusion, and the fusion formula is as follows: ； in Indicates the characteristics after fusion. This indicates element-wise multiplication.

5. The method for glioma subregion segmentation based on multimodal collaborative perception according to claim 1, characterized in that, In step 2, the processing flow of the HCR Block is as follows: Define input features First, input the features. Intermediate features are obtained by channel mapping using 1×1×1 convolution, normalization layers, and the LeakyReLU activation function; Then, the intermediate features are input into three parallel 3×3×3 three-dimensional convolutional branches to extract multi-scale contextual features. The outputs of the three 3×3×3 three-dimensional convolutional branches are respectively... , , ; To reduce redundancy and parameter bloat caused by direct concatenation, learnable branch weights are introduced. , , The aggregation coefficients were obtained by normalizing them using the Softmax function. , , ; The outputs of the three 3×3×3 three-dimensional convolutional branches are then weighted and aggregated to obtain the fused features. The calculation formula is as follows: ； in Indicates the first The aggregation coefficients corresponding to each 3×3×3 three-dimensional convolution branch Indicates the first Multi-scale contextual features output by a 3×3×3 three-dimensional convolutional branch; Then, 1×1×1 convolutions are used to fuse the features. Mapped to the number of output channels, and with The features after identity mapping or 1×1×1 convolution are summed by residual addition, and then activated by LeakyReLU to obtain the basic context-enhanced features. ; Will The input is parallel to the channel recalibration unit and the spatial recalibration unit for recalibration; The processing flow of the channel recalibration unit is as follows: right Global average pooling and global max pooling are performed separately to obtain two channel statistical description vectors; Combine the two channel statistical description vectors with the modal weight vector from the MCM module. The layers are spliced ​​and input into a mapping network consisting of two fully connected layers to obtain the channel recalibration coefficients. ; The processing flow for the space recalibration unit is as follows: right Average compression and maximum compression are performed along the channel dimension to obtain two single-channel spatial description maps. The two single-channel spatial description maps are then concatenated and processed by a 3×3×3 convolution and a sigmoid activation function to obtain a spatial recalibration map. ;Will , and Perform element-wise multiplication to obtain the recalibrated features. Its expression is: ; Introducing learnable scaling parameters into residual connections And output it as follows: ; in This indicates the output characteristics of the HCR Block.

6. The method for glioma subregion segmentation based on multimodal collaborative perception according to claim 1, characterized in that, In step 2, the processing flow of the GRS module is as follows: Define shallow skip connection features of the encoder The guiding features after upsampling at the current decoding layer First, use 1×1×1 convolution to respectively... and Perform channel mapping to obtain the mapped skip features. and the mapped guiding features ; Simultaneously, a linear mapping is used to transform the modal weight vector. Mapping to the intermediate channel space yields a condition vector; then, the condition vector undergoes dimensional expansion and spatial broadcasting to align it with... and With consistent spatial dimensions, modal condition features are obtained. ; Will , and Element-wise addition is performed, followed by processing with LeakyReLU activation, 3×3×3 convolution, and Sigmoid activation to generate a region filtering map. Its expression is as follows: ； in This represents a 3×3×3 convolution operation; This represents the LeakyReLU activation function operation. This represents the Sigmoid activation function operation; Finally, the region filtering map is used. Features of shallow skip connections Perform residual gating to obtain output features The calculation formula is as follows: 。 7. The method for glioma subregion segmentation based on multimodal collaborative perception according to claim 1, characterized in that, In step 2, the processing flow of the HRD Head decoder is as follows: The HRD Head decoder includes a parallel basic category segmentation branch and a region-level auxiliary prediction branch; The basic category segmentation branch performs a 1×1×1 convolution on the input features and outputs a four-channel voxel-level classification score map, which corresponds to the background, necrotic / non-enhanced tumor core area, edema area and enhanced tumor area, respectively. The regional auxiliary prediction branch performs a 3×3×3 convolution on the input features for local fusion, and then outputs a three-channel regional classification score map after a 1×1×1 convolution, which corresponds to the enhanced tumor area, the tumor core area and the whole tumor area respectively. Define voxel-level label map voxel-level labeling The category coding is as follows: 0 represents background, 1 represents necrotic / non-enhancing tumor core area, 2 represents edema area, and 3 represents enhancing tumor area; based on the voxel-level label map The derived regional-level oversight objectives are as follows: ； ； ； in This indicates an enhanced regional surveillance objective for tumor areas. This indicates the regional surveillance target of the tumor core area. This indicates the regional surveillance target for the entire tumor area; This indicates an indicator function that, when the condition within the parentheses is true, The value is 1, when the condition inside the parentheses is not met. The value is 0.

8. The method for glioma subregion segmentation based on multimodal collaborative perception according to claim 1, characterized in that, The total loss function during model training includes the main segmentation loss, region-level loss, boundary loss, and deep supervision loss, and its calculation formula is as follows: ； in Main segmentation loss, This is a regional-level loss. For boundary loss, The losses were due to in-depth supervision; Main segmentation loss weight, For regional loss weights, For boundary loss weights, Weighting for losses in deep monitoring; The calculation formula is: ; in For multiple types of Soft Dice loss, The weighted cross-entropy loss; Multiple types of Soft Dice loss The calculation formula is: ； in Indicates the total number of basic categories. This represents the total number of voxels involved in the calculation. Indicates the first Individual elements belong to the first Predicted probability of class This indicates the corresponding one-hot encoded tag; Indicates the smoothing term; Weighted cross-entropy loss The calculation formula is: ； in Indicates the first The category weight corresponding to the class; The calculation formula is: ; in This represents the region-level binary cross-entropy loss. This indicates Focal Tversky's loss; Regional-level binary cross-entropy loss The calculation formula is: ； in This represents the total number of voxels participating in the calculation within the current region. Represents the binary label of the corresponding region. This represents the region prediction probability obtained by performing a sigmoid transformation on the classification score map; Focal Tversky loss The calculation formula is: ； in Indicates the focus parameter, The Tversky index; Boundary loss The calculation formula is: ； in This represents the total number of voxels involved in the boundary loss calculation. and These represent the soft boundary responses corresponding to the predicted probability map and the label map, respectively. Deep monitoring of losses The calculation formula is: ； in Indicates the number of auxiliary monitoring output heads. Indicates the first The prediction results corresponding to each auxiliary output head This represents a voxel-level label map.

9. The method for glioma subregion segmentation based on multimodal collaborative perception according to claim 1, characterized in that, In step 3, the processing flow of the regional consistency correction strategy is as follows: First, classification is performed on the four-channel voxel-level classification score map output by the basic category segmentation branch in the HRD Head to obtain the initial label map. Then, based on the initial label map... The three clinically commonly used areas are as follows: ； ； ； in, Indicates enhancement of the tumor area, Indicates the core area of ​​the tumor. Indicates the entire tumor area; Perform region consistency correction using the following steps: For the entire tumor area Perform connected component analysis, retain only the largest connected component or the top k largest connected components, and remove isolated pseudo-regions far from the main lesion; For the tumor core area Perform small connected component removal if the number of connected components is less than a preset value. The connected components of the threshold are no longer retained as the tumor core region; For enhancing tumor area Perform small region detection when the volume of the enhanced tumor region connected domain is smaller than the preset value. When the threshold is reached, the region is relabeled as the necrotic / non-enhancing tumor core area; Perform hierarchical consistency correction to ensure that the enhanced tumor region is located within the tumor core region, and the tumor core region is located within the entire tumor region, thus satisfying the following conditions: ; The entire tumor region after the above connected component screening, region relabeling, and hierarchical consistency correction and tumor core area Perform hole filling; finally, recombine the labels and output the glioma subregion segmentation results.

10. A glioma subregion segmentation system based on multimodal collaborative perception, characterized in that, Includes the following modules: The preprocessing module is used to acquire magnetic resonance images of the four modalities of the object to be segmented, preprocess the magnetic resonance images of the four modalities, and construct a training dataset. The image segmentation module is used to build a glioma subregion segmentation model based on a multimodal collaborative segmentation network. The model includes a 3D convolutional shallow feature extraction unit, a modal collaborative perception module (MCM), an encoder, a decoder, a bottleneck layer, and a decoding head. There are four 3D convolutional shallow feature extraction units, each of which is used to extract shallow features of a magnetic resonance image of a certain modality. The MCM module evaluates the modal contribution and modulates the spatial response of the shallow features output by the 3D convolutional shallow feature extraction unit, and then performs adaptive fusion to obtain fused features. The encoder uses multiple hierarchical context recalibration blocks (HCR Blocks) to extract hierarchical features from the fused features, resulting in multi-scale features, shallow skip connection features, and deep semantic features. HCR Block enhances features by extracting contextual information under different receptive fields and combining it with the modal weights output by the MCM module; The encoder and decoder are connected through a bottleneck layer, which uses HCR Block to enhance deep semantic features; The decoder uses the Guided Region Filtering (GRS) module to filter shallow skip connection features. The decoding head includes a hierarchical region decoding head (HRD Head) and an auxiliary supervision output head; the HRD Head is used to output voxel-level classification results and region-level classification results. A glioma subregion segmentation model based on a multimodal collaborative segmentation network was trained. The trained model was used to segment the input image, and the segmentation results were processed by a region consistency correction strategy to obtain the glioma subregion segmentation results.

Images