A method for image segmentation of a three-dimensional glioma, an electronic device, and a storage medium

Abstract

This invention proposes a three-dimensional glioma image segmentation method, electronic device, and storage medium. The method includes: acquiring three-dimensional volume data from multimodal magnetic resonance imaging; fusing the three-dimensional volume data with prior information on orientation boundaries to obtain orientation-boundary-enhanced three-dimensional volume data; decomposing the orientation-boundary-enhanced three-dimensional volume data into a two-dimensional slice sequence and performing inter-slice context modeling to aggregate information from adjacent slices to generate a target feature volume; dynamically inferring discrete token combinations based on the target feature volume using a discrete token vocabulary and an attribute predictor to generate high-level semantic cue features; inputting the target feature volume into an orientation-aware dual-domain enhancement branch to obtain enhanced features; and inputting the enhanced features and high-level semantic cue features together into a sparse hybrid expert decoder to output segmentation probability maps of three nested tumor regions. This invention significantly improves the boundary accuracy and small target recognition capability of glioma subregion segmentation, thereby enhancing the accuracy of image segmentation.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a three-dimensional glioma image segmentation method, electronic device, and storage medium. Background Technology

[0002] Multiparameter magnetic resonance imaging (MRI) is a crucial tool for the diagnosis and treatment of gliomas. Accurate segmentation of the whole tumor (WT), tumor core (TC), and enhancing tumor (ET) subregions is of significant clinical value for surgical planning and efficacy evaluation. While the Brain Tumor Segmentation (BraTS) benchmark has driven the development of this field, severe class imbalance arises due to the blurred boundaries of glioma images, frequent omissions of ET regions, and their extremely small size. Existing deep learning methods are highly prone to generating scattered false positive predictions when dealing with small targets like ET. Commonly used surface distance metrics (such as HD95) are extremely sensitive to these tiny false positives, potentially leading to assessment results that deviate significantly from clinical reality. Therefore, overcoming the problems of blurred boundaries and class imbalance, effectively suppressing false positives, and achieving robust segmentation consistent with anatomical logic are urgent technical bottlenecks that need to be overcome. Summary of the Invention

[0003] This invention provides a three-dimensional glioma image segmentation method, electronic device, and storage medium to solve the problems existing in related technologies. The technical solution is as follows: In a first aspect, embodiments of the present invention provide an image segmentation method for three-dimensional gliomas, comprising: The three-dimensional volume data of the multimodal magnetic resonance imaging to be segmented is obtained, and the three-dimensional volume data is fused with the prior information of the directional boundary obtained by parametric high-frequency direction extraction to obtain three-dimensional volume data with enhanced directional boundary. The 3D volume data enhanced by directional boundaries is decomposed into a sequence of 2D slices along the depth dimension, and the inter-slice context modeling of the 2D slice sequence is performed to aggregate the information of adjacent slices, generating a target feature body containing 3D spatial context information. Based on the target feature body, high-level semantic cue features are generated by dynamically inferring discrete token combinations through a fixed discrete token vocabulary and an attribute predictor. The target feature volume is input into the orientation-aware dual-domain enhancement branch, which includes a parallel frequency domain enhancement branch and a spatial domain multi-scale orientation enhancement branch. This branch is used to simultaneously enhance the features of the target feature volume in both the spatial and frequency domains to obtain the enhanced features. The enhanced features and high-level semantic cue features are input into a task-structured sparse hybrid expert decoder, which outputs a segmentation probability map of three nested tumor regions for each voxel. The nested tumor regions include the overall tumor region, the tumor core region, and the enhanced tumor region.

[0004] In one implementation, fusing three-dimensional volume data with prior information about directional boundaries obtained through parametric high-frequency direction extraction to obtain three-dimensional volume data with enhanced directional boundaries includes: The modal average volume is obtained by averaging the three-dimensional volume data of multimodal magnetic resonance imaging along the modal dimension. A parameterless fixed central difference kernel along each coordinate axis is used to convolve the modal average volume to extract the high-frequency response map in each direction; The absolute values ​​of the high-frequency response maps in each direction are taken and stitched along the channel dimension. Then, the spatial dimension of the stitched results is normalized sample by sample and channel by channel to form a directional prior stack. By stitching the directional prior stack with the original multimodal magnetic resonance imaging 3D volume data along the channel dimension, we obtain 3D volume data with enhanced directional boundaries and an increased number of channels.

[0005] In one implementation, generating a target feature body containing three-dimensional spatial context information by aggregating adjacent slice information through inter-slice context modeling of a two-dimensional slice sequence includes: The label embedding vector of each two-dimensional slice in the two-dimensional slice sequence is extracted by a two-dimensional image encoder; For each target slice, its corresponding label embedding vector is used as the query, and the feature concatenation result of all two-dimensional slices in the adjacent slice window is used as the key and value. The information of adjacent slices is aggregated through a cross-attention mechanism to obtain the target features after fusing the three-dimensional context. The target features after fusing all two-dimensional slices are organized in depth order to form a target feature body with three-dimensional spatial context information.

[0006] In one implementation, the method for generating high-level semantic prompts includes: A discrete token vocabulary is predefined, which includes region tokens for identifying segmentation tasks, existence tokens for indicating the presence of enhanced tumor regions, fragmentation level tokens for quantifying the spatial dispersion of enhanced tumor regions, and scale level tokens for quantifying the volume of enhanced tumor regions. A global volume descriptor is obtained by aggregating from the target feature volume. The global volume descriptor is input into the attribute predictor. The attribute predictor outputs the predicted concept label. The concept label includes at least the existence category, fragmentation level category and scale level category of the enhanced tumor region. The predicted concept labels are mapped to corresponding discrete token embedding vectors via a discrete token vocabulary index, and then the discrete token embedding vectors are converted into the final high-level semantic cue features by a specified cue encoder.

[0007] In one implementation, the method for generating the enhanced features includes: By using a two-level feature calibration fusion gating, the target feature volume is adaptively fused with the multi-scale orientation perception enhancement features output by the multi-scale orientation enhancement branch in the spatial domain in turn to obtain the fusion result. The fusion result is then adaptively fused with the spectral enhancement features output from the frequency domain enhancement branch to obtain the final enhanced features; Among them, the frequency domain enhancement branch transforms the target feature volume to the frequency domain through three-dimensional Fourier transform, uses cross-channel shared learnable radial gating to adaptively modulate the amplitude spectrum, and then reconstructs it through inverse Fourier transform to obtain the spectrum enhancement feature. The spatial domain multi-scale orientation enhancement branch uses directional deep convolution kernels of multiple scales to extract features along the three orthogonal directions of x, y, and z, respectively. It then adaptively weights and fuses the feature maps of each scale and direction by using attention weights based on global feature learning to obtain multi-scale orientation perception enhancement features.

[0008] In one implementation, the enhanced features and high-level semantic cue features are jointly input into a task-structured sparse hybrid expert decoder, and the output segmentation probability map of three nested tumor regions corresponding to each voxel includes: Feature linear modulation is employed, and the enhanced features are modulated using high-level semantic cue features to obtain modulated features; The global volume descriptor and high-level semantic cue features are input into the gating network, and the routing weights of a preset number of experts are generated by softmax normalization, and the sum of the routing weights of each expert is 1. Based on the routing weights, a sparse top-k routing mechanism is used to select the k experts with the highest weights for activation, while the remaining experts are skipped, resulting in the selected k lightweight 3D convolutional expert decoders. The modulated features are input into k selected lightweight 3D convolutional expert decoders, and each expert decoder outputs three-channel logits corresponding to the overall tumor region, the tumor core region, and the enhanced tumor region. The output logits of each selected expert are weighted and fused according to their corresponding routing weights to obtain the final regional logits. The three channels are respectively denoted as the overall tumor region logits, the tumor core region logits, and the enhanced tumor region logits.

[0009] In one implementation, it further includes: The original predicted value of the enhanced tumor region output by the sparse hybrid expert decoder is converted into an output probability value by the sigmoid function, and then gating is performed according to the probability of the existence of the enhanced tumor region output by the attribute predictor. The gating process includes forcibly setting the output probability value of the enhanced tumor region to zero when the probability of the existence of the enhanced tumor region is lower than a preset threshold.

[0010] In one implementation, it further includes: Obtain multiple outputs from the model prediction, including: segmentation probability maps of three nested tumor regions, enhanced tumor region existence probability output by the attribute predictor, concept labels for fragmentation-level and scale-level categories output by the attribute predictor, and expert routing weights output by the gating network. Obtain the corresponding ground truth labels, including: three region ground truth masks obtained from the gold standard segmentation mask, existence ground truth labels calculated from the gold standard enhanced tumor mask, fragmentation level ground truth labels, and scale level ground truth labels; The overall loss function value is calculated based on the multiple outputs predicted by the model and the true labels, and the model parameters are updated through backpropagation.

[0011] Secondly, embodiments of the present invention provide an electronic device comprising a memory and a processor. The memory and the processor communicate with each other via an internal connection path. The memory stores instructions, and the processor executes the instructions stored in the memory. When the processor executes the instructions stored in the memory, it causes the processor to perform the method described in any of the above embodiments.

[0012] Thirdly, embodiments of the present invention provide a computer-readable storage medium that stores a computer program, wherein when the computer program is run on a computer, the methods in any of the above-described embodiments are executed.

[0013] The advantages or beneficial effects of the above technical solutions include at least the following: This invention effectively enhances the boundary perception capability of tumor regions, especially enhanced tumors (ET) regions with blurred boundaries, by fusing parameter-free high-frequency directional prior information into multimodal MRI 3D volumetric data. By decomposing the 3D volumetric data into 2D slice sequences and performing inter-slice context modeling, the advantages of 2D encoding are maintained while restoring 3D spatial continuity, ensuring consistent segmentation results in the depth direction. A fixed discrete token vocabulary and an attribute predictor that dynamically infers token combinations based on input features generate high-level semantic cues encoding tumor morphological attributes, enabling the segmentation process to adaptively adjust according to sample characteristics. Dual-domain feature enhancement, combining spatial multi-scale directional enhancement and frequency spectral modulation, improves the model's sensitivity to small ET regions and complex boundaries. A sparse hybrid expert decoding structure based on conditional routing dynamically selects a subset of experts for weighted fusion based on sample features, achieving efficient inference while maintaining model capacity. Overall, this invention significantly improves the boundary accuracy and small target recognition capability of glioma subregion segmentation, effectively suppresses clinically sensitive false positive errors, and improves image segmentation accuracy.

[0014] The above overview is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of the invention will become readily apparent from the accompanying drawings and the following detailed description. Attached Figure Description

[0015] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments disclosed in the invention and should not be construed as limiting the scope of the invention.

[0016] Figure 1 This is a diagram illustrating the overall architecture of the GliomaSAM3D-MoE brain tumor image segmentation model of the present invention. Figure 2 This is a flowchart illustrating the three-dimensional glioma image segmentation method of the present invention; Figure 3 The results of comparative experiments based on the BraTS 2023 validation set are for this invention. Figure 4 The results of comparative experiments based on the BraTS 2018 validation set are for this invention. Figure 5 This is a qualitative comparison diagram of representative cases of BraTS in this invention; Figure 6 This invention enhances the performance results of the tumor (ET) presence classification task; Figure 7 These are the ablation experiment results of this invention; Figure 8 This is a comparison diagram of the ET missing sample experiment of the present invention; Figure 9 This is a comparison diagram of the effects of the ET gating mechanism of the present invention; Figure 10 This is a comparison chart showing how the Dice index is used to evaluate boundary accuracy in this invention. Figure 11 This is a structural block diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0017] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of the invention. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0018] Example 1 This embodiment provides a three-dimensional glioma image segmentation method. Based on the GliomaSAM3D-MoE brain tumor image segmentation model, this method automatically and accurately segments nested glioma regions from multimodal MRI volumes, and meets the clinical need for fine three-dimensional segmentation through voxel-level output.

[0019] like Figure 1 As shown, the goal of the GliomaSAM3D-MoE brain tumor image segmentation model in this embodiment is to automatically segment three key subregions of glioma from multi-parameter magnetic resonance imaging data, corresponding to the whole tumor region (WT), the tumor core region (TC), and the enhancing tumor region (ET). These three regions must spatially satisfy a nested relationship where the enhancing tumor is contained within the tumor core, and the tumor core is contained within the whole tumor (ET≤TC≤WT).

[0020] To achieve the above prediction goals, such as Figure 2 As shown, the three-dimensional glioma image segmentation method used in this embodiment includes: Step S1: Obtain the three-dimensional volume data of the multimodal magnetic resonance imaging (MRI) to be segmented, and fuse the three-dimensional volume data with the prior information of the directional boundary obtained by parametric high-frequency directional extraction to obtain the three-dimensional volume data with enhanced directional boundary.

[0021] Acquiring three-dimensional volume data X∈R for multimodal magnetic resonance imaging C×H×W×D .

[0022] It should be explained that R represents that each basic unit (i.e., the value of each voxel) constituting data X is a real number; variable X is a four-dimensional array composed of real numbers, C is the modality number, C=4 in this embodiment, H is the height of the three-dimensional volume data, W is the width of the three-dimensional volume data, and D is the depth of the three-dimensional volume data.

[0023] To enhance sensitivity to boundary cues and small fragments, this embodiment employs a parameter-free directional high-frequency prior injection mechanism, the specific steps of which include: Step S11: Average the three-dimensional volume data of multimodal magnetic resonance imaging along the modal dimension to obtain the modal average volume X∈R. H×W×D : .

[0024] The purpose of calculating the modal average volume in this embodiment is that, although different MRI modalities (such as T1, T1ce, T2, and FLAIR) highlight different tissue characteristics, they all share boundary information of anatomical structures. By averaging across modalities, the noise and artifacts specific to each modality can be effectively suppressed, while enhancing the true anatomical boundaries shared by all modalities. This results in a stable and pure edge prior, providing a more reliable input for subsequent high-frequency directional extraction.

[0025] Step S12: Using parameterless fixed central difference kernels along each coordinate axis, convolve the modal average volume respectively to extract the high-frequency response map in each direction.

[0026] A fixed 3D central difference kernel (kernel size 3, fill 1, and coefficients for each axis) is used along the x, y, and z coordinate axes. [1,0,1] / 2), respectively for modal average volume Convolution is performed to obtain high-frequency response maps in three directions: ; in, d represents the gradient operator along direction d, specifically implemented as a parameterless fixed central difference kernel (size 3, padding 1, coefficients [ ] ). [1, 0, 1] / 2), used to calculate the spatial gradient of three-dimensional data in the x, y, or z directions; G d The high-frequency response diagram represents the direction d, where d∈{x, y, z} represents three orthogonal spatial directions.

[0027] The design principle of the central difference kernel is to approximate the spatial gradient by calculating the gray-level difference between adjacent voxels. Positive values ​​represent the rate of change along the positive direction, and negative values ​​represent the rate of change along the negative direction. The coefficient is divided by 2 to maintain the scale consistency of the gradient magnitude. Since this kernel is fixed and parameter-free, it does not introduce additional learning burden and can accurately capture high-frequency boundary information in each direction. In this way, spatial gradient maps corresponding to the x, y, and z directions can be obtained, and these gradient maps reflect the intensity variation characteristics of the tumor boundary in different directions.

[0028] Step S13: Take the absolute value of the high-frequency response map in each direction and stitch it along the channel dimension. Then, normalize the spatial dimension of the stitched result sample by sample and channel by channel to form a directional prior stack.

[0029] The absolute values ​​of |Gx|, |Gy|, and |Gz| are taken for the high-frequency response maps in each direction to eliminate the influence of the direction sign and retain only the boundary intensity information. Then, these three values ​​are concatenated along the channel dimension to obtain a three-channel direction feature map. To eliminate the influence of differences in grayscale distribution between different samples, the concatenated feature map is normalized by z-score for each sample and each channel. .

[0030] The normalization operation adjusts the feature values ​​of each channel to zero mean and unit variance, enabling the subsequent network to utilize boundary information in all three directions equally and avoiding training instability caused by differences in numerical ranges. The resulting directional prior stack H encodes the boundary strengths in the three orthogonal directions, providing a clear directional boundary prior for brain tumor image segmentation models.

[0031] Step S14: The directional prior stack is stitched together with the original three-dimensional volume data of multimodal magnetic resonance imaging along the channel dimension to obtain three-dimensional volume data with enhanced directional boundaries and increased channel number.

[0032] The directional prior stack H is stitched together with the original multimodal MRI volume X along the channel dimension to form directional boundary-enhanced 3D volume data with an increased number of channels: X + =Concat(X, H)∈R (C+3)×H×W×D ; The enhanced volumetric data retains the rich texture information of the original multimodal model while explicitly introducing a directional boundary prior. Since the entire process requires no learnable parameters, it does not increase model complexity and effectively guides the network to focus on the boundary location of the tumor region. In particular, for regions with fragmented edges and irregular shapes, such as enhanced tumors (ET), this directional prior can significantly improve the accuracy of boundary localization.

[0033] Step S2: Decompose the 3D volume data enhanced by directional boundaries into a sequence of 2D slices along the depth dimension, and aggregate the information of adjacent slices by performing inter-slice context modeling on the 2D slice sequence to generate a target feature volume containing 3D spatial context information.

[0034] The three-dimensional volume data X after directional boundary enhancement + ∈R (C+3)×H×W×D Decomposed along the depth dimension (usually the axial direction) into D two-dimensional slices, forming a two-dimensional slice sequence. ,in ∈R (C+3)×H×W Let represent the t-th slice. This decomposition operation transforms the three-dimensional problem into a two-dimensional sequence problem, allowing full utilization of a mature two-dimensional image encoder while preserving the natural ordering relationships between slices.

[0035] In this embodiment, inter-slice context modeling to aggregate information from adjacent slices specifically includes: Step S21: Extract the label embedding vector of each two-dimensional slice in the two-dimensional slice sequence using a two-dimensional image encoder.

[0036] Each two-dimensional slice Input a two-dimensional image encoder E img (·), This encoder can use a pre-trained SAM architecture (such as ViT) to generate the label embedding vectors corresponding to the slices: F t =E img ( )∈R N×d ; Where N is the number of tags and d is the tag dimension.

[0037] The purpose of this step is to extract rich local semantic features from each slice, providing a foundation for subsequent 3D context modeling. Because the encoder is pre-trained on a large number of 2D natural images, it can effectively capture key information such as texture and shape within the slices, and can be adapted to medical image features through fine-tuning.

[0038] Step S22: For each target slice, use its corresponding label embedding vector as the query, and use the feature concatenation result of all two-dimensional slices in the adjacent slice window as the key and value. Aggregate the information of adjacent slices through the cross-attention mechanism to obtain the target features after fusing the three-dimensional context.

[0039] Since independently processed slices lack depth-dimensional correlation, this embodiment employs a memorized cross-attention mechanism to inject 3D context, aggregating information from adjacent slices and endowing the feature volume with 3D perception capabilities. Specifically, for each target slice t, a short window is defined: ; This involves taking the K consecutive slices preceding the current slice as the context source (K is the window size, defaulted to 4). The target slice's labeled embedding vector F... t As a query, the embedded vectors of all slices within the window are arranged sequentially according to slice order and concatenated along the dimension of the number of labels to form a larger feature matrix (F). t-K F t-1 (), used as keys and values ​​for cross-attention, to obtain target features after fusing the 3D context through cross-attention computation: ; This operation allows each slice to "read" relevant information from neighboring slices. For example, when the tumor boundary in a slice is unclear, it can be supplemented by more defined boundary information from adjacent slices, thereby enhancing the continuity and consistency of the segmentation. Using window constraints instead of global sequences can significantly reduce computational complexity, while short windows are sufficient to capture the continuity of local anatomical structures, which is consistent with the high correlation between adjacent slices in medical images.

[0040] Furthermore, to eliminate the bias that might be introduced by a fixed traversal direction, the aggregation direction is randomly selected during the training phase: a forward order (i.e., taking the first K slices of the window) is used with a 50% probability, and a backward order (i.e., taking the last K slices of the window) is used with a 50% probability. This random reversal strategy forces the model to be independent of a specific slice order, enhancing its robustness to changes in direction. During the inference phase, to maintain efficiency and avoid randomness, a fixed forward order is used for aggregation.

[0041] Step S23: Organize the target features after fusing all two-dimensional slices in depth order to form a target feature body with three-dimensional spatial context information.

[0042] Through the above steps, the 2D slice feature sequence is efficiently converted into a feature body with 3D contextual information, providing a rich representation that simultaneously includes intra-slice details and inter-slice continuity. This operation has a small overall parameter count, is easy to train, and can be seamlessly integrated into the backend of existing 2D encoders, significantly improving 3D segmentation performance.

[0043] Step S3: Based on the target feature body, dynamically infer the combination of discrete tokens through a fixed discrete token vocabulary and an attribute predictor to generate high-level semantic cue features.

[0044] To provide high-level semantic guidance for brain tumor image segmentation models without human intervention, a dynamic cueing mechanism based on discrete concept tokens is proposed. This mechanism uses a fixed discrete token vocabulary and a learnable attribute predictor to automatically infer the morphological attributes of the current sample based on the features of the input data during testing, and encodes these attributes as cue features to inject into the subsequent decoder.

[0045] Specifically, methods for generating high-level semantic prompts include: Step S31: Predefine a discrete token vocabulary.

[0046] To avoid uncertainty during reasoning and to ensure that the prompt information strictly corresponds to the pathological features of medical images, this embodiment predefines a fixed discrete token vocabulary. The vocabulary contains 13 tokens, divided into four types: region tokens for identifying segmentation tasks, presence tokens for indicating the presence of enhanced tumor regions, fragmentation level tokens for quantifying the spatial dispersion of enhanced tumor regions, and scale level tokens for quantifying the volume of enhanced tumor regions.

[0047] The 13 tokens include 3 region tokens: WT represents the entire tumor region, TC represents the tumor core region, and ET represents the enhancing tumor region. There are 2 existence tokens: ET_PRESENT indicates that the currently input 3D MRI volume contains an enhancing tumor region, and ET_ABSENT indicates that the currently input 3D MRI volume does not contain an enhancing tumor region. There are 4 fragmentation level tokens: FRAG_BIN... i (i∈{0, 1, 2, 3}), the spatial dispersion (i.e., fragmentation) of the enhanced tumor region is quantified through four levels (0 to 3), reflecting the discreteness of tumor invasion. Four size-level tokens, namely SCALE_BIN j (j∈{0,1,2,3}), the volume of the enhanced tumor region is quantified through four levels (0 to 3). The different levels are based on the total number of voxels in the ET region, reflecting the size of the tumor.

[0048] To generate the supervision signal corresponding to the above token, the gold standard ET mask Y from the training samples is used. ET ∈{0,1} H×W×D The calculation involves three concept labels, including the existence label y. pres Fragmentation level tag y frag and size grade label y scale The specific calculation process is as follows: (1) Existence of EF (y) pres ) Determine whether a sample contains enhanced tumor regions by counting the total number of voxels in the ET mask: ; Where v represents the position of each voxel in three-dimensional space, Y ET (v) represents the value of the gold standard enhanced tumor (ET) mask at location v.

[0049] The binary label distinguishes between "tumors with enhancement" and "tumors without enhancement." It iterates through all voxels, counting the total number of voxels belonging to the enhanced tumor region. If the total number is greater than 0, the sample is determined to contain an enhanced tumor (y). pres =1), otherwise it is determined that it does not contain (y). pres =0).

[0050] (2) Fragmentation level (y) frag ) Fragmentation level labels are determined based on the number of three-dimensional connected components in the enhanced tumor region, by using the number of connected components N. cc The results are compared with a preset threshold range and mapped to a preset number of discrete levels. Specifically, a 3D connected component analysis is performed on the ET mask using 26 connectivity parameters, and the number of connected components N is counted. cc Based on the distribution characteristics of the ET region in the BraTS 2023 training set, the fragmentation level is quantified into four levels: ; This attribute reflects the degree of spatial dispersion of the tumor region; tumors with a high degree of fragmentation often have complex boundaries and strong invasiveness.

[0051] (3) Scale level (y) scale ) Scale rating labels are based on the total number of voxels N in the enhanced tumor region. vox This is determined by comparing the total number of voxels with a preset threshold range, mapping it to a preset number of discrete levels. Specifically, this is done by counting the total number of voxels N in the ET region. vox Their size is also quantified and divided into four levels: ; The above thresholds were selected based on the quartile distribution of the ET volume in the BraTS 2023 training set to ensure a balanced number of samples at each level.

[0052] When the presence label of the enhanced tumor region is "absent", both the fragmentation level label and the scale level label are set to the lowest value corresponding to the specified level by default, i.e., y. pres =0, y is defined as 0 frag=0 and y scale =0 corresponds to the lowest level.

[0053] Step S32: Aggregate the global volume descriptor from the target feature volume, input the global volume descriptor into the attribute predictor, and output the predicted concept label through the attribute predictor. The concept label includes at least the existence category, fragmentation level category and scale level category of the enhanced tumor region.

[0054] During the reasoning phase, the aforementioned concept labels are generated by the attribute predictor h. attr (·) Dynamic prediction based on input features avoids inconsistencies caused by using the gold standard label during the training-testing phase. The specific process is as follows: The global volume descriptor z is obtained by aggregating the target feature volumes: ; Pool(·) represents mean pooling of the slice dimension and the token dimension to obtain a global feature vector of fixed length; The target feature body.

[0055] Input the global feature vector into the attribute predictor h attr (·), This predictor employs a two-layer multilayer perceptron structure, with the hidden layer dimension being the same as the token dimension d, using GELU as the activation function, and incorporating a dropout layer to prevent overfitting. Its output is divided into three branches: Existence prediction branch: Outputs binary classification logic values, which are then converted into existence probabilities π using a sigmoid function. ET ∈[0,1]; Fragmentation level prediction branch: Outputs four-category logical values; Scale-level prediction branch: Outputs four-class logical values.

[0056] The prediction process can be represented as: ; Where z is the global volume descriptor; The predicted concept labels include fragmentation level and scale level; π ET To enhance the probability of tumor presence, i.e., the probability value of presence output by the attribute predictor, the presence prediction is binarized using a default threshold of 0.5. ET If the value is greater than 0.5, it is determined that ET exists and the ET_PRESENT token is selected; otherwise, the ET_ABSENT token is selected.

[0057] Step S33: The predicted concept labels are mapped to corresponding discrete token embedding vectors via the discrete token vocabulary index, and the discrete token embedding vectors are converted into the final high-level semantic cue features through the specified cue encoder.

[0058] Based on the predicted concept labels, query the corresponding token identifier in the discrete token vocabulary. Here, a concept label is actually one or more category indexes. For example, if the predicted fragmentation level is FRAG_BIN2, then its corresponding token identifier is the index value corresponding to FRAG_BIN2 in the vocabulary.

[0059] Based on the predicted concept label and the corresponding token identifier, the corresponding row vector is extracted from the learnable embedding matrix through a lookup operation (i.e., row lookup by index). This yields the discrete token embedding vector for the concept label, which incorporates high-level semantic information of the corresponding concept. The embedding matrix has a dimension of |V|×d, where |V| is the size of the discrete token vocabulary (13 in this example), and d is the dimension of the embedding vector (e.g., 256). Each row of the embedding matrix stores a fixed-length vector, corresponding one-to-one with each token in the discrete token vocabulary. These vector parameters are optimized end-to-end along with other parts of the network during training.

[0060] These tokens are then embedded into a lightweight prompt encoder E. prm (·), thus obtaining the final high-level semantic cue features: ; in, The concept labels to be predicted, i.e., the attribute predictor h attr (·) outputs the discrete category prediction results; Embed(·) is the embedding layer, which maps discrete category indices to continuous, learnable dense vectors.

[0061] The high-level semantic cue feature p will be fused with the features in the decoder to guide the decoder to generate a segmentation result that is more consistent with the morphological attributes of the current sample.

[0062] Furthermore, to prevent the model from incorrectly predicting enhanced tumor regions on samples where ET does not exist, an existence gating mechanism is introduced. Specifically, let the logits of the enhanced tumor region output by the decoder be I. ET ∈R H×W×D After converting it into an output probability value using the sigmoid function, it is compared with the enhanced tumor presence probability π. ET Perform gating operation: when π ETWhen the value is below the threshold, the output probability value of the ET region is forcibly reset to zero or a very low value, thereby effectively suppressing false positive predictions.

[0063] Through the above design, discrete concept tokens are combined with attribute predictors to achieve the function of dynamically generating semantic prompts based on input data during testing. These prompts include both task-level region identifiers and embedded sample-specific morphological attributes, providing high-level semantic guidance for subsequent direction-aware dual-domain enhancement and sparse MoE decoding, significantly improving the segmentation accuracy of enhanced tumor regions.

[0064] Step S4: Input the target feature volume into the orientation-aware dual-domain enhancement branch. The orientation-aware dual-domain enhancement branch includes a parallel frequency domain enhancement branch and a spatial domain multi-scale orientation enhancement branch, which are used to simultaneously enhance the features of the target feature volume in the spatial domain and frequency domain to obtain the enhanced features.

[0065] To simultaneously enhance feature representations in both the spatial and frequency domains, particularly to improve the perception of tumor boundary directionality and multi-scale morphology, this embodiment proposes a direction-aware dual-domain enhancement module. This module comprises two parallel processing branches: a frequency domain enhancement branch and a spatial domain multi-scale direction enhancement branch.

[0066] The frequency domain enhancement branch transforms the target feature volume to the frequency domain using a three-dimensional Fourier transform, adaptively modulates the amplitude spectrum using cross-channel shared learnable radial gating, and then reconstructs it using an inverse Fourier transform to obtain the spectral enhancement feature. Specifically, in this embodiment, the target feature volume is set as U∈R. C×D×H′×W ′ (Where C is the number of channels, and D, H′, W′ are the depth, height, and width, respectively). First, perform a three-dimensional Fourier transform independently on each channel to obtain the amplitude spectrum A and the phase spectrum Φ: ; The amplitude spectrum A reflects the intensity of each frequency component, while the phase spectrum Φ contains spatial structure information. To achieve selective frequency enhancement, the radial frequency magnitude r is discretized into B frequency bands (B=16 in this embodiment), and a cross-channel shared learnable radial gating u is introduced. θ (r)∈[0,1], this gating is generated by a set of learnable logit vectors using the sigmoid function. This is achieved by combining the amplitude spectrum A with the radially gated u. θ (r) Element-by-element multiplication yields the modulated amplitude spectrum: ; This operation enables the model to automatically learn which frequency components are more important (e.g., high-frequency boundaries correspond to large gradients) based on the training task and assign them higher weights. Subsequently, the modulated amplitude spectrum A′ is combined with the original phase spectrum Φ, and reconstructed back into the spatial domain through a three-dimensional inverse Fourier transform to obtain the spectral enhancement features. ; This feature, while preserving the original spatial structure, enhances task-related frequency information, providing richer boundary cues for subsequent segmentation.

[0067] Brain tumors exhibit diverse morphologies, with their boundaries displaying varying characteristics in different directions and varying scales. To enhance the model's ability to perceive directionality and multi-scale structures, a spatial domain multi-scale orientation enhancement branch is proposed. This branch employs directional deep convolutional kernels at multiple scales to extract features along the three orthogonal directions (x, y, z). Furthermore, it adaptively weights and fuses the feature maps at each scale and direction using attention weights based on global feature learning, resulting in multi-scale orientation perception enhancement features. Specifically, The target feature volume U is considered as a feature distribution on a 3D mesh. For multiple preset scales k∈K (K={3, 5} in this embodiment) and three orthogonal directions d∈{x, y, z}, a depthwise convolution kernel with specific directions is used to perform convolution operations on U: In the x-direction, the convolution kernel size is k×1×1, which is a single point along the height and depth directions and expands only along the width direction, capturing the context in the horizontal direction; In the y-direction, the convolution kernel size is 1×k×1, capturing the context in the vertical direction; In the z-direction, the convolution kernel size is 1×1×k, capturing the context in the depth direction.

[0068] Through the above operations, feature maps U at various scales and in various directions are obtained. k，d The feature maps enhance the modeling of local relationships in specific directions: .

[0069] To adaptively fuse these multi-scale directional features, the original target feature volume U is first globally pooled to obtain a global descriptor; this descriptor is then input into a multilayer perceptron, which outputs the original weights for each scale and direction. These weights are then normalized using a softmax function to obtain the attention weights. ; The attention weight a k，d This reflects the importance of the current sample to the final segmentation at different scales and orientations. Finally, the feature maps are summed with their corresponding weights to obtain the multi-scale orientation-aware enhancement features: ; This branch, while maintaining directional selectivity, achieves adaptive fusion of multi-scale information, significantly improving the ability to model irregular tumor boundaries.

[0070] Furthermore, in order to fully utilize the complementary advantages of the original features, multi-scale directional sensing features, and spectral enhancement features, this embodiment adopts a lightweight feature calibration fusion mechanism. This mechanism uses two-level feature calibration fusion gating to adaptively fuse the target feature volume with the multi-scale directional sensing enhancement features output from the spatial domain multi-scale directional enhancement branch to obtain the fusion result, rather than simply splicing or adding them together. Then, the fusion result is adaptively fused with the spectral enhancement features output from the frequency domain enhancement branch to obtain the final enhanced features.

[0071] Specifically: First, a first-level fusion is performed: the original target feature volume U is combined with the multi-scale orientation perception enhancement feature U. msda The concatenation is performed along the channel dimension, and the concatenation result is globally pooled to obtain pooled features. These pooled features are then input into a multilayer perceptron and activated by the sigmoid function to obtain the gating coefficient η. ; The gating coefficient η ranges from [0, 1], representing the proportion of the original target feature volume U retained in the fusion, while 1-η represents the contribution proportion of the multi-scale directional sensing features. The fused features are: ; This mechanism enables the network to dynamically adjust the fusion weights of the two features based on the characteristics of the input samples, thereby achieving adaptive calibration.

[0072] Then, a second-level fusion is performed: using the same gating mechanism, the first-level fusion result U... fuse With spectral enhancement feature U spec Adaptive weighted fusion is performed to obtain the final enhanced feature U. final This feature integrates multi-scale directional information in the spatial domain and spectral enhancement information in the frequency domain, providing rich and high-quality feature representations for subsequent sparse hybrid expert decoders.

[0073] To further enhance the model's robustness to differences in acquisition devices, image noise, and style variations, a Fourier amplitude mixing enhancement strategy is introduced during the training phase. This method is based on the observation that the amplitude spectrum of an image primarily carries style and texture information, while the phase spectrum primarily carries structure and contour information.

[0074] For two randomly paired training samples (a, b), calculate their amplitude spectra A. (a) and A (b)And retain the phase spectrum Φ of sample a (a) Generate a mixed amplitude spectrum: ; The mixing coefficients α ~ Beta(β, β) are set to β = 1.0, indicating a uniform distribution, and all channels within the same volume share the same α value. The mixed amplitude spectrum is combined with the phase spectrum of sample a, and the enhanced sample is reconstructed using inverse Fourier transform. ; This operation introduces diverse texture variations without disrupting the main structure of the sample, forcing the model to learn phase-related structural invariances, thereby improving generalization ability. This enhancement is used only during the training phase and is not mixed during inference.

[0075] Through the aforementioned collaborative work, the orientation-aware dual-domain enhancement module provides the segmentation model with feature representations that combine spatial orientation sensitivity and frequency selectivity, laying a solid foundation for the final high-precision voxel-level segmentation.

[0076] Step S5: Input the enhanced features and high-level semantic cue features into a task-structured sparse hybrid expert decoder, and output the segmentation probability map of three nested tumor regions for each voxel. The nested tumor regions include the overall tumor region, the tumor core region, and the enhanced tumor region.

[0077] To achieve efficient inference while maintaining model capacity, and to fully leverage the guiding role of high-level semantic cue features in the segmentation process, this embodiment employs a task-structured sparse Mixture-of-Experts (MoE) decoder. This decoder uses a conditional routing mechanism to dynamically select the most suitable subset of experts for activation based on the global features and semantic cue features of each sample, achieving adaptive combination of expert capabilities.

[0078] In this embodiment, M lightweight 3D convolutional expert decoders are provided. In this embodiment, M=5. Each expert decoder uses the same lightweight architecture to ensure computational efficiency. First layer: 3×3×3 three-dimensional convolution, with d output channels, followed by group normalization (GroupNorm) and GELU activation function; Second layer: 3×3×3 three-dimensional convolution, with the number of output channels kept at d, followed by group normalization and GELU activation function; The third layer: a 1×1×1 three-dimensional convolution that projects the number of channels to 3, corresponding to the segmentation logits of the three regions: the overall tumor (WT), the tumor core (TC), and the enhanced tumor (ET).

[0079] This structural design ensures that each expert possesses complete 3D segmentation capabilities, but different experts automatically develop functional differentiations during training through a routing mechanism. For example, some experts excel at fine boundary characterization, while others excel at suppressing false positive predictions. This functional differentiation is emergent rather than pre-specified.

[0080] To dynamically select experts based on the characteristics of the input samples, a gating network G(·) is designed, whose input consists of two parts: Global volume descriptor z: A global feature obtained from pooling in the feature volume, reflecting the overall properties of the sample; High-level semantic cue feature p: A cue vector generated by the concept cue module that encodes enhanced tumor morphological attributes.

[0081] The global volume descriptor z and the high-level semantic cue features p are concatenated or fused and input into a gating network G(·). The gating network G(·) uses a multilayer perceptron to process the concatenated or fused features, mapping them to an M-dimensional original weight vector. This vector is then normalized using a softmax function to obtain the routing weights y∈R for each expert. M ,satisfy: .

[0082] This weight reflects the degree to which the current sample depends on each expert. Since the routing weights depend on both global visual features and semantic cues, they can adaptively select the most suitable expert combination based on the presence, fragmentation, and scale of the tumor.

[0083] To further enhance the guiding role of high-level semantic cues in the decoding process, Feature Linear Modulation (FiLM) is used to inject the cue feature p into the feature volume before inputting the enhanced features into each expert decoder. Specifically, scaling coefficient γ and offset coefficient β are generated through p, and an affine transformation is performed on the feature volume to obtain the modulated features: ; Where U represents the enhanced input feature, The modulated features are then input into each expert decoder. This operation allows the cue information to directly adjust the activation intensity of the features, achieving fine-grained conditional control.

[0084] To achieve efficient inference and avoid computational redundancy caused by activating all experts, a sparse top-k routing mechanism is adopted. Only the k experts with the highest routing weights are activated (k=2 in this embodiment), and the calculation of the remaining experts is skipped. The output logits of the selected experts are weighted and fused according to their corresponding routing weights to obtain the final region logits: ; Among them, L (m)Let L be the three-channel logits output by the m-th expert, and L be the final logits after fusion. The three channels are denoted as {I...} WT I TC I ET This corresponds to three regions: the overall tumor, the tumor core, and the enhanced tumor.

[0085] The sparse routing mechanism has the following advantages: (1) Only a small number of experts are activated during inference, which greatly reduces the computational cost compared with full expert activation; (2) Each sample can customize the expert combination according to its own characteristics to achieve adaptive inference; (3) Each expert naturally differentiates different functional preferences during training, which improves the overall expressive power of the model.

[0086] In sparse MoE training, a "routing collapse" phenomenon can easily occur, where a few experts are frequently activated while most experts fail to receive effective training. To avoid this problem, a load balancing regularization term is introduced. For a small set of experts (M=5), a simple uniformity penalty can achieve a stable effect: a regularization constraint is applied to the average route weight to encourage balanced utilization of each expert. Specifically, the average route weight of each expert is calculated within a training batch. m The variance is calculated as an additional loss term in the total loss function, either by calculating the KL divergence with a uniform distribution or by directly applying a variance penalty. This regularization term ensures that all experts are adequately trained, maintaining the overall capacity of the model.

[0087] Through the above design, the task-structured sparse hybrid expert decoder achieves the following technical effects: Dynamic expert portfolio: Based on the global features and semantic cues of the samples, the most suitable subset of experts is customized for each sample; Efficient reasoning: The sparse activation mechanism significantly reduces computational overhead, making the model more feasible in practical applications; Expert functional differentiation: The routing mechanism encourages each expert to spontaneously learn different processing preferences, thereby improving the overall expressive power of the model; Hint-guided decoding: High-level semantic cues are incorporated into features through FiLM modulation, enabling semantic information to accurately guide the segmentation process.

[0088] Finally, the sparse hybrid expert decoder outputs voxel-level three-region segmentation logits, which are then processed (such as sigmoid activation and existence gating) to obtain the final segmentation probability map, completing the end-to-end fully automated processing from multimodal MRI input to three-dimensional brain tumor segmentation.

[0089] Furthermore, to train the entire GliomaSAM3D-MoE brain tumor image segmentation model end-to-end, a multi-task joint loss function is used to supervise the final segmentation result, while also supervising the intermediate concept attribute predictions. Hierarchical consistency constraints and hybrid expert load balancing regularization are introduced to improve the model's accuracy, robustness, and training stability. This allows the model to segment the input multimodal MRI volume data automatically during the inference phase, without any manual intervention. In this embodiment, the overall loss function is defined as follows: ; Among them, L seg The segmentation loss is used to supervise the prediction results of the three tumor regions (whole tumor WT, tumor core TC, and enhanced tumor ET) output by the decoder. Considering the class imbalance problem common in medical image segmentation, this embodiment combines Dice loss and binary cross-entropy loss, and introduces optional Focal loss for the enhanced tumor region to further focus hard-to-separate samples. The specific definitions are as follows: ; in: The Dice loss for region r is used to maximize the overlap between the predicted segmented region and the gold standard mask; The binary cross-entropy loss for region r provides dense supervision from a pixel-level perspective; To enhance the Focal loss in the tumor region, the weight of easily distinguishable samples is reduced, making the model pay more attention to enhanced tumor regions with blurred boundaries or complex shapes, thereby improving the sensitivity to small fragmented regions. λ r The weights for the Dice loss in each region can be adjusted based on the sample size or importance of each region; λ BCE and λ Focal It serves as the global weight, used to balance the magnitudes of different loss terms.

[0090] This combined loss ensures that the model is effectively supervised at both the global structure (Dice) and local detail (BCE) levels, while the Focal loss specifically enhances the segmentation ability of the most challenging tumor regions.

[0091] L pres The existence loss is used to supervise the attribute predictor's prediction of the presence of enhanced tumor regions. This task is a binary classification problem, and a binary cross-entropy loss is employed. ; Where π ET∈[0,1] represents the enhanced tumor presence probability output by the attribute predictor (after sigmoid activation). ∈{0,1} represents the true presence label calculated from the gold standard mask. This loss-guided model accurately determines whether the current sample contains enhanced tumor regions, providing a reliable basis for subsequent prompt generation and gating mechanisms.

[0092] L attr The concept attribute loss is used to supervise the attribute predictor's prediction of enhanced tumor region morphological attributes (fragmentation level and scale level). Both attributes are multi-class classification tasks, employing multi-class cross-entropy loss: ; in This represents the set of conceptual attributes other than existence, namely the fragmentation bin and the scale bin. c represents the predicted class of attribute u by the model (normalized by softmax). u The true category label is calculated from the gold standard mask. This loss forces the attribute predictor to learn a mapping from visual features to high-level semantic concepts, enabling the generated cue features to accurately encode the morphological information of the tumor, thereby more effectively guiding the decoder.

[0093] L hier For the hierarchical consistency loss, based on medical priors, the three regions of a brain tumor exhibit a strict nested containment relationship: the enhanced tumor region (ET) is completely contained within the tumor core region (TC), and the tumor core region is completely contained within the overall tumor region (WT). To incorporate this logical constraint into model training, this embodiment employs a soft penalty term to penalize voxels in the prediction probability graph that violate this relationship: ; in These are probability maps (values ​​ranging from 0 to 1) for the three regions predicted by the model. The loss is calculated on all voxels. Greater than The difference (i.e., the portion of ET exceeding TC) and Greater than The difference between TC and WT (i.e., the portion of TC that exceeds WT) is summed as a penalty. This constraint enforces the inclusion relationship between regions in a soft way, avoiding gradient problems that may be caused by hard truncation, while improving the anatomical rationality of the segmentation results.

[0094] L moeTo address the load balancing loss for hybrid experts, a "routing collapse" phenomenon can easily occur in sparse hybrid expert (MoE) training, where the gating network tends to activate only a few experts, resulting in insufficient training for the remaining experts and a decrease in the overall model capacity. To promote balanced utilization of all experts, this embodiment introduces a load balancing regularization term. Specifically, the average routing weight of each expert is calculated within a training batch. And punish it for being uniformly distributed. Degree of deviation: ; Where M is the total number of experts (M=5 in this embodiment). This loss term encourages each expert to obtain a similar average activation probability in the form of squared error, thereby ensuring that all experts can play a role in the training process and maintain the expressive power and stability of the model. This regularization, together with the sparse routing mechanism, ensures both inference efficiency and avoids expert degradation.

[0095] Through joint optimization of the above multi-task loss functions, the model is comprehensively improved in terms of segmentation accuracy, attribute prediction accuracy, anatomical rationality, and training stability, ultimately achieving high-precision fully automatic three-dimensional brain tumor segmentation.

[0096] To evaluate the effectiveness of the method in this embodiment (GliomaSAM3D-MoE), quantitative and qualitative comparative experiments were conducted on the publicly available brain tumor segmentation benchmarks BraTS 2023 and BraTS 2018 validation sets. The experiments used the Dice similarity coefficient (%) and the 95% Hausdorff distance (HD95, mm) as evaluation metrics. The Dice similarity coefficient (%) measures the overlap between the segmented region and the gold standard, with higher values ​​being better; HD95 (mm) measures the maximum error between the segmentation boundary and the gold standard boundary, with lower values ​​being better. It should be noted that the BraTS 2018 and BraTS 2023 validation sets are well-known in the field of brain tumor segmentation research.

[0097] like Figure 3 As shown, Figure 3Quantitative results on the BraTS 2023 validation set (250 cases) are presented. Compared with various methods, including classic and state-of-the-art segmentation networks such as U-Net++, MedNeXt, SegResNet, TransUnet, UNETR, SwinUNETR, and SegMamba, as well as general-purpose large models such as SAM, MedSAM, and SAM2, the method in this embodiment achieves the best or second-best Dice coefficients in all three tumor regions (WT, TC, ET), with an average Dice of 88.86% and an average HD95 of 5.25 mm, both superior to the strongest baseline SegMamba (87.21% Dice, 5.78 mm HD95). Especially in the most challenging enhanced tumor (ET) region, the method in this embodiment effectively suppresses false positives and improves boundary accuracy through presence-aware gating and boundary focusing enhancement, achieving a Dice of 84.20% and an HD95 of 4.62 mm, significantly outperforming the comparison methods.

[0098] like Figure 4 As shown, Figure 4 Quantitative results on the BraTS 2018 validation set (66 cases) are presented. The method in this embodiment also performs excellently, with an average Dice of 88.48% and an average HD95 of 3.98 mm, both ranking first. In the ET region, the method in this embodiment achieves a Dice of 83.51% and an HD95 as low as 2.85 mm, significantly outperforming comparative methods such as SegMamba, validating the method's generalization ability and accurate segmentation capability for small targets.

[0099] like Figure 5 As shown, Figure 5 A qualitative comparison of representative BraTS cases is presented, with representative cases selected for visualization. Each row corresponds to one case, displaying from left to right the original images of four modalities: T1, T1ce, FLAIR, and T2; the gold standard (GT) segmentation mask; the prediction results of the method in this embodiment (GliomaSAM3D-MoE); and the prediction results of the baseline method SegMamba. The whole tumor (WT), tumor core (TC), and enhanced tumor (ET) regions are marked in cyan, magenta, and yellow, respectively. The comparison shows that the segmentation results of the method in this embodiment are closer to the gold standard in terms of regional integrity, boundary consistency, and preservation of nesting relationships. Especially in the ET region, it can more completely capture fragmented morphology while reducing false positive predictions, directly demonstrating its advantages in quantitative indicators.

[0100] The above experiments demonstrate that the GliomaSAM3D-MoE brain tumor image segmentation model in this embodiment exhibits excellent performance on both the BraTS2023 and BraTS2018 validation sets. On the BraTS2023 dataset, the GliomaSAM3D-MoE brain tumor image segmentation model achieved an average Dice value of 88.86% and an average HD95 value of 5.25 mm, surpassing the strongest baseline model in the existing architecture (SegMamba: 87.21% Dice). The improvements are most significant in the enhanced tumor (ET) region, thanks to presence-aware gating and boundary-focusing enhancement, which effectively optimizes the boundary-sensitive HD95 metric in this region.

[0101] To comprehensively evaluate the effectiveness of each module in the method of this embodiment, supplementary experiments such as attribute classification verification, ablation experiments, and boundary accuracy analysis were further designed on the BraTS 2023 validation set.

[0102] like Figure 6 As shown, Figure 6 This study demonstrates the performance of the enhanced tumor (ET) presence classification task. On a validation subset comprising 122 samples (120 containing ET and 2 not containing ET), with the Youden index used to determine a threshold of 0.995, the method achieved an AUROC (area under the ROC curve) of 0.896, an accuracy of 0.795, a sensitivity of 0.792, and a specificity of 1.000. The specificity of 1.000 indicates that the method can accurately identify all samples without ET, providing a reliable basis for presence gating mechanisms and effectively avoiding false positive predictions on samples lacking ET.

[0103] like Figure 7 As shown, Figure 7 To validate the ablation experiment results, the contribution of the core module was verified by stepwise removal. Compared with the complete model (mean Dice 88.86%, mean HD95 5.25 mm, ET HD95 4.62 mm): After removing the concept cue token, the average Dice dropped to 88.45%, and the ET HD95 increased to 4.71 mm; After removing the ET gate, the average Dice decreased to 88.28%, and the ET HD95 increased to 4.80 mm; After removing the dual-domain enhancement module, the performance degradation was more pronounced, with the average Dice dropping to 87.50% and the ET HD95 increasing to 4.97 mm. The removal of the Sparse Hybrid Expert (MoE) decoder resulted in the greatest performance loss, with the average Dice dropping to 87.02%, the average HD95 increasing to 5.96 mm, and the ET HD95 increasing to 5.03 mm.

[0104] The ablation results confirmed that each module made a positive contribution to the final performance. Among them, the dual-domain enhancement and MoE decoder were particularly critical to improving the overall accuracy, while concept hints and ET gating specifically optimized the boundary accuracy and false positive suppression of the ET region.

[0105] like Figure 8 As shown, Figure 8 A case study of missing ET samples. Figure 8 (a) is the result after removing the ET gate. Figure 8 (b) shows the segmentation result of the complete model. Combined with... Figure 9 As shown, Figure 9 This is a comparative chart of the effects of ET gating mechanisms, used to visually demonstrate the impact of presence gating on the prediction results of enhanced tumor (ET) regions. The comparison shows that the ET gating mechanism effectively suppressed false positive predictions of yellow (ET) regions that should not have appeared, while fully preserving the cyan (WT) and magenta (TC) regions, visually demonstrating the effect of the gating mechanism in eliminating erroneous predictions while maintaining the nested relationship between regions. This result is consistent with... Figure 7 The quantitative results of removing the enhanced tumor gating mechanism in the middle corroborate each other.

[0106] like Figure 10 As shown, Figure 10 The boundary accuracy was further evaluated using the Dice index. The Dice coefficient was calculated within an expansion / erosion band with a radius of 3 voxels (26 connectivity). This index eliminates overlap within the region, focusing on the characterization accuracy near the boundary. Results showed that the boundary Dice values ​​of the proposed method in the WT, TC, and ET regions were 0.789, 0.766, and 0.697, respectively, with an average of 0.750. Combined with the aforementioned qualitative comparisons, these results confirm the fine segmentation capability of the proposed method at tumor boundaries, especially in the morphologically complex ET region, where boundary accuracy is effectively guaranteed.

[0107] In summary, the supplementary experiments verified the effectiveness of the method of this invention and the necessity of each core component from multiple dimensions such as attribute classification, module contribution, and boundary accuracy.

[0108] Example 2 This embodiment provides an electronic device. Figure 11 A structural block diagram of an electronic device according to an embodiment of the present invention is shown. Figure 11 As shown, the electronic device includes a memory 100 and a processor 200. The memory 100 stores a computer program that can run on the processor 200. When the processor 200 executes the computer program, it implements the three-dimensional glioma image segmentation method in the above embodiment. The number of memories 100 and processors 200 can be one or more.

[0109] The electronic device also includes: The communication interface 300 is used to communicate with external devices and perform data exchange and transmission.

[0110] If the memory 100, processor 200, and communication interface 300 are implemented independently, they can be interconnected via a bus to communicate with each other. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc.

[0111] Optionally, in a specific implementation, if the memory 100, processor 200, and communication interface 300 are integrated on a single chip, then the memory 100, processor 200, and communication interface 300 can communicate with each other through an internal interface.

[0112] This invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method provided in this invention.

[0113] This invention also provides a chip, which includes a processor for calling and executing instructions stored in a memory, causing a communication device on which the chip is installed to perform the method provided in this invention.

[0114] This invention also provides a chip, including: an input interface, an output interface, a processor, and a memory. The input interface, output interface, processor, and memory are connected through an internal connection path. The processor is used to execute code in the memory. When the code is executed, the processor is used to execute the method provided in this invention.

[0115] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting the Advanced Reduced Instruction Set Computing (RISC) machine (ARM) architecture.

[0116] Further, optionally, the aforementioned memory may include read-only memory and random access memory, and may also include non-volatile random access memory. The memory may be volatile or non-volatile, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. Many forms of RAM are available by way of example, but not limitation. Examples include static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0117] In the above embodiments, implementation can be achieved, in whole or in part, by software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the present invention is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0118] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0119] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0120] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in the present invention, and these should all be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for image segmentation of three-dimensional gliomas, characterized in that, include: The three-dimensional volume data of the multimodal magnetic resonance imaging to be segmented is obtained, and the three-dimensional volume data is fused with the directional boundary prior information obtained by parameterless high-frequency direction extraction to obtain directional boundary enhanced three-dimensional volume data. The 3D volume data enhanced by directional boundaries is decomposed into a sequence of 2D slices along the depth dimension, and the inter-slice context modeling of the 2D slice sequence is performed to aggregate the information of adjacent slices, thereby generating a target feature body containing 3D spatial context information. Based on the target feature body, a high-level semantic cue feature is generated by dynamically inferring discrete token combinations through a fixed discrete token vocabulary and an attribute predictor. The target feature body is input to the orientation-aware dual-domain enhancement branch, which includes a parallel frequency domain enhancement branch and a spatial domain multi-scale orientation enhancement branch, for simultaneously enhancing the features of the target feature body in the spatial domain and frequency domain to obtain the enhanced features. The enhanced features and the high-level semantic cue features are input together into a task-structured sparse hybrid expert decoder, which outputs a segmentation probability map of three nested tumor regions corresponding to each voxel. The nested tumor regions include the overall tumor region, the tumor core region, and the enhanced tumor region.

2. The image segmentation method for three-dimensional gliomas according to claim 1, characterized in that, The step of fusing the three-dimensional volume data with the prior information of the orientation boundary obtained through parameterless high-frequency orientation extraction to obtain three-dimensional volume data with enhanced orientation boundaries includes: The modal average volume is obtained by averaging the three-dimensional volume data of the multimodal magnetic resonance imaging along the modal dimension. A parameterless fixed center difference kernel along each coordinate axis is used to convolve the modal average volume to extract the high-frequency response map in each direction; The absolute values ​​of the high-frequency response maps in each direction are taken and stitched along the channel dimension. Then, the spatial dimension of the stitched results is normalized sample by sample and channel by channel to form a directional prior stack. The directional prior stack is stitched together with the original three-dimensional volume data of multimodal magnetic resonance imaging along the channel dimension to obtain three-dimensional volume data with enhanced directional boundaries and an increased number of channels.

3. The image segmentation method for three-dimensional gliomas according to claim 1, characterized in that, By performing inter-slice context modeling on the two-dimensional slice sequence to aggregate information from adjacent slices, a target feature body containing three-dimensional spatial context information is generated, including: The label embedding vector of each two-dimensional slice in the two-dimensional slice sequence is extracted by a two-dimensional image encoder; For each target slice, its corresponding label embedding vector is used as the query, and the feature concatenation result of all two-dimensional slices in the adjacent slice window is used as the key and value. The information of adjacent slices is aggregated through a cross-attention mechanism to obtain the target features after fusing the three-dimensional context. The target features after fusing all two-dimensional slices are organized in depth order to form a target feature body with three-dimensional spatial context information.

4. The image segmentation method for three-dimensional gliomas according to claim 1, characterized in that, The method for generating high-level semantic prompts includes: The discrete token vocabulary is predefined, which includes region tokens for identifying segmentation tasks, existence tokens for indicating the presence of enhanced tumor regions, fragmentation level tokens for quantifying the spatial dispersion of enhanced tumor regions, and scale level tokens for quantifying the volume of enhanced tumor regions. A global volume descriptor is obtained by aggregating from the target feature volume. The global volume descriptor is input into the attribute predictor. The attribute predictor outputs the predicted concept label. The concept label includes at least the existence category, fragmentation level category and scale level category of the enhanced tumor region. The predicted concept labels are mapped to corresponding discrete token embedding vectors via the discrete token vocabulary index, and the discrete token embedding vectors are converted into final high-level semantic cue features by a specified cue encoder.

5. The image segmentation method for three-dimensional gliomas according to claim 1, characterized in that, The method for generating the enhanced features includes: By using a two-level feature calibration fusion gating, the target feature volume is adaptively fused with the multi-scale orientation perception enhancement features output by the spatial domain multi-scale orientation enhancement branch to obtain the fusion result. The fusion result is then adaptively fused with the spectral enhancement features output by the frequency domain enhancement branch to obtain the final enhanced features. The frequency domain enhancement branch transforms the target feature volume to the frequency domain through a three-dimensional Fourier transform, adaptively modulates the amplitude spectrum using cross-channel shared learnable radial gating, and then reconstructs it through an inverse Fourier transform to obtain the spectrum enhancement feature. The spatial domain multi-scale orientation enhancement branch uses directional deep convolution kernels of multiple scales to extract features along the three orthogonal directions of x, y, and z, respectively. It then performs adaptive weighted fusion of the feature maps of each scale and direction by attention weights based on global feature learning to obtain the multi-scale orientation perception enhancement features.

6. The image segmentation method for three-dimensional gliomas according to claim 1, characterized in that, The enhanced features and the high-level semantic cue features are jointly input into a task-structured sparse hybrid expert decoder, and the output segmentation probability map of three nested tumor regions corresponding to each voxel includes: Feature linear modulation is employed, and the enhanced features are modulated using the high-level semantic cue features to obtain modulated features; The global volume descriptor and the high-level semantic cue features are input into the gating network, and the routing weights of a preset number of experts are generated by softmax normalization, and the sum of the routing weights of each expert is 1. Based on the routing weights, a sparse top-k routing mechanism is used to select the k experts with the highest weights for activation, while the remaining experts are skipped, resulting in the selected k lightweight 3D convolutional expert decoders. The modulated features are input into k selected lightweight 3D convolutional expert decoders, and each expert decoder outputs three-channel logits corresponding to the overall tumor region, the tumor core region, and the enhanced tumor region. The output logits of each selected expert are weighted and fused according to their corresponding routing weights to obtain the final regional logits. The three channels are respectively denoted as the overall tumor region logits, the tumor core region logits, and the enhanced tumor region logits.

7. The image segmentation method for three-dimensional gliomas according to claim 1, characterized in that, Also includes: The original predicted value of the enhanced tumor region output by the sparse hybrid expert decoder is converted into an output probability value through a sigmoid function, and gating processing is performed according to the probability of the existence of the enhanced tumor region output by the attribute predictor; the gating processing includes forcibly setting the output probability value of the enhanced tumor region to zero when the probability of the existence of the enhanced tumor region is lower than a preset threshold.

8. The image segmentation method for three-dimensional gliomas according to claim 1, characterized in that, Also includes: Obtain multiple outputs from the model prediction, including: segmentation probability maps of three nested tumor regions, enhanced tumor region existence probability output by the attribute predictor, concept labels for fragmentation-level and scale-level categories output by the attribute predictor, and expert routing weights output by the gating network. Obtain the corresponding ground truth labels, including: three region ground truth masks obtained from the gold standard segmentation mask, existence ground truth labels calculated from the gold standard enhanced tumor mask, fragmentation level ground truth labels, and scale level ground truth labels; Based on the multiple outputs predicted by the model and the true labels, the overall loss function value is calculated, and the model parameters are updated through backpropagation.

9. An electronic device, characterized in that, include: A processor and a memory, wherein the memory stores instructions that are loaded and executed by the processor to implement the image segmentation method for three-dimensional gliomas as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the image segmentation method for three-dimensional gliomas as described in any one of claims 1 to 8.

Images