A brain tumor segmentation method and device based on TriSAM-UNet

Abstract

The application relates to a brain tumor segmentation method and device based on TriSAM-UNet, which comprises the following steps: S1, collecting a multi-modal MRI brain tumor image dataset, and performing pretreatment and data enhancement; S2, constructing a hybrid segmentation network TriSAM-UNet, designing a hybrid module SAHM, capturing three-dimensional dependence and cross-region jump correlation, and realizing all-around global context perception; S3, constructing an FUE module, inhibiting fuzzy and noise response to ensure that high-confidence features are preferentially transmitted; S4, constructing a WFAB module, and restoring the boundary and texture details lost in up-sampling; and S5, training and optimizing the network model and performing full-automatic segmentation on the brain tumor image. The application aims to solve the problems of insufficient global context capture, indiscriminate forwarding of noise features in jump connection and loss of boundary details in up-sampling in brain tumor segmentation, and is suitable for the field of brain tumor image analysis.

Description

Technical Field

[0001] This invention relates to the field of medical image processing, specifically to a brain tumor segmentation method and apparatus based on TriSAM-UNet. Background Technology

[0002] Brain tumors are the most common type of tumor in the central nervous system and have a highly heterogeneous prognosis. Precise segmentation is crucial for preoperative planning, radiotherapy targeting, and follow-up evaluation. Clinically, this often relies on experts to delineate the tumor region layer by layer on multimodal MRI, which is time-consuming and suffers from significant subjectivity and poor reproducibility.

[0003] In recent years, deep learning-based methods have made significant progress in brain tumor segmentation. While hybrid architectures such as Transformer-based UNETR and Swin UNETR possess global modeling capabilities, the time and space complexity of standard self-attention increases approximately quadratically with the number of tokens, drastically raising the computational and memory costs of 3D multimodal MRI volumetric segmentation, making it difficult to deploy in resource-constrained clinical environments. Slim-UNETR, as a lightweight 3D medical image segmentation framework, significantly reduces the number of parameters and improves inference throughput while maintaining accuracy through sparse global attention. On the other hand, the Mamba selective state-space model exhibits linear complexity and good global modeling capabilities in long sequence modeling.

[0004] However, the existing methods still have the following shortcomings: (1) Although the scheme of directly replacing attention with Mamba reduces the complexity, the simple one-way or one-axis recursion cannot model the non-axial cross-regional jump dependency, resulting in insufficient correlation ability of the far-end symmetrical structure, and it is a simple replacement without innovation; (2) The existing jump connection adopts the direct splicing or summation method, which cannot distinguish between reliable regions and noise or blurred regions in the encoder features, resulting in low-quality features being passed to the decoder; (3) The decoder relies on transposed convolution or interpolation upsampling, which is a low-pass operation, resulting in the blurring of tumor boundaries and fine-grained texture structures during the decoding process. Summary of the Invention

[0005] To address the aforementioned technical issues, this invention provides a brain tumor segmentation method and apparatus based on TriSAM-UNet, aiming to further enhance the modeling ability for complex morphology and long-range dependencies of brain tumors by using dual-paradigm fusion coding and spatial-frequency joint decoding enhancement while maintaining lightweight and high throughput.

[0006] Specifically, the method includes the following steps: S1: Acquire a multimodal MRI brain tumor image dataset, and perform preprocessing and data augmentation; S2: Construct a hybrid segmentation network TriSAM-UNet (Tri-directional Sparse Attention-MambaUNet), and design a hybrid module SAHM (Sparse Attention-Hybrid Mamba Module) in the encoder to capture three-dimensional axial continuous dependencies and cross-regional skip associations simultaneously with linear computational complexity, thereby achieving comprehensive global context awareness; S3: Deploy the Uncertainty Guided Feature Gating (FUE) module in skip connections to suppress fuzzy and noisy responses and ensure that high-confidence features are passed first; S4: Deploy the Space-Frequency Joint Enhancement Module (WFAB) in the decoder to restore the boundary and texture details lost during upsampling; S5: Train and optimize the network model and perform fully automatic segmentation of brain tumor images.

[0007] Preferably, S2 includes the following steps: S2.1: Construct the SAHM module (SparseAttention-Hybrid Mamba Module) with a parallel structure of three-way Mamba branches and sparse attention branches. Preferably, S2.1 includes the following steps: S2.1.1: TriSSM Branch – Three-Dimensional Spatial Serialization and Selective State Recursion. This involves processing the 3D feature map output from the current layer of the encoder. Unfold the feature maps along the x, y, and z axes into one-dimensional sequence features, and rearrange the feature maps along the x-axis into a sequence set of length W, denoted as W. The sequence is rearranged along the y-axis into a set of sequences of length H, denoted as H. The sequence is rearranged along the z-axis into a set of sequences of length D, denoted as... The channel dimension C is maintained as the feature dimension. Selective state-space recursion is performed independently for the sequence features in each direction. A selective mechanism for input dependency is introduced into the state recursion to adaptively enhance key spatial locations and channels, suppress redundant or noise responses, and complete the forward recursion along the sequence dimension to capture long-range spatial dependencies along that axis, resulting in SSM-enhanced features in three directions. , , The SSM outputs from the three directions are then concatenated along the channel dimension, fused using a direction-adaptive weighting operator, and reweighted along the channels to obtain the final output of the TriSSM branch. Simultaneously encodes long-range dependencies in the x, y, and z directions; S2.1.2: Sparse Attention Branch – Retains the original sparse global attention mechanism of Slim-UNETR. It applies the same input features... The input is processed according to the original sparse global attention module. By establishing an explicit cross-regional jump relationship between any two spatial locations through sparse global attention, the output of the sparse attention branch is obtained. ; S2.2: The outputs of the two branches are fused using AGF (Adaptive Gated Fusion); Preferably, S2.2 includes the following steps: S2.2.1: Adaptive Gated Fusion (AGF). This involves outputting the TriSSM branch. With sparse attention branch output After concatenation along the channel dimension, a 1×1×1 3D convolution is performed with... Activation function generates space-channel joint gating graph : The two outputs are weighted and fused using a gating graph, and residual connections are applied: Where σ is the Sigmoid function, ‖ is the channel dimension concatenation operation, and ⊙ is the element-wise multiplication operation. The gating graph G is learned autonomously by the network, enabling each spatial location and channel to adaptively determine whether to rely more on the axial continuous recursion capability of the TriSSM branch or the cross-regional jump association capability of the sparse attention branch, thereby achieving a complementary fusion of two global modeling paradigms.

[0008] Preferably, S3 includes the following steps: S3.1: Introduce the Feature Uncertainty-guided Enhancement Module (FUE) to obtain the uncertainty estimate for each spatial location; Preferably, S3.1 includes the following steps: S3.1.1: The first encoder output... Scale multi-channel feature map Input the FUE module, where They are the first The scale feature map includes the number of channels, depth dimension, height dimension, and width dimension. First, mean aggregation is performed along the channel dimension to obtain a single-channel response map that retains only spatial structure information; then... function After normalization, the normalized channel mean characteristics are obtained: in, For the first Scale at the The characteristic values ​​of each channel, The normalized single-channel response plot is given, ranging from (0,1). Based on this, the entropy-type uncertainty for each spatial location of the normalized response plot is calculated: in, For the first Scale uncertainty map, and input features Having the same space size Entropy The larger the value, the more ambiguous the feature at that location, and the less stable the model's decision, thus leading to a discrepancy with the input feature. Uncertainty maps of the same spatial size provide a quantitative basis for subsequent location-adaptive weighting. S3.2: Construct a position-adaptive weighting factor based on uncertainty; Preferably, S3.2 includes the following steps: S3.2.1: Uncertainty map obtained from S3.1.1 Constructing location-adaptive weighting factors The encoder output features are reweighted position by position: for regions with low uncertainty (high confidence), Larger values ​​are assigned greater weights to enhance them; for regions of high uncertainty (low confidence), Smaller values ​​are suppressed by assigning smaller weights. The original features are then linked using residual connections. With weighted features By merging, we obtain the final output: This residual form avoids excessive information attenuation and preserves the basic representation of the original features. The final output is a feature map enhanced by FUE. Then, through skip connections, the information is passed to the decoder, providing more stable and reliable contextual support for tumor boundaries and fine-grained structures.

[0009] Preferably, S4 includes the following steps: S4.1: Deploy WFAB (Wavelet-Frequency Augmented Block) enhancement modules, which include spatial domain dual attention branches and frequency domain wavelet enhancement branches, in each layer of the decoder. Preferably, S4.1 includes the following steps: S4.1.1: Spatial Domain Dual Attention Branch. This involves applying the feature map of the current layer of the decoder... The input spatial domain branch contains the Position Attention Module (PAM) and the Channel Attention Module (CAM): Position Attention (PAM) submodule: Let the total number of spatial locations N = D × H × W. First, it uses three 1 × 1 × 1 convolutions to... Mapped to queries respectively ,key Sum ,in , , r is the channel compression ratio; then the correlation matrix between spatial locations is calculated and normalized to an attention weight map: The response of key lesion areas is explicitly enhanced, while the response of background and noise areas is suppressed. The result is obtained through weighted aggregation and residual connection: in It is a learnable scalar parameter, initialized to 0, and its contribution strength to spatial attention is gradually learned during training.

[0010] Channel Attention (CAM) submodule: Similarly, it is generated through three 1×1×1 convolutions. , , Calculate the inter-channel correlation matrix: Enhancing channels with higher semantic relevance to brain tumors and suppressing redundant or irrelevant channels yields: The outputs of positional attention and channel attention are added element-wise to form a spatial domain enhancement feature that simultaneously takes into account both spatial and channel long-range dependencies: S4.1.2: Frequency domain wavelet enhancement branch. Using the same feature map... Applying three-dimensional discrete wavelet transform It is decomposed into one low-frequency approximate subband A and seven high-frequency detail subbands {D_LLH, D_LHL, D_LHH, D_HLL, D_HLH, D_HHL, D_HHH}; Taking Haar wavelets as an example, low-pass filter kernels are applied sequentially along the three spatial dimensions. and high-pass filter kernel With a step size of 2, 8 sub-bands are obtained; Channel attention weighting is applied independently to each high-frequency sub-band (SE style), i.e., global average pooling → fully connected layer → ReLU → fully connected layer → for each high-frequency sub-band. → Channel weighting selectively enhances high-frequency channel responses related to tumor boundaries and texture, while suppressing noise-dominated high-frequency components; The low-frequency approximate subband A remains unchanged because the global structural information has been processed by the spatial domain branch; The attention-enhanced high-frequency subband and the original low-frequency subband are then subjected to a three-dimensional inverse discrete wavelet transform. Reconstructing spatial domain features ; S4.2: The enhanced features from both are adaptively fused to obtain an enhanced output; Preferably, S4.2 includes the following steps: S4.2.1: Spatial-Frequency Adaptive Fusion. This involves branching the spatial domain output... frequency domain branch output Learnable channel weight vector , Perform weighted fusion and apply residual connections: in , The learnable parameter vectors are jointly optimized during training. This fusion enables the decoder to enhance the saliency of lesion regions in the spatial domain while recovering high-frequency boundaries and texture details lost during upsampling in the frequency domain, achieving joint spatial-frequency enhancement.

[0011] Preferably, S5 includes the following steps: S5.1: The TeLU activation function is used as a nonlinear activation operator to alleviate gradient vanishing and enhance nonlinear expressive power; Preferably, S5.1 includes the following steps: S5.1.1: The TeLU activation function is defined as follows: , where x is the input feature and f(x) is the activation output. This form maintains a near-linear response on the positive half-axis and passes through ... Combining with the exponential function achieves a smooth non-zero gradient transition, avoiding the neuron inactivation problem caused by the traditional ReLU function having a gradient that is always zero on the negative half-axis; S5.1.2: During training, the parameters of TeLU are jointly optimized so that the response intensity is adaptively adjusted according to the feature distribution of different levels, the information propagation is enhanced for deep modules where the gradient is easy to vanish, and irrelevant perturbations are suppressed for shallow or high-noise features. S5.2: A weighted combination of Dice loss and cross-entropy loss is used as a hybrid loss function to train the TriSAM-UNet (Tri-directional Sparse Attention-Mamba UNet) network model; S5.3: Using the trained TriSAM-UNet model, predict the results and label the predicted brain tumor tissue locations to obtain the brain tumor segmentation results of the patient's MRI images.

[0012] The second technical solution adopted in this invention is: a brain tumor segmentation device based on TriSAM-UNet, comprising: SAHM encoding module: In the encoder of the TriSAM-UNet network, it is used to fuse two complementary features through the three-way selective state space branch TriSSM and the sparse global attention branch in parallel, and then through the adaptive gating fusion module AGF to capture three-dimensional axial continuous dependencies and cross-region jump associations with linear computational complexity. FUE feature gating module: used to suppress fuzzy and noise responses based on entropy uncertainty in skip connections, ensuring that high-confidence features are preferentially passed to the decoder; WFAB Decoding Enhancement Module: Used to restore the boundary and texture details lost during upsampling by fusing the spatial domain dual attention branch and the frequency domain wavelet enhancement branch in each layer of the decoder. Training and Segmentation Module: Used to train and optimize the TriSAM-UNet network using the TeLU activation function and a hybrid loss function, and to perform fully automatic segmentation of brain tumor images using the trained model.

[0013] The beneficial effects of the method and apparatus of this invention are as follows: This invention proposes the TriSAM-UNet network architecture, which achieves multi-dimensional collaborative innovation based on the lightweight backbone of Slim-UNETR. First, the SAHM module is designed to fuse a three-dimensional selective state space branch and a sparse global attention branch in parallel, enabling the encoder to possess both the ability to capture axial continuous dependencies with linear complexity and the ability to model explicit cross-regional skip associations. The two complementary paradigms achieve synergistic effects through adaptive gating fusion, effectively solving the problem of insufficient representation of complex morphological features of brain tumors by a single modeling paradigm. Second, a FUE-WFAB collaborative enhancement mechanism is formed in the feature transfer pathway: FUE performs reliability screening and weighting of encoder features based on uncertainty on the skip connection side, while WFAB restores the high-frequency boundaries and texture details lost in upsampling through spatial-frequency joint enhancement on the decoder side. The combination of the two effectively improves the segmentation quality of tumor boundaries and small volumes. Finally, the hierarchical adaptive TeLU activation strategy adaptively adjusts the nonlinear response for different network depths, taking into account both deep gradient smoothness and shallow noise suppression, thereby improving the stability and robustness of training convergence. Thanks to the lightweight backbone of Slim-UNETR and the linear complexity of Mamba, the overall solution has low computational and memory requirements, making it more suitable for deployment on conventional hospital workstations and real-time clinical applications. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention, the following is a brief introduction to the drawings used in the prior art and embodiments. The following drawings are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a flowchart illustrating a brain tumor segmentation method based on TriSAM-UNet according to the present invention. Figure 2 This is a schematic diagram of the overall architecture of the TriSAM-UNet network in the brain tumor segmentation method based on TriSAM-UNet of the present invention. Figure 3 This is a schematic diagram of the SAHM module structure of a brain tumor segmentation method based on TriSAM-UNet according to the present invention, which includes a TriSSM branch, a sparse attention branch and an AGF fusion module. Figure 4 This is a schematic diagram of the WFAB module structure of a brain tumor segmentation method based on TriSAM-UNet according to the present invention, which includes a spatial domain PAM / CAM branch and a frequency domain wavelet transform branch. Figure 5 This is a schematic diagram of the FUE module structure of a brain tumor segmentation method based on TriSAM-UNet according to the present invention. Detailed Implementation

[0016] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of this application. Unless otherwise specified, all other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0017] This application provides a brain tumor segmentation method and apparatus based on TriSAM-UNet, which is used to effectively extract brain tumor feature information from brain MRI images, thereby achieving accurate segmentation of brain tumors.

[0018] As a typical embodiment of the present invention, the BRATS2021 dataset is used as an example for illustration. Figure 1 As shown, the method includes the following steps: S1: Acquire a multimodal MRI brain tumor image dataset, and perform preprocessing and data augmentation; S2: Construct a hybrid segmentation network TriSAM-UNet (Tri-directional Sparse Attention-MambaUNet), and design a hybrid module SAHM (Sparse Attention-Hybrid Mamba Module) in the encoder to capture three-dimensional axial continuous dependencies and cross-regional skip associations simultaneously with linear computational complexity, thereby achieving comprehensive global context awareness; S3: Deploy the Feature Uncertainty-guided Enhancement Module (FUE) in skip connections to suppress fuzzy and noisy responses and ensure that high-confidence features are passed first; S4: Deploy the Space-Frequency Joint Enhancement Module (WFAB) in the decoder to restore the boundary and texture details lost during upsampling; S5: Train and optimize the network model and perform fully automatic segmentation of brain tumor images.

[0019] Furthermore, S2 includes the following steps: S2.1: Construct the SAHM module (SparseAttention-Hybrid Mamba Module) with a parallel structure of three-way Mamba branches and sparse attention branches. Furthermore, S2.1 includes the following steps: S2.1.1: TriSSM Branch – Three-Dimensional Spatial Serialization and Selective State Recursion. This involves processing the 3D feature map output from the current layer of the encoder. Unfold the feature maps along the x, y, and z axes into one-dimensional sequence features, and rearrange the feature maps along the x-axis into a sequence set of length W, denoted as W. The sequence is rearranged along the y-axis into a set of sequences of length H, denoted as H. The sequence is rearranged along the z-axis into a set of sequences of length D, denoted as... The channel dimension C is maintained as the feature dimension. Selective state-space recursion is performed independently for the sequence features in each direction. A selective mechanism for input dependency is introduced into the state recursion to adaptively enhance key spatial locations and channels, suppress redundant or noise responses, and complete the forward recursion along the sequence dimension to capture long-range spatial dependencies along that axis, resulting in SSM-enhanced features in three directions. , , The SSM outputs from the three directions are then concatenated along the channel dimension, fused using a direction-adaptive weighting operator, and reweighted along the channels to obtain the final output of the TriSSM branch. Simultaneously encodes long-range dependencies in the x, y, and z directions; S2.1.2: Sparse Attention Branch – Retains the original sparse global attention mechanism of Slim-UNETR. It applies the same input features... The input is processed according to the original sparse global attention module. By establishing an explicit cross-regional jump relationship between any two spatial locations through sparse global attention, the output of the sparse attention branch is obtained. ; S2.2: The outputs of the two branches are fused using AGF (Adaptive Gated Fusion); Furthermore, S2.2 includes the following steps: S2.2.1: Adaptive Gated Fusion (AGF). This involves outputting the TriSSM branch. With sparse attention branch output After concatenation along the channel dimension, a 1×1×1 3D convolution is performed with... Activation function generates space-channel joint gating graph : The two outputs are weighted and fused using a gating graph, and residual connections are applied: Where σ is the Sigmoid function, ‖ is the channel dimension concatenation operation, and ⊙ is the element-wise multiplication operation. The gating graph G is learned autonomously by the network, enabling each spatial location and channel to adaptively determine whether to rely more on the axial continuous recursion capability of the TriSSM branch or the cross-regional jump association capability of the sparse attention branch, thereby achieving a complementary fusion of two global modeling paradigms.

[0020] Furthermore, S3 includes the following steps: S3.1: Introduce the Feature Uncertainty-guided Enhancement Module (FUE) to obtain the uncertainty estimate for each spatial location; Furthermore, S3.1 includes the following steps: S3.1.1: The first encoder output... Scale multi-channel feature map Input the FUE module, where They are the first The scale feature map includes the number of channels, depth dimension, height dimension, and width dimension. First, mean aggregation is performed along the channel dimension to obtain a single-channel response map that retains only spatial structure information; then... function After normalization, the normalized channel mean characteristics are obtained: in, For the first Scale at the The characteristic values ​​of each channel, The normalized single-channel response plot is given, ranging from (0,1). Based on this, the entropy-type uncertainty for each spatial location of the normalized response plot is calculated: in, For the first Scale uncertainty map, and input features Having the same space size Entropy The larger the value, the more ambiguous the feature at that location, and the less stable the model's decision, thus leading to a discrepancy with the input feature. Uncertainty maps of the same spatial size provide a quantitative basis for subsequent location-adaptive weighting. S3.2: Construct a position-adaptive weighting factor based on uncertainty; Furthermore, S3.2 includes the following steps: S3.2.1: Uncertainty map obtained from S3.1.1 Constructing location-adaptive weighting factors The encoder output features are reweighted position by position: for regions with low uncertainty (high confidence), Larger values ​​are assigned greater weights to enhance them; for regions of high uncertainty (low confidence), Smaller values ​​are suppressed by assigning smaller weights. The original features are then linked using residual connections. With weighted features By merging, we obtain the final output: This residual form avoids excessive information attenuation and preserves the basic representation of the original features. The final output is a feature map enhanced by FUE. Then, through skip connections, the information is passed to the decoder, providing more stable and reliable contextual support for tumor boundaries and fine-grained structures.

[0021] Furthermore, S4 includes the following steps: S4.1: Deploy WFAB (Wavelet-Frequency Augmented Block) enhancement modules, which include spatial domain dual attention branches and frequency domain wavelet enhancement branches, in each layer of the decoder. Furthermore, S4.1 includes the following steps: S4.1.1: Spatial Domain Dual Attention Branch. This involves applying the feature map of the current layer of the decoder... The input spatial domain branch contains the Position Attention Module (PAM) and the Channel Attention Module (CAM): Position Attention (PAM) submodule: Let the total number of spatial locations N = D × H × W. First, it uses three 1 × 1 × 1 convolutions to... Mapped to queries respectively ,key Sum ,in , , r is the channel compression ratio; then the correlation matrix between spatial locations is calculated and normalized to an attention weight map: The response of key lesion areas is explicitly enhanced, while the response of background and noise areas is suppressed. The result is obtained through weighted aggregation and residual connection: in It is a learnable scalar parameter, initialized to 0, and its contribution strength to spatial attention is gradually learned during training.

[0022] Channel Attention (CAM) submodule: Similarly, it is generated through three 1×1×1 convolutions. , , Calculate the inter-channel correlation matrix: Enhancing channels with higher semantic relevance to brain tumors and suppressing redundant or irrelevant channels yields: The outputs of positional attention and channel attention are added element-wise to form a spatial domain enhancement feature that simultaneously takes into account both spatial and channel long-range dependencies: S4.1.2: Frequency domain wavelet enhancement branch. Using the same feature map... Applying three-dimensional discrete wavelet transform It is decomposed into one low-frequency approximate subband A and seven high-frequency detail subbands {D_LLH, D_LHL, D_LHH, D_HLL, D_HLH, D_HHL, D_HHH}; Taking Haar wavelets as an example, low-pass filter kernels are applied sequentially along the three spatial dimensions. and high-pass filter kernel With a step size of 2, 8 sub-bands are obtained; Channel attention weighting is applied independently to each high-frequency sub-band (SE style), i.e., global average pooling → fully connected layer → ReLU → fully connected layer → for each high-frequency sub-band. → Channel weighting selectively enhances high-frequency channel responses related to tumor boundaries and texture, while suppressing noise-dominated high-frequency components; The low-frequency approximate subband A remains unchanged because the global structural information has been processed by the spatial domain branch; The attention-enhanced high-frequency subband and the original low-frequency subband are then subjected to a three-dimensional inverse discrete wavelet transform. Reconstructing spatial domain features ; S4.2: The enhanced features from both are adaptively fused to obtain an enhanced output; Furthermore, S4.2 includes the following steps: S4.2.1: Spatial-Frequency Adaptive Fusion. This involves branching the spatial domain output... frequency domain branch output Learnable channel weight vector , Perform weighted fusion and apply residual connections: in , The learnable parameter vectors are jointly optimized during training. This fusion enables the decoder to enhance the saliency of lesion regions in the spatial domain while recovering high-frequency boundaries and texture details lost during upsampling in the frequency domain, achieving joint spatial-frequency enhancement.

[0023] Furthermore, S5 includes the following steps: S5.1: The TeLU activation function is used as a nonlinear activation operator to alleviate gradient vanishing and enhance nonlinear expressive power; Furthermore, S5.1 includes the following steps: S5.1.1: The TeLU activation function is defined as follows: , where x is the input feature and f(x) is the activation output. This form maintains a near-linear response on the positive half-axis and passes through ... Combining with the exponential function achieves a smooth non-zero gradient transition, avoiding the neuron inactivation problem caused by the traditional ReLU function having a gradient that is always zero on the negative half-axis; S5.1.2: During training, the parameters of TeLU are jointly optimized so that the response intensity is adaptively adjusted according to the feature distribution of different levels, the information propagation is enhanced for deep modules where the gradient is easy to vanish, and irrelevant perturbations are suppressed for shallow or high-noise features. S5.2: A weighted combination of Dice loss and cross-entropy loss is used as a hybrid loss function to train the TriSAM-UNet (Tri-directional Sparse Attention-Mamba UNet) network model; S5.3: Using the trained TriSAM-UNet model, predict the results and label the predicted brain tumor tissue locations to obtain the brain tumor segmentation results of the patient's MRI images.

[0024] The above is a detailed description of the implementation of the present invention. However, the present invention is not limited to the described embodiments. Those skilled in the art should know that although the present invention has been described according to the above specific embodiments, the inventive concept of the present invention is not limited to this invention. Any modification that uses the inventive concept will be included within the scope of protection of this patent.

Claims

1. A brain tumor segmentation method based on TriSAM-UNet, characterized in that, The method includes the following steps: S1: Acquire a multimodal MRI brain tumor image dataset, and perform preprocessing and data augmentation; S2: Construct a hybrid segmentation network TriSAM-UNet (Tri-directional Sparse Attention-MambaUNet) and design a hybrid module SAHM (Sparse Attention-Hybrid Mamba Module) to capture 3D dependencies and cross-regional jump associations, achieving comprehensive global context awareness; S3: Construct the FUE (Feature Uncertainty-guided Enhancement Module) to suppress blurry and noisy responses and ensure that high-confidence features are delivered first; S4: Construct the WFAB (Wavelet-Frequency Augmented Block Module) to restore the boundary and texture details lost during upsampling; S5: Train and optimize the network model and perform fully automatic segmentation of brain tumor images.

2. The brain tumor segmentation method based on TriSAM-UNet according to claim 1, characterized in that, S2 includes the following steps: S2.1: Construct the SAHM module (SparseAttention-Hybrid Mamba Module) with a parallel structure of three-way Mamba branches and sparse attention branches. S2.1 includes the following steps: S2.1.1: TriSSM branch -- three-dimensional spatial serialization and selective state recursion; converting the 3D feature map output by the current layer of the encoder... Unfold the feature maps along the x, y, and z axes into one-dimensional sequence features, and rearrange the feature maps along the x-axis into a sequence set of length W, denoted as W. The sequence is rearranged along the y-axis into a set of sequences of length H, denoted as H. The sequence is rearranged along the z-axis into a set of sequences of length D, denoted as... The channel dimension C is maintained as the feature dimension. Selective state-space recursion is performed independently for the sequence features in each direction. A selective mechanism for input dependence is introduced in the state recursion to adaptively enhance key spatial locations and channels, suppress redundant or noise responses, and complete the forward recursion along the sequence dimension to capture long-range spatial dependencies along that axis, resulting in SSM-enhanced features in three directions. , , The SSM outputs from the three directions are then concatenated along the channel dimension, fused using a direction-adaptive weighting operator, and reweighted along the channels to obtain the final output of the TriSSM branch. Simultaneously encodes long-range dependencies in the x, y, and z directions; S2.1.2: Sparse Attention Branch – Retains the original sparse global attention mechanism of Slim-UNETR; uses the same input features The input is processed according to the original sparse global attention module. By establishing an explicit cross-regional jump relationship between any two spatial locations through sparse global attention, the output of the sparse attention branch is obtained. ; S2.2: The outputs of the two branches are fused using AGF (Adaptive Gated Fusion); S2.2 includes the following steps: S2.2.1: Adaptive Gated Fusion (AGF); This involves adjusting the TriSSM branch output. With sparse attention branch output After concatenation along the channel dimension, a 1×1×1 3D convolution is performed with... Activation function generates space-channel joint gating graph : The two outputs are weighted and fused using a gating graph, and residual connections are applied: Where σ is the Sigmoid function, ‖ is the channel dimension concatenation operation, and ⊙ is the element-wise multiplication operation; the gating graph G is learned autonomously by the network, enabling each spatial location and channel to adaptively determine whether to rely more on the axial continuous recursion capability of the TriSSM branch or the cross-regional jump association capability of the sparse attention branch, thereby achieving the complementary fusion of the two global modeling paradigms.

3. The brain tumor segmentation method based on TriSAM-UNet according to claim 1, characterized in that, S3 includes the following steps: S3.1: Introduce the Feature Uncertainty-guided Enhancement Module (FUE) to obtain the uncertainty estimate for each spatial location; S3.1 includes the following steps: S3.1.1: The first encoder output... Scale multi-channel feature map Input the FUE module, where They are the first The scale feature map is analyzed by considering the number of channels, depth dimension, height dimension, and width dimension. First, mean aggregation is performed along the channel dimension to obtain a single-channel response map that retains only spatial structure information. Then, through... function After normalization, the normalized channel mean characteristics are obtained: in, For the first Scale at the The characteristic values ​​of each channel, The normalized single-channel response plot is given, ranging from (0,1). Based on this, the entropy-type uncertainty corresponding to each spatial location of the normalized response plot is calculated: in, For the first Scale uncertainty map, and input features Having the same space size Entropy The larger the value, the more ambiguous the feature at that location, and the less stable the model's decision, thus leading to a discrepancy with the input features. Uncertainty maps of the same spatial size provide a quantitative basis for subsequent location-adaptive weighting. S3.2: Construct a position-adaptive weighting factor based on uncertainty; S3.2 includes the following steps: S3.2.1: Uncertainty map obtained from S3.1.1 Constructing location-adaptive weighting factors The encoder output features are reweighted position by position: for regions with low uncertainty (high confidence), Larger values ​​are assigned greater weights to enhance them; for regions of high uncertainty (low confidence), Smaller values ​​are suppressed by assigning smaller weights; the original features are then linked using residual connections. Weighted features By merging, we obtain the final output: This residual form avoids excessive information attenuation and preserves the basic representation of the original features; the final output is a feature map enhanced by FUE. Then, through skip connections, the information is passed to the decoder, providing more stable and reliable contextual support for tumor boundaries and fine-grained structures.

4. The brain tumor segmentation method based on TriSAM-UNet according to claim 1, characterized in that, S4 includes the following steps: S4.1: Deploy WFAB (Wavelet-Frequency Augmented Block) enhancement modules, which include spatial domain dual attention branches and frequency domain wavelet enhancement branches, in each layer of the decoder. S4.1 includes the following steps: S4.1.1: Spatial domain dual attention branch; This involves using the feature map of the current layer of the decoder... The input spatial domain branch contains the Position Attention Module (PAM) and the Channel Attention Module (CAM): Position Attention (PAM) submodule: Let the total number of spatial locations N = D × H × W. First, it uses three 1 × 1 × 1 convolutions to... Mapped to queries respectively ,key Sum ,in , , r is the channel compression ratio; then the correlation matrix between spatial locations is calculated and normalized to an attention weight map: The response of key lesion areas is explicitly enhanced, while the response of background and noise areas is suppressed. The result is obtained through weighted aggregation and residual connection: in As a learnable scalar parameter, it is initialized to 0, and the contribution strength of spatial attention is gradually learned during training. Channel Attention (CAM) submodule: Similarly, it is generated through three 1×1×1 convolutions. , , Calculate the inter-channel correlation matrix: Enhancing channels with higher semantic relevance to brain tumors and suppressing redundant or irrelevant channels yields: The outputs of positional attention and channel attention are added element-wise to form a spatial domain enhancement feature that simultaneously takes into account both spatial and channel long-range dependencies: S4.1.2: Frequency domain wavelet enhancement branch; using the same feature map Applying three-dimensional discrete wavelet transform It is decomposed into one low-frequency approximate subband A and seven high-frequency detail subbands {D_LLH, D_LHL, D_LHH, D_HLL, D_HLH, D_HHL, D_HHH}; Taking Haar wavelets as an example, low-pass filter kernels are applied sequentially along the three spatial dimensions. and high-pass filter kernel With a step size of 2, 8 sub-bands are obtained; Channel attention weighting is applied independently to each high-frequency sub-band (SE style), i.e., global average pooling → fully connected layer → ReLU → fully connected layer → for each high-frequency sub-band. → Channel weighting selectively enhances high-frequency channel responses related to tumor boundaries and texture, while suppressing noise-dominated high-frequency components; The low-frequency approximate subband A remains unchanged because the global structural information has been processed by the spatial domain branch; The attention-enhanced high-frequency subband and the original low-frequency subband are then subjected to a three-dimensional inverse discrete wavelet transform. Reconstructing spatial domain features ; S4.2: The enhanced features from both are adaptively fused to obtain an enhanced output; S4.2 includes the following steps: S4.2.1: Spatial-Frequency Adaptive Fusion; This involves branching the spatial domain output. frequency domain branch output Learnable channel weight vector , Perform weighted fusion and apply residual connections: in , The learnable parameter vectors are jointly optimized during training; this fusion enables the decoder to enhance the saliency of lesion regions in the spatial domain while recovering the high-frequency boundaries and texture details lost during upsampling in the frequency domain, thus achieving joint spatial-frequency enhancement.

5. The brain tumor segmentation method based on TriSAM-UNet according to claim 1, characterized in that, S5 includes the following steps: S5.1: The TeLU activation function is used as a nonlinear activation operator to alleviate gradient vanishing and enhance nonlinear expressive power; S5.1 includes the following steps: S5.1.1: The TeLU activation function is defined as follows: Where x is the input feature and f(x) is the activation output; this form maintains a near-linear response on the positive half-axis and passes through the negative half-axis. Combining with the exponential function achieves a smooth non-zero gradient transition, avoiding the neuron inactivation problem caused by the traditional ReLU function having a gradient that is always zero on the negative half-axis; S5.1.2: During training, the parameters of TeLU are jointly optimized so that the response intensity is adaptively adjusted according to the feature distribution of different levels, the information propagation is enhanced for deep modules where the gradient is easy to vanish, and irrelevant perturbations are suppressed for shallow or high-noise features. S5.2: A weighted combination of Dice loss and cross-entropy loss is used as a hybrid loss function to train the TriSAM-UNet (Tri-directional Sparse Attention-Mamba UNet) network model; S5.3: Using the trained TriSAM-UNet model, predict the results and label the predicted brain tumor tissue locations to obtain the brain tumor segmentation results of the patient's MRI images.

6. A brain tumor segmentation device based on TriSAM-UNet, characterized in that, include: SAHM encoding module: In the encoder of the TriSAM-UNet network, it is used to fuse two complementary features through the three-way selective state space branch TriSSM and the sparse global attention branch in parallel, and then through the adaptive gating fusion module AGF to capture three-dimensional axial continuous dependencies and cross-region jump associations with linear computational complexity. FUE feature gating module: used to suppress fuzzy and noise responses based on entropy uncertainty in skip connections, ensuring that high-confidence features are preferentially passed to the decoder; WFAB Decoding Enhancement Module: Used to restore the boundary and texture details lost during upsampling by fusing the spatial domain dual attention branch and the frequency domain wavelet enhancement branch in each layer of the decoder. Training and Segmentation Module: Used to train and optimize the TriSAM-UNet network using the TeLU activation function and a hybrid loss function, and to perform fully automatic segmentation of brain tumor images using the trained model.

Images