Glioma segmentation method based on XLSTM-HVED network

Abstract

The invention discloses a brain glioma deletion mode segmentation method based on an XLSTM-HVED neural network, and the method comprises the following steps: 1, obtaining a glioma data set, and dividing the data set into a training set and a test set; step 2, constructing an XLSTM-HVED neural network model used for glioma segmentation, wherein the XLSTM-HVED neural network model comprises an SAVE encoder, a Vision XLSTM attention module, a segmentation decoder, a reconstruction decoder and a DuSFE module; step 3, training the XLSTM-HVED neural network model by using the training set, and adjusting and optimizing the parameters of the XLSTM-HVED neural network model in combination with the characteristics of the glioma image; and step 4, using a test set to test the XLSTM-HVED network model obtained by evaluation, and finally realizing brain glioma deletion mode segmentation.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision deep learning and medical image processing, and specifically refers to a glioma segmentation method based on an XLSTM-HVED network. Background Art

[0002] Glioma is one of the most deadly types of cancer and the most common malignant primary brain tumor in adults, accounting for about 25% of all primary brain tumors and 80% of malignant primary brain and central nervous system tumors. Among them, diffuse glioma is the most common malignant subtype, but due to its variability in biological behavior and prognostic response, it poses great challenges to treatment and monitoring.

[0003] Magnetic resonance imaging (MRI) is the imaging gold standard for the diagnosis and treatment of diffuse gliomas, which provides important information about tumor size, location, and morphology over time. Different MRI modes, such as T1, T1-Gd, T2, and FLAIR, provide complementary information about brain tumors. T1 and T2 modes can provide information about vasogenic edema in subacute stroke; T1-Gd, enhanced by contrast agents, reveals information related to blood vessels and the blood-brain barrier; while FLAIR mode provides general information about stroke lesions. Therefore, in clinical practice, the use of multimodal MRI for segmentation helps to reduce uncertainty and improve segmentation performance compared with the use of single-modality MRI. However, in practice, the loss of modalities often occurs. Due to limited patient tolerance, limited scanning time, and image damage, some modalities may be lost. In this case, the segmentation performance of traditional algorithms will be greatly reduced. Therefore, in recent years, accurate segmentation of MRI under missing modal conditions has attracted the attention of many researchers, and its practical significance is self-evident. For example, Sharma et al. designed a multimodal generative adversarial network (MM-GAN) for synthesizing missing MRI pulse sequences. Dorent et al. designed a heterogeneous modality variational encoder-decoder (HVED) for tumor segmentation and modality completion in the absence of multimodal information. Chen et al. proposed a multimodal brain tumor segmentation framework based on feature decoupling and gated fusion, aiming to enhance robustness in the absence of multimodal information. Liu et al. designed a multimodal masked autoencoder (M3AE) for brain tumor segmentation in the absence of multimodal information. Zhang et al. proposed a multimodal medical Transformer model (mmFormer) for brain tumor segmentation in the absence of multimodal information. However, these methods still have problems such as limited robustness to missing modalities, low efficiency of information fusion between modalities, and low segmentation accuracy. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention proposes a glioma segmentation method based on an XLSTM-HVED network.

[0005] In order to solve the above technical problems, the technical solution of the present invention is:

[0006] A glioma segmentation method based on XLSTM-HVED network includes the following steps:

[0007] Step 1, obtaining a glioma data set, and dividing the data set into a training set and a test set; the data set includes multimodal data, and the multimodal data includes at least one of four magnetic resonance imaging modalities: T1, T1c, T2 and FLAIR;

[0008] Step 2, constructing an XLSTM-HVED neural network model for glioma segmentation, wherein the XLSTM-HVED neural network model includes a SAVE encoder, a Vision XLSTM attention module, a segmentation decoder, a reconstruction decoder and a DuSFE module;

[0009] Step 3: Use the training set to train the XLSTM-HVED neural network model, and optimize the parameters of the XLSTM-HVED neural network model based on the characteristics of glioma images;

[0010] Step 4: Use the test set to test and evaluate the obtained XLSTM-HVED network model, and finally achieve the missing modality segmentation of brain glioma.

[0011] Preferably, the SAVE encoder adopts an MVAE encoder.

[0012] Preferably, the MVAE encoder comprises a DRB dimensionality reduction block.

[0013] Preferably, the Vision XLSTM attention module adopts the Vision-lstm module.

[0014] Preferably, the DuSFE module includes a cSFE module and a sSFE module.

[0015] Preferably, the SAVE encoder encodes the multimodal input through the MVAE encoder, then processes the features obtained from the encoding through the convolution layer and spatial attention, and uses the input multimodal features to perform attention processing on the obtained features to obtain MVAE multimodal features; at the same time, the encoded MVAE multimodal features are decoded;

[0016] The MVAE encoder fuses the mean μ and covariance ∑ of each modality into the latent variable z using Gaussian product, and the sampled latent variable z is decoded into the image space by a reparameterization method, and the features of the obtained image space are optimized by the DRB dimensionality reduction block to obtain the DRB multimodal features;

[0017] The multimodal data input is input of at least one magnetic resonance imaging modality among four magnetic resonance imaging modalities (T1, T1c, T2 and FLAIR).

[0018] The latent variable z is obtained by sampling the mean μ and covariance ∑ of each mode through Gaussian product fusion.

[0019] Finally, the MVAE and DRB multimodal features are added to the unimodal latent variable z through the downsampling layer to obtain the shape features (B, C, H, W, D) and input into the Vision XLSTM encoder.

[0020] Where B refers to the number of input data in a batch, C refers to the number of channels of the input data, H refers to the height of the data, W refers to the width of the data, and D refers to the depth of the data.

[0021] Preferably, in the Vision XLSTM attention module, the shape features (B, C, H, W, D) are first flattened to obtain features (B, H*W*D, C), and then the features (B, H*W **D, C) are input into the Vision-lstm module; the Vision-lstm module first divides the input feature map with a shape of (B, H*W*D, C) into non-overlapping H*W*D tags through a shared linear projection; then, a learnable position embedding is added to each tag to obtain a ViL feature.

[0022] Preferably, the ViL feature and the DRB multimodal feature are connected to a segmentation decoder and a reconstruction decoder, and the segmentation decoder and the reconstruction decoder output modality F respectively. 1 and mode F 2 , then the modal F1 and modal F 2 Input to the DuSFE module.

[0023] Preferably, the cSFE module first uses global average pooling to transform the modal F 1 and mode F 2 Compressed into vector V 1 and V 2 , and concatenate them, and pass through the fully connected layer to obtain the fusion vector V fuse ; Then use V fuse The vector R is obtained by two fully connected layers respectively. 1 and vector R 2, and finally use channel multiplication to transform the vector R 1 and vector R 2 Applying back the two input modes, generates F 1_cSFE and F 2_cSFE , the expression is as follows:

[0024]

[0025] V fuse =w[V 1 , V 2 ]+b

[0026] F 1_cSFE =δ(w 1 V fuse +b 1 )⊙F 1 , F 2_sSFE =δ(w 2 V fuse +b 2 )⊙F 1

[0027] Where c refers to F 1 and F 2 The c-th channel of , σ(·) refers to the sigmoid function, ⊙ refers to the channel-wise multiplication, [·] refers to the channel-level concatenation operator, b, b1, b2 refer to the bias terms in the neural network, and w1, w2 refer to the weight terms in the neural network.

[0028] Preferably, in the sSFE module, the mode F 1 and mode F 2 It is first processed through a convolutional layer, then concatenated, and then another layer of convolution to generate M fuse ; Then, M fuse is fed into two separate convolutional layers, where spatial multiplication is applied to the input pattern output F 1_sSFE and F 2_sSFE , the expression is as follows:

[0029] M fuse =K fuse *[K in1 *F 1 , K in2 *F 2 ]

[0030]

[0031] Where * refers to the convolution operator, refers to the spatial multiplication operator, K in1 , K in2 Refers to two modes for processing F 1and mode F 2 The convolution kernel of the convolution layer, K out1 , K out2 Refers to two for processing M fuse The convolution kernels of two independent convolutional layers.

[0032] Preferably, the training method of the XLSTM-HVED neural network model is: the XLSTM-HVED neural network model is trained using a training set, combined with the characteristics of glioma images, implemented through a PyTorch framework, and trained using NVIDIA Tesla V100; during the training process, the batch is set to 2, the learning rate is set to 0.0001, and the training duration is set to 72h.

[0033] Preferably, in step 4, two methods are used for evaluation: Dice score to measure the degree of overlap of regions and HD95 score to measure the degree of overlap of boundaries.

[0034] The present invention has the following characteristics and beneficial effects:

[0035] By adopting the above technical solution, the present invention integrates the multimodal features and multimodal coding features obtained by variation through Vision XLSTM, deeply fuses multimodal information and enhances the capture of temporal and spatial features. The present invention interactively upsamples information through a cross-perception module on the encoder of the multi-task missing modality reconstruction and segmentation task, and freezes part of the pre-trained reconstruction task encoder to enhance cross-modal feature fusion and improve the performance of multi-task learning. The present invention proposes a single-multimodal variational encoder, which uses multimodal feature attention to perceive the important features of a single modality and finally fuses the single modal features through a variational encoder, thereby improving the downsampling capability of missing data and capturing the intrinsic structure and distribution characteristics of the data. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0037] Figure 1 This is the overall structure diagram of the XLSTM-HVED neural network model of the present invention;

[0038] Figure 2 This is a schematic diagram of the Vision XLSTM attention module of the present invention;

[0039] Figure 3Schematic diagram of the channel squeeze fusion excitation (cSFE) and space squeeze fusion excitation (sSFE) modules in the squeeze-fusion-excitation cross-perception module of the present invention. DETAILED DESCRIPTION

[0040] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0041] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and the like are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first", "second", and the like may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0042] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood by specific circumstances.

[0043] The present invention provides a glioma segmentation method based on XLSTM-HVED network, such as Figure 1 As shown, the following steps are included:

[0044] Step 1: Obtain the glioma dataset and divide the dataset into a training set and a test set.

[0045] Specifically, in this embodiment, a glioma Brats2024 dataset is established. The glioma Brats2024 dataset is a multimodal brain tumor segmentation challenge (BraTS) 2024 dataset. The dataset contains imaging data of T1, T1ce, T2 and FLAIR MRI modes of about 4,500 patients with various types of gliomas. The imaging data is resampled to an isotropic 1 mm 3The images were aligned with the anatomical template and then preprocessed with skull stripping. The labels were manually marked by experts, and the segmentation targets were three regions: whole tumor, enhanced tumor and tumor core. The dataset was randomly divided into 80% training set and 20% test set.

[0046] Step 2, construct an XLSTM-HVED neural network model for glioma segmentation, the XLSTM-HVED neural network model includes a SAVE encoder, a Vision XLSTM attention module, a segmentation decoder, a reconstruction decoder and a squeeze-fusion-excitation cross-perception module (DuSFE module); wherein the SAVE encoder includes an MVAE encoder. The Vision XLSTM attention module adopts the Vision-lstm module. The DuSFE module includes channel squeeze fusion excitation (cSFE) and spatial squeeze fusion excitation (sSFE).

[0047] Step 3: Use the training set to train the XLSTM-HVED neural network model, and optimize the parameters of the XLSTM-HVED neural network model based on the characteristics of glioma images.

[0048] Specifically, in this embodiment, the XLSTM-HVED neural network model is trained using the training set, combined with the characteristics of glioma images, and has been implemented through the PyTorch framework and trained using NVIDIA Tesla V100. During the training process, the batch size is set to 2, the learning rate is set to 0.0001, and the training duration is set to 72h.

[0049] Step 4: Use the test set to test and evaluate the obtained XLSTM-HVED network model, and finally achieve the missing modality segmentation of brain glioma.

[0050] Specifically, in order to evaluate the model's classification performance on images, XLSTM-HVED is compared with other classification models on the glioma Brats2024 dataset. The present invention uses two standard indicators to evaluate the classification performance. They are: Dice scores, which measure the degree of overlap of regions, and HD95 scores, which measure the degree of overlap of boundaries. Their formulas are as follows:

[0051]

[0052] Where |X∩Y| is the intersection between X and Y, |X| and |Y| represent the number of elements in X and Y, x and y are the elements in X and Y, d represents the distance, and HD95 is the calculation based on the 95th percentile of the distance between the boundary points in X and Y.

[0053] Finally, the trained and evaluated XLSTM-HVED model was used to perform missing modality segmentation of brain glioma.

[0054] The specific method is as follows: first, the SAVE encoder encodes the multimodal data input through the MVAE encoder, processes the features obtained from the encoding through convolutional layers and spatial attention, and uses the input multimodal features to pay attention to the obtained features to obtain MVAE multimodal features; at the same time, the encoded MVAE multimodal features are decoded.

[0055] It should be noted that the MAVE encoder is in the condition-independent mode X = x 1 ...x n Developed in , this module is based on variational autoencoders (VAEs) and handles missing data by expanding multimodal inputs.

[0056] The MVAE encoder includes a DRB dimension reduction block.

[0057] The MVAE encoder obtains a latent variable z by fusing the mean μ and covariance ∑ of each modality using Gaussian product sampling, and the sampled latent variable z is decoded into the image space by a reparameterization method, and the features of the obtained image space are optimized by the DRB dimensionality reduction block to obtain the DRB multimodal features;

[0058] The multimodal input is an input of at least one of four magnetic resonance imaging modalities (T1, T1c, T2 and FLAIR).

[0059] Finally, the MVAE and DRB multimodal features are added to the unimodal latent variable z through the downsampling layer to obtain the shape features (B, C, H, W, D) and input into the Vision XLSTM encoder.

[0060] Where B refers to the number of input data in a batch, C refers to the number of channels of the input data, H refers to the height of the data, W refers to the width of the data, and D refers to the depth of the data.

[0061] Specifically, in this embodiment, the SAVE encoder is provided with four layers, each layer is connected in sequence, the output of the previous layer of SAVE encoder is used as the input of the next layer of SAVE encoder, and the output of the last layer of SAVE encoder is input to the Vision XLSTM attention module.

[0062] Further, such as Figure 2As shown, in the Vision XLSTM attention module, the shape features (B, C, H, W, D) are first flattened to obtain the features (B, H*W*D, C), and then the features (B, H*W*D, C) are input into the Vision-lstm module; the Vision-lstm module first divides the input feature map with the shape of (B, H*W*D, C) into non-overlapping H*W*D tags through shared linear projection; then, a learnable position embedding is added to each tag to obtain the ViL feature.

[0063] It can be understood that the core of the Vision-lstm module is the mLSTM block with matrix storage and covariance update rule.

[0064] Further configuration of this embodiment is as follows: Figure 3 As shown, the ViL feature and the DRB multimodal feature are connected to a segmentation decoder and a reconstruction decoder, and the segmentation decoder and the reconstruction decoder output modality F respectively. 1 and mode F 2 , then the modal F 1 and mode F 2 Input to the DuSFE module.

[0065] Specifically, the cSFE module first uses global average pooling to transform the modal F 1 and mode F 2 Compressed into vector V 1 and V 2 , and concatenate them, and pass through the fully connected layer to obtain the fusion vector V fuse ; Then use V fuse The vector R is obtained by two fully connected layers respectively. 1 and vector R 2 , and finally use channel multiplication to transform the vector R 1 and vector R 2 Applying back the two input modes, generates F 1_cSFE and F 2_cSFE , the expression is as follows:

[0066]

[0067] V fuse =w[V 1 , V 2 ]+b

[0068] F 1_cSFE =δ(w 1 V fuse +b 1 )⊙F 1 , F 2_sSFE =δ(w 2 Vfuse +b 2 )⊙F 1

[0069] Where c refers to F 1 and F 2 The c-th channel of , σ(·) refers to the sigmoid function, ⊙ refers to the channel-wise multiplication, [·] refers to the channel-level concatenation operator, b, b1, b2 refer to the bias terms in the neural network, and w1, w2 refer to the weight terms in the neural network.

[0070] Specifically, in the SSFE module, the mode F 1 and mode F 2 It is first processed through a convolutional layer, then concatenated, and then another layer of convolution to generate M fuse ; Then, M fuse is fed into two separate convolutional layers, where spatial multiplication is applied to the input pattern output F 1_sSFE and F 2_sSFE , the expression is as follows:

[0071] M fuse =K fuse *[K in1 *F 1 , K in2 *F 2 ]

[0072]

[0073] Where * refers to the convolution operator, refers to the spatial multiplication operator, K in1 , K in2 Refers to two modes for processing F 1 and mode F 2 The convolution kernel of the convolution layer, K out1 , K out2 Refers to two for processing M fuse The convolution kernels of two independent convolutional layers.

[0074] Specifically, in this embodiment, the DuSFE module is provided with two layers, and three groups of segmentation decoders and reconstruction decoders are provided. A group of segmentation decoders and reconstruction decoders are respectively provided at the front end of the first-layer DuSFE module, between the two-layer DuSFE modules, and at the output end of the second-layer DuSFE module. The output of the first group of segmentation decoders and reconstruction decoders is used as the input of the first-layer DuSFE module, the output of the first-layer DuSFE module is used as the input of the second group of segmentation decoders and reconstruction decoders, the output of the second group of segmentation decoders and reconstruction decoders is used as the output of the second-layer DuSFE module, and the output of the second-layer DuSFE module is used as the input of the third group of segmentation decoders and reconstruction decoders. The output of the third group of segmentation decoders and reconstruction decoders is discriminated by a discriminator, and finally the classification result is output by generating a loss.

[0075] In addition, the DRB features input to the first group of segmentation decoders and reconstruction decoders come from the output of the DRB dimensionality reduction block in the third-layer SAVE encoder; the DRB features input to the second group of segmentation decoders and reconstruction decoders come from the output of the DRB dimensionality reduction block in the second-layer SAVE encoder; and the DRB features input to the third group of segmentation decoders and reconstruction decoders come from the output of the DRB dimensionality reduction block in the first-layer SAVE encoder.

[0076] It can be understood that in the above technical solution, the squeeze-fusion-excitation cross-sensing module inputs the ViL features into the segmentation and reconstruction decoders, and the downsampled SAVE features are connected to the decoding process. During upsampling, the 8-dimensional and 16-dimensional features from the two decoders interact through the DuSFE module. The final feature tensor is obtained through the respective modules for segmentation and reconstruction. In pre-training, only the reconstruction decoder is trained, and the 2nd, 3rd and last modules are frozen. The DuSFE module integrates segmentation and reconstruction functions using channel squeeze fusion excitation (cSFE) and spatial squeeze fusion excitation (sSFE) for spatial recalibration.

[0077] Comparative Example:

[0078] The RA-HVED model and the RMBTS model are used as evaluation baselines, and the model designed by the present invention is used to evaluate their performance. The baseline and the model designed by the present invention use the same backbone network for training and testing during the evaluation phase. The present invention selects U-HVED as the backbone network. It is worth noting that the model of the present invention performs well in all evaluation indicators and is particularly effective in multimodal fusion scenarios.

[0079] In addition, the XLSTM-HVED model is compared with several baseline models and state-of-the-art models. Compared with other advanced models such as mmformer, the HD95 index of XLSTM-HVED performs well, especially in the case of multimodal fusion.

[0080] Ablation experiments analyzed the impact of SAVE, ViLA, and SFECA modules on model performance. The results showed that after removing ViLA, the segmentation performance of T1 modality decreased significantly, and both HD95 and Dice indicators deteriorated. After removing SAVE, the overall performance of multimodal fusion decreased, especially when segmenting the entire tumor (WT). After removing the SFECA module, the segmentation performance of T1c and T1 modes decreased significantly, especially the HD95 indicator deteriorated. This shows that these three modules are crucial to improving the accuracy of the model.

[0081] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions and variations of these embodiments including components are made without departing from the principles and spirit of the present invention, and still fall within the scope of protection of the present invention.

Claims

1. A glioma segmentation method based on XLSTM-HVED network, characterized in that: The steps include: Step 1, obtaining a glioma data set, and dividing the data set into a training set and a test set; the data set includes multimodal data, and the multimodal data includes at least one of four magnetic resonance imaging modalities: T1, T1c, T2 and FLAIR; Step 2, constructing an XLSTM-HVED neural network model for glioma segmentation, wherein the XLSTM-HVED neural network model includes a SAVE encoder, a Vision XLSTM attention module, a segmentation decoder, a reconstruction decoder and a DuSFE module; Step 3: Use the training set to train the XLSTM-HVED neural network model, and optimize the parameters of the XLSTM-HVED neural network model based on the characteristics of glioma images; Step 4: Use the test set to test and evaluate the obtained XLSTM-HVED network model, and finally achieve the missing modality segmentation of brain glioma.

2. According to claim 1, a glioma segmentation method based on XLSTM-HVED network is characterized in that: The SAVE encoder adopts an MVAE encoder, and the MVAE encoder includes a DRB dimension reduction block.

3. The glioma segmentation method based on XLSTM-HVED network according to claim 1, characterized in that: The Vision XLSTM attention module adopts the Vision-lstm module.

4. According to the glioma segmentation method based on XLSTM-HVED network in claim 1, the DuSFE module includes a cSFE module and a sSFE module.

5. The glioma segmentation method based on XLSTM-HVED network according to claim 2, characterized in that: The SAVE encoder encodes the multimodal input through the MVAE encoder, then processes the features obtained from the encoding through the convolution layer and spatial attention, and uses the input multimodal data to pay attention to the obtained features to obtain MVAE multimodal features; at the same time, the encoded MVAE multimodal features are decoded; The MVAE encoder obtains a latent variable z by fusing the mean z and covariance ∑ of each modality using Gaussian product sampling, and the sampled latent variable z is decoded into the image space by a reparameterization method to obtain image space features, and the image space features are optimized by the DRB dimensionality reduction block to obtain DRB multimodal features; Finally, the MVAE and DRB multimodal features are added to the unimodal latent variable z through the downsampling layer to obtain the shape features (B, C, H, W, D) and input into the Vision XLSTM encoder, where B refers to the number of a batch of input data, C refers to the number of channels of the input data, H refers to the height of the data, W refers to the width of the data, and D refers to the depth of the data.

6. The glioma segmentation method based on XLSTM-HVED network according to claim 5, characterized in that: In the Vision XLSTM attention module, the shape features (B, C, H, W, D) are first flattened to obtain the features (B, H*W*D, C), and then the features (B, H*W*D, C) are input into the Vision-lstm module; the Vision-lstm module first divides the input feature map with the shape of (B, H*W*D, C) into non-overlapping H*W*D tags through shared linear projection; then, a learnable position embedding is added to each tag to obtain the ViL feature.

7. The glioma segmentation method based on XLSTM-HVED network according to claim 6, characterized in that: The ViL features and DRB multimodal features are connected to a segmentation decoder and a reconstruction decoder, which output modality F1 and modality F2 respectively, and then the modality F1 and modality F2 are input to the DuSFE module.

8. The glioma segmentation method based on XLSTM-HVED network according to claim 7, characterized in that: The cSFE module first uses global average pooling to compress the modal F1 and modal F2 into vectors V1 and V2, and then connects them in series and passes through a fully connected layer to obtain a fusion vector V fuse ; Then use V fuse Vectors R1 and R2 are obtained through two fully connected layers respectively, and finally channel multiplication is used to apply vectors R1 and R2 back to the two input modes to generate F 1_cSFE and F 2_cSFE , the expression is as follows: V fuse =w[V1,V2]+b F 1_cSFE =δ(w1V fuse +b1)⊙F1,F 2_sSFE =δ(w2V fuse +b2)☉F1 Where c refers to the c-th channel of F1 and F2, σ(·) refers to the sigmoid function, ⊙ refers to the channel multiplication, [·] refers to the channel-level concatenation operator, b, b1, b2 refer to the bias terms in the neural network, and w1, w2 refer to the weight terms in the neural network.

9. The glioma segmentation method based on XLSTM-HVED network according to claim 7, characterized in that: In the sSFE module, modality F1 and modality F2 are first processed by a convolutional layer, then concatenated, and then subjected to another convolution layer to generate M fuse ; Then, M fuse is fed into two separate convolutional layers, where spatial multiplication is applied to the input pattern output F 1_sSFE and F 2_sSFE , the expression is as follows: M fuse =K fuse *[K in1 *F1,K in2 *F2] Where * refers to the convolution operator, refers to the spatial multiplication operator, K in1 , K in2 Refers to two convolution kernels used to process the convolution layers of modality F1 and modality F2 respectively, K out1 , K out2 Refers to two for processing M fuse The convolution kernels of two independent convolutional layers.

10. The glioma segmentation method based on XLSTM-HVED network according to claim 1, characterized in that: The training method of the XLSTM-HVED neural network model is as follows: the XLSTM-HVED neural network model is trained using a training set, combined with the characteristics of glioma images, implemented through a PyTorch framework, and trained using NVIDIA Tesla V100; during the training process, the batch size is set to 2, the learning rate is set to 0.0001, and the training duration is set to 72 hours.

11. The glioma segmentation method based on XLSTM-HVED network according to claim 1, characterized in that: In step 4, two methods are used for evaluation: the Dice score is used to measure the degree of overlap of regions, and the HD95 score is used to measure the degree of overlap of boundaries.

Images