Medical image segmentation method based on parallel bidirectional Mama and neural memory decoding
The medical image segmentation network model PDMamba, constructed using parallel bidirectional Mamba and a neural memory decoder, solves the problems of large parameter quantity and high memory consumption in high-dimensional medical image segmentation, and achieves a significant improvement in segmentation accuracy.
Patent Information
- Application Number
- CN202511596766.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-04
- Publication Date
- 2026-02-03
AI Technical Summary
Existing medical image segmentation methods face challenges in processing high-dimensional images, including large model parameters, high memory consumption, and insufficient modeling of long-distance dependencies. In particular, the computational complexity is high in high-resolution medical image segmentation, and the traditional Mamba model presents challenges in terms of parameter efficiency and memory consumption.
We employ a parallel bidirectional Mamba and a neural memory decoder. By using a parallel bidirectional scanning mechanism and a neural memory decoder, we construct a medical image segmentation network model PDMamba. We utilize the parallel bidirectional Mamba module for feature channel splitting and bidirectional Hilbert scanning, and combine it with the neural memory ODE module for decoding, thereby reducing model parameters and improving segmentation accuracy.
It significantly reduces the number of model parameters and memory usage, while improving the accuracy and efficiency of medical image segmentation. It can effectively capture long-distance dependencies and improve segmentation performance.
Smart Images
Figure CN121458737A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the technical field of image segmentation, and specifically relates to a medical image segmentation method based on parallel bidirectional Mamba and neural memory decoding. BACKGROUND
[0002] Medical image segmentation refers to the process of extracting regions of interest such as tissues, organs or lesions from medical images. It can assist doctors in disease diagnosis, analysis and treatment, and has important application value in clinical practice.
[0003] In recent years, the rapid development of deep learning technology has provided an effective solution for medical image segmentation. Among them, convolutional neural networks (CNN) and Transformers dominate the field. However, CNN relies on local convolution operations, resulting in limited receptive fields. Although the receptive field can be expanded by stacking convolution layers or using dilated convolutions, the modeling ability of long-distance dependencies is still insufficient. In addition, the morphology of organs or lesions in medical images is variable, and the segmentation effect of CNN on small targets or fuzzy boundary regions often decreases due to local computation. Transformer achieves global dependency modeling through self-attention mechanism, bringing new ideas to medical image segmentation. However, its computational complexity grows quadratically with the size of the input image, and in actual deployment, especially when segmenting high-resolution medical images, it often faces the challenge of limited resources on embedded devices. Therefore, it is very important to effectively reduce the model parameter quantity and computational complexity without affecting the model accuracy.
[0004] Mamba, as a state space sequence model, scans the entire image at a time while maintaining linear complexity, making it a key to achieving model lightweight. Mamba introduces a selective state space mechanism, allowing it to dynamically adjust the state update strategy based on the importance of input tokens. In addition, Mamba can effectively capture long-distance dependencies by parameterizing the input of the state space model. Although Mamba has achieved good results in medical image segmentation tasks, it still has the following problems: (1) The number of channels in Mamba has an explosive impact on model parameters, resulting in challenges in parameter efficiency and memory occupation when processing high-dimensional medical images based on Mamba-based segmentation models; (2) Mamba has weak linear scanning space correlation and missing one-way context information in long sequence modeling, affecting the segmentation accuracy of the model. SUMMARY
[0005] To overcome the problems of existing methods, this invention proposes a medical image segmentation method based on parallel bidirectional Mamba and neural memory decoding. By introducing a parallel bidirectional scanning mechanism and a neural memory decoder, the segmentation accuracy of medical images is improved while significantly reducing model parameters.
[0006] The objective of this invention is achieved through the following technical solution: A medical image segmentation method based on parallel bidirectional Mamba and neural memory decoding includes the following steps: Step 1: Construct the medical image segmentation network model PDMamba, perform hyperparameter initialization and model training; The network model PDMamba includes an encoder and a neural memory decoder based on parallel bidirectional Mamba. The encoder comprises four hierarchical Mamba modules: First, shallow features are extracted using depthwise separable convolution; second, the feature maps are input layer by layer into the hierarchical Mamba modules to obtain the output feature maps of the corresponding layers; wherein, the hierarchical Mamba modules include pixel-level parallel bidirectional Mamba and region-level parallel bidirectional Mamba; pixel-level parallel bidirectional Mamba enhances the ability to capture local details by dividing the feature map into multiple large sub-kernels and performing parallel bidirectional Mamba operations on these sub-kernels respectively; region-level parallel bidirectional Mamba effectively models global dependencies by dividing the feature map into multiple representative regions and exchanging information between these regions; the parallel bidirectional Mamba modules split the input feature channels into four groups and process them respectively through a bidirectional Hilbert scan state space model to enhance the ability to model spatial continuity; the bidirectional Hilbert scan state space model simultaneously processes feature sequences in two completely opposite directions along the forward and backward directions of the Hilbert space curve, and then uses the state space model to perform recursive state calculations along their respective sequence directions to capture and fuse bidirectional long-range dependencies in the image; The neural memory encoder includes four neural memory ODE modules; the neural memory ODE module receives features from the encoder jump connection as external input, fuses them with the output of the neural memory ODE module of the next layer, and then passes them to the neural memory ODE module of the previous layer. Step 2: The medical image to be segmented Preprocessing is required; Step 3: Input the preprocessed medical image into the PDMamba network model, and encode it using an encoder based on parallel bidirectional Mamba, extracting the first image from the network in parallel. Feature map of layer , ; Step 4: Combine the feature maps obtained from different layers of the encoder. They are directly transmitted to the corresponding layer's neural memory ODE module of the neural memory decoder via jump connections; Step 5: Decode the feature map using a neural memory decoder to gradually restore the resolution and obtain the feature map. ; Step 6: During the decoding and recovery of the feature map size... Feature map of layer and To integrate with each other; Step 7: Output the final segmentation result image.
[0007] Furthermore, the hyperparameter initialization in step 1 includes: using a combination of cross-entropy loss and Dice loss as the loss function, using the Adam optimizer, setting the training batch size to 24 and the number of training iterations to 1000, and during training, using a learning rate decay strategy, setting the initial learning rate to 0.001 and the weight decay coefficient to 3e-5.
[0008] Furthermore, the medical images from step 2 The specific process of preprocessing is as follows: Step 2.1: The medical image to be segmented Perform normalization processing and resize the image to 256×256; Step 2.2: Perform data augmentation on the normalized medical images, including random rotation, random scaling, elastic deformation, random flipping, and random adjustment of brightness and contrast. Furthermore, the specific processing steps of the parallel bidirectional Mamba encoder in step 3 include: Step 3.1, Set the number of channels to... Input feature map Split into four sub-feature maps The number of channels in each sub-feature map is The process is represented as: Step 3.2: Input each sub-feature map obtained in Step 3.1 into the bidirectional Hilbert scan Mamba module (DH-Mamba) to perform bidirectional sequence modeling; Step 3.3: Take the output of step 3.2 A residual connection is made with the original input, and the skip connection is adaptively scaled by introducing a learnable adjustment factor; Step 3.4: Restore the original number of channels by concatenating the outputs of the four sub-feature maps.
[0009] Further, the specific steps of the bidirectional Hilbert scan Mamba module (DH-Mamba) in step 3.2 for bidirectional sequence modeling are as follows: Step 1: input feature map After the linear layer, the preliminary feature transformation and dimension adaptation are realized; Step 2: the feature map processed in Step 1 is further enhanced through a convolution layer and a nonlinear activation function SiLU to enhance the feature expression ability; Step 3: the feature map obtained in Step 2 is input into the bidirectional Hilbert scan state space model (DH-SSM), which adopts a bidirectional scanning mechanism to process the feature sequence along the forward and backward directions of the Hilbert space filling curve in opposite directions; Step 4: traverse the two-dimensional feature map in the clockwise direction of the Hilbert curve, and convert the pixel points in the spatial dimension into a one-dimensional forward sequence with spatial continuity ; At the same time, the counterclockwise scanning mode is adopted to capture the sequence information in the opposite direction, generating an ordered reverse sequence ; Step 5: linearly project the input on the two paths to generate dynamic parameters , and use the zero-order hold method to discretize the parameters and on the forward and reverse paths, respectively: wherein, is the time scale parameter; is the state matrix of the model; is the input matrix; and are the discretized state matrix and input matrix, respectively; Step 6: use the discretized matrix to obtain the final output feature map , the specific calculation is as follows: wherein, represents the output matrix; represents the skip connection weight; represents the mapping input sequence; represents the predicted output sequence; represents the features with context information generated dynamically inside the model during the scanning image process; Step 7: Output features from both directions and Perform fusion and predict the output sequence Its spatial structure is recovered using the inverse mapping of Hilbert curves; Step 8: Input feature map Dimension matching is performed through linear mapping, followed by the SiLU activation function to obtain the output feature map; Step 9: The outputs of the two branches are multiplied element-wise to interact with each other, thereby enhancing the feature representation capability; Step 10: Transform the input feature map by using a linear layer to compress the number of channels back to their original size, generating an output feature map with the same size as the input feature map.
[0010] Furthermore, the specific steps for the neural memory decoder to decode the feature map in step 5 are as follows: Step 5.1: Construct the Neural Memory ODE (NMO) module to model the feature evolution process as a continuous dynamic system; Step 5.2: Discretize the continuous dynamic system using the explicit Euler discretization method to obtain the discrete state update equation: In the formula, Indicates the level of the current operation; Indicates the first uplink path The layer's input is used as the solution. The initial value; for ; Image features representing skip connections; The parameters represent skip connections; BN represents batch normalization; ReLU is the modified linear unit activation function. The function is a parameterized projection function consisting of convolution and upsampling operations. It extracts spatial features from the input feature map and performs channel dimension transformation through learnable convolution kernels. Step 5.3: Stack multiple NMO modules to form a neural memory decoder, wherein each NMO module receives skip connection features from the encoder. As external input; Step 5.4: Solve the discrete state update equation using numerical integration to realize the continuous evolution of the hidden state from the initial state to the target segmentation result.
[0011] Furthermore, in step 5.2 The specific implementation method of the function is as follows: (a) To Feature dimension reduction and channel number reduction are performed through convolution; (b) performing nonlinear feature mapping on the feature map processed in step (a) using a ReLU function; (c) performing upsampling operation on the feature map processed in step (b) using a bilinear interpolation method, so that it is the same size as .
[0012] Further, the specific steps for fusing the feature maps and with each other in step 6 are as follows: Step 6.1: adding the feature and the feature map obtained in step 5 to fuse the features from different paths; Step 6.2: performing batch normalization and nonlinear activation on the feature map processed in step 6.1; Step 6.3: performing residual connection on the feature map processed in step 6.2 and .
[0013] Compared with the prior art, the present application has the following beneficial effects: Firstly, the input feature channels are split into four parallel groups for processing by the parallel bidirectional Mamba module (PDM), and each group uses an independent state space model for bidirectional Hilbert scanning, which significantly reduces the parameter explosion problem caused by the increase in the number of channels, improves the computing efficiency, and reduces the memory occupation. Through nonlinear scanning path, the spatial continuity modeling capability is enhanced, effectively alleviating the problem of context information loss in traditional linear scanning. Secondly, a lightweight neural memory decoder is designed, which models the decoding process as a dynamic system evolution process driven by ordinary differential equations, replaces the traditional convolution operation through the parameter sharing mechanism, avoids the storage demand of intermediate activation values, and greatly compresses the model parameter quantity. The decoder maintains the feature reconstruction quality while achieving efficient memory utilization. BRIEF DESCRIPTION OF DRAWINGS
[0014] Figure 1 is the network structure diagram of the PDMamba of the present application; Figure 2 is the structure diagram of the DH-Mamba module of the present application; Figure 3 is the structure diagram of the state space model of the bidirectional Hilbert scanning of the present application; Figure 4 is a visualization diagram of segmentation results on an abdominal multi-organ data set; Figure 5 is a visualization diagram of segmentation results on a heart data set. DETAILED DESCRIPTION
[0015] The specific content of the present application is further explained in detail in combination with the following examples.
[0016] The present application is further described in detail in combination with the accompanying drawings and specific examples.
[0017] Referring to Figures 1 to 3 , a medical image segmentation method based on parallel bidirectional Mamba and neural memory decoding, the specific segmentation steps include: Step 1, constructing a medical image segmentation network model PDMamba, and performing hyperparameter initialization and model training, the specific steps include: The combination of cross-entropy loss and Dice loss is used as the loss function, the Adam optimizer is used, the training batch size is set to 24, the training iteration number is set to 1000, in the training process, the learning rate decay strategy is used, the initial learning rate is set to 0.001, and the weight decay coefficient is 3e-5.
[0018] Step 2, preprocessing the medical image to be segmented , the specific steps mainly include: Step 2.1, normalizing the medical image to be segmented , and adjusting the size of the image and the label to 256x256.
[0019] Step 2.2, data augmentation is performed on the normalized medical image, including random rotation, random scaling, elastic deformation, random mirror flipping, and random adjustment of brightness, contrast, etc.
[0020] Step 3, inputting the preprocessed medical image into the network model PDMamba, encoding in the encoder based on parallel bidirectional Mamba, and extracting the feature map of the first layer in the network in parallel, the specific steps mainly include: Step 3.1, splitting the input feature map with a channel number of into four sub-feature maps , each sub-feature map has a channel number of , and the process is represented as: Step 3.2, inputting each sub-feature map obtained in step 3.1 into a bidirectional Hilbert scanning Mamba module (DH-Mamba) respectively, and performing bidirectional sequence modeling, the specific steps mainly include: Step 1: input feature map A linear layer is used to implement the initial feature transformation and dimension adaptation; Step 2: The feature map processed in Step 1 is further enhanced by a convolutional layer and a nonlinear activation function SiLU to enhance the feature expression capability; Step 3: The feature map obtained in Step 2 is input into the state space model of bidirectional Hilbert scanning (DH-SSM), which uses a bidirectional scanning mechanism to process the feature sequence in two completely opposite directions along the forward and backward directions of the Hilbert space filling curve; Step 4: Traverse the two-dimensional feature map in the clockwise direction of the Hilbert curve, and convert the pixel points in the spatial dimension into a one-dimensional forward sequence with spatial continuity ; At the same time, use the counterclockwise scanning mode to capture sequence information in the opposite direction to generate an ordered reverse sequence ; Step 5: Linearly project the input on the two paths to generate dynamic parameters , and and respectively use zero-order hold method for discretization: where, is the time scale parameter; is the state matrix of the model; is the input matrix; and are the discretized state matrix and input matrix, respectively; Step 6: Use the discretized matrix to obtain the final output feature map , the specific calculation is: where, represents the output matrix; represents the skip connection weight; represents the mapping input sequence; represents the predicted output sequence; represents the dynamically generated features with context information inside the model during the scanning image process; Step 7: Fuse the output features of the two directions and . Finally, the output sequence uses the inverse mapping of the Hilbert curve to restore its spatial structure; Step 8: input feature map After a linear mapping for dimension matching, the output feature map is obtained through the activation function SiLU; Step 9: the outputs of the two branches are multiplied element by element to interact with the features, thereby enhancing the feature expression ability; Step 10: a linear layer is used for transformation to compress the channel number back to the original size, generating an output feature map with the same size as the input feature map.
[0021] Step 3.3, the output of step 3.2 is connected with the original input in residual connection, and the jump connection is adaptively scaled by introducing a learnable adjustment factor; Step 3.4, the outputs of the four sub-feature maps are restored to the original channel number through splicing operation.
[0022] Step 4, the feature maps obtained by the encoder at different layers are respectively transmitted through jump connection to the neural memory ODE module of the corresponding layer of the neural memory decoder; Step 5, the feature map is decoded by the neural memory decoder to gradually restore the resolution, and the feature map is obtained. Its specific steps mainly include: Step 5.1: build a neural memory ODE (NMO) module to model the feature evolution process as a continuous dynamic system; Step 5.2: discretize the continuous dynamic system by explicit Euler discretization method to obtain the discrete state update equation: In the formula, represents the number of layers of the current operation; represents the input of the first layer of the uplink path, which is used as the initial value of ; is ; represents the image feature of the jump connection; represents the parameter of the jump connection; BN represents batch normalization operation; ReLU is the rectified linear unit activation function; The function is a parameterized projection function composed of convolution and upsampling operation, which extracts spatial features of the input feature map through a learnable convolution kernel and realizes channel dimension transformation; the specific steps mainly include: Step 1: for , perform feature dimension reduction and reduce the number of channels through convolution; Step 2: Apply the ReLU function to the feature map processed in Step 1 to perform non-linear feature mapping; Step 3: Upsample the feature map processed in Step 2 using bilinear interpolation to make it consistent with... They are the same size.
[0023] Step 5.3: Stack multiple NMO modules to form a neural memory decoder, wherein each NMO module receives skip connection features from the encoder. As an external input; Step 5.4: Solve the discrete state update equation using numerical integration to realize the continuous evolution of the hidden state from the initial state to the target segmentation result.
[0024] Step 6: During the decoding and recovery of the feature map size... Feature map of layer and The specific steps for mutual integration mainly include: Step 6.1: For features The feature map obtained in step 5 Add them together to fuse features from different paths; Step 6.2: Perform batch normalization and nonlinear activation on the feature maps processed in Step 6.1; Step 6.3: Combine the feature map processed in Step 6.2 with... Perform residual connection.
[0025] Step 7: Output the final segmentation result image.
[0026] Verification of the present invention: For ease of description, PDMamba will be used to refer to the present invention: a medical image segmentation method based on parallel bidirectional Mamba and neural memory decoding.
[0027] To evaluate the segmentation performance of PDMamba on medical images, lesion multi-organ medical images in public medical image datasets were used for segmentation comparison experiments and ablation experiments. The images in the existing public datasets Abdomen MRI, Synapse and ACDC were used for network training and testing. The Abdomen MRI dataset is a publicly available 2D segmentation dataset from the MICCAI 2022 challenge, including 110 MRI cases and 13 abdominal organs. The size of the 2D MRI image is 320x320. Among them, 60 labeled samples are used for training, and another 50 are used for testing. The Synapse dataset is a 3D multi-organ segmentation dataset, containing 30 abdominal CT scans, a total of 3779 axial clinical images. These images cover 8 abdominal organs, including aorta, gallbladder, spleen, left kidney, right kidney, liver, pancreas and stomach. Each CT image sequence contains 85-198 slices with a size of 512x512. Among them, 18 samples are used for training, and 12 samples are used for testing. The ACDC dataset is a cardiac MRI image containing 100 patients, and the labels include left ventricle, right ventricle and myocardium. The dataset is divided into 70 training samples (1930 axial slices), 10 validation samples and 20 test samples.
[0028] The verification environment is: Intel(R) Xeon(R) Gold 6326 CPU, NVIDIA GeForce RTX 3090 GPU, and 24GB of video memory. The deep learning framework used to implement PDMamba is PyTorch.
[0029] To further compare the segmentation performance of different network models, Intersection over Union (IoU), Dice Similarity Coefficient (DSC), Normalized Surface Distance (NSD), and parameter quantity were selected as four widely used indicators to quantify and evaluate the segmentation performance of the model. The calculation methods are as follows: In the formula, G and S represent the predicted segmentation area and the true segmentation area, respectively; and G and S represent the size of the predicted segmentation area and the true segmentation area, respectively; and G and S represent the boundary of the true segmentation and the predicted segmentation, respectively; and These represent the boundary regions of the actual segmentation and the predicted segmentation within the tolerance τ, respectively. The values of IoU, DSC, and NSD range from [0, 1], with values closer to 1 indicating better segmentation performance.
[0030] First, the effectiveness of the parallel bidirectional scanning Mamba, bidirectional Hilbert scanning state space model, and neural memory decoder designed in this invention was verified through ablation experiments on the Abdomen MRI medical abdominal dataset. LambaKM-UM was selected as the baseline for the ablation experiments in this paper. Specifically, the baseline model removed the PDM, DH-SSM, and NMO modules. In the encoder stage, Mamba using a traditional row-scanning strategy was used for feature extraction, and in the decoder stage, a traditional U-shaped network decoder was used for feature fusion. Based on this, the innovative modules proposed in this invention were gradually integrated for performance testing to rigorously evaluate the independent contribution and synergistic effect of each module. The ablation experiment results are shown in Table 1.
[0031] Table 1. Ablation experiments of various organs on the Abdomen MRI dataset (best results are shown in bold). Table 1 shows the DSC comparison results, mean DSC, mean NSD, and parameter count comparison results of PDMamba in ablation experiments on thirteen organs. Compared with the baseline model, when only the PDM module is introduced, the mean IoU, mean DSC, and mean NSD are improved by 0.95%, 0.72%, and 0.26%, respectively, achieving the best performance on seven organs: liver, right kidney, spleen, aorta, left kidney, right adrenal gland, and esophagus. The number of parameters is reduced from 52.359M to 40.559M, significantly reducing the number of parameters while improving segmentation accuracy. Adding the NMO module on top of the PDM module, using a lightweight neural memory decoder to replace the traditional decoder, reduces the number of parameters by 35.17%, significantly reducing computational complexity and storage requirements. Despite the significant reduction in the number of parameters, the model's segmentation accuracy suffers only a very small loss; on the Abdomen MRI dataset, the model still achieves a mean DSC of 76.87% and a mean NSD of 83.71%, achieving a good balance between efficient segmentation performance and computational complexity. Integrating the DH-SSM module further improved the average IoU, average DSC, and average NSD to 62.80%, 77.15%, and 84.39%, respectively, resulting in improved model performance. Therefore, ablation experiments demonstrate that the PDM and NMO modules can significantly reduce the number of model parameters with almost no loss of accuracy, while the DH-SSM module effectively improves segmentation accuracy.
[0032] Secondly, the superiority of the present invention was verified through comparative experiments. To ensure the fairness of the comparative experiments, all models used the same experimental settings, input images of the same size, and the same data augmentation strategies. On the Abdomen MRI dataset, PDMamba was compared with current mainstream medical image segmentation methods, and the quantitative results are shown in Table 2.
[0033] Table 2 Comparison with other state-of-the-art networks on the Abdomen MRI dataset (bold indicates best results, underline indicates second-best results). As shown in Table 2, compared with other models, PDMamba achieved the best segmentation performance for seven organs: liver, right kidney, spleen, aorta, left kidney, right adrenal gland, and esophagus. It achieved the second-best segmentation performance for pancreas, gallbladder, and left adrenal gland, and the third-best performance for stomach, inferior vena cava, and duodenum. In terms of average IoU, average DSC, average NSD, and number of parameters, PDMamba achieved the best segmentation performance compared to other models. The parallel bidirectional Mamba module, through global receptive field and long-range dependency modeling, can integrate global spatial information when segmenting organs with irregular shapes or complex structures, significantly improving the integrity of the segmentation. On the Abdomen MRI dataset, the average IoU reached 62.80%, the average DSC reached 77.15%, significantly higher than other networks, and the average NSD reached 84.39%, also surpassing other models, further demonstrating that the present invention has better accuracy at the segmentation boundary. Meanwhile, the parallel bidirectional Mamba module, with its efficient sequence scanning mechanism, avoids dependence on deep stacking or large convolutional kernels in terms of computational paradigm, achieving parameter simplification at the feature extraction level. Secondly, the neural memory decoder utilizes a lightweight shared memory to replace the repetitive and dense convolutional layers in traditional decoders. PDMamba's parameter count is only 26.29M, a 46.05% reduction compared to the second-best performing xLSTM-UNet, and a 68.26% reduction compared to U-Mamba. Therefore, PDMamba significantly reduces the number of parameters while maintaining good segmentation performance.
[0034] Visualization results on the Abdomen MRI dataset are as follows: Figure 4 As shown. By Figure 4It can be seen that PDMamba can obtain segmentation results very close to the real target region. Its segmentation results are better than other methods in terms of region coverage integrity and boundary accuracy. Especially when dealing with organs with large deformation, small volume and blurred tissue edges, PDMamba can also accurately capture subtle structural changes to obtain more accurate segmentation regions.
[0035] To further verify the effectiveness of PDMamba, comparative experiments were conducted on the abdominal cavity dataset 3D Synapse and the heart segmentation dataset ACDC, and the results are shown in Tables 3 and 4.
[0036] Table 3 Comparison with other advanced networks on Synapse dataset (the optimal result is in bold, and the suboptimal result is underlined) As can be seen from Table 3, the average IoU of PDMamba on the Synapse dataset is 78.35%, the average DSC is 87.36%, and the average NSD is 90.76%, which is slightly lower than that of MultiSegMamba. However, the parameter amount of PDMamba is the lowest, only 36.86M, which is reduced by 46.16% compared with that of MultiSegMamba and by 78.76% compared with that of U-Mamba. In summary, compared with other advanced methods, PDMamba achieves comparable segmentation accuracy with large models at a lower model parameter amount, realizing better overall performance.
[0037] Table 4 Comparison results with other advanced methods on ACDC dataset (the optimal result is in bold, and the suboptimal result is underlined) As can be seen from Table 4, the average IoU of PDMamba on this dataset is 85.58%, the average DSC is 92.23%, and the highest DSC is achieved on the left ventricle. At the same time, the parameter amount of PDMamba is only 36.86M, which is reduced by 44.82% compared with that of unetr++ method and by 78.76% compared with that of U-Mamba method. The segmentation result visualization on the ACDC dataset is shown in Figure 5 Figure 5 It can be seen that PDMamba can accurately locate the target region and obtain better segmentation results.
Claims
1. A medical image segmentation method based on parallel bidirectional Mamba and neural memory decoding, characterized in that The method comprises the following steps: Step 1, constructing a medical image segmentation network model PDMamba, performing hyperparameter initialization and model training; The network model PDMamba comprises an encoder based on parallel bidirectional Mamba and a neural memory decoder; The encoder comprises four hierarchical Mamba modules: firstly, shallow layer features are extracted by using a depth separable convolution; secondly, feature maps are input into the hierarchical Mamba modules layer by layer to obtain output feature maps of corresponding layers; wherein the hierarchical Mamba modules comprise a pixel-level parallel bidirectional Mamba and a region-level parallel bidirectional Mamba; the pixel-level parallel bidirectional Mamba divides the feature map into a plurality of large sub-kernels, and performs parallel bidirectional Mamba operations on the sub-kernels to enhance the ability to capture local details; the region-level parallel bidirectional Mamba divides the feature map into a plurality of representative regions, and performs information interaction between the regions to effectively model global dependencies; the parallel bidirectional Mamba module splits the input feature channels into four groups, and processes them by using a bidirectional Hilbert scanning state space model to enhance the ability to model spatial continuity; the bidirectional Hilbert scanning state space model simultaneously processes feature sequences in two completely opposite directions along a Hilbert space curve, and then performs recursive state calculation along the respective sequence directions by using a state space model to capture and fuse bidirectional long-range dependencies in the image; The neural memory decoder comprises four neural memory ODE modules; the neural memory ODE module receives features from the encoder jump connection as external input, fuses the features with the output of the neural memory ODE module of the next layer, and then passes the features to the neural memory ODE module of the previous layer; Step 2, preprocessing the medical image to be segmented performing preprocessing; Step 3, input the preprocessed medical image into the network model PDMamba, encode through the parallel bidirectional Mamba-based encoder, and extract the features of the first layer of the network in parallel , ; Step 4, the feature maps obtained by different layers of the encoder are directly transmitted to the neural memory ODE modules of the corresponding layers of the neural memory decoder through the skip connection respectively; Step 4, the feature maps obtained by different layers of the encoder are directly transmitted to the neural memory ODE modules of the corresponding layers of the neural memory decoder through the skip connection respectively; Step 5, decode the feature map by the neural memory decoder, recover the resolution step by step, get the feature map ; Step 6, the feature map size in the decoding recovery process is recovered to the original size of the feature map of the first layer layer of the feature map and are fused with each other; Step 7, outputting a final segmentation result image.
2. The medical image segmentation method based on parallel bidirectional Mamba and neural memory decoding according to claim 1, characterized in that The hyperparameter initialization of step 1 comprises the following steps: using a combination of cross-entropy loss and Dice loss as a loss function, using an Adam optimizer, setting the training batch size to 24 and the training iteration number to 1000, using a learning rate decay strategy during the training process, setting the initial learning rate to 0.001, and setting the weight decay coefficient to 3e-5.
3. The medical image segmentation method based on parallel bidirectional Mamba and neural memory decoding according to claim 1, characterized in that the medical image of step 2 The specific process of pre-processing is: Step 2.1, on the medical image to be segmented Normalization is performed and the image size is adjusted to 256x256; Step 2.2, performing data enhancement on the normalized medical image, including random rotation, random scaling, elastic deformation, random flipping, and random adjustment of brightness and contrast.
4. The medical image segmentation method based on parallel bidirectional Mamba and neural memory decoding according to claim 1, characterized in that The specific processing process of the parallel bidirectional Mamba encoder in step 3 for encoding comprises the following steps: Step 3.1, Set the number of channels to... Input feature map Split into four sub-feature maps The number of channels in each sub-feature map is The process is represented as: Step 3.2, inputting each sub-feature map obtained in step 3.1 into a bidirectional Hilbert scanning Mamba module (DH-Mamba) to perform bidirectional sequence modeling; Step 3.
3. Output of step 3.2 Residual connection with the original input and adaptive scaling of this skip connection by introducing a learnable adjustment factor; Step 3.4, restoring the original channel number by splicing the outputs of the four sub-feature maps.
5. The medical image segmentation method based on parallel bidirectional Mamba and neural memory decoding according to claim 4, characterized in that The specific steps of the bidirectional Hilbert scanning Mamba module (DH-Mamba) in step 3.2 for bidirectional sequence modeling are as follows: Step 1: input feature map The linear layer is used to achieve the initial feature transformation and dimension adaptation. Step 2: The feature map processed in Step 1 is further enhanced in feature expression capability by a convolutional layer and a nonlinear activation function SiLU; Step 3: The feature map obtained in Step 2 is input into a state space model of bidirectional Hilbert scanning (DH-SSM), and a bidirectional scanning mechanism is adopted to process the feature sequence along the forward and backward directions of the Hilbert space-filling curve which are completely opposite to each other; Step 4: traverse the two-dimensional feature map in the clockwise direction of the Hilbert curve to convert the pixel points in the spatial dimension into a one-dimensional forward sequence with spatial continuity ; at the same time, a counterclockwise scanning mode is adopted to capture sequence information in the opposite direction to generate a sorted reverse sequence ; Step 5: Linearly project the inputs on both paths to generate dynamic parameters , and discretize the parameters on both forward and backward paths and using zero-order hold method, respectively: wherein is a time scale parameter; is a state matrix of the model; is an input matrix; and are the discretized state matrix and input matrix, respectively; Step 6: Obtain the final output feature map using the discretized matrix The specific calculation is: wherein represents an output matrix; represents a skip connection weight; represents a mapped input sequence; represents a predicted output sequence; represents a feature with context information dynamically generated inside the model during the process of scanning the image; Step 7: Fuse the output features of both directions and recover their spatial structure using the inverse mapping of the Hilbert curve. and Step 7: Fuse the output features of both directions and recover their spatial structure using the inverse mapping of the Hilbert curve. Step 8: input feature map After dimension matching through linear mapping, the output feature map is obtained through the activation function SiLU. Step 9: The outputs of the two branches are multiplied element by element to interact the features, thereby enhancing the feature expression capability; Step 10: The channel number is compressed back to the original size through a linear layer to generate an output feature map with the same size as the input feature map.
6. The parallel bidirectional Mamba and neural memory decoding based medical image segmentation method according to claim 1, characterized in that The specific steps of the neural memory decoder of step 5 for decoding the feature map are: Step 5.1: Construct a neural memory ODE (NMO) module to model the feature evolution process as a continuous dynamic system; Step 5.2: Discretize the continuous dynamic system by an explicit Euler discretization method to obtain a discrete state update equation: In the formula, denotes the number of layers of the current operation; denotes the input of the uplink path layer , as an initial value for solving ; ; denotes the image features of the skip connection; denotes the parameters of the skip connection; BN denotes a batch normalization operation; ReLU is a rectified linear unit activation function; The function is a parameterized projection function composed of convolution and up-sampling operations, which extracts spatial features from the input feature map through a learnable convolution kernel and realizes channel dimension transformation; Step 5.3: stacking multiple said NMO modules to form a neural memory decoder, wherein each NMO module receives the skip-connection features from the encoder as an external input; Step 5.4: Solve the discrete state update equation by a numerical integration method to realize the continuous evolution of the hidden state from the initial state to the target segmentation result.
7. The medical image segmentation method based on parallel bidirectional Mamba and neural memory decoding according to claim 6, characterized in that The step 5.2 of The specific implementation method of the function is: (a) to Feature dimension reduction and channel number reduction are performed through convolution; (b) using a ReLU function for nonlinear feature mapping on the feature map processed in step (a); (c) using bilinear interpolation to up-sample the feature map after step (b) to make it the same size as the input image.
8. The parallel bidirectional Mamba and neural memory decoding based medical image segmentation method of claim 1, wherein The feature maps in step 6 are fused with each other in a specific way as follows: and The specific method of fusing the feature maps with each other is as follows: Step 6.1: Feature matching Addition of the feature maps resulting from step 5 Fusion of features from different paths Step 6.2: Batch normalization and nonlinear activation are performed on the feature map processed in step 6.1; Step 6.3: The feature map from step 6.2 is concatenated with the residual connection.