Visual Mangban high-order morphological image segmentation method and system based on state space
By employing a state-space-based visual Mamba higher-order morphological image segmentation method, combined with a U-shaped network and a multi-scale attention mechanism, the shortcomings of image edge regions in medical image segmentation are addressed, achieving efficient and accurate image segmentation results.
Patent Information
- Application Number
- CN202411123931.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-16
- Publication Date
- 2024-11-26
AI Technical Summary
Existing medical image segmentation methods have shortcomings when segmenting image edge regions. In particular, fully convolutional neural networks lack the ability to model global information, and transformer-based methods suffer from a surge in computational cost and insufficient robustness.
We employ a state-space-based Visual Mamba higher-order morphological image segmentation method. By extracting features through the HMM-VSS module and combining a U-shaped network and a multi-scale attention mechanism, we utilize morphological methods to emphasize the boundary features of lesion areas, reduce noise interference, and preserve morphological details. The resulting image segmentation model is designed as a 6-layer structure.
It improves the segmentation accuracy of image edge regions, significantly enhances the accuracy and robustness of medical image segmentation, while maintaining computational efficiency, making it suitable for medical image analysis.
Smart Images

Figure CN119027430A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of medical image segmentation technology, and in particular to a state-space-based visual Mamba high-order morphological image segmentation method and system. Background Art
[0002] With the development of deep learning in the field of medical image segmentation, convolutional neural networks have gradually been applied to cancer lesion segmentation. U-Net captures rich feature information through encoder-decoder and skip connections. Attention U-Net introduces an attention mechanism in the fully convolutional network to suppress low-sensitivity features and enhance lesion area information. Researchers proposed a channel-level contrast single-domain generalization method to perform subsequent contrast training by performing low-level feature enhancement. MHorUNet introduced high-level spatial interaction operations in the fully convolutional network to alleviate feature information redundancy and achieved excellent performance in skin cancer lesion segmentation. However, due to the inherent limitations of convolution, fully convolutional neural networks lack the ability to model global information. For methods based on the Transformer framework, Swin Transformer enhances the effectiveness of Vision Transformer by focusing on global information through a hierarchical design, but due to the scarcity of medical images, some researchers began to combine CNN with Transformer to enhance the robustness of Transformer. TransNorm integrates the Transformer module into the standard U-Net encoder and skip connections to improve the segmentation effect by adaptively recalibrating the skip path, thereby avoiding the limited localization ability of the Transformer due to insufficient underlying features. However, due to the quadratic complexity of the self-attention mechanism, transformer-based methods often lead to a surge in computation. Therefore, it is necessary to design a medical image segmentation architecture that can effectively capture global context information while maintaining linear computational complexity.
[0003] Currently, State Selection Space (SSM) has shown superior performance in medical image segmentation while reducing computational requirements compared to Transformers. Mamb not only provides an efficient solution for processing long sequence data, but also greatly reduces the number of model parameters that need to be trained through parallel training. VM-UNet and VM-UNet-V2 have also achieved excellent results in skin cancer segmentation. In addition, UltraLight VM-UNet confirmed that Vision Mamba can achieve excellent performance in medical image segmentation with a lower computational load compared to Transformer. Although these models have made great progress in segmenting medical images, they still have shortcomings in segmenting edge areas of images. Summary of the invention
[0004] One of the objectives of the present invention is to provide a state-space based visual Mamba high-order morphological image segmentation method to improve the segmentation accuracy of image edge areas.
[0005] In order to solve the above technical problems, the present invention adopts the following technical solution: a state-space-based visual Mamba high-order morphological image segmentation method: establishing an image segmentation model, loading a training set image, and then inputting the image to be segmented into the trained image segmentation model for image segmentation, wherein the segmentation steps of the image segmentation model include:
[0006] (1) Use the high-order morphological method visual state space module (HMM-VSS module) to extract the features of the input image;
[0007] (2) further extracting features through a U-shaped network feature extraction encoder;
[0008] (3) Add HMM-VSS modules to the 3rd to 6th layers of the branch encoder and shared decoder, and set different orders in the 3rd to 6th layers to improve the ability to capture contextual information;
[0009] (4) processing feature information based on scan expansion operation, scan merging operation and S6 module operation using visual state space block (VSS);
[0010] (5) The decoder gradually restores the spatial resolution of the image and fuses the features extracted by the encoder with the features of the decoder through skip connections;
[0011] (6) Applying the external multi-scale attention mechanism (EMWS Attention) and the skip-based channel and spatial attention bridge (SCA Bridge) module to achieve multi-level and multi-scale information fusion;
[0012] (7) Using morphological methods of erosion and dilation operations to emphasize the boundary features of the lesion area, reduce noise interference and retain morphological details;
[0013] (8) Upsampling and feature fusion are performed in the final layer of the decoder to output the final image segmentation result.
[0014] Preferably, the image segmentation model is designed as a 6-layer structure, and the number of channels is set to 8, 16, 32, 64, 128, or 256.
[0015] More preferably, the HMM-VSS module is a high-order morphological method visual state space module, and its structure includes an HMM-SS2D module, and the structure of the HMM-SS2D module includes a Local-SS2D module and an MM-SS2D module.
[0016] More preferably, the HMM-SS2D module retains the ability of SS2D to capture global context information and focuses on local feature information of the image.
[0017] More preferably, the HMM-VSS module is represented as:
[0018]
[0019]
[0020] More preferably, the feature information is cut into two parts on the channel and then input into the deep convolution and HMM-SS2D modules respectively, and the features output by the MM-SS2D module and the Local-SS2D module are Hadamard-multiplied to refocus the global perception field and increase the weight of local features.
[0021] More preferably, in step (7), the MM-SS2D module is used to reduce noise interference in the original image and selectively retain relevant feature information, thereby retaining sufficient morphological details. The MM-SS2D module is represented as:
[0022]
[0023] y=Wave(x)+SS2D(x)+DW(x)+φ(x).
[0024] More preferably, in step (4), the scan expansion operation expands the original input feature image from four directions into different sequences, which are then input into the S6 module, wherein the S6 module allows different sequences to be scanned in parallel in four directions and effectively stores intermediate results, and the scan merging operation re-sums and reversely merges the different feature sequences extracted from the four directions by the S6 module, thereby restoring the input feature image to its original size.
[0025] In addition, the present invention also provides a state-space-based visual Mamba high-order morphological image segmentation system, which includes:
[0026] A processor, used to execute the above-mentioned state-space based visual Mamba high-order morphological image segmentation method;
[0027] A memory that stores data and instructions needed by a processor during execution.
[0028] Compared with the prior art, the present invention has the following beneficial effects:
[0029] 1. The state-space-based visual Mamba high-order morphological image segmentation method proposed in the present invention uses the SS2D morphological method to ensure global and local receptive fields, while reducing the aliasing of feature information to a certain extent;
[0030] 2. The morphological method is incorporated into the Vision Mamba structure for the first time by introducing Selective Scan-2D (SS2D). In addition, the External Multi-Scale Attention Mechanism (EMWS) combines the feature information of the convolution and SS2D modules to achieve multi-space and multi-scale fusion, and mitigates aliasing through wavelet transform and selective state space;
[0031] 3. Experiments on three public medical image cancer segmentation datasets show that HMM-VMamba maintains strong competitiveness and the segmentation accuracy of image edge areas is significantly improved, demonstrating the effectiveness of the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1 It is a schematic diagram of the method flow of the present invention;
[0033] Figure 2 It is a schematic diagram of the structure of the image segmentation model of the present invention;
[0034] Figure 3 This is a schematic diagram of the S6 module structure of the present invention;
[0035] Figure 4 It is a schematic diagram of the HMM-VSS module structure of the present invention;
[0036] Figure 5 It is a schematic diagram of the MM Block module structure of the present invention;
[0037] Figure 6 A schematic diagram showing the comparison of image segmentation results between the method of the present invention and the existing method;
[0038] Figure 7 This is a diagram of the actual image effect during the image segmentation process of the method of the present invention. DETAILED DESCRIPTION
[0039] In order to facilitate the understanding of those skilled in the art, the present invention is further described below in conjunction with embodiments and drawings. The contents mentioned in the implementation modes are not intended to limit the present invention.
[0040] like Figure 1 As shown in FIG, the state-space based visual Mamba high-order morphological image segmentation method mainly includes the following steps: establishing an image segmentation model, loading a training set of images, and then inputting the image to be segmented into the trained image segmentation model for image segmentation. The image segmentation model is the high-order morphological method visual Mamba (HMM-VMamba) model, and the structure of the model is shown in FIG. Figure 2 shown.
[0041] The model combines morphological methods and VMamba for U-shaped networks, including a basic framework consisting of a high-order morphological method visual state space (HMM-VSS) module and a U-shaped network feature extraction encoder. In addition, the external multi-scale attention mechanism (EMWS Attention) proposed in this embodiment uses the morphological method Mamba, wavelet transform and the introduced jump-based channel and spatial attention bridge (SCA Bridge) module to achieve multi-level and multi-scale information fusion (cite{21}). HMM-VMamba is designed as a 6-layer structure with the number of channels set to [8, 16, 32, 64, 128, 256]. This embodiment adds HMM-VSS modules to the 3rd to 6th layers of the branch encoder and shared decoder. In particular, this embodiment sets different orders in the 3rd to 6th layers to improve the ability to capture contextual information.
[0042] This embodiment introduces the visual state space block (VSS) derived from VMamba. The VSS block is based on scan extension operation, scan merge operation and S6 module operation. Figure 3 As shown in part (a) of Figure 1, the scan expansion operation expands the original input feature image from four directions into different sequences, which are then input to the S6 module. The S6 module developed by Mamba allows different sequences to be scanned in parallel in four directions and the (intermediate) results to be stored efficiently. The scan merge operation re-sums and reversely merges the different feature sequences extracted from the four directions by the S6 module, as shown in Figure 1. Figure 3 As shown in (b), the input feature image is restored to its original size.
[0043] HMM-VSS module such as Figure 4 As shown. The HMM-SS2D proposed in this embodiment combines the SS2D module with a morphological method. This method not only retains the ability of SS2D to capture global contextual information, but also focuses on the local feature information of the image. The HMM-VSS module is based on SSM and Transformer optimization improvements. The Transformer framework processes the input feature information by utilizing all sequential tag information. However, capturing more global information makes the attention mechanism very memory-intensive when processing long feature sequences. Although the attention mechanism in the Transformer framework is calculated in parallel to speed up training, each tag generated during the reasoning process requires recalculating the attention of the entire feature sequence. To solve this problem, this embodiment adopts a framework similar to the Transformer and replaces the original self-attention layer with the proposed HMM-SS2D module.
[0044] The SS2D module is different from the Transformer attention mechanism. It benefits from Mamba's ability to process long sequence feature data in linear time while retaining the efficient computational characteristics of the state-space model. However, although SS2D is optimized for computational efficiency and processing long sequence feature information, in medical image processing, it is often necessary to focus on local lesion features at the boundary. To solve this problem, this embodiment proposes HMM-SS2D, which combines the SS2D module with a morphological method. This method not only retains the ability of SS2D to capture global contextual information, but also focuses on the local feature information of the image. The proposed morphological method will be introduced in detail in the EMWS-Attention section. The HMM-VSS module can be described by the formula:
[0045]
[0046]
[0047] The feature information is cut into two parts on the channel and then input into the deep convolution and SS2D respectively to focus on the global feature information. However, the local-SS2D cannot fully focus on the high-level feature information. Therefore, this embodiment performs Hadamard product on the features that have been output by the MM-SS2D module and the Local-SS2D module, refocuses the global perception field, and increases the weight of the local features.
[0048] Morphological methods are essential for processing local feature information in medical images. The proposed MM-SS2D focuses on global information through SS2D and emphasizes the boundary features of the lesion area using the erosion and expansion methods unique to morphological methods. Figure 5 The MM Block in is shown in Figure 1. The erosion method can reduce the interference of subtle noise and remove noise points. The dilation method can enhance local feature details and fill the empty points in the prediction results. The proposed MM-SS2D module can reduce the noise interference in the original image and selectively retain the relevant feature information, thereby retaining sufficient morphological details. The detailed analysis of the morphological method is demonstrated in the ablation experiment. The proposed MM-SS2D is specifically shown in the following formula:
[0049]
[0050] y=Wave(x)+SS2D(x)+DW(x)+φ(x).
[0051] By conducting image segmentation experiments on three public medical image cancer segmentation datasets, the results are as follows Figure 6As shown in the last column of , it can be seen from the image comparison analysis that the method of this embodiment has significantly improved the segmentation accuracy in the edge area of the image compared with other existing image segmentation methods, demonstrating the effectiveness of the model. Figure 7 The model is shown how, during the image segmentation process, the erosion method is used to reduce the interference of subtle noise and remove noise points, and the dilation method is used to enhance local feature details to fill the void points in the prediction results. The image segmentation process shows that this method has an obvious positive effect on the segmentation of image edge areas.
[0052] The state-space-based visual Mamba high-order morphological image segmentation method and system proposed in the present invention realizes the accurate segmentation of subtle lesion areas in medical images by integrating advanced high-order morphological technology and state-space models. The method utilizes the HMM-VSS module and the U-shaped network feature extraction encoder, combined with the multi-scale attention mechanism and morphological operations, to effectively capture and enhance the local and global features in the image while reducing noise interference. In addition, through the optimized Transformer framework and the introduced SCA Bridge module, the present invention significantly improves the ability of feature fusion and information extraction while maintaining computational efficiency. Experimental results show that this method has significantly improved the accuracy, robustness and processing speed of image segmentation, providing an efficient and reliable technical solution for the field of medical image analysis.
[0053] In order to make it easier for ordinary technicians in the field to understand the improvements of the present invention over the prior art, some drawings and descriptions of the present invention have been simplified, and the above-mentioned embodiments are preferred implementation schemes of the present invention. In addition, the present invention can also be implemented in other ways. Any obvious replacement without departing from the concept of the present technical solution is within the protection scope of the present invention.
Claims
1. A state-space based visual Mamba high-order morphological image segmentation method, characterized by: An image segmentation model is established, a training set image is loaded, and then the image to be segmented is input into the trained image segmentation model for image segmentation. The segmentation steps of the image segmentation model include: (1) Use the HMM-VSS module to extract the features of the input image; (2) further extracting features through a U-shaped network feature extraction encoder; (3) Add HMM-VSS modules to the 3rd to 6th layers of the branch encoder and shared decoder, and set different orders in the 3rd to 6th layers to improve the ability to capture contextual information; (4) processing feature information based on scan expansion operation, scan merging operation and S6 module operation using visual state space blocks; (5) The decoder gradually restores the spatial resolution of the image and fuses the features extracted by the encoder with the features of the decoder through skip connections; (6) Applying external multi-scale attention mechanism and skip-based channel and spatial attention bridge modules to achieve multi-level and multi-scale information fusion; (7) Using morphological methods of erosion and dilation operations to emphasize the boundary features of the lesion area, reduce noise interference and retain morphological details; (8) Upsampling and feature fusion are performed in the final layer of the decoder to output the final image segmentation result.
2. The state-space-based visual Mamba high-order morphological image segmentation method according to claim 1, characterized in that: The image segmentation model is designed as a 6-layer structure, and the number of channels is set to 8, 16, 32, 64, 128, and 256.
3. The state-space-based visual Mamba high-order morphological image segmentation method according to claim 1, characterized in that: The HMM-VSS module is a high-order morphological method visual state space module, and its structure includes an HMM-SS2D module. The structure of the HMM-SS2D module includes a Local-SS2D module and an MM-SS2D module.
4. The state-space-based visual Mamba high-order morphological image segmentation method according to claim 3, characterized in that: The HMM-SS2D module retains the ability of SS2D to capture global context information and focuses on the local feature information of the image.
5. The state-space-based visual Mamba high-order morphological image segmentation method according to claim 3, characterized in that: The HMM-VSS module is represented as:
6. The state-space-based visual Mamba high-order morphology image segmentation method according to claim 3, characterized in that: The feature information is cut into two parts on the channel, and then input into the deep convolution and HMM-SS2D modules respectively. The features output by the MM-SS2D module and the Local-SS2D module are Hadamard-multiplied to refocus the global perception field and increase the weight of local features.
7. The state-space-based visual Mamba high-order morphology image segmentation method according to claim 3, characterized in that: In step (7), the MM-SS2D module is used to reduce noise interference in the original image and selectively retain relevant feature information, thereby retaining sufficient morphological details. The MM-SS2D module is expressed as:
8. The state-space based visual Mamba high-order morphology image segmentation method according to claim 1, characterized in that: In step (4), the scan expansion operation expands the original input feature image from four directions into different sequences, which are then input into the S6 module. The S6 module allows different sequences to be scanned in parallel in four directions and effectively stores intermediate results. The scan merging operation re-sums and reversely merges the different feature sequences extracted from the four directions by the S6 module, thereby restoring the input feature image to its original size.
9. A state-space based visual Mamba high-order morphological image segmentation system, characterized in that: include: A processor, used to execute the state-space based visual Mamba high-order morphological image segmentation method according to any one of claims 1 to 8; A memory that stores data and instructions needed by a processor during execution.
10. An application of the state-space-based visual Mamba high-order morphological image segmentation system according to claim 9, characterized in that: Used for medical image segmentation to help clinicians accurately locate the location and extent of lesions.
Citation Information
Patent Citations
Medical image segmentation method based on edge optimization and attention mechanism
CN116958160A
MRI (Magnetic Resonance Imaging) brain tumor segmentation method based on multi-scale feature fusion of improved U-Net
CN117876399A
Kettle defect semantic segmentation detection system and method based on UMamba
CN118396973A
Cited By
U-shaped neural network skin lesion segmentation method and equipment
CN119228827A
Method for automatically generating medical report by using multi-modal feature fusion Mamba
CN119479969A
Cancer image processing method and system based on Mamba structure, and medium
CN120070355A
Medical image segmentation method based on multi-scanning visual state space
CN120953621A
A medical image segmentation method based on multi-scan visual state space
CN120953621B
