A medical image segmentation method based on multi-scan visual state space

Abstract

The application relates to the field of medical image segmentation, in particular to a medical image segmentation method based on a multi-scan visual state space, an MSMamba model is constructed, the MSMamba model is used for medical image segmentation, the MSMamba model comprises a wavelet transform flow module, an encoder module, an intermediate layer module and a decoder module which are sequentially connected, the encoder module comprises a Patch Embedding module and three Patch Merging modules which are sequentially connected, and a multi-scan visual state space module is connected after the Patch Embedding module and each Patch Merging module; the intermediate layer module is a Transpose Conv module; the decoder module comprises four Up-sample Block modules which are sequentially connected; the wavelet transform flow module is introduced in the application, the design synchronously extracts multi-scale space features and multi-frequency detail features of a medical image, and enhances local feature extraction and a receptive field expansion of medical image segmentation.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of medical image segmentation, and in particular to a medical image segmentation method based on a multi-scan visual state space. BACKGROUND

[0002] Existing medical image segmentation methods have the ability to efficiently extract local feature details, but are limited by the fixed receptive field, making it difficult to capture global context information and long-distance pixel dependency relationships in medical images. The scene adaptability is insufficient in the face of variable organ size and fuzzy boundary in medical images, and the multi-scale feature integration capability is weak, which cannot balance the segmentation accuracy of different size anatomical structures.

[0003] Although the existing method can effectively construct long-distance pixel dependency relationships, it has the problem of high computational complexity and memory consumption. When processing high-resolution medical images (such as high-resolution MRI / CT scans), a large amount of computing resources are required, which is difficult to meet the requirements of clinical real-time.

[0004] The existing model has poor coordination between global modeling and local feature extraction, making it difficult to effectively combine global long-distance dependency capture and local feature detail extraction. SUMMARY

[0005] The purpose of the present application is to provide a medical image segmentation method based on a multi-scan visual state space to improve the segmentation accuracy of medical images.

[0006] In order to achieve the above purpose, the technical scheme adopted by the present application is as follows:

[0007] A medical image segmentation method based on a multi-scan visual state space, a MSMamba model is constructed, and the MSMamba model is used for segmentation of medical images. The MSMamba model includes a wavelet transform stream module, an encoder module, an intermediate layer module and a decoder module connected in turn. A skip connection is provided between the encoder module and the decoder module;

[0008] The encoder module includes a Patch Embedding module and three Patch Merging modules connected in turn. The Patch Embedding module and each Patch Merging module are connected with a multi-scan visual state space module;

[0009] The intermediate layer module is a Transpose Conv module, and the decoder module includes four Up-sample Block modules connected in turn;

[0010] The wavelet transform stream module is used for enhancing local feature extraction and receptive field expansion of medical image segmentation, the encoder module adopts multi-scale state space modeling to refine hierarchical feature representation and capture long-range dependencies, the intermediate layer module is used for up-sampling feature maps output by the encoder module, for increasing spatial size of the feature maps output by the encoder, and the decoder module is used for gradually recovering spatial dimensions of features.

[0011] Preferably, the wavelet transform stream module adopts a first convolution module to adjust channels of the input feature maps, performs average pooling operation, wavelet convolution operation and maximum pooling operation on the adjusted feature maps respectively, and performs feature fusion on the feature maps output by the average pooling operation, the wavelet convolution operation and the maximum pooling operation respectively and the adjusted feature maps.

[0012] Preferably, the multi-scan visual state space module specifically processes the input data in the following steps:

[0013] Linear layers and depthwise separable convolutions are adopted to sequentially pre-process the input data;

[0014] A multi-scan module is adopted to scan the pre-processed features according to a preset scan path, and output a plurality of 1D sequences;

[0015] The plurality of sequences output by the multi-scan module are input into a plurality of SSMs modules one by one respectively, and each SSMs module models long-distance dependencies of the input sequence according to its own learnable parameters;

[0016] A multi-merge module is adopted to restore the sequences output by the plurality of SSMs modules into 2D features;

[0017] An efficient channel attention module is adopted to adaptively recalibrate feature information in the channel dimension of the 2D features;

[0018] A residual inverted fully connected layer is adopted to extract features from the features output by the efficient channel attention module.

[0019] Preferably, the skip connection setting includes inputting the feature maps output by the wavelet transform stream module into a last Up-sample Block module of the decoder, and inputting the feature maps output by the first, second and third multi-scan visual state space modules of the encoder into the third, second and first Up-sample Block modules of the decoder one by one respectively.

[0020] By adopting the foregoing design scheme, the application has the following beneficial effects: the wavelet transform stream module is introduced in the application, which synchronously extracts multi-scale spatial features and multi-frequency detail features of medical images, enhances local feature extraction and receptive field expansion of medical image segmentation;

[0021] The application proposes a multi-scan visual state space module, which converts 2D features into 1D sequences by presetting multiple direction scanning paths, captures long distance dependencies through multiple independent state space models, utilizes the advantages of the state space model, and models long distance pixel dependencies through linear complexity;

[0022] The application constructs a coding and decoding structure based on the wavelet transform stream module and the multi-scan visual state space module, realizes efficient collaborative capture of local features and global long distance dependencies, and improves the medical image segmentation precision in the organ size variable and boundary fuzzy scene. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 Fig. 1 is a structural schematic diagram of the MSMamba model of the application;

[0024] Figure 2 Fig. 2 is a structural schematic diagram of the wavelet transform stream module of the application;

[0025] Figure 3 Fig. 3 is a structural schematic diagram of the multi-scan visual state space module of the application;

[0026] Figure 4 Fig. 4 is a comparison result of the medical image segmentation method of the application in organ segmentation of an abdomen MRI (Abdomen MRI data set);

[0027] Figure 5 Fig. 5 is a comparison result of the medical image segmentation method of the application in endoscope instrument segmentation (Endoscopy data set);

[0028] Figure 6 Fig. 6 is a comparison result of the medical image segmentation method of the application in microscope cell segmentation (Microscopy data set). DETAILED DESCRIPTION

[0029] In order to make the purpose, technical scheme and advantages of the application clearer, the application will be further described in detail below in combination with the drawings. Obviously, the described embodiments are only some of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the application.

[0030] The terms "first", "second", "third", and the like in the description and the claims of the present application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a particular order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or device.

[0031] A medical image segmentation method based on multi-scan visual state space, a MSMamba model as shown in Figure 1 is constructed, and the MSMamba model is used for medical image segmentation.

[0032] The wavelet transform stream module is used to enhance local feature extraction and receptive field expansion of medical image segmentation, the encoder module uses multi-scale state space modeling to refine hierarchical feature representation and capture long-range dependencies, the intermediate layer module is used to up-sample the feature map output by the encoder module, and is used to increase the spatial size of the feature map output by the encoder, and the decoder module is used to gradually restore the spatial dimension of the feature.

[0033] The present application designs a wavelet transform stream module (WTStream) as shown in Figure 2 , aiming at enhancing local feature extraction and receptive field expansion of medical image segmentation.

[0034] The wavelet transform stream module uses a first convolution module to adjust the channels of the input feature map, and performs average pooling operation, wavelet convolution operation and maximum pooling operation on the adjusted feature map respectively, and performs feature fusion on the feature maps output by the average pooling operation, the wavelet convolution operation and the maximum pooling operation respectively.

[0035] In this embodiment, it is assumed that the feature map input into the wavelet transform stream module is , and the channel number is adjusted by 1x1 convolution to obtain the adjusted feature map , which aims to provide a unified channel basis for subsequent multi-path feature processing and avoid information loss caused by channel mismatch between different paths.

[0036] ;

[0037] The feature map Averaging pooling is performed to statistically analyze the global information of the medical image, resulting in an average pooling feature map. :

[0038] ;

[0039] For feature maps Perform wavelet convolution operation on the feature map Frequency decomposition is performed to extract low-frequency approximate features and high-frequency detail features, focusing on capturing multi-scale frequency information of medical images and outputting wavelet convolution feature maps. :

[0040] ;

[0041] For feature maps Max pooling is performed to retain the maximum values ​​in local regions of the feature map, extracting salient structures from the medical image to obtain the max pooled feature map. :

[0042] ;

[0043] Finally, the skip links and the output feature maps are combined. , and Feature fusion and channel attention enhancement are performed to provide a high-quality, multi-dimensional feature foundation for hierarchical feature extraction in medical image segmentation. This process is represented by the following formula:

[0044] .

[0045] The encoder module includes a Patch Embedding module and three Patch Merging modules connected in sequence. The Patch Embedding module and each Patch Merging module are then connected to a Multi-Scanning Vision State Space (MSVSS) module.

[0046] The decoder module comprises four Up-sample Block modules connected in sequence; a skip connection is arranged between the encoder module and the decoder module. In this embodiment, the skip connection arrangement comprises inputting the feature map output by the wavelet transform stream module into the last Up-sample Block module of the decoder, and inputting the feature maps output by the first, second and third multi-scan visual state space modules of the encoder into the third, second and first Up-sample Block modules of the decoder respectively one by one; the decoder follows a traditional U-shaped architecture, and is enhanced by dense skip connections to facilitate effective gradient flow and improve cross-scale feature fusion.

[0047] In this embodiment, the intermediate layer module is a Transpose Conv module, and the role of the Transpose Conv is to realize up-sampling to restore a feature map of a smaller size to a larger size, so as to be able to generate a high-resolution output related to the input subsequently. In combination with up-sampling and the like, it helps to gradually restore the spatial dimension of the feature in the decoding stage of the network.

[0048] The present application provides a hybrid architecture MSMamba model based on a multi-scan visual state space block as shown in Figure 1 The MSMamba model is designed to effectively capture local spatial details and global contextual dependencies by integrating multi-scale features and state space modeling. The MSMamba model first processes the input image using a wavelet transform stream. It enhances perceptual feature extraction by decomposing the image into multiple frequency components. This allows for more effective multi-scale feature representation before encoding. The encoder of the MSMamba model consists of four hierarchical stages, each gradually extracting higher-level semantic features while reducing spatial resolution. The first stage employs a 4x4 Patch Embedding module, which reduces the feature resolution to 1 / 4 of the original image size by marking non-overlapping patches. Subsequent stages apply Patch Merging modules for 2x downsampling, effectively reducing spatial dimensions while doubling feature channels. It utilizes multi-scale state space modeling to refine hierarchical feature representation and capture long-range dependencies. Among them, the number of MSVSS modules at each stage is {2, 2, 9, 2}, gradually increasing the model's ability to capture complex patterns. Correspondingly, the feature dimension is {96, 192, 384, 768}, which is doubled after each stage to enhance feature abstraction and expression ability.

[0049] To improve the native up-sampling block in the U-shaped architecture, we integrate additional convolution blocks with residual connections, which enhances feature refinement and preserves spatial information when processing skip-connection features. The specific structure of the up-sampling is as followsFigure 1 As shown in FIG. 12, given the skip-connection features from the i-th encoder and the features from the previous upsampling block , the detailed process is as follows:

[0050] ;

[0051] wherein, represents processing the skip-connection features, represents refining the fused features before upsampling, represents the feature concatenation operation.

[0052] In this embodiment, the multi-scan visual state space module is optimized and constructed based on a state space model (SSM). The multi-scan visual state space module adopts a multi-directional scanning strategy, and converts 2D spatial features into 1D spatial sequences through four different scanning paths. This design can obtain image information from multiple angles, not only capturing global context information, but also preserving the relationship between spatial neighboring pixels, effectively overcoming the limitations of traditional methods in capturing long-distance dependencies and local details. The 1D sequences generated by the four scanning paths are respectively input into four independent state space models (SSMs). SSMs are good at processing sequence data, can effectively capture long-distance dependency relationships, and have linear complexity in computation. In the multi-scan visual state space module, SSMs process sequences of different scanning paths to analyze spatial clues from a global perspective, further enhancing the model's ability to model long-distance dependencies in images, allowing the model to better understand the relationships between different regions in the image. The four sequences processed by SSMs are restored to 2D features and integrated through a multi-merge module. Subsequently, an efficient channel attention mechanism (ECA) is applied to adaptively adjust the channel weights, highlighting features that are more important to the current task and suppressing unnecessary information, thereby improving the network's feature representation ability and making the model more accurate and efficient when processing complex images.

[0053] As shown in FIG. 13, the specific processing steps of the multi-scan visual state space module for input data are as follows: Figure 3 A linear layer (Linear) and a depthwise separable convolution (DWConv) are used to sequentially preprocess the input data; the linear layer performs linear transformation on the data to adjust the dimensions of the data; the depthwise separable convolution is used to enhance the correlation between local features, preparing for subsequent multi-directional scanning.

[0054]

[0055] ​​The preprocessed features are scanned using a multi-scan module according to four preset scan paths, outputting four 1D sequences. The specific scanning process is as follows:

[0056] ;

[0057] in, Indicates the first row and number The features in the column, the four scan paths, and the four 1D sequences produced are as follows:

[0058] Scanning along the row from the top left corner feature to the bottom right corner feature, the resulting sequence is: ;

[0059] Scanning along the column from the top left feature to the bottom right feature, the resulting sequence is: ;

[0060] and : along respectively and Scan in the opposite direction. , .

[0061] The four 1D sequences output by the multi-scan module are input one-to-one into four SSMs modules. Each SSMs module performs long-range dependency modeling on the input sequences based on its own learnable parameters and outputs the result after long-range dependency modeling. The specific process can be represented as follows:

[0062] ;

[0063] ;

[0064] in, , , This represents the learnable parameters in SSM. Indicates the first The first sequence A hidden state, Indicates the first i The first sequence j-1 A hidden state, Indicates the first i One input, Indicates the first i The first input j One input element.

[0065] The four sequences after SSMs are processed by a multi-merge module. To restore 2D features, the specific operation involves converting the four sequences... Returning along their respective scan paths, four 2D features with their original spatial shapes are obtained. These features are then integrated into a final spatial feature map through element-wise addition, achieving the fusion of information from different scan directions.

[0066] An efficient channel attention module is employed to adaptively recalibrate the feature information along the channel dimension of the 2D feature map, highlighting important channel features. First, global pooling is performed on the input feature map. Then, one-dimensional convolution operations are used to construct local dependencies between channels. Next, the weights for each channel are obtained through the Sigmoid activation function. Finally, these weights are multiplied channel-by-channel by the original input feature map. The specific process is as follows:

[0067] ;

[0068] in, Indicates input, Indicates the output. This indicates a global pooling operation. Represents one-dimensional convolution. This represents the Sigmoid activation function.

[0069] A residual inverted fully connected layer is used to extract features from the output of this high-efficiency channel attention module. That is, the features are first increased in dimensionality and then reduced in dimensionality, so that key features can be extracted more fully in the high-dimensional space. The specific process is as follows:

[0070] ;

[0071] in, Indicates input, Indicates the output. and They represent and convolution.

[0072] like Figures 4-6 As shown, the medical image segmentation method of this application demonstrates its performance and scalability across three different medical image segmentation tasks: organ segmentation, instrument segmentation, and cell segmentation. These datasets cover varying image resolutions, modalities, and segmentation challenges, providing valuable insights into the effectiveness and adaptability of the model across diverse medical imaging scenarios. Table 1 shows the overall performance of various methods on different datasets. The method of this application achieves a state-of-the-art average DSC of 79.55% and an NSD of 84.95% on the AbdomenMRI dataset, outperforming various baseline methods. Figure 4In particular, the medical image segmentation method of the present application accurately identifies the shape and type of the target organ, while other methods can fail to detect, misclassify, or inaccurately delineate certain regions of interest. Figure 5 In particular, the medical image segmentation method of the present application outperforms existing models in terms of DSC and NSD metrics on the Endoscopy dataset, as shown in Table 1. Figure 6 As shown in Table 1, the medical image segmentation method of the present application can clearly segment surgical instruments. On the Microscopy dataset, the method of the present application shows improvement over other models in terms of F1-Score, demonstrating its ability to segment cells. As shown in Table 1, the method of the present application can accurately segment most target cells. Figure 6

[0073] Experimental results show that the medical image segmentation method of the present application significantly outperforms existing models in terms of performance. Specifically, on the Abdomen MRI dataset, the average Dice similarity coefficient reaches 79.55%; on the Endoscopy dataset, the average DSC reaches 56.5% and the NSD reaches 58.73%, outperforming existing models. In addition, on the Microscopy dataset validation set, the F1-Score reaches 59.35%.

[0074] Table 1 Comparison of the medical image segmentation method of the present application and other methods on three datasets:

[0075]

[0076] To evaluate whether the improvements proposed in the present application are effective, we conducted ablation experiments, as shown in Table 2. The wavelet transform stream and the multi-scan visual state space block individually improve performance, and their combination achieves the best results. To demonstrate the impact of the number of scan directions on performance, we designed ablation experiments based on the number of scan directions in the visual state space block, as shown in Table 3. The first configuration uses a single scan direction, the second involves two scan directions, and the third applies four scan directions. As the number of scan directions increases, we observe a consistent improvement in model performance, as indicated by metrics such as DSC and NSD. This suggests that merging multiple scan directions helps the visual state space block better capture long-range dependencies and enhance feature extraction.

[0077] Table 2 Ablation experiments of the medical image segmentation method of the present application:

[0078]

[0079] Table 3 Ablation experiments of the multi-scan mechanism of the medical image segmentation method of the present application:

[0080] ​

[0081] The experimental results show that the medical image segmentation method of the present application has achieved significant improvement in performance over existing models. Specifically, on the Abdomen MRI dataset, the average Dice similarity coefficient reaches 79.55%; on the Endoscopy dataset, the average DSC reaches 56.5%, and the NSD reaches 58.73%, which is superior to existing models. In addition, on the Microscopy dataset validation set, the F1-Score reaches 59.35%.

[0082] In summary, the present application introduces a wavelet transform flow module, which simultaneously extracts multi-scale spatial features and multi-frequency detail features of medical images, enhancing local feature extraction and receptive field expansion for medical image segmentation.

[0083] The present application proposes a multi-scan visual state space module, which converts 2D features into 1D sequences using a preset multiple direction scanning path, captures long distance dependencies through multiple independent state space models, and uses the advantages of state space models to model long distance pixel dependencies with linear complexity.

[0084] The present application constructs a coding and decoding structure based on the wavelet transform flow module and the multi-scan visual state space module, which realizes efficient collaborative capture of local features and global long distance dependencies, and improves the accuracy of medical image segmentation in organ size variable and boundary fuzzy scenarios.

[0085] The above specific embodiments further illustrate the purpose, technical solutions and advantages of the present application. It should be understood that the above description is only a specific embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A medical image segmentation method based on multi-scan visual state space, characterized in that: An MSMamba model is constructed and used for medical image segmentation. The MSMamba model includes a wavelet transform stream module, an encoder module, an intermediate layer module, and a decoder module connected in sequence. A skip connection is set between the encoder module and the decoder module. The encoder module includes a Patch Embedding module and three Patch Merging modules connected in sequence. Each Patch Embedding module and each Patch Merging module is then connected to a multi-scan visual state space module. The intermediate layer module is the Transpose Conv module, and the decoder module includes four Up-sampleBlock modules connected in sequence. The wavelet transform flow module is used to enhance local feature extraction and receptive field expansion in medical image segmentation. The encoder module uses multi-scale state space modeling to refine hierarchical feature representation and capture long-range dependencies. The intermediate layer module is used to upsample the feature map output by the encoder module to increase the spatial size of the feature map output by the encoder. The decoder module is used to gradually restore the spatial dimension of the features. The wavelet transform flow module uses the first convolution module to adjust the channels of the input feature map, and then performs average pooling, wavelet convolution and max pooling operations on the adjusted feature map. The feature maps output by the average pooling operation, the wavelet convolution operation and the max pooling operation are then fused with the adjusted feature map. The specific processing steps for the input data by the multi-scan vision state space module are as follows: The input data is preprocessed sequentially using linear layers and depthwise separable convolutions. The preprocessed features are scanned using a multi-scan module according to a preset scanning path, and multiple 1D sequences are output. The multiple sequences output by the multi-scan module are input into multiple SSMs modules one by one, and each SSMs module performs long-distance dependency modeling on the input sequences based on its own learnable parameters. A multi-merging module is used to restore the 2D features of the sequences output by multiple SSMs modules; An efficient channel attention module is used to adaptively recalibrate the feature information in the channel dimension of this 2D feature; A residual inverted fully connected layer is used to extract features from the output of this high-efficiency channel attention module. The skip connection setup includes inputting the feature map output by the wavelet transform stream module into the last Up-sample Block module of the decoder, and inputting the feature maps output by the first, second, and third multi-scan visual state space modules of the encoder into the third, second, and first Up-sample Block modules of the decoder, respectively.

Images