A lung CT image segmentation method based on hybrid deep convolution and state space model

Abstract

The present application relates to the field of medical image auxiliary interpretation, in particular to a lung CT image segmentation method based on hybrid deep convolution and state space model, comprising: obtaining a lung CT image to be processed; the lung CT image is divided, mapped, the embedded image is obtained, and the embedded image is normalized; the normalized embedded image is sent to the encoder layer, the height and width of the feature map are reduced and the number of channels is increased; the encoded image is sent to the decoder layer, the height and width of the feature are restored; the decoded image is sent to the linear projection layer, the number of channels is restored, and the segmentation target is matched; the present application can effectively improve the ability of the model to capture remote information under the premise of low calculation cost by combining state space model and hybrid deep convolution.

Description

Technical Field

[0001] This invention relates to the field of medical image-assisted interpretation, specifically to a lung CT image segmentation method based on hybrid depth convolution and state space model. Background Technology

[0002] Lung image segmentation is an important research area in medical image processing. Traditional diagnosis of lung diseases mainly relies on the experience of professional doctors and visual observation, which can lead to misdiagnosis. Introducing deep learning models into medical image interpretation can effectively assist doctors in making judgments, save time and manpower costs, and optimize the allocation of medical resources.

[0003] Currently, CNN-based or Transformer-based models are commonly used in lung image segmentation tasks. However, both CNN-based and Transformer-based models have inherent limitations: CNN-based models can effectively extract local features, but their ability to capture long-range information is limited by the local receptive field; Transformer-based models can effectively capture long-range dependencies, but their high computational cost poses a challenge because their self-attention mechanism scales twice with the input size.

[0004] Therefore, there is an urgent need for a medical segmentation model that can effectively capture remote information and has low computational cost. Summary of the Invention

[0005] To address the problems of insufficient segmentation accuracy due to limited local receptive fields and high computational cost and low efficiency when processing high-resolution images in existing lung image segmentation methods, this invention designs an efficient medical segmentation model by fusing a state-space model and hybrid depthwise convolution.

[0006] The lung CT image segmentation method based on hybrid depth convolution and state space model described in this invention includes:

[0007] Acquire lung CT images to be processed; input the lung CT images to be processed into the trained lung CT image segmentation model to obtain the segmentation result; wherein the lung CT image segmentation model adopts the MCMamba network; the MCMamba network is composed of a Patch Embedding layer, an encoder layer, a decoder layer, and a Linear Projection layer connected in sequence; the encoder layer is composed of an MC-SSM module and a Patch Merging module; the MC-SSM module includes a MixConv module, an SSM module, a fusion layer, two 1×1 convolutional layers, and a normalization layer.

[0008] The beneficial effects of this invention include:

[0009] This invention combines state-space models and hybrid depthwise convolutions, enabling models to accurately extract local features and establish long-distance dependencies, thereby effectively capturing global information and improving segmentation accuracy; it provides a design approach for model research in medical image segmentation. Attached Figure Description

[0010] Figure 1 This is a structural diagram of the MCMamba network described in this invention;

[0011] Figure 2 This is a structural diagram of the MC-SSM module described in this invention;

[0012] Figure 3 This is a structural diagram of the MixConv module described in this invention;

[0013] Figure 4 This is a structural diagram of the SSM module described in this invention;

[0014] Figure 5 The figures show test results under different performance indicators for embodiments of the present invention. Detailed Implementation

[0015] To make the objectives, technical solutions, features, and advantages of the present invention clearer, the present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0016] Obtain the lung CT image to be processed; input the lung CT image to be processed into the trained lung CT image segmentation model to obtain the segmentation result.

[0017] Specifically, the two lung CT images used for training in this embodiment are a CXR lung contour image and a segmentation image of pneumonia caused by Convd19 COVID-19; the images in the selected dataset are all 256×256 pixels in size, and after random flipping and rotation, they are normalized to obtain the training dataset; training the lung CT image segmentation model includes:

[0018] S1. Input the lung CT images from the training data into the Patch Embedding layer for segmentation and mapping to obtain embedded images; normalize the embedded images.

[0019] The partitioning and mapping are processed in the Patch Embedding layer. The Patch Embedding layer divides the lung CT image into 4*4 non-overlapping blocks, each block being 64*64 pixels in size. The non-overlapping blocks are mapped to a fixed dimension, defined as the mapping dimension C, which is preferably 96, to obtain the embedded image X′. The embedded image is then normalized using Layer Normalization.

[0020] S2. The normalized embedded image is fed into the encoder layer and processed by the MC-SSM module and the Patch Embedding module to obtain the encoded image.

[0021] The encoder layer consists of an MC-SSM module and a Patching Merging module. The MC-SSM module extracts local and global features of the image, while the Patching Merging module reduces the height and width of the feature map and increases the number of channels. The encoder and decoder layers are skip-connected, and the data processing process includes four stages. The skip connections use addition operations, and each stage processes and abstracts information from the input feature map to different degrees, including:

[0022] The first stage mainly involves shallow feature extraction, preserving more spatial information. The input of the encoder layer passes sequentially through two connected MC-SSM modules and a Patch Merging module to obtain features with C channels. Figure 1 .

[0023] Phase Two: Focusing on extracting more structural information and features Figure 1 After passing through two connected MC-SSM modules and one Patch Merging module, a feature with 2C channels is obtained. Figure 2 .

[0024] The third stage: further capturing global semantic information and features. Figure 2 After passing through two connected MC-SSM modules and one Patch Merging module, a feature with 4C channels is obtained. Figure 3 .

[0025] Phase Four: Focusing on the abstract representation of high-level features. Figure 3 After passing through two connected MC-SSM modules in sequence, a feature with 8C channels is obtained. Figure 4 .

[0026] The information processing process of the decoder layer includes: First stage: Features Figure 4 After passing through two connected MC-SSM modules in sequence, a feature map with 8C channels is obtained. This feature map is then compared with the feature map... Figure 3 Adding together yields the features Figure 5 Second stage: Characteristics Figure 5 After passing through two consecutive MC-SSM modules and a Patch Merging module, a feature map with 4C channels is obtained. This feature map is then compared with... Figure 2The sum is used to obtain feature map 6; Third stage: Feature map 6 passes through two connected MC-SSM modules and a Patch Merging module to obtain a feature map with 2C channels. This feature map is then combined with the feature map... Figure 1 The summation yields feature map 7; Fourth stage: Feature map 7 passes through a connected MC-SSM module and a Patch Merging module to obtain a feature map with C channels. This feature map is then added to the input of the encoder layer to obtain feature map 8, which is used as the output of the decoder layer.

[0027] Furthermore, the MC-SSM module includes a MixConv module and an SSM module, and the structure of the MC-SSM module is as follows: Figure 2 As shown, the feature maps obtained by the MC-SSM module are input into the MixConv module and the SSM module, respectively.

[0028] The MixConv module has the following module structure diagram: Figure 3 As shown, the feature maps obtained by the MixConv module are input into convolutional kernels of sizes 3*3, 5*5, and 7*7 respectively to group the input feature maps; each group is convolved using a different convolutional kernel and then normalized; the outputs of each group are concatenated; to enhance the feature representation capability, the input of the MixConv module is added to the concatenated vector to obtain the MixConv output.

[0029] Specifically, the input channel tensor is divided into g groups, and all the resulting tensors X have the same spatial height h and spatial width w. The final output tensor is... Concatenate all the partitioned tensors:

[0030]

[0031] c = c1 + c2 + ... + c g

[0032] z0 = z1 + ... + z g =m·c

[0033] Where c is the channel size, X (h,w,c) z0 represents the input tensor of shape (h, w, c), z0 represents the final output channel width, and m represents the channel multiplier, which is used to control the size of the output channel.

[0034] The SSM module has the following module structure diagram: Figure 4As shown, the input features are divided into two paths when passing through the LayerNorm (Layer Normalization) layer. In branch 1, the input passes through a linear layer and then an activation function to obtain the output of branch 1. In branch 2, the input passes through a linear layer, an activation function, and a depthwise separable convolution (DWConv) for processing, and then enters the SS2D (2D-Selective-Scan) module for feature extraction. Furthermore, the SS2D module also includes: scan expansion, S6 block (S6 Block, selective scan mechanism), and scan merging.

[0035] The scan expansion includes unfolding the input image into a sequence along four different directions: top left to bottom right, bottom right to top left, top right to bottom left, and bottom left to top right.

[0036] The S6 block includes: feature extraction of the expanded sequence to ensure thorough scanning of information in all directions, thereby capturing diverse features; furthermore, the S6 block is derived from the Mamba model, adding a selection mechanism based on input-adjusted SSM parameters to the original S4, enabling the model to filter out irrelevant information while distinguishing and retaining relevant information; the operations performed by the S6 block are represented as follows:

[0037] Δ,B,C=Linear(x),Linear(x),Linear(x)

[0038]

[0039] y t =Ch t +Dx t

[0040] y = [y1, y2, ..., y t ,···,y L ]

[0041] Here, x represents the input of the S6 blocks with shape [B,L,D], where B, L, and D represent the batch size, batch length, and batch dimension, respectively. Three linear transformations are performed on x to obtain Δ, B, and C, respectively, and the hidden state h is updated. t , This indicates that A has zero-order hold (ZOH), at which point... It is a discretization parameter. This indicates that B is preserved at order zero, using the hidden state h from the previous time step. t-1 and the input x at the current time t Calculate the output y at the current time. t Combined with the current hidden state h tAnd input x t Output y at all times t The features are concatenated to form the final output y, where y represents a feature of shape [B,L,D].

[0042] The scan merging process involves summing the sequence from four directions and restoring the image to the same size as the input.

[0043] The features are normalized by a horizontal normalization layer and multiplied element-wise with the output of branch 1. The result of the multiplication is passed through a linear layer. The feature map after passing through the linear layer is added to the input of the SSM module to obtain the SSM output. Preferably, SiLU is used as the activation function of the SSM module.

[0044] The feature map output by the MixConv module is merged with the feature map output by the SSM module to obtain the merged feature map.

[0045] Furthermore, the Mixconv ​​output shape is (B, C) MixConv Feature map Y of H,W) MixConv The SSM output shape is (B, C) SSM The feature map of (H,W) is Y SSM Where B represents the channel size, C represents the number of channels, H represents the height of the feature map, W represents the width of the feature map, and Y represents the concatenation. MixConv With Y SSM The resulting shape is (B, C) MixConv +C SSM Feature map Y of H,W) concat .

[0046] A 1×1 convolutional layer is used to perform deep feature extraction on the merged feature map to obtain a deep feature map.

[0047] Furthermore, the original input features are subjected to a 1×1 convolution. The function of this convolutional layer is to perform weighted fusion on each channel through the convolutional kernel, so as to better integrate feature maps from different sources. The output shape is (B, C). out Feature map Y of H,W) fused Perform batch normalization.

[0048] The feature maps obtained by the MC-SSM module are convolved with 1×1 and batch normalized to obtain shallow feature maps.

[0049] The deep feature map is added to the shallow feature map to obtain the fused feature map, which is then used as the output of the MC-SSM module.

[0050] S3. The encoded image is sent to the decoder layer and processed by the MC-SSM module and the Patch Embedding module to obtain the decoded image.

[0051] S4. Send the decoded image to the Linear Projection layer to restore the number of channels.

[0052] Specifically, the projection layer compresses the number of channels in the feature map to match the number of channels in the target segmentation map, so that the feature map output by the decoder matches the target segmentation map in both spatial size and number of channels.

[0053] Figure 5 This is a test graph for this embodiment under different metrics. The graph records different metrics when different models are tested on two datasets, showing the test results of different models.

[0054] S5. Calculate the model's loss function based on the matched segmentation target, adjust the model parameters, and complete the model training when the loss function converges.

[0055] Specifically, the model uses two loss functions, one for binary segmentation and the other for multi-class segmentation tasks.

[0056] Furthermore, for binary segmentation, the BceDice loss function is selected, which consists of binary cross-entropy (Bce) and the Sorenson-Dice loss function (Dice Loss). The BceDice loss function can measure the difference between the probability distribution predicted by the model and the true label.

[0057] For multi-class segmentation, the CeDice loss function is used, which consists of cross-entropy (Ce) and Sorenson-Dies loss function;

[0058] The formula for the BceDic loss function is L. BceDice The formula for CeDice loss function L CeDice Represented as:

[0059]

[0060] L BceDice =λ1L Bce +λ2L Dice

[0061] L CeDice =λ1L Ce +λ2L Dice

[0062] Among them, L Bce L represents the binary cross-entropy function. Ce L represents the cross-entropy function. Dice Let represent the Sorenson-Dies loss function, N represent the total number of samples, C represent the total number of classes, and y represent the total number of classes. i and These represent the true label and the predicted label, respectively. i,c It is an indicator; if sample i belongs to category c, then y i,c It equals 1 if it is equal to 1, otherwise it is 0. Let |X| and |Y| represent the true value and the predicted value, respectively, and λ1 and λ2 represent the probability that sample i belongs to class c. Bce The weights and L Dice The weights are λ1 and λ2, with λ1 and λ2 both being 1 by default.

[0063] Furthermore, the entire network uses AdamW as the optimizer, combined with the Cosine AnnealingLR scheduler for dynamic adjustment of the learning rate; preferably, the initial learning rate is set to 0.001, and the minimum learning rate is set to 1e-5.

[0064] Finally, it should be noted that the above description is only one embodiment of the present invention. For those skilled in the art, various changes, modifications, substitutions and variations can be conceived of these embodiments without departing from the principles and spirit of the present invention. The scope of protection of the present invention is defined by the appended claims and their equivalents, and all the above-mentioned behaviors should be covered within the scope of protection of the present invention.

Claims

1. A lung CT image segmentation method based on hybrid depthwise convolution and state-space model, characterized in that, include: Acquire the lung CT image to be processed; input the lung CT image to be processed into the trained lung CT image segmentation model to obtain the segmentation result; The lung CT image segmentation model uses the MCMamba network; the MCMamba network consists of a PatchEmbedding layer, an encoder layer, a decoder layer, and a Linear Projection layer connected in sequence; the encoder layer consists of an MC-SSM module and a Patch Merging module; the MC-SSM module includes a MixConv module, an SSM module, a fusion layer, two 1×1 convolutional layers, and a normalization layer. Training the lung CT image segmentation model includes: S1. Input the lung CT images from the training data into the Patch Embedding layer to obtain the embedded images; normalize the embedded images. S2. The normalized embedded image is fed into the encoder layer and processed by the MC-SSM module and the Patch Embedding module to obtain the encoded image. The MC-SSM module's data processing includes the following steps: S21. Input the input feature map into the MixConv module and the SSM module respectively; the SSM module includes: 3 linear layers, 2 lateral normalization layers, depthwise separable convolutional layers, and SS2D layers, wherein SiLU is used as the activation function for the linear layers; S22. Merge the feature map output by the MixConv module with the feature map output by the SSM module to obtain the merged feature map; the MixConv module includes: 3 convolutional kernels of sizes 3×3, 5×5, and 7×7, 3 normalization modules, and a residual concatenation module; the MixConv module processes data in the following steps: S221. Input the input feature maps into convolutional kernels of sizes 3×3, 5×5, and 7×7 respectively, and group the input feature maps accordingly; S222. Perform convolution on each group using a different kernel and then normalize; S223. Concatenate the outputs of each group; S224. Add the input feature map to the concatenated vector; S23. Use a 1×1 convolutional layer to perform deep feature extraction on the merged feature map to obtain a deep feature map; S24. Perform a 1×1 convolution on the input feature map and then batch normalize it to obtain a shallow feature map; S25. Add the deep feature map and the shallow feature map to obtain the fused feature map; S3. Send the encoded image to the decoder layer, and process it through the MC-SSM module and the Patch Embedding module to obtain the decoded image; S4. Send the decoded image to the Linear Projection layer to restore the number of channels; S5. Calculate the model's loss function based on the matched segmentation target, adjust the model parameters, and complete the model training when the loss function converges.

2. The lung CT image segmentation method based on hybrid depth convolution and state space model according to claim 1, characterized in that, The Patch Embedding layer processes lung CT images by dividing the lung CT image into non-overlapping blocks, mapping the non-overlapping blocks to a fixed dimension, and obtaining the embedded image.

3. The lung CT image segmentation method based on hybrid depth convolution and state space model according to claim 1, characterized in that, The encoder and decoder layers are skip connections, and the data processing process for each layer includes four stages, with the skip connections employing an addition operation.

4. The lung CT image segmentation method based on hybrid depth convolution and state space model according to claim 3, characterized in that, The encoder layer processes the data in the following ways: First stage: The input is passed through two connected MC-SSM modules and a Patch Merging module in sequence to perform shallow feature extraction, resulting in feature map 1 with C channels; Second stage: Feature map 1 passes through two connected MC-SSM modules and a Patch Merging module in sequence to obtain feature map 2 with 2C channels; Third stage: Feature map 2 passes through two connected MC-SSM modules and a Patch Merging module in sequence to obtain feature map 3 with 4C channels; Fourth stage: Feature map 3 passes through two connected MC-SSM modules in sequence to obtain feature map 4 with 8 channels; Where C is the dimension of the mapping.

5. The lung CT image segmentation method based on hybrid depth convolution and state space model according to claim 4, characterized in that, The decoder layer processes data in four stages: First stage: Feature map 4 passes through two connected MC-SSM modules in sequence to obtain a feature map with 8 channels. This feature map is then added to feature map 3 to obtain feature map 5. Second stage: Feature map 5 passes through two connected MC-SSM modules and a Patch Expanding module in sequence to obtain a feature map with 4C channels. This feature map is then added to feature map 2 to obtain feature map 6. Third stage: Feature map 6 passes through two connected MC-SSM modules and a Patch Expanding module in sequence to obtain a feature map with 2C channels. This feature map is then added to feature map 1 to obtain feature map 7. Fourth stage: Feature map 7 passes through a connected MC-SSM module and a Patch Expanding module to obtain a feature map with C channels. This feature map is added to the input of the encoder layer to obtain feature map 8, which is then used as the output of the decoder layer. Where C is the dimension of the mapping.

6. The lung CT image segmentation method based on hybrid depth convolution and state space model according to claim 1, characterized in that, The operations performed by the Linear Projection layer include compressing the number of channels in the output feature map of the decoder layer so that the number of channels in the output feature map is the same as the number of channels in the target segmentation map.

Images