Pancreas CT image segmentation method and device based on mamba

Abstract

The invention discloses a pancreas CT image segmentation method and device based on mamba. The method comprises the following steps of S1, obtaining NIH, MSD and clinical data set images; s2, carrying out data preprocessing and data enhancement; s3, a multi-scale mamba UNet segmentation model is designed; s4, training the model, and loading the picture subjected to data enhancement processing into the model to start training; and S5, verifying the performance of the model. According to the invention, the edge enhancement path is introduced into the model, so that the segmentation effect of the model on the pancreatic contour is improved.

Description

Technical Field

[0001] The invention belongs to the field of image processing, and in particular relates to a pancreatic CT image segmentation method and device based on mamba. Background Art

[0002] Pancreatic cancer is significantly challenging to diagnose and treat, with an extremely low survival rate, so early detection and treatment are crucial. CT scanning is the primary means of detecting pancreatic lesions, however, the pancreas occupies a very small proportion of abdominal CT images, usually less than 1.5% of a slice, and the pancreas and pancreatic tumors are closely adjacent to surrounding organs and blood vessels, with textures similar to adjacent tissues, resulting in unclear boundaries and low contrast, making accurate segmentation very difficult. Therefore, a segmentation technology that can accurately divide pancreatic lesions is crucial for the effective treatment of pancreatic cancer.

[0003] Currently, models that fuse convolutional neural networks (CNNs) and visual transformers (ViTs) based on the UNet structure dominate visual representation learning and have been widely used in various medical image segmentation tasks. However, although CNNs can effectively extract local features, they have difficulty capturing global context and long-term dependencies, resulting in insufficient feature extraction. Although ViTs can effectively capture long-distance dependencies, the high quadratic complexity of its self-attention mechanism in long sequence modeling leads to a huge computational burden. How to balance model parameters and segmentation performance has become a challenge.

[0004] The second problem is the poor performance of pancreatic contour segmentation caused by the low contrast of abdominal CT images and the blurred pancreatic boundary. Currently, there is no particularly effective solution. Summary of the invention

[0005] In order to overcome the shortcomings of the prior art and solve the problems of excessive model parameters and poor pancreatic lesion contour segmentation effect, the present invention provides a pancreatic CT image segmentation method and device based on mamba. The present invention introduces an edge enhancement path in the model, so that the model improves the segmentation effect of the pancreatic contour.

[0006] The present invention achieves the above object through the following technical solutions:

[0007] A pancreatic CT image segmentation method based on Mamba comprises the following steps:

[0008] Step S1: Obtain NIH, MSD, and clinical dataset images;

[0009] Step S2: perform data preprocessing and data enhancement;

[0010] Step S3: design a multi-scale mamba UNet segmentation model;

[0011] Step S4: training the model, loading the data-enhanced images into the model to start training;

[0012] Step S5: Verify model performance.

[0013] Further, in step S1, NIH, MSD2018 and clinical data sets are selected as data sets for pancreas and pancreatic tumor segmentation experiments. The NIH data set contains 82 cases of contrast-enhanced abdominal CT scan data, and the 82 CT scan data are randomly divided into 61 cases for training and 21 cases for verification. The MSD data set contains 281 cases of contrast-enhanced abdominal CT scan data, of which 211 are used for training and 70 are used for testing. The clinical data set contains 89 cases of enhanced CT scan data with pancreatic tumor labels, of which 71 are used for training and 18 are used for testing.

[0014] Furthermore, the process of step S2 is as follows:

[0015] 2.1 The 3D CT data were cut into a series of slices along the cross section, and 7309, 9073, and 1476 2D slices were obtained for the three datasets, respectively.

[0016] 2.2 Adjust the window level and window width to accurately capture the grayscale values. The grayscale values ​​of the data in the NIH and MSD datasets were clipped to the range of [-100, +240] HU, and the grayscale values ​​in the clinical dataset were clipped to the range of [-100, +140] HU. All values ​​were then normalized to the range of [0, 255].

[0017] 2.3 Apply random axis flipping, random 90-degree rotation, random Gaussian noise addition, random contrast adjustment, random Gaussian smoothing, and random histogram shift data enhancement operations to the three data.

[0018] Furthermore, in step S3, a segmentation model is designed, and the process is as follows:

[0019] 3.1 Define the backbone network of the model, which consists of four multi-scale mamba modules, each of which consists of four branches. In the first to third branches, 1, 2, and 3 deformable convolutions are applied respectively to capture receptive fields of different scales. In addition, VSS blocks are introduced to capture different fine-grained features. The fourth branch is composed of ordinary residual blocks for better model training.

[0020] 3.2 Define the edge enhancement path of the model. This path introduces residual blocks and attention mechanisms to improve the learning of boundary contours in the pancreas segmentation task. Specifically, the edge enhancement path accurately extracts boundary information by weighting feature maps at different levels, generating attention coefficients, and processing them in combination with residual blocks. Finally, the edge contours are extracted by the Canny operator and learned with the true labels, which strengthens the backbone network's ability to extract edge information.

[0021] 3.3 Definition of multi-layer decoder. In order to address the problem of small target details loss of pancreatic lesions caused by the traditional U-shaped decoder when fusing deep and shallow information, a multi-layer decoder (MLD) module is introduced. This module restores the encoder output to the same 1×H×W dimension as the original input feature size by applying multiple upsampling blocks at different levels, thereby effectively capturing features of different scales. On this basis, in order to further improve the expressiveness of the features, the feature output of each level is first processed by the convolutional block attention module (CBAM) to obtain more refined edge information. Finally, the feature maps of four different scales are fused through the connection operation to generate the final decoder output.

[0022] 3.4 Definition of loss. ACLoss and DiceLoss are selected as the final loss. This combination method can handle category imbalance, optimize detail segmentation, and improve the overall performance of the segmentation results.

[0023] The second aspect of the present invention relates to a mamba-based pancreatic CT image segmentation device, characterized in that it includes a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement a mamba-based pancreatic CT image segmentation method of the present invention.

[0024] A third aspect of the present invention relates to a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements a mamba-based pancreatic CT image segmentation method of the present invention.

[0025] The beneficial effects of the present invention are:

[0026] 1. A multi-scale Mamba module was designed to balance the number of model parameters and segmentation accuracy. Through the multi-scale feature extraction of the Mamba module, the model can simultaneously capture the details of pancreatic lesions at different scales while maintaining a low model complexity. This design effectively controls the use of computing resources while ensuring high-precision segmentation.

[0027] 2. An edge enhancement path is designed to specifically process the edge information of pancreatic lesions. It combines low-level and high-level features and introduces the AttentionGate mechanism to enhance the model's ability to capture edge information at all scales, effectively solving the problem of blurred boundary contours in pancreatic segmentation.

[0028] 3. Compared with the traditional U-shaped encoding and decoding structure, the present invention innovatively proposes an E-type multi-path decoder, which aims to extract features of different scales through different decoding paths, avoiding the problem of small target detail loss that may occur in the traditional U-shaped structure when fusing deep and shallow information. It is particularly suitable for the fine segmentation of small organs such as the pancreas. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a schematic diagram of the overall process of the present invention.

[0030] Figure 2 It is a model diagram of the present invention. DETAILED DESCRIPTION

[0031] The present invention will be further described below in conjunction with the accompanying drawings.

[0032] Example 1

[0033] Reference Figure 1 , a pancreatic CT image segmentation method based on Mamba, the overall flow chart is as follows Figure 1 As shown, the following steps are included:

[0034] Step S1: NIH, MSD2018 and clinical datasets were selected as datasets. The NIH dataset contains 82 contrast-enhanced abdominal CT scan data, which are randomly divided into 61 for training and 21 for validation. The MSD dataset contains 281 contrast-enhanced abdominal CT scan data, of which 211 are used for training and 70 are used for testing. The clinical dataset contains 89 contrast-enhanced CT scan data with pancreatic tumor labels, of which 71 are used for training and 18 are used for testing.

[0035] Step S2:

[0036] 2.1 The 3D CT data were cut into a series of slices along the cross section, and 7309, 9073, and 1476 2D slices were obtained for the three datasets, respectively.

[0037] 2.2 Adjust the window level and window width to accurately capture the grayscale values. The grayscale values ​​of the data in the NIH and MSD datasets were clipped to the range of [-100, +240] HU, and the grayscale values ​​in the clinical dataset were clipped to the range of [-100, +140] HU. All values ​​were then normalized to the range of [0, 255].

[0038] 2.3 Apply random axis flipping, random 90-degree rotation, random Gaussian noise addition, random contrast adjustment, random Gaussian smoothing, and random histogram shift data enhancement operations to the three data.

[0039] Step S3:

[0040] 3.1 Define the backbone network of the model, which consists of four multi-scale mamba modules, each of which consists of four branches. In the first to third branches, 1, 2, and 3 deformable convolutions are applied respectively to capture receptive fields of different scales. In addition, VSS blocks are introduced to capture different fine-grained features. The fourth branch is composed of ordinary residual blocks for better model training.

[0041] 3.2 Define the edge enhancement path of the model. This path introduces residual blocks and attention mechanisms to improve the learning of boundary contours in the pancreas segmentation task. Specifically, the edge enhancement path accurately extracts boundary information by weighting feature maps at different levels, generating attention coefficients, and processing them in combination with residual blocks. Finally, the edge contours are extracted by the Canny operator and learned with the true labels, which strengthens the backbone network's ability to extract edge information.

[0042] 3.3 Definition of multi-layer decoder. In order to address the problem of small target details loss of pancreatic lesions caused by the traditional U-shaped decoder when fusing deep and shallow information, a multi-layer decoder (MLD) module is introduced. This module restores the encoder output to the same 1×H×W dimension as the original input feature size by applying multiple upsampling blocks at different levels, thereby effectively capturing features of different scales. On this basis, in order to further improve the expressiveness of the features, the feature output of each level is first processed by the convolutional block attention module (CBAM) to obtain more refined edge information. Finally, the feature maps of four different scales are fused through the connection operation to generate the final decoder output.

[0043] 3.4 Definition of loss. ACLoss and DiceLoss are selected as the final loss. This combination method can handle category imbalance, optimize detail segmentation, and improve the overall performance of the segmentation results.

[0044] Step S4: training the model, loading the data-enhanced images into the model to start training;

[0045] Step S5: Verify model performance.

[0046] The pancreatic CT image segmentation method based on Mamba proposed in this example has the following advantages:

[0047] 1. This method designs a multi-scale Mamba module to balance the number of model parameters and segmentation accuracy. Through the multi-scale feature extraction of the Mamba module, the model can simultaneously capture the details of pancreatic lesions at different scales while maintaining a low model complexity. This design effectively controls the use of computing resources while ensuring high-precision segmentation.

[0048] 2. This method designs an edge enhancement path to specifically process the edge information of pancreatic lesions. It combines low-level and high-level features and introduces the AttentionGate mechanism to enhance the model's ability to capture edge information at all scales, effectively solving the problem of blurred boundary contours in pancreatic segmentation.

[0049] 3. Compared with the traditional U-shaped encoding and decoding structure, this method innovatively proposes an E-type multi-path decoder, which aims to extract features of different scales through different decoding paths, avoiding the problem of small target detail loss that may occur in the traditional U-shaped structure when fusing deep and shallow information. It is especially suitable for the fine segmentation of small organs such as the pancreas.

[0050] Example 2

[0051] The present embodiment relates to a pancreatic CT image segmentation device based on mamba, characterized in that it includes a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement a pancreatic CT image segmentation method based on mamba in Example 1.

[0052] Example 3

[0053] This embodiment relates to a computer-readable storage medium on which a program is stored. When the program is executed by a processor, a pancreatic CT image segmentation method based on Mamba in Embodiment 1 is implemented.

[0054] The contents described in the embodiments of this specification are merely an enumeration of the implementation forms of the inventive concept. The protection scope of the present invention should not be regarded as limited to the specific forms described in the embodiments. The protection scope of the present invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A pancreatic CT image segmentation method based on Mamba, comprising the following steps: Step S1: Obtain NIH, MSD, and clinical dataset images; Step S2: perform data preprocessing and data enhancement; Step S3: design a multi-scale mamba UNet segmentation model; Step S4: training the model, loading the data-enhanced images into the model to start training; Step S5: Verify model performance.

2. The pancreatic CT image segmentation method based on Mamba as claimed in claim 1, characterized in that: In step S1, NIH, MSD2018 and clinical datasets are selected as datasets for pancreas and pancreatic tumor segmentation experiments; the NIH dataset contains 82 contrast-enhanced abdominal CT scan data, and the 82 CT scan data are randomly divided into 61 for training and 21 for validation; the MSD dataset contains 281 contrast-enhanced abdominal CT scan data, of which 211 are used for training and 70 are used for testing; the clinical dataset contains 89 contrast-enhanced CT scan data with pancreatic tumor labels, of which 71 are used for training and 18 are used for testing.

3. The pancreatic CT image segmentation method based on Mamba as claimed in claim 1, characterized in that: The process of step S2 is as follows: 2.1 The 3D CT data were cut into a series of slices along the cross section, and 7309, 9073, and 1476 2D slices were obtained for the three datasets, respectively; 2.2 Adjust the window level and window width to accurately capture the grayscale values; clip the grayscale values ​​of the data in the NIH and MSD datasets to the range of [-100, +240] HU, and clip the grayscale values ​​in the clinical dataset to the range of [-100, +140] HU; then normalize all values ​​to the range of [0, 255]; 2.3 Apply random axis flipping, random 90-degree rotation, random Gaussian noise addition, random contrast adjustment, random Gaussian smoothing, and random histogram shift data enhancement operations to the three data.

4. The pancreatic CT image segmentation method based on Mamba as claimed in claim 1, characterized in that: The process of designing the segmentation model described in step S3 is as follows: 3.1 Define the backbone network of the model, which consists of four multi-scale mamba modules, each of which consists of four branches. In the first to third branches, 1, 2, and 3 deformable convolutions are applied respectively to capture receptive fields of different scales. In addition, VSS blocks are introduced to capture different fine-grained features. The fourth branch is composed of ordinary residual blocks for better model training. 3.2 Define the edge enhancement path of the model; this path introduces residual blocks and attention mechanisms to improve the boundary contour learning in the pancreas segmentation task; specifically, the edge enhancement path accurately extracts boundary information by weighting feature maps at different levels, generating attention coefficients, and processing them in combination with residual blocks; finally, the edge contour is extracted by the Canny operator and learned with the true label, which strengthens the backbone network's ability to extract edge information; 3.3 Define a multi-layer decoder; In order to address the problem of small target details of pancreatic lesions being lost when the traditional U-shaped decoder fuses deep and shallow information, a multi-layer decoder MLD module is introduced; This module restores the encoder output to the same 1×H×W dimension as the original input feature size by applying multiple upsampling blocks at different levels, thereby effectively capturing features of different scales; On this basis, in order to further improve the expressiveness of features, the feature output of each level is first processed by the convolutional block attention module CBAM to obtain more refined edge information; Finally, the feature maps of four different scales are fused through the connection operation to generate the final decoder output; 3.4 Definition of loss; ACLoss and DiceLoss are selected as the final loss.

5. A pancreatic CT image segmentation device based on mamba, characterized in that: The invention comprises a memory and one or more processors, wherein the memory stores executable codes, and when the one or more processors execute the executable codes, the method for pancreatic CT image segmentation based on Mamba according to any one of claims 1 to 4 is implemented.

6. A computer-readable storage medium, characterized in that: A program is stored thereon, and when the program is executed by a processor, a pancreatic CT image segmentation method based on Mamba according to any one of claims 1 to 4 is implemented.

Images