A pancreatic CT image segmentation method and device based on Mamba

Abstract

A Mamba-based pancreatic CT image segmentation method and device comprises the following steps: Step S1: Acquiring images from NIH, MSD, and clinical datasets; Step S2: Performing data preprocessing and data enhancement; Step S3: Designing a multi-scale Mamba UNet segmentation model; Step S4: Training the model by loading the data-enhanced images into the model for training; and Step S5: Verifying model performance. The present invention introduces an edge enhancement path into the model, improving the segmentation of pancreatic contours.

Description

Technical Field

[0001] The present invention belongs to the field of image processing, and in particular relates to a pancreatic CT image segmentation method and device based on Mamba. Background Art

[0002] Pancreatic cancer presents significant challenges in diagnosis and treatment, with an extremely low survival rate, making early detection and treatment crucial. CT scans are the primary means of detecting pancreatic lesions. However, the pancreas occupies a very small proportion of abdominal CT images, typically less than 1.5% of a slice. Furthermore, the pancreas and pancreatic tumors are closely adjacent to surrounding organs and blood vessels, and their texture is similar to that of adjacent tissues, resulting in unclear boundaries and low contrast, making accurate segmentation very difficult. Therefore, a segmentation technology that can accurately delineate pancreatic lesions is crucial for the effective treatment of pancreatic cancer.

[0003] Currently, models based on the UNet architecture, which fuses convolutional neural networks (CNNs) and visual transformers (ViTs), dominate visual representation learning and have been widely used in various medical image segmentation tasks. However, while CNNs can effectively extract local features, they struggle to capture global context and long-term dependencies, resulting in insufficient feature extraction. While ViTs can effectively capture long-range dependencies, the high quadratic complexity of their self-attention mechanism in modeling long sequences leads to a significant computational burden. Balancing model parameters and segmentation performance has become a challenge.

[0004] The second problem is the poor performance of pancreatic contour segmentation caused by the low contrast of abdominal CT images and the blurred pancreatic boundary. There is currently no particularly effective solution. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, including the excessive number of model parameters and poor pancreatic lesion contour segmentation, the present invention provides a pancreatic CT image segmentation method and device based on Mamba. This invention introduces an edge enhancement path into the model, improving the model's pancreatic contour segmentation performance.

[0006] The present invention achieves the above-mentioned object through the following technical solutions:

[0007] A pancreatic CT image segmentation method based on Mamba includes the following steps:

[0008] Step S1: Acquire images from NIH, MSD, and clinical datasets;

[0009] Step S2: perform data preprocessing and data enhancement;

[0010] Step S3: Design a multi-scale mamba UNet segmentation model;

[0011] Step S4: training the model, loading the data-enhanced images into the model to start training;

[0012] Step S5: Verify model performance.

[0013] Furthermore, in step S1, the NIH, MSD2018, and clinical datasets were selected as datasets for the pancreas and pancreatic tumor segmentation experiment. The NIH dataset contains 82 contrast-enhanced abdominal CT scans, which were randomly divided into 61 for training and 21 for validation. The MSD dataset contains 281 contrast-enhanced abdominal CT scans, of which 211 were used for training and 70 for testing. The clinical dataset contains 89 contrast-enhanced CT scans with pancreatic tumor labels, of which 71 were used for training and 18 for testing.

[0014] Furthermore, the process of step S2 is as follows:

[0015] 2.1 The 3D CT data were cut into a series of slices along the cross section, and 7309, 9073, and 1476 2D slices were obtained for the three datasets, respectively.

[0016] 2.2 Adjust the window level and window width to accurately capture grayscale values. Grayscale values ​​in the NIH and MSD datasets were clipped to the range of [-100, +240] HU, and grayscale values ​​in the clinical dataset were clipped to the range of [-100, +140] HU. All values ​​were then normalized to the range of [0, 255].

[0017] 2.3 Apply random axis flipping, random 90-degree rotation, random Gaussian noise addition, random contrast adjustment, random Gaussian smoothing, and random histogram shift data enhancement operations to the three data.

[0018] Furthermore, in step S3, a segmentation model is designed, and the process is as follows:

[0019] 3.1 Define the model's backbone network, which consists of four multi-scale Mamba modules, each with four branches. The first, second, and third branches respectively apply one, two, and three deformable convolutions to capture receptive fields of varying scales. Furthermore, a VSS block is introduced to capture fine-grained features. The fourth branch consists of a standard residual block to facilitate better model training.

[0020] 3.2 Define the model's edge enhancement path. This path introduces residual blocks and an attention mechanism to improve boundary contour learning in the pancreas segmentation task. Specifically, the edge enhancement path accurately extracts boundary information by weighting feature maps at different levels, generating attention coefficients, and combining them with residual blocks. Finally, edge contours are extracted using the Canny operator and learned from the ground-truth labels, strengthening the backbone network's ability to extract edge information.

[0021] 3.3 Definition of Multi-layer Decoder. To address the problem of small target details of pancreatic lesions being lost when fusing deep and shallow information in the traditional U-shaped decoder, a multi-layer decoder (MLD) module is introduced. This module restores the encoder output to the same 1×H×W dimension as the original input feature size by applying multiple upsampling blocks at different levels, thereby effectively capturing features of different scales. On this basis, to further improve the expressiveness of features, the feature output of each level is first processed by the convolutional block attention module (CBAM) to obtain more refined edge information. Finally, the feature maps of four different scales are fused through a concatenation operation to generate the final decoder output.

[0022] 3.4 Definition of Loss. ACLoss and DiceLoss are selected as the final loss. This combination method can handle category imbalance, optimize detail segmentation, and improve the overall performance of the segmentation results.

[0023] A second aspect of the present invention relates to a mamba-based pancreatic CT image segmentation device, characterized in that it includes a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement a mamba-based pancreatic CT image segmentation method of the present invention.

[0024] A third aspect of the present invention relates to a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements a mamba-based pancreatic CT image segmentation method of the present invention.

[0025] The beneficial effects of the present invention are:

[0026] 1. We designed a multi-scale Mamba module to balance model parameter count and segmentation accuracy. The Mamba module's multi-scale feature extraction enables the model to simultaneously capture details of pancreatic lesions at different scales while maintaining low model complexity. This design ensures high-precision segmentation while effectively controlling the use of computing resources.

[0027] 2. An edge enhancement path was designed to specifically process pancreatic lesion edge information. It combines low-level and high-level features and introduces the AttentionGate mechanism, enhancing the model's ability to capture edge information at all scales and effectively addressing the issue of blurred borders in pancreatic segmentation.

[0028] 3. Compared with the traditional U-shaped encoding and decoding structure, this paper innovatively proposes an E-type multi-path decoder, which aims to extract features of different scales through different decoding paths, avoiding the problem of small target detail loss that may occur when the traditional U-shaped structure fuses deep and shallow information. It is particularly suitable for the fine segmentation of small organs such as the pancreas. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a schematic diagram of the overall process of the present invention.

[0030] Figure 2 It is a model diagram of the present invention. DETAILED DESCRIPTION

[0031] The present invention will be further described below with reference to the accompanying drawings.

[0032] Example 1

[0033] Reference Figure 1 , a pancreatic CT image segmentation method based on mamba, the overall flow chart is as attached Figure 1 As shown, the following steps are included:

[0034] Step S1: The NIH, MSD2018, and clinical datasets were selected as datasets. The NIH dataset contains 82 contrast-enhanced abdominal CT scans, which were randomly divided into 61 for training and 21 for validation. The MSD dataset contains 281 contrast-enhanced abdominal CT scans, of which 211 were used for training and 70 for testing. The clinical dataset contains 89 contrast-enhanced CT scans with pancreatic tumor labels, of which 71 were used for training and 18 for testing.

[0035] Step S2:

[0036] 2.1 The 3D CT data were cut into a series of slices along the cross section, and 7309, 9073, and 1476 2D slices were obtained for the three datasets, respectively.

[0037] 2.2 Adjust the window level and window width to accurately capture grayscale values. Grayscale values ​​in the NIH and MSD datasets were clipped to the range of [-100, +240] HU, and grayscale values ​​in the clinical dataset were clipped to the range of [-100, +140] HU. All values ​​were then normalized to the range of [0, 255].

[0038] 2.3 Apply random axis flipping, random 90-degree rotation, random Gaussian noise addition, random contrast adjustment, random Gaussian smoothing, and random histogram shift data enhancement operations to the three data.

[0039] Step S3:

[0040] 3.1 Define the model's backbone network, which consists of four multi-scale Mamba modules, each with four branches. The first, second, and third branches respectively apply one, two, and three deformable convolutions to capture receptive fields of varying scales. Furthermore, a VSS block is introduced to capture fine-grained features. The fourth branch consists of a standard residual block to facilitate better model training.

[0041] 3.2 Define the model's edge enhancement path. This path introduces residual blocks and an attention mechanism to improve boundary contour learning in the pancreas segmentation task. Specifically, the edge enhancement path accurately extracts boundary information by weighting feature maps at different levels, generating attention coefficients, and combining them with residual blocks. Finally, edge contours are extracted using the Canny operator and learned from the ground-truth labels, strengthening the backbone network's ability to extract edge information.

[0042] 3.3 Definition of Multi-layer Decoder. To address the problem of small target details of pancreatic lesions being lost when fusing deep and shallow information in the traditional U-shaped decoder, a multi-layer decoder (MLD) module is introduced. This module restores the encoder output to the same 1×H×W dimension as the original input feature size by applying multiple upsampling blocks at different levels, thereby effectively capturing features of different scales. On this basis, to further improve the expressiveness of features, the feature output of each level is first processed by the convolutional block attention module (CBAM) to obtain more refined edge information. Finally, the feature maps of four different scales are fused through a concatenation operation to generate the final decoder output.

[0043] 3.4 Definition of Loss. ACLoss and DiceLoss are selected as the final loss. This combination method can handle category imbalance, optimize detail segmentation, and improve the overall performance of the segmentation results.

[0044] Step S4: Train the model, load the data-augmented images into the model and start training;

[0045] Step S5: Verify model performance.

[0046] The pancreatic CT image segmentation method based on Mamba proposed in this example has the following advantages:

[0047] 1. This method incorporates a multi-scale Mamba module to balance model parameter count and segmentation accuracy. The Mamba module's multi-scale feature extraction enables the model to simultaneously capture details of pancreatic lesions at different scales while maintaining low model complexity. This design ensures high-precision segmentation while effectively limiting the use of computing resources.

[0048] 2. This method designs an edge enhancement path specifically for processing pancreatic lesion edge information. It combines low-level and high-level features and introduces the AttentionGate mechanism, enhancing the model's ability to capture edge information at all scales and effectively addressing the issue of blurred borders in pancreatic segmentation.

[0049] 3. Compared with the traditional U-shaped encoding and decoding structure, this method innovatively proposes an E-type multi-path decoder, which aims to extract features of different scales through different decoding paths, avoiding the problem of small target detail loss that may occur when the traditional U-shaped structure fuses deep and shallow information. It is particularly suitable for the fine segmentation of small organs such as the pancreas.

[0050] Example 2

[0051] This embodiment relates to a mamba-based pancreatic CT image segmentation device, characterized in that it includes a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, it is used to implement a mamba-based pancreatic CT image segmentation method of Example 1.

[0052] Example 3

[0053] This embodiment relates to a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, the pancreatic CT image segmentation method based on Mamba in Example 1 is implemented.

[0054] The contents described in the embodiments of this specification are merely an enumeration of the implementation forms of the inventive concept. The scope of protection of the present invention should not be regarded as limited to the specific forms described in the embodiments. The scope of protection of the present invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A pancreatic CT image segmentation method based on Mamba, comprising the following steps: Step S1: Acquire images from NIH, MSD, and clinical datasets; Step S2: perform data preprocessing and data enhancement; Step S3: Design a multi-scale mamba UNet segmentation model; specifically including: 3.1 Define the model's backbone network, which consists of four multi-scale Mamba modules. Each module has four branches. The first, second, and third branches respectively use 1, 2, and 3 deformable convolutions to capture receptive fields of different scales. In addition, a VSS block is introduced to capture different fine-grained features. The fourth branch consists of a standard residual block to facilitate better model training. 3.2 Define the model's edge enhancement path; this path introduces residual blocks and an attention mechanism to improve boundary contour learning in the pancreas segmentation task. Specifically, the edge enhancement path accurately extracts boundary information by weighting feature maps at different levels, generating attention coefficients, and processing them with residual blocks. Finally, edge contours are extracted using the Canny operator and learned from the ground truth labels, strengthening the backbone network's ability to extract edge information. 3.3 Defining a Multi-Layer Decoder: To address the problem of small object detail loss in pancreatic lesions caused by the traditional U-shaped decoder when fusing deep and shallow layer information, a multi-layer decoder (MLD) module is introduced. This module applies multiple upsampling blocks at different levels to restore the encoder output to the same 1×H×W dimension as the original input feature, effectively capturing features at different scales. To further improve the expressiveness of features, the feature output of each level is first processed by the convolutional block attention module (CBAM) to obtain more refined edge information. Finally, the feature maps of four different scales are fused through a concatenation operation to generate the final decoder output. 3.4 Define loss; select ACLoss and DiceLoss as the final loss; Step S4: training the model, loading the data-enhanced images into the model to start training; Step S5: Verify model performance.

2. The pancreatic CT image segmentation method based on Mamba according to claim 1, characterized in that: In step S1, the NIH, MSD2018, and clinical datasets are selected as datasets for the pancreas and pancreatic tumor segmentation experiment; the NIH dataset contains 82 contrast-enhanced abdominal CT scan data, and the 82 CT scan data are randomly divided into 61 for training and 21 for validation; the MSD dataset contains 281 contrast-enhanced abdominal CT scan data, of which 211 are used for training and 70 are used for testing; the clinical dataset contains 89 contrast-enhanced CT scan data with pancreatic tumor labels, of which 71 are used for training and 18 are used for testing.

3. The pancreatic CT image segmentation method based on Mamba according to claim 1, characterized in that: The process of step S2 is as follows: 2.1 The 3D CT data were cut into a series of slices along the cross section, resulting in 7309, 9073, and 1476 2D slices for the three datasets, respectively; 2.2 Adjust the window level and window width to accurately capture the grayscale values; clip the grayscale values ​​of the data in the NIH and MSD datasets to the range of [-100, +240] HU, and clip the grayscale values ​​in the clinical dataset to the range of [-100, +140] HU; then normalize all values ​​to the range of [0, 255]; 2.3 Apply random axis flipping, random 90-degree rotation, random Gaussian noise addition, random contrast adjustment, random Gaussian smoothing, and random histogram shift data enhancement operations to the three data.

4. A pancreatic CT image segmentation device based on Mamba, characterized in that: The invention comprises a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, the method for pancreatic CT image segmentation based on Mamba is implemented as described in any one of claims 1 to 3.

5. A computer-readable storage medium, characterized in that A program is stored thereon, and when the program is executed by a processor, a pancreatic CT image segmentation method based on Mamba according to any one of claims 1 to 3 is implemented.

Images