Polyp image segmentation method based on Mamba-convolution double branches and boundary enhancement
By employing the Mamba-convolutional dual-branch method and boundary enhancement, combined with Laplacian pyramid and multi-scale feature fusion, the problem of balancing global semantics and local details in polyp image segmentation is solved, achieving higher accuracy polyp region segmentation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING UNIV OF TECH
- Filing Date
- 2026-01-14
- Publication Date
- 2026-04-21
Smart Images
Figure CN121904064A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image segmentation technology, and more specifically, to a polyp image segmentation method based on Mamba-convolutional dual-branch and boundary enhancement. Background Technology
[0002] Colorectal cancer (CRC) is a common malignant tumor of the digestive tract, primarily occurring in the colon and rectum. Colonoscopy, as the main method for screening and diagnosing CRC, can effectively identify and locate intestinal polyps, which are considered important precursor lesions for the development and progression of CRC. However, traditional colonoscopy procedures heavily rely on the physician's clinical experience. Manual detection and localization of polyps is not only physically demanding and time-consuming, but also susceptible to subjective judgment and fatigue, leading to a high risk of missed diagnoses and misdiagnoses. Therefore, developing precise and efficient computer-aided polyp segmentation methods is of great significance for improving the early detection rate of CRC, assisting clinical diagnosis and treatment decisions, and optimizing the allocation of medical resources.
[0003] With the continuous maturation of deep learning in the medical field, using deep learning methods to segment medical images has become a mainstream trend in the medical community. Among them, the U-Net network based on Convolutional Neural Network (CNN) has significantly promoted the development of automatic polyp image segmentation technology and spawned a variety of improved network models based on the CNN architecture. However, limited by the local receptive field characteristics of convolution operations, CNN-based polyp image segmentation methods have limited ability to model long-distance dependencies and acquire global contextual information, which to some extent restricts further improvement in segmentation performance. The Mamba model proposed in recent years performs well in modeling long sequences and long-distance dependencies, effectively making up for the shortcomings of CNN in capturing global information, and has significant advantages over Transformer in terms of lower computational complexity and less resource consumption. However, similar to Transformer, Mamba also has limitations in extracting local detail features, making it difficult to fully capture the fine structural information in polyp images.
[0004] Current polyp image segmentation methods mostly employ single-branch feature extraction frameworks based on CNNs, Transformers, or Mamba, which have relatively limited feature representation capabilities and struggle to simultaneously capture global semantic information and local details, thus affecting segmentation accuracy. Furthermore, existing methods do not adequately utilize the boundary details of polyp regions, resulting in insufficiently precise localization of polyp contours. Based on these issues, there is an urgent need to explore a novel polyp image segmentation method to achieve more complete and sufficient semantic feature representation, ultimately improving the segmentation performance of polyp images. Summary of the Invention
[0005] To address the aforementioned problems, this invention provides a polyp image segmentation method based on Mamba-convolutional dual-branch and boundary enhancement. This invention encodes the input image using Mamba and convolutional dual-branch methods, fully extracting local features and global information; it constructs a Laplacian pyramid on the original image to supplement high-frequency detail information of the encoder portion of the image features; and in the decoder portion, it introduces the Laplacian operator and reverse attention to better locate the boundaries of the polyp region. This invention employs the following technical solutions:
[0006] A polyp image segmentation method based on Mamba-convolutional dual-branch and boundary enhancement includes the following steps:
[0007] Step 1: Collect colonoscopy polyp images and polyp mask labels dataset, and divide the dataset into training set and test set;
[0008] Step 2: Construct a polyp image segmentation network model based on Mamba-convolutional dual-branch and boundary enhancement;
[0009] Step 3: Perform data augmentation on the training data obtained in Step 1, and input it into the network model in Step 2 for training;
[0010] Step 4: Use the best model obtained after training to test the colonoscopy images in the test set, obtain the polyp segmentation results after testing, and evaluate the segmentation results of the model.
[0011] Furthermore, the dataset in step 1 comprises three sub-datasets: the publicly available colonoscopy polyp image datasets CVC-ClinicDB, CVC-ColonDB, and Kvasir-SEG. Each dataset is randomly divided in an 8:2 ratio to obtain a training set and a test set. The model is trained on the training set of each dataset, and finally, the segmentation performance of the final model is evaluated using the test set.
[0012] Furthermore, the polyp image segmentation network model based on Mamba-convolutional dual-branch and boundary enhancement proposed in step 2 is used to perform the following steps:
[0013] Step 2.1: Using Mamba-based VSS blocks and convolution-based ConvNeXt blocks as dual branches for multi-level feature extraction, four-stage multi-level features are extracted from the original image. The features extracted by the Mamba branch are denoted as follows: The features extracted by the convolutional branch are denoted as ;
[0014] Step 2.2: Use the dual-branch fusion module designed in this invention to analyze the characteristics of the Mamba branch output in Step 2.1. Features of convolution branch output Weights are learned, then adaptively fused, and the fused features are denoted as... ;
[0015] Step 2.3: Using the detail feature enhancement module designed in this invention, construct a Laplacian pyramid on the original image, extract high-frequency detail information from the original image, and compare it with the features obtained in Step 2.2. By fusing and enhancing the details and edges of features, detailed enhanced features are obtained. ;
[0016] Step 2.4: Enhance the details obtained in Step 2.3 The input is fed into a multi-scale feature fusion module, which extracts and learns image features at multiple scales using convolutions with different kernel sizes. Then, the mined semantic information is progressively fused to obtain a coarse segmentation map, denoted as... ;
[0017] Step 2.5: The results obtained in Step 2.4 Downsampling, to obtain Combine it with the product generated in step 2.3 The input is fed into the first-level enhanced reverse attention module, which uses this module to mine complementary regions and boundary information, and then... A residual connection is performed with the boundary detail information output by the first-level enhanced back attention module to obtain... ;
[0018] Step 2.6: For Upsampling Combine it with the product generated in step 2.3 The input is fed into the second-level enhanced reverse attention module, whose output boundary detail information is... After adding, we get ;
[0019] Step 2.7: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full context.] Upsampled image and the product generated in step 2.3 The input is fed into the third-level enhanced reverse attention module, whose output boundary detail information is... Obtained after residual connection ;
[0020] Step 2.8: For , , and Conduct in-depth supervision; The image is upsampled to the size of the input image, and then binarized using the sigmoid function to obtain the prediction map of the network model, i.e., the segmentation result.
[0021] Furthermore, the dual-branch fusion module proposed in step 2.2 will integrate the features output by the Mamba branches. Features of convolution branch output Adaptive fusion, the specific process is as follows: First let and Features of each branch are extracted using 1×1 convolutions, then concatenated along the channel dimension and weighted using a normalized exponential function. and Mamba branch weights Finally, the features of each stage of convolutional branches are... and Mamba branching features Use the corresponding weights and Weighted summation ( ), to obtain fusion features The specific expression method is as follows:
[0022] ,
[0023] ,
[0024] ,
[0025] in , This indicates splicing along the channel dimension. Represents a 1×1 convolution. For normalized exponential functions, This indicates a channel partitioning operation on a tensor. For element-wise multiplication, Add elements together.
[0026] Furthermore, the detail feature enhancement module proposed in step 2.3 includes the following steps:
[0027] Step 2.3.1: Construct a Laplacian pyramid for the original image to obtain high-frequency detail information in four stages. The specific process is as follows: First, convert the original image to grayscale. Then, construct a five-layer Gaussian pyramid through Gaussian smoothing and downsampling operations. Next, upsample the output image of each layer of the Gaussian pyramid. Subtract the upsampled image of the current layer from the output image of the previous layer of the Gaussian pyramid to obtain the output image of the current layer of the Laplacian pyramid. By analogy, we can obtain The specific expression method is as follows:
[0028] ,
[0029] In the formula , This is the image output for this layer of the Gaussian pyramid. This is the image output for this layer of the Laplacian pyramid. This is an upsampling operation;
[0030] Step 2.3.2: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation The fusion features obtained after normalization and step 2.2 Multiply element by element, then multiply Residual connections ultimately yield features with strong high-frequency detail information. The specific expression method is as follows:
[0031] ,
[0032] In the formula , This is the normalization function.
[0033] Furthermore, the multi-scale feature fusion module proposed in step 2.4 includes the following steps:
[0034] Step 2.4.1: with As input, let respectively Through four branches of 1×1 convolution, strip convolution, and dilated convolution at different scales, multi-scale feature information is fully extracted. Furthermore, concatenation, convolution, and residual connections are used to enrich the feature information. The specific representation method is as follows:
[0035] ,
[0036] ,
[0037] ,
[0038] ,
[0039] ,
[0040] ,
[0041] in , It is a 3×1 convolution. This represents a 3×3 convolution with a hole rate of 3. This represents a 3×3 convolution with a hole rate of 5. , , and Representing features respectively Based on the results of each branch Representation of features Summary characteristics after each branch It is a 3×3 convolution. For activation functions;
[0042] Step 2.4.2: Take the result obtained in step 2.4.1 A progressive fusion was performed to obtain an image with the approximate area of the polyp. It employs a phased upsampling strategy, the specific process of which is as follows: First, high-level features... Upsampling The size, after convolution adjustment, is the same as... Element-wise multiplication generates modulation features. Then Upsampling to The scale, at the same time After upsampling and convolution adjustment, both together with Element-wise multiplication generates modulation features. Secondly, the modulation characteristics and The upsampled features are concatenated along the channel dimension and then fused by convolution to generate intermediate features. Then the modulation features With intermediate features Upsampled and convolutionally adjusted features are concatenated through channels and then convolved to generate intermediate features. Finally, the intermediate features The image is refined using convolutional layers and the number of channels is adjusted to 1, outputting an image with the approximate area of the polyp. The specific expression method is as follows:
[0043] ,
[0044] ,
[0045] ,
[0046] ,
[0047] ,
[0048] in, This indicates an upsampling of 2 times. For convolution operations, This indicates splicing along the channel dimension.
[0049] Furthermore, the enhanced reverse attention module in steps 2.5, 2.6, and 2.7 effectively enhances the image boundary region by mining background information and boundary features. Specifically, the... The input to the enhanced reverse attention module at level 1 is and the detail feature enhancement module , Through the reverse attention branch and the Laplace operator branch respectively, for the reverse attention branch, for Normalization is applied and the image is inverted to obtain the approximate background region; for the Laplacian operator branch, the normalization function ... Laplacian operator branch is inverted and the Laplacian operator branch is inverted and the Laplacian operator branch is inverted and the Laplacian operator branch is inverted and the Laplacian operator branch is invert The boundary information is obtained after processing with the Laplacian operator. Then, the outputs of the two branches are concatenated along the channel dimension and subjected to convolution before being combined with the input. Element-wise multiplication to enhance features The boundary details are then refined using convolutional operations, as shown in the following specific representation:
[0050] ,
[0051] ,
[0052] ,
[0053] ,
[0054] in , This indicates the negation operation. For normalization function, For convolution operations, This represents the Laplace operator operation. and They represent the first The outputs of the back attention branch and the Laplacian operator branch in the enhanced back attention module of the level, For the first The enhanced reverse attention module at the level integrates the spatial weight representation of the two branches. Indicates the first The output of the level ERA.
[0055] Furthermore, in step 3, the model training process is optimized using a weighted sum of Dice similarity coefficient loss and binary cross-entropy loss (BCE loss), as mentioned in step 2.8. , , and Deep supervision is performed during training; therefore, the overall loss function is expressed as follows:
[0056] ,
[0057] ,
[0058] In this invention and The value of is 1. To Upsampled to the size of the label image, Represents a real label image. and These represent Dice loss and BCE loss, respectively. This is the final loss function of this invention.
[0059] Compared with the prior art, the present invention has the following advantages:
[0060] (1) This invention is a polyp image segmentation method based on Mamba-convolution dual branch and boundary enhancement. In the encoder part, Mamba and convolution dual branch are used to fully combine the advantages of Mamba modeling global information with convolution capturing local features, thereby improving the feature expression ability of the model.
[0061] (2) Through the dual-branch fusion module, the network model can learn the weights of the two branches autonomously and perform adaptive fusion, thereby generating semantically rich fusion features.
[0062] (3) The detail feature enhancement module integrates the high-frequency detail information of the original image into the features output by the dual-branch fusion module by constructing the Laplacian pyramid, thereby further enhancing the detail information of the features.
[0063] (4) The multi-scale feature fusion module extracts multi-scale information through multi-branch convolution with different kernel sizes and gradually fuses it to obtain an image containing the approximate area of the polyp, so as to guide the decoding process of the subsequent decoder.
[0064] (5) The decoder consists of an enhanced back attention module. The enhanced back attention module uses back attention and Laplacian operator to obtain background and boundary information, thereby enhancing the boundary region of the decoded features and ultimately making polyp segmentation more accurate. Attached Figure Description
[0065] Figure 1 This is the overall network architecture diagram of the present invention;
[0066] Figure 2 This is a structural diagram of the dual-branch fusion module;
[0067] Figure 3 This is a structural diagram of the detail feature enhancement module;
[0068] Figure 4 It is the construction process of the Pyramid of Laplace;
[0069] Figure 5 This is a structural diagram of the multi-scale feature fusion module;
[0070] Figure 6 This is a structural diagram of the enhanced reverse attention module;
[0071] Figure 7 This is a comparison chart of experiments;
[0072] Figure 8 It is a visualization of the experimental comparison. Detailed Implementation
[0073] To better understand the purpose, structure, and function of this invention, the following description, in conjunction with the accompanying drawings, further elaborates on the polyp image segmentation method based on Mamba-convolutional dual-branch and boundary enhancement provided by this invention.
[0074] The present invention provides a polyp image segmentation method based on Mamba-convolutional dual-branch and boundary enhancement, comprising the following steps:
[0075] Step 1: Collect colonoscopy polyp images and polyp mask labels dataset, and divide the dataset into training set and test set;
[0076] The dataset consists of three sub-datasets: the publicly available colonoscopy polyp image datasets CVC-ClinicDB, CVC-ColonDB, and Kvasir-SEG. Each dataset is randomly divided in an 8:2 ratio to obtain training and testing sets. The model is trained on the training set of each dataset, and the segmentation performance of the final model is evaluated using the testing set.
[0077] Step 2: Construct a polyp image segmentation network model based on Mamba-convolutional dual-branch and boundary enhancement;
[0078] like Figure 1 As shown, in step 2, the proposed polyp image segmentation network model based on Mamba-convolutional dual-branch and boundary enhancement includes the following steps:
[0079] Step 2.1: Using Mamba-based VSS blocks and convolution-based ConvNeXt blocks as dual branches for multi-level feature extraction, four-stage multi-level features are extracted from the original image. The features extracted by the Mamba branch are denoted as follows: The features extracted by the convolutional branch are denoted as ;
[0080] Step 2.2: Use the Dual-Branch Fusion Module (DBF) designed in this invention to analyze the characteristics of the Mamba branch output in Step 2.1. Features of convolution branch output Weights are learned, then adaptively fused, and the fused features are denoted as... ;
[0081] like Figure 2 As shown, the DBF module will display the characteristics of the Mamba branch output. Features of convolution branch output Adaptive fusion, firstly let and Features of each branch are extracted using 1×1 convolutions, then concatenated along the channel dimension and weighted using a normalized exponential function. and Mamba branch weights Finally, the features of each stage of convolutional branches are... and Mamba branching features Use the corresponding weights and Weighted summation ( ), to obtain fusion features The specific expression method is as follows:
[0082] ,
[0083] ,
[0084] ,
[0085] in , This indicates splicing along the channel dimension. This represents a 1×1 convolution operation. For normalized exponential functions, This indicates a channel partitioning operation on a tensor. For element-wise multiplication, Add elements together.
[0086] Step 2.3: Using the Detailed Feature Enhancement Module (DFE) designed in this invention, construct a Laplacian pyramid on the original image, extract high-frequency detail information from the original image, and combine it with the features obtained in Step 2.2. By fusing and enhancing the details and edges of features, detailed enhanced features are obtained. ;
[0087] like Figure 3 As shown, the DFE module proposed in step 2.3 includes the following steps:
[0088] Step 2.3.1: Construct a Laplacian pyramid on the original image to obtain the four-stage high-frequency detail information of the original image, denoted as... ,like Figure 4 As shown, the specific process is as follows: First, the original image is converted to grayscale. Then, a five-layer Gaussian pyramid is constructed through Gaussian smoothing and downsampling operations. Next, the output image of each layer of the Gaussian pyramid is upsampled. The output image of the current layer is obtained by subtracting the upsampled image of the current layer from the output image of the previous layer of the Gaussian pyramid. By analogy, we can obtain The specific expression method is as follows:
[0089] ,
[0090] in , This is the image output for this layer of the Gaussian pyramid. This is the image output for this layer of the Laplacian pyramid. This is an upsampling operation;
[0091] Step 2.3.2: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation The fusion features obtained after normalization and step 2.2 Multiply element by element, then multiply Residual connections ultimately yield features with strong high-frequency detail information. The specific expression method is as follows:
[0092] ,
[0093] in , This is the normalization function.
[0094] Step 2.4: Enhance the details obtained in Step 2.3 The input is fed into the Multi-Scale Feature Fusion Module (MSFF) designed in this invention. Through convolutions with different kernel sizes, multi-scale image features are extracted and learned. The mined semantic information is then progressively fused to obtain a coarse segmentation map, denoted as... ;
[0095] like Figure 5 As shown, the proposed MSFF module includes the following steps:
[0096] Step 2.4.1: with As input, let respectively Through four branches of 1×1 convolution, strip convolution, and dilated convolution at different scales, multi-scale feature information is fully extracted. Furthermore, concatenation, convolution, and residual connections are used to enrich the feature information. The specific representation method is as follows:
[0097] ,
[0098] ,
[0099] ,
[0100] ,
[0101] ,
[0102] ,
[0103] in , It is a 3×1 convolution. This represents a 3×3 convolution with a hole rate of 3. This represents a 3×3 convolution with a hole rate of 5. , , and They represent The result after each branch express Summary characteristics after each branch It is a 3×3 convolution. For activation functions;
[0104] Step 2.4.2: Take the result obtained in step 2.4.1 Perform progressive fusion to obtain an image with the approximate area of the polyp. It employs a phased upsampling strategy to avoid information loss caused by a large-scale upsampling at once, and enhances the feature information of the polyp region by weighting the feature importance through element-wise multiplication. The specific process is as follows: First, high-level features are... Upsampling The size, after convolution adjustment, is the same as... Element-wise multiplication generates modulation features. Then Upsampling to The scale, at the same time After upsampling and convolution adjustment, both together with Element-wise multiplication generates modulation features. Secondly, the modulation characteristics and The upsampled features are concatenated along the channel dimension and then fused by convolution to generate intermediate features. Then the modulation features With intermediate features Upsampled and convolutionally adjusted features are concatenated through channels and then convolved to generate intermediate features. Finally, the intermediate features The image is refined using convolutional layers and the number of channels is adjusted to 1, outputting an image with the approximate area of the polyp. The specific expression method is as follows:
[0105] ,
[0106] ,
[0107] ,
[0108] ,
[0109] ,
[0110] in, To upsample by 2 times, This represents the convolution operation. This indicates a splicing operation.
[0111] Step 2.5: The results obtained in Step 2.4 Downsampling, to obtain Combine it with the product generated in step 2.3 The input is fed into the first-level Enhanced Reverse Attention Module (ERA). The ERA module is used to mine complementary regions and boundary information, effectively enhancing polyp boundary detection. Then... By performing a residual concatenation with the boundary detail information output from the first-level ERA, we obtain... ;
[0112] like Figure 6 As shown, the ERA module effectively enhances image boundary regions by mining background information and boundary features. Specifically, the first... The input of the level ERA is and generated by the DFE module , Through the reverse attention branch and the Laplace operator branch respectively, for the reverse attention branch, for Normalization is applied and the image is inverted to obtain the approximate background region; for the Laplacian operator branch, the normalization function ... Laplacian operator branch is inverted and the Laplacian operator branch is inverted and the Laplacian operator branch is inverted and the Laplacian operator branch is inverted and the Laplacian operator branch is invert The boundary information is obtained after processing with the Laplacian operator. Then, the outputs of the two branches are concatenated along the channel dimension and subjected to convolution before being combined with the input. Element-wise multiplication to enhance features The boundary details are then refined using convolutional operations, as shown in the following specific representation:
[0113] ,
[0114] ,
[0115] ,
[0116] ,
[0117] in , This indicates the negation operation. For normalization function, This represents the Laplace operator operation. For convolution operations, and They represent the first The outputs of the reverse attention branch and the Laplacian operator branch in a level ERA. For the first Spatial weight representation of the two branches in ERA synthesis. Indicates the first The output of the level ERA.
[0118] Step 2.6: For Upsampling Combine it with the product generated in step 2.3 The input is fed into the second-level ERA, and its output boundary detail information is consistent with... After adding, we get ;
[0119] Step 2.7: [The sentence is incomplete and requires more context to be translated accurately.] Upsampled image and the product generated in step 2.3 The input is fed into the third-level ERA, and the boundary detail information output by the third-level ERA is compared with... Perform residual join to obtain ;
[0120] Step 2.8: For , , and Conduct in-depth supervision, The image is upsampled to the size of the input image, and then binarized using the sigmoid function to obtain the prediction map of the network model, i.e., the segmentation result.
[0121] Step 3: Perform data augmentation on the training data obtained in Step 1, and input it into the network model in Step 2 for training;
[0122] Specifically, the model training process is optimized using a weighted sum of Dice similarity coefficient loss and binary cross-entropy loss (BCE loss), as mentioned in step 2.8. , , and Deep supervision is performed during training; therefore, the overall loss function is expressed as follows:
[0123] ,
[0124] ,
[0125] In this invention and The value of is 1. To Upsampled to the size of the label image, Represents a real label image. and These represent Dice loss and BCE loss, respectively. This is the final loss function of this invention.
[0126] The training process for the model in this invention is as follows: end-to-end training is performed using an NVIDIA GeForce RTX 3090 GPU with a batch size of 8 and a total of 100 training rounds. The AdamW optimizer is used with a weight decay of 0.0001 and an initial learning rate of 0.0002, which gradually decreases as training progresses. The parameters of the VSS and ConvNeXt blocks are initialized using weights pre-trained on ImageNet.
[0127] Step 4: Use the best model obtained after training to test the colonoscopy images in the test set, obtain the polyp segmentation results after testing, and evaluate the segmentation results of the model.
[0128] In step 4, the polyp images from the test set are input into the trained polyp image segmentation model, and the output segmentation mask is used for evaluation. The proposed network model is trained and tested using three classic polyp segmentation datasets: CVC-ClinicDB, CVC-ColonDB, and Kvasir-SEG. The test results are compared with existing classic or advanced medical image segmentation network models to demonstrate the effectiveness of the invention. The proposed network model is implemented using PyTorch, and all image inputs are uniformly set to 384×384. The test results on the CVC-ClinicDB, CVC-ColonDB, and Kvasir-SEG datasets are shown in Tables 1, 2, and 3, respectively. The evaluation metrics used in this invention include: mean Dice similarity coefficient (Dice), mean IoU similarity coefficient (mIoU), accuracy (ACC), 95% Hausdorff distance (HD95), and mean absolute error (MAE). To ensure fairness, all other classic or advanced network models maintain the same data processing methods and training strategies as the network model in this invention.
[0129] Table 1. Test results of this invention and 10 other methods on the CVC-ClinicDB dataset.
[0130]
[0131] Table 2. Test results of this invention and 10 other methods on the CVC-ColonDB dataset.
[0132]
[0133] Table 3. Test results of this invention and 10 other methods on the Kvasir-SEG dataset.
[0134]
[0135] The bolded text represents the optimal value for each indicator. Up indicates that the higher the result for that indicator, the better the performance, and down indicates that the lower the result for that indicator, the better the performance.
[0136] Figure 7 This is a comparison chart of the experimental results of this invention with other network models. Figure 8 The visualization shows the experimental comparison. The green area represents the overlap between the model's predicted segmentation map and the ground truth label map; the yellow area represents the portion of the real polyp region that was not detected by the network model; and the red area represents the region where the network model mistakenly predicted normal tissue as a polyp. The figure demonstrates that the polyp segmentation method proposed in this invention can obtain segmentation results with more accurate boundaries and a more complete semantic structure.
[0137] It is understood that this invention has been described through some embodiments, and those skilled in the art will recognize that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the invention. Furthermore, under the teachings of this invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the invention. Therefore, this invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of this invention.
Claims
1. A polyp image segmentation method based on Mamba-convolutional dual-branch and boundary enhancement, characterized in that... Includes the following steps: Step 1: Collect colonoscopy polyp images and polyp mask labels dataset, and divide the dataset into training set and test set; Step 2: Construct a polyp image segmentation network model based on Mamba-convolutional dual-branch and boundary enhancement; Step 3: Perform data augmentation on the training data obtained in Step 1, and input it into the network model in Step 2 for training; Step 4: Use the best model obtained after training to test the colonoscopy images in the test set, obtain the polyp segmentation results after testing, and evaluate the segmentation results of the model.
2. The polyp image segmentation method based on Mamba-convolutional dual-branch and boundary enhancement according to claim 1, characterized in that: In step 1, the dataset contains three sub-datasets: the publicly available colonoscopy polyp image datasets CVC-ClinicDB, CVC-ColonDB, and Kvasir-SEG. Each dataset is randomly divided in an 8:2 ratio to obtain a training set and a test set. The model is trained on the training set of each dataset, and finally the segmentation performance of the final model is evaluated using the test set.
3. The polyp image segmentation method based on Mamba-convolutional dual-branch and boundary enhancement according to claim 1, characterized in that: The polyp image segmentation network model proposed in step 2, based on Mamba and convolutional dual-branch and boundary enhancement, is used to perform the following steps: Step 2.1: Using Mamba-based VSS blocks and convolution-based ConvNeXt blocks as dual branches for multi-level feature extraction, four-stage multi-level features are extracted from the original image. The features extracted by the Mamba branch are denoted as follows: The features extracted by the convolutional branch are denoted as ; Step 2.2: Use the dual-branch fusion module to analyze the characteristics of the Mamba branch output in Step 2.
1. Features of convolution branch output Weights are learned, then adaptively fused, and the fused features are denoted as... ; Step 2.3: Using the detail feature enhancement module, construct a Laplacian pyramid on the original image, extract high-frequency detail information from the original image, and combine it with the features obtained in Step 2.
2. By fusing and enhancing the details and edges of features, detailed enhanced features are obtained. ; Step 2.4: Enhance the details obtained in Step 2.3 The input is fed into a multi-scale feature fusion module, which uses convolutions with different kernel sizes to extract and learn image features at multiple scales. The mined semantic information is then progressively fused to obtain a coarse segmentation map, denoted as... ; Step 2.5: [Referring to the information obtained in Step 2.4] Downsampling, to obtain Combine it with the product generated in step 2.3 The input is fed into the first-level enhanced reverse attention module, which uses this module to mine complementary regions and boundary information, effectively enhancing polyp boundary detection. Then... A residual connection is performed with the boundary detail information output by the first-level enhanced back attention module to obtain... ; Step 2.6: For Upsampling Combine it with the product generated in step 2.3 The input is fed into the second-level enhanced reverse attention module, whose output boundary detail information is... After adding, we get ; Step 2.7: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require Upsampled image and the product generated in step 2.3 The input is fed into the third-level enhanced reverse attention module, whose output boundary detail information is... Obtained after residual connection ; Step 2.8: For , , and Conduct in-depth supervision; The image is upsampled to the size of the input image, and then binarized using the sigmoid function to obtain the prediction map of the network model, i.e., the segmentation result.
4. The polyp image segmentation method based on Mamba-convolutional dual-branch and boundary enhancement according to claim 3, characterized in that: The dual-branch fusion module proposed in step 2.2 describes the characteristics of the Mamba branch output. Features of convolution branch output To perform adaptive fusion, specifically, first let and Features of each branch are extracted using 1×1 convolutions, then concatenated along the channel dimension and weighted using a normalized exponential function. and Mamba branch weights Finally, the features of each stage of convolutional branches are... and Mamba branching features Use the corresponding weights and Weighted summation ( ), to obtain fusion features .
5. The polyp image segmentation method based on Mamba-convolutional dual-branch and boundary enhancement according to claim 3, characterized in that: The detailed feature enhancement module proposed in step 2.3 includes the following steps: Step 2.3.1: Construct a Laplacian pyramid for the original image to obtain high-frequency detail information in four stages. The specific process is as follows: Convert the original image to grayscale, construct a five-layer Gaussian pyramid through Gaussian smoothing and downsampling operations, then upsample the output image of each layer of the Gaussian pyramid, and subtract the upsampled image of the current layer from the output image of the previous layer of the Gaussian pyramid to obtain the output image of the current layer of the Laplacian pyramid. By analogy, we can obtain ; Step 2.3.2: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation The fusion features obtained after normalization and step 2.2 Multiply element by element, then multiply Residual connections ultimately yield features with strong high-frequency detail information. .
6. The polyp image segmentation method based on Mamba-convolutional dual-branch and boundary enhancement according to claim 3, characterized in that: The multi-scale feature fusion module proposed in step 2.4 includes the following steps: Step 2.4.1: with As input, respectively let Through four branches of 1×1 convolution, strip convolution, and dilated convolution at different scales, multi-scale feature information is fully extracted. Furthermore, concatenation, convolution, and residual connections are used to enrich the feature information, resulting in... ; Step 2.4.2: Take the result obtained in step 2.4.1 A progressive fusion was performed to obtain an image with the approximate area of the polyp. The specific process is as follows: First, high-level features... Upsampling The size, after convolution adjustment, is the same as... Element-wise multiplication generates modulation features. Then Upsampling to The scale, at the same time After upsampling and convolution adjustment, both together with Element-wise multiplication generates modulation features. ; Secondly, the modulation features and The upsampled features are concatenated along the channel dimension and then fused by convolution to generate intermediate features. Modulation features With intermediate features Upsampled and convolutionally adjusted features are concatenated through channels and then convolved to generate intermediate features. Finally, the intermediate features The image is refined using convolutional layers and the number of channels is adjusted to 1, outputting an image with the approximate area of the polyp. .
7. The polyp image segmentation method based on Mamba-convolutional dual-branch and boundary enhancement according to claim 3, characterized in that: The enhanced reverse attention module mentioned in steps 2.5, 2.6, and 2.7 effectively enhances the image boundary region by mining background information and boundary features. Specifically, the... The input to the enhanced reverse attention module at level 1 is and the detail feature enhancement module , Through the reverse attention branch and the Laplacian operator branch respectively, for the reverse attention branch, for Normalization is applied and the image is inverted to obtain the approximate background region; for the Laplacian operator branch, the normalization function is applied and the image is inverted to obtain the approximate background region; for the Laplacian operator branch, the normalization function is applied and the Laplacian operator branch is inverted ... The boundary information is obtained after processing with the Laplacian operator. Then, the outputs of the two branches are concatenated along the channel dimension, followed by convolution, and then combined with the input. Element-wise multiplication to enhance features The boundary details are then refined by convolutional operations.
8. The polyp image segmentation method based on Mamba-convolutional dual-branch and boundary enhancement according to claim 1, characterized in that: In step 3, the model training process is optimized by using a weighted sum of Dice similarity coefficient loss and binary cross-entropy loss (BCE loss), namely: In addition, , , and Deep supervision is performed during training; therefore, the total loss function is... In this invention and The value of is 1. To Upsample to the same scale as the label image. Represents the actual label image. and These represent Dice loss and BCE loss, respectively. This is the final loss function of this invention.