A light polyp image segmentation method based on a double-branch adaptive fusion

Abstract

The present application belongs to the technical field of medical image processing based on deep learning, and particularly relates to a lightweight polyp image segmentation method based on double-branch adaptive fusion. The method first extracts feature maps of multiple scales by using a backbone network, and unifies the channel dimension by a channel alignment module; then a multi-scale double-branch adaptive fusion module is used to enhance each scale feature, perform cross-resolution interaction between the high-resolution branch and the low-resolution branch, and realize adaptive fusion combined with channel-spatial attention; then an attention-guided feature interaction module is used to gradually fuse the multi-scale features from the deep layer to the shallow layer; finally, the polyp segmentation result is output through a prediction head. The present application can improve the segmentation capability for polyp boundaries and multi-scale targets under lower computational complexity, and is more lightweight and efficient.

Description

Technical Field

[0001] This invention belongs to the field of deep learning-based medical image processing technology, specifically relating to a lightweight polyp image segmentation method based on dual-branch adaptive fusion. Background Technology

[0002] Colorectal cancer is one of the most common malignant tumors of the digestive tract, and colorectal polyps, especially adenomatous polyps, are closely related to the occurrence and development of colorectal cancer. Timely detection and removal of polyps through colonoscopy is of great significance in reducing the risk of colorectal cancer. During colonoscopy, accurately locating and segmenting the polyp area helps doctors determine the location, size, and boundaries of the lesion, providing a basis for subsequent diagnosis and treatment. However, polyps in colonoscopy images vary greatly in shape, size, color, and texture, and are often affected by factors such as intestinal folds, mucosal texture, uneven lighting, reflections, and low-contrast boundaries, making the boundary between the polyp area and the surrounding normal tissue unclear, increasing the difficulty of automatic segmentation.

[0003] Early polyp segmentation methods primarily relied on manually designed features or traditional image processing techniques, such as color, texture, edge, region growing, and thresholding. These methods typically require feature or parameter design based on the specific image scene and are highly sensitive to image quality, lighting conditions, and changes in background texture. In complex colonoscopy images, polyps may have highly similar colors and textures to the background mucosa. Traditional methods cannot fully utilize high-level semantic information and global contextual information, easily leading to problems such as incomplete segmentation regions, inaccurate boundary localization, and insufficient generalization ability.

[0004] With the development of deep learning, medical image segmentation methods based on encoder-decoder structures have been widely applied to polyp segmentation tasks. Typical UNet and its improved networks extract multi-level features through the encoder and gradually restore resolution using the decoder. Simultaneously, skip connections pass shallow detail features to the decoding stage, effectively preserving spatial details and achieving good results in medical image segmentation. However, conventional skip connections typically fuse shallow and deep features through direct concatenation or element-wise addition, failing to adequately handle semantic differences between features at different levels. Furthermore, the multi-scale features output during the encoding stage are often not sufficiently enhanced, resulting in a lack of effective interaction between shallow detail information and deep semantic information. In scenarios involving small-scale polyps, polyps with blurred boundaries, or polyps with irregular shapes, problems such as missed segmentation, boundary breaks, or incomplete region segmentation may still occur.

[0005] To improve polyp segmentation performance, previous studies have introduced mechanisms such as multi-scale feature fusion, inverse attention, boundary enhancement, and context modeling. For example, PraNet uses a parallel local decoder and inverse attention mechanism to progressively refine the polyp region, showing advantages in boundary correction; CaraNet improves the segmentation ability of small-target medical images through context-axis inverse attention; and Polyp-PVT uses a pyramid visual Transformer as the encoder to enhance feature representation. These methods have achieved good results in polyp segmentation tasks, but they also have shortcomings: some methods focus more on the decoding stage or the utilization of high-level semantic features, and do not sufficiently enhance the features at each scale in the encoding stage; attention or context enhancement modules may introduce additional computational overhead, making them less suitable for deployment in resource-constrained scenarios.

[0006] In recent years, the Transformer architecture has been increasingly adopted for medical image segmentation and polyp segmentation tasks due to its strong ability to model long-range dependencies. This type of method can enhance global contextual representation, helping to address the limited receptive field problem when relying solely on convolutional networks. However, Transformer or hybrid Transformer-CNN models typically have high model complexity and computational overhead, making deployment costly in real-time colonoscopy-assisted analysis, edge devices, or clinical assistance scenarios with limited computing resources.

[0007] Therefore, it is necessary to propose a lightweight polyp image segmentation method based on bi-branch adaptive fusion to enhance multi-scale feature representation with lower model complexity and improve the segmentation ability of small-scale polyps, polyps with blurred boundaries and polyps with irregular shapes. Summary of the Invention

[0008] To address the aforementioned problems in existing technologies, this invention provides a lightweight polyp image segmentation method based on dual-branch adaptive fusion. This method, based on multi-scale feature extraction from the backbone network, enhances the encoded features through a multi-scale dual-branch adaptive fusion module, and achieves step-by-step fusion in the decoding stage through an attention-guided feature interaction module, thereby improving the segmentation capability for polyps and boundary regions at different scales.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: A lightweight polyp image segmentation method based on dual-branch adaptive fusion includes the following steps: Step S1: Obtain the polyp image to be segmented, and input the polyp image into the backbone network for multi-scale feature extraction to obtain multiple feature maps of different resolutions; Step S2: Perform channel alignment processing on the multiple feature maps respectively, so that feature maps of different resolutions are mapped to the same channel dimension; Step S3: Input the channel-aligned feature maps into the multi-scale bi-branch adaptive fusion module for feature enhancement. The multi-scale bi-branch adaptive fusion module includes multiple bi-branch adaptive fusion sub-modules. Each bi-branch adaptive fusion sub-module constructs a high-resolution branch and a low-resolution branch for the input features at the corresponding scale. Multi-scale features after feature enhancement are obtained through cross-resolution feature interaction and channel-space attention fusion. Step S4: Input the multi-scale features after feature enhancement into the decoder. The decoder uses multiple attention-guided feature interaction modules to fuse deep features and shallow features step by step from deep to shallow to obtain the final fused features. Step S5: Input the final fused features into the segmentation output module, output a segmentation prediction map, and upsample the segmentation prediction map to the resolution of the input polyp image to obtain the polyp segmentation result.

[0010] Furthermore, the backbone network mentioned in step S1 is an EfficientNet network, which outputs four feature maps at different resolutions. ,in, and Arranged from shallowest to deepest network layers, their scales are 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input image, respectively. and These are shallow features. and This represents a deep feature.

[0011] Furthermore, the channel alignment process in step S2 employs 1×1 convolution and batch normalization operations, respectively, for the feature map. and Channel dimension mapping is performed to uniformly map the number of channels in each feature map to a preset number of channels, resulting in one-to-one channel-aligned feature maps. ,in and These are shallow features after channel alignment. and This represents the deep features after channel alignment.

[0012] Furthermore, the multi-scale bi-branch adaptive fusion module MSDAF in step S3 includes four bi-branch adaptive fusion sub-modules DAFM, and feature maps and Each corresponds to a DAFM module, and each dual-branch adaptive fusion submodule (DAFM) receives input features at the corresponding scale. This is used as the input feature X of the current DAFM submodule, and the corresponding scale of the enhanced features are output. Where i = 1, 2, 3, 4 and In a one-to-one correspondence, the dual-branch adaptive fusion submodule DAFM is divided into a standard structure and a lightweight structure. Both the standard structure and the lightweight structure achieve feature enhancement based on the following three stages: The first stage is the dual-branch initial feature construction stage. Branch 1 performs convolution mapping on the input feature X to obtain the branch-1 initial feature that maintains the resolution of the input feature. Branch two downsamples the same input feature X and then performs convolution mapping to obtain branch two initial features with a resolution lower than that of the initial features of branch one. This leads to two branches: in, and This represents the convolution mapping operation for the corresponding branch. Indicates average pooling; The second stage is the cross-resolution interaction and intra-branch fusion stage, which involves the initial features of branch one. and the initial features of branch two Perform cross-resolution feature interaction, which includes transform features within branch one. Transformation characteristics of downsampling from branch one to branch two Transformation characteristics within branch two and the transformation characteristics of branch 2 to branch 1 upsampling : in, ~ This represents the convolution mapping operation for the corresponding branch. Indicates an upsampling operation; Will and As a branch-1 fusion input, and As the input for branch two fusion, the features after branch one fusion are then obtained through channel-spatial attention fusion. Features after fusion with branch two : Wherein, CSAF represents the channel-spatial attention fusion module; The third stage is the final fusion and residual output stage, which first processes the fused branch features. Perform output convolution mapping on the fused branch two features First, upsample to the same resolution as the feature in branch one, then perform output convolution mapping. Next, perform channel-spatial attention fusion on the two output convolution mapping results, and finally add them element-wise with the input feature X to obtain the final output feature Y of DAFM: in, This represents element-wise addition. and This represents the convolution mapping operation for the corresponding branch, and thus, and After being processed by their respective dual-branch adaptive fusion submodules, enhanced features are obtained. and .

[0013] Furthermore, the multi-scale dual-branch adaptive fusion module MSDAF employs differentiated processing strategies for features at different levels. and It adopts a standard structure of dual-branch adaptive fusion submodule DAFM; for features and The system employs a lightweight dual-branch adaptive fusion submodule DAFM. The overall processing stages are the same for both the standard and lightweight structures. In the standard structure, the convolution mapping operation... ~ This is a convolutional mapping unit that sequentially performs 3×3 convolution, batch normalization, and ReLU activation; in the lightweight architecture, the convolutional mapping operation... and It is a 1×1 convolution, and the number of intermediate channels in the convolution operation of the lightweight structure is less than the corresponding number of intermediate channels in the standard structure.

[0014] Furthermore, the channel-spatial attention fusion module includes: The two input features are added element-wise to obtain the initial fused features; The preliminary fused features are subjected to global average pooling, and then channel attention weights are generated through two layers of convolution and activation functions. The preliminary fused features are subjected to channel-dimensional average pooling and max pooling respectively, and then spatial attention weights are generated by convolution and activation functions after concatenation. The preliminary fusion features are multiplied sequentially by the channel attention weight and the spatial attention weight to obtain the attention-weighted fusion output.

[0015] Furthermore, the Attention-Guided Feature Interaction (AGFI) module in step S4 is used to progressively fuse the two input features of the current stage during the decoding phase, wherein the first input feature... The second input feature is the deep input feature in the current stage. For the shallow input features in the current stage, the attention-guided feature interaction module includes: The upsampling submodule is used for feature sampling. Upsampling is performed to obtain features To make its resolution match the characteristics of the current stage match; The feature concatenation submodule is used to concatenate the upsampled features. With the second input features The splicing is performed along the channel dimension to obtain the spliced ​​features. ; The fusion attention submodule is used to process the spliced ​​features. Perform 1×1 convolutional channel dimensionality reduction and ReLU sequentially. Activation, 1×1 convolution, and sigmoid activation generate fused attention weights A, which are then combined with concatenated features. Element-wise multiplication yields the weighted features. ; The fusion convolution submodule is used to process the weighted features. The fused features are obtained by sequentially performing two-dimensional convolution, batch normalization, ReLU activation, two-dimensional convolution, and batch normalization operations. ; The residual connection submodule is used to connect features After convolution adjustment and fusion features By adding elements one by one, the output characteristics of the AGFI module at the current stage can be obtained. .

[0016] Furthermore, the decoder in step S4 includes three cascaded attention-guided feature interaction modules, with the first stage used for fusing and enhancing features. With enhanced features The first-level output features are obtained. The second stage is used to fuse the output features of the first stage with the enhanced features. The second-level output features are obtained. The third stage is used to fuse the output features of the second stage with the enhanced features. The final fusion features are obtained. The step-by-step fusion process can be represented as: Furthermore, the segmentation output module in step S5 consists of a 3×3 convolutional layer, a normalization layer, a Dropout layer, and a 1×1 convolutional layer connected in sequence. It is used to generate a single-channel segmentation prediction map based on the final fusion features. The single-channel segmentation prediction map represents the prediction result that each pixel in the input polyp image belongs to the polyp region.

[0017] The beneficial effects of this invention are: This invention addresses the problems of large target scale variations, blurred boundaries, irregular shapes, and interference from complex background textures in polyp images. It introduces a multi-scale bi-branch adaptive fusion module in the encoding enhancement stage of a lightweight polyp image segmentation network, and combines a channel-spatial attention fusion module in the bi-branch adaptive fusion sub-module. This enables the model to adaptively enhance the feature responses related to the polyp region from both channel and spatial dimensions, suppressing interference from background textures and non-polyp regions, thereby improving the stability of polyp region localization and boundary prediction.

[0018] In the dual-branch adaptive fusion submodule, this invention constructs two branches for the same input feature. Through cross-scale feature interaction, it simultaneously utilizes local detail information in the original scale branch and contextual semantic information in the down-resolution branch. Furthermore, channel-spatial attention fusion is adopted in both the intra-branch fusion stage and the final dual-branch fusion stage, enabling information at different scales to be adaptively integrated. This helps reduce the cases of missed segmentation of small-scale polyps, boundary breaks, and incomplete segmentation of irregular regions.

[0019] This invention targets shallow enhancement features , and deep enhancement features , The differences in their roles in polyp segmentation tasks are addressed using a differentiated processing strategy: shallow enhancement features are generated. , The dual-branch adaptive fusion submodule adopts a standard structure to preserve the ability to express details such as boundaries, textures, and local morphology; it is used to generate deep enhancement features. , The dual-branch adaptive fusion submodule adopts a lightweight structure to reduce redundant computations in the deep semantic enhancement process, thereby achieving a better balance between segmentation accuracy and model complexity.

[0020] The lightweight structure of this invention reduces the number of parameters and computational load in the deep feature enhancement process by replacing the convolutional mapping used for the initial features of branch 1 in the first stage and the output convolutional mapping in the third stage with 1×1 convolutions and reducing the number of intermediate channels. This is beneficial for deployment and application in resource-constrained medical image-assisted diagnosis scenarios.

[0021] This invention enhances features by progressively fusing them through cascaded attention-guided feature interaction modules. and This allows deep semantic information to gradually align and complement shallow detail information, which helps improve the integrity of segmented regions and the stability of boundary prediction for multi-scale polyps, polyps with blurred boundaries, and polyps with irregular shapes.

[0022] Overall, this invention maintains good polyp segmentation performance with relatively low parameter count and computational complexity, and is suitable for colonoscopy polyp image-assisted analysis scenarios that require both lightweight models and high segmentation accuracy. Attached Figure Description

[0023] Figure 1 This is a flowchart of the polyp segmentation method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the overall network structure of a lightweight polyp image segmentation method based on dual-branch adaptive fusion provided in an embodiment of the present invention. The network is referred to as LEAFNet in this embodiment and includes an EfficientNet backbone network, a channel alignment module, a multi-scale dual-branch adaptive fusion module, an attention-guided feature interaction module, and a segmentation output module. Figure 3 This is a structural diagram of the dual-branch adaptive fusion module DAFM provided in an embodiment of the present invention; Figure 4 This is a structural diagram of the Channel-Spatial Attention Fusion Module (CSAF) provided in an embodiment of the present invention; Figure 5 This is a structural diagram of the attention-guided feature interaction module AGFI provided in an embodiment of the present invention; Figure 6 This is a comparison of segmentation results on five public datasets provided in this embodiment of the invention. Detailed Implementation

[0024] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings and technical solutions.

[0025] like Figure 1 The diagram shown is a flowchart of a polyp image segmentation method provided in an embodiment of the present invention, including the following steps: Step S1: Obtain the polyp image to be segmented, and input the polyp image into the backbone network for multi-scale feature extraction to obtain multiple feature maps of different resolutions; Step S2: Perform channel alignment processing on the multiple feature maps respectively, so that feature maps of different resolutions are mapped to the same channel dimension; Step S3: Input the channel-aligned feature maps into the multi-scale bi-branch adaptive fusion module for feature enhancement. The multi-scale bi-branch adaptive fusion module includes multiple bi-branch adaptive fusion sub-modules. Each bi-branch adaptive fusion sub-module constructs a high-resolution branch and a low-resolution branch for the input features at the corresponding scale. Multi-scale features after feature enhancement are obtained through cross-resolution feature interaction and channel-space attention fusion. Step S4: Input the multi-scale features after feature enhancement into the decoder. The decoder uses multiple attention-guided feature interaction modules to fuse deep features and shallow features step by step from deep to shallow to obtain the final fused features. Step S5: Input the final fused features into the segmentation output module, output a segmentation prediction map, and upsample the segmentation prediction map to the resolution of the input polyp image to obtain the polyp segmentation result.

[0026] The specific process of step S1 is as follows: In one embodiment, the collected publicly available polyp image dataset includes Kvasir-SEG, CVC-ClinicDB, CVC-ColonDB, ETIS, and CVC-300. Preferably, the input image size is uniformly adjusted to 352×352. For dataset partitioning, 1450 images (900 from Kvasir-SEG and 550 from CVC-ClinicDB) are used as training samples, further divided into training and validation sets for model training and parameter selection. The remaining 100 images from Kvasir-SEG, 62 images from CVC-ClinicDB, and images from ETIS, CVC-ColonDB, and CVC-300 are used as the test dataset for final performance evaluation. The final test set is not used in the model parameter selection process.

[0027] Table 1 below provides detailed information on each colon polyp dataset in this embodiment: Table 1

[0028] like Figure 2 The diagram shown illustrates the overall network structure of a lightweight polyp segmentation method based on dual-branch adaptive fusion constructed in this embodiment. Preferably, the backbone network is first initialized using pre-trained weights (efficientnet-b4), and each module of the backbone network outputs four feature maps of different resolutions at four preset locations. Their resolutions are 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input image resolution, corresponding to 32, 56, 160, and 448 channels, respectively. and These are shallow features. and This represents a deep feature.

[0029] The specific process of step S2 is as follows: Subsequently, the feature map is aligned using the channel alignment module. Unified mapping to the same channel dimension, preferably 32 channels, yields channel-aligned feature maps. This is to facilitate subsequent feature enhancement and fusion, among which and These are shallow features after channel alignment. and This represents the deep features after channel alignment. Each channel alignment module consists of 1×1 convolutions and batch normalization, which reduces computational complexity while maintaining the discriminative power of the features.

[0030] The specific process of step S3 is as follows: The multi-scale bi-branch adaptive fusion module MSDAF is used to process the channel-aligned feature maps. Enhance them separately. For example Figure 2 As shown, the MSDAF module contains four dual-branch adaptive fusion submodules (DAFM). The input features at each scale are enhanced by the corresponding dual-branch adaptive fusion submodule to obtain the enhanced features. ,in and This is a shallow enhancement feature. and This is a deep enhancement feature.

[0031] Used to generate deep enhancement features and The dual-branch adaptive fusion submodule employs a lightweight structure to generate shallow enhancement features. and The dual-branch adaptive fusion submodule adopts a standard structure.

[0032] like Figure 3 As shown, the dual-branch adaptive fusion submodule DAFM specifically includes the following three stages: The first stage is the dual-branch initial feature construction stage. Branch 1 performs convolution mapping on the input feature X to obtain the branch-1 initial feature that maintains the resolution of the input feature. Branch two downsamples the same input feature X and then performs convolution mapping to obtain branch two initial features with a resolution lower than that of the initial features of branch one. This leads to two branches: in, and This represents the convolution mapping operation for the corresponding branch. The average pooling method is preferably used with a step size of 2. The second stage is the cross-resolution interaction and intra-branch fusion stage, which involves the initial features of branch one. and the initial features of branch two Perform cross-resolution feature interaction, which includes transform features within branch one. Transformation characteristics of downsampling from branch one to branch two Transformation characteristics within branch two and the transformation characteristics of branch 2 to branch 1 upsampling : in, ~ This represents the convolution mapping operation for the corresponding branch. This indicates an upsampling operation, preferably bilinear interpolation upsampling; Will and As a branch-1 fusion input, and As the input for branch two fusion, the features after branch one fusion are then obtained through channel-spatial attention fusion. Features after fusion with branch two : Wherein, CSAF represents the channel-spatial attention fusion module; The third stage is the final fusion and residual output stage, which first processes the fused branch features. Perform output convolution mapping on the fused branch two features First, upsample to the same resolution as the feature in branch one, then perform output convolution mapping. Next, perform channel-spatial attention fusion on the two output convolution mapping results, and finally add them element-wise with the input feature X to obtain the final output feature Y of DAFM: in, This represents element-wise addition. and This represents the convolution mapping operation for the corresponding branch, and thus, and After being processed by their respective dual-branch adaptive fusion submodules, enhanced features are obtained. and .

[0033] Through the above design, DAFM can simultaneously model local detail information and large receptive field context information, and use residual connections to preserve the original feature representation.

[0034] Preferably, for the feature map after channel alignment and The corresponding dual-branch adaptive fusion submodule adopts a lightweight DAFM structure to reduce the number of parameters and computational overhead in the deep feature enhancement process; for feature maps and The corresponding dual-branch adaptive fusion submodule adopts the standard DAFM structure to maintain the detailed representation ability of shallow features.

[0035] The overall processing stages are the same for both the standard and lightweight structures. In the standard DAFM structure, the convolution mapping operation... ~ This is a convolutional mapping unit that sequentially performs 3×3 convolution, batch normalization, and ReLU activation; in the lightweight architecture, the convolutional mapping operation... and Replace with 1×1 convolution; at the same time, the number of intermediate channels in the lightweight structure is less than the corresponding number of intermediate channels in the standard structure. Preferably, the number of intermediate channels is half of the corresponding number of intermediate channels in the standard structure.

[0036] like Figure 4 As shown, the Channel-Spatial Attention Fusion (CSAF) module is used to adaptively fuse two input features. First, it performs element-wise addition on the two input features to obtain preliminary fused features: in, Indicates preliminary fusion characteristics. and This represents the two input feature maps to be fused; during the branch-one fusion process, and These correspond to the feature transformation results within branch one and the feature transformation results from branch two to branch one, respectively; during the fusion process in branch two... and These correspond to the feature transformation results within branch two and the feature transformation results from branch one to branch two, respectively; in the final fusion process... and The results are shown separately after performing output convolution mapping on the features of the fused branch one and after upsampling and performing output convolution mapping on the features of the fused branch two.

[0037] Then, channel attention weights are generated based on the preliminary fusion features: in, This indicates a global average pooling operation. This represents a 1×1 convolution used for channel dimensionality reduction. Represents the ReLU activation function. This represents a 1×1 convolution used for channel upscaling. This represents the Sigmoid activation function. This represents the channel attention weight.

[0038] Next, calculate the spatial attention weights: in, This represents the average pooling result along the channel dimension. This represents the result of max pooling along the channel dimension. This indicates a channel-level concatenation operation. This represents a 7×7 convolution operation. This represents the spatial attention weights.

[0039] Finally, the output of the CSAF module is expressed as: in, denoted as element-wise multiplication, and Z represents the attention-weighted fused feature. Through the combined effect of channel attention and spatial attention, CSAF can enhance the responses of critical channels and critical spatial regions.

[0040] The specific process of step S4 is as follows: like Figure 5 As shown, the Attention-Guided Feature Interaction (AGFI) module is used to progressively fuse the two input features of the current stage during the decoding phase. The first input feature is a deep input feature of the current stage, and the second input feature is a shallow input feature of the current stage. First, the first input feature of the current stage is upsampled to obtain the upsampled feature. This ensures that its spatial dimensions match the second input feature of the current stage: in, This represents the first input feature of the current stage. This represents the second input feature at the current stage. When the upsampled feature size is inconsistent with the second input feature size, further interpolation can be used to... Adjust to the same resolution as the second input feature.

[0041] Subsequently, the upsampled first input features are compared with the second input features. The splicing is performed along the channel dimension to obtain the spliced ​​features. : Then, fusion attention weights are generated based on the concatenated features, and the concatenated features are weighted to obtain weighted features. : in, This represents the fusion attention weights generated from the concatenated features. This indicates that the first 1×1 convolution operation is performed on the spliced ​​features for channel dimensionality reduction. This indicates the second 1×1 convolution operation, used to generate a single-channel attention weight map.

[0042] Next, the weighted features are convolutionally fused to obtain the fused features. The second input feature is adjusted by convolution and then added element-wise to the fused feature to obtain the output feature for the current stage: in, , and This represents a two-dimensional convolution operation. Indicates batch normalization, express , This indicates the output characteristics of the AGFI module.

[0043] In this embodiment, the decoder uses three cascaded AGFI modules to gradually fuse deep semantic features with shallow detail features, ultimately generating a high-resolution segmentation prediction: The first level is used for fusing and enhancing features. With enhanced features The first-level output features are obtained. : The second stage is used to fuse the output features and enhanced features of the first stage. The second-level output features are obtained. : The third stage is used to fuse the output features and enhanced features of the second stage. The final fusion features are obtained. : The specific process of step S5 is as follows: like Figure 2 The final fusion features The input segmentation output module generates the segmentation result. The segmentation output module sequentially includes a 3×3 convolutional layer, a normalization layer, a Dropout layer, and a 1×1 convolutional layer to output a single-channel segmentation prediction map. This prediction map is then upsampled 4 times to restore the input image resolution, yielding the final segmentation result. in, This represents a 1×1 convolution operation. This represents a 3×3 convolution operation. This represents a single-channel segmentation prediction map with a resolution of 1 / 4 of the input image. This indicates a 4x bilinear interpolation upsampling. This represents the final output polyp segmentation diagram.

[0044] In one embodiment, the model is implemented in the PyTorch framework and trained on an NVIDIA RTX3090 GPU with 24GB of VRAM. Preferably, the AdamW optimizer is used, with an initial learning rate of 1e-4, a weight decay of 1e-4, a batch size of 16, and a gradient pruning strategy to improve training stability. Parameter optimization is performed in conjunction with a dynamic learning rate adjustment strategy, and a multi-scale training strategy is employed to enhance the model's adaptability to targets of different sizes. Preferably, the input image is scaled using three scaling ratios: 0.75, 1.0, and 1.25 before training.

[0045] In this embodiment, a structural loss function is used, which includes weighted binary cross-entropy loss and weighted intersection-over-union (IoU) loss. By assigning higher weights to boundary regions, the model's ability to learn about polyp boundary regions can be enhanced. The loss function is defined as follows: in, This indicates the weighted average loss. This represents the weighted binary cross-entropy loss; The weighted binary cross-entropy loss is calculated as follows: in, Let P represent the predicted segmentation map and M represent the ground truth mask label, where M is the pixel-level binary mask label corresponding to the input polyp image, with pixels in the polyp region labeled as 1 and pixels in the background region labeled as 0. Represents a binary cross-entropy with logical output; The weighted intersection-union ratio loss is calculated as follows: in, Indicates the weighted intersection. This represents the weighted union; The weight W is calculated as follows: in, This represents an average pooling operation with a kernel size of k. To enhance the coefficient, This represents absolute value operation; in the preferred embodiment, k=31. =5; The weight W is calculated by comparing the local mean with the original mask, giving edge regions a higher weight value, thereby enhancing the model's segmentation accuracy of polyp boundaries.

[0046] During training, optimal model parameters can be selected based on the performance of the validation set, and the corresponding weight file can be saved. It should be noted that the validation set and the final test set are independent of each other; the final test set does not participate in the model parameter selection process.

[0047] In one embodiment, the method of the present invention is experimentally verified on multiple publicly available polyp segmentation datasets. The trained model weights are loaded into the network, and inference is performed on the test images to output segmentation result maps. The evaluation metrics can be the average Dice coefficient (mDice) and the average IoU (mIoU) to measure the model's segmentation accuracy. Furthermore, with an input size of 352×352, the complexity of a single forward inference process of the complete network can be statistically analyzed to obtain the number of model parameters and floating-point operations, which is used to evaluate the model's lightweight nature. Examples of experimental results are shown in the table below: Table 2 presents a comparison of the average Dice and average IoU for different methods: Table 2

[0048] Table 3 presents the model complexity results of the embodiments of the present invention and the comparative methods under the condition of an input size of 352×352: Table 3

[0049] Based on the experimental results of this embodiment, combined with Figure 6The visualization results show that the method of this invention can obtain relatively complete polyp region prediction results on different publicly available polyp datasets. For small-scale polyps, polyps with blurred boundaries, and polyp samples with irregular shapes, the method of this invention has fewer missed segments, boundary breaks, and oversegments compared to some comparative methods, and the segmentation results have good consistency with the real mask in terms of region morphology and boundary contours.

[0050] As shown in Table 2, the method of this invention achieves superior results on the CVC-300 and CVC-ClinicDB datasets, with mDice coefficients of 0.9118 and 0.9457, and mIoU of 0.8465 and 0.9009, respectively. In the Kvasir-SEG, CVC-ColonDB, and ETIS datasets, although some large models with complex structures, such as PVT-Cascade, achieve slightly higher metrics, their parameter count and computational cost are high. In contrast, this invention maintains good segmentation performance with lower model complexity.

[0051] As shown in Table 3, the complexity metrics of this invention's model have 0.5M parameters and 1.35G floating-point operations (FLOPs). Taking PVT-Cascade as an example, the number of parameters in this invention is approximately 1 / 70th, and the computational cost is approximately 1 / 11th, demonstrating its feasibility for engineering applications. Compared to similar lightweight models like LV-UNet, this invention achieves higher mDice and mIoU metrics on multiple datasets with a smaller number of parameters, with significant improvements on the CVC-300, CVC-ColonDB, and CVC-ClinicDB datasets. The above quantitative results are consistent with... Figure 6 The visualization results corroborate each other, demonstrating that the lightweight polyp image segmentation method based on dual-branch adaptive fusion proposed in this invention can improve the polyp image segmentation effect under lightweight conditions.

Claims

1. A lightweight polyp image segmentation method based on dual-branch adaptive fusion, characterized in that, Includes the following steps: Step S1: Obtain the polyp image to be segmented, and input the polyp image into the backbone network for multi-scale feature extraction to obtain multiple feature maps of different resolutions; Step S2: Perform channel alignment processing on the multiple feature maps respectively, so that feature maps of different resolutions are mapped to the same channel dimension; Step S3: Input the channel-aligned feature maps into the multi-scale bi-branch adaptive fusion module for feature enhancement. The multi-scale bi-branch adaptive fusion module includes multiple bi-branch adaptive fusion sub-modules. Each bi-branch adaptive fusion sub-module constructs a high-resolution branch and a low-resolution branch for the input features at the corresponding scale. Multi-scale features after feature enhancement are obtained through cross-resolution feature interaction and channel-space attention fusion. Step S4: Input the multi-scale features after feature enhancement into the decoder. The decoder uses multiple attention-guided feature interaction modules to fuse deep features and shallow features step by step from deep to shallow to obtain the final fused features. Step S5: Input the final fused features into the segmentation output module, output a segmentation prediction map, and upsample the segmentation prediction map to the resolution of the input polyp image to obtain the polyp segmentation result.

2. The lightweight polyp image segmentation method based on dual-branch adaptive fusion according to claim 1, characterized in that, The backbone network mentioned in step S1 is the EfficientNet network, which outputs four feature maps at different resolutions. ,in, and Arranged from shallowest to deepest network layers, their scales are 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input image, respectively. and These are shallow features. and This represents a deep feature.

3. The lightweight polyp image segmentation method based on dual-branch adaptive fusion according to claim 1, characterized in that, The channel alignment process in step S2 uses 1×1 convolution and batch normalization operations to perform the feature map alignment. and Channel dimension mapping is performed to uniformly map the number of channels in each feature map to a preset number of channels, resulting in one-to-one channel-aligned feature maps. ,in and These are shallow features after channel alignment. and This represents the deep features after channel alignment.

4. The lightweight polyp image segmentation method based on dual-branch adaptive fusion according to claim 1, characterized in that, The multi-scale bi-branch adaptive fusion module MSDAF in step S3 includes four bi-branch adaptive fusion sub-modules DAFM, and feature maps. and Each corresponds to a DAFM module, and each dual-branch adaptive fusion submodule (DAFM) receives input features at the corresponding scale. This is used as the input feature X of the current DAFM submodule, and the corresponding scale of the enhanced features are output. Where i = 1, 2, 3, 4 and In a one-to-one correspondence, the dual-branch adaptive fusion submodule DAFM is divided into a standard structure and a lightweight structure. Both the standard structure and the lightweight structure achieve feature enhancement based on the following three stages: The first stage is the dual-branch initial feature construction stage. Branch 1 performs convolution mapping on the input feature X to obtain the branch-1 initial feature that maintains the resolution of the input feature. Branch two downsamples the same input feature X and then performs convolution mapping to obtain branch two initial features with a resolution lower than that of the initial features of branch one. This leads to two branches: in, and This represents the convolution mapping operation for the corresponding branch. Indicates average pooling; The second stage is the cross-resolution interaction and intra-branch fusion stage, which involves the initial features of branch one. and the initial features of branch two Perform cross-resolution feature interaction, which includes transform features within branch one. Transformation characteristics of downsampling from branch one to branch two Transformation characteristics within branch two and the transformation characteristics of branch 2 to branch 1 upsampling : in, ~ This represents the convolution mapping operation for the corresponding branch. Indicates an upsampling operation; Will and As a branch-1 fusion input, and As the input for branch two fusion, the features after branch one fusion are then obtained through channel-spatial attention fusion. Features after fusion with branch two : Wherein, CSAF represents the channel-spatial attention fusion module; The third stage is the final fusion and residual output stage, which first processes the fused branch features. Perform output convolution mapping on the fused branch two features First, upsample to the same resolution as the feature in branch one, then perform output convolution mapping. Next, perform channel-spatial attention fusion on the two output convolution mapping results, and finally add them element-wise with the input feature X to obtain the final output feature Y of DAFM: in, This represents element-wise addition. and This represents the convolution mapping operation for the corresponding branch, and thus, and After being processed by their respective dual-branch adaptive fusion submodules, enhanced features are obtained. and .

5. A lightweight polyp image segmentation method based on dual-branch adaptive fusion according to claim 4, characterized in that, The multi-scale dual-branch adaptive fusion module MSDAF employs differentiated processing strategies for features at different levels. and It adopts a standard structure of dual-branch adaptive fusion submodule DAFM; for features and The system employs a lightweight dual-branch adaptive fusion submodule DAFM. The overall processing stages are the same for both the standard and lightweight structures. In the standard structure, the convolution mapping operation... ~ This is a convolutional mapping unit that sequentially performs 3×3 convolution, batch normalization, and ReLU activation; in the lightweight architecture, the convolutional mapping operation... and It is a 1×1 convolution, and the number of intermediate channels in the convolution operation of the lightweight structure is less than the corresponding number of intermediate channels in the standard structure.

6. A lightweight polyp image segmentation method based on dual-branch adaptive fusion according to claim 4, characterized in that, The channel-spatial attention fusion module includes: The two input features are added element-wise to obtain the initial fused features; The preliminary fused features are subjected to global average pooling, and then channel attention weights are generated through two layers of convolution and activation functions. The preliminary fused features are subjected to channel-dimensional average pooling and max pooling respectively, and then spatial attention weights are generated by convolution and activation functions after concatenation. The preliminary fusion features are multiplied sequentially by the channel attention weight and the spatial attention weight to obtain the attention-weighted fusion output.

7. A lightweight polyp image segmentation method based on dual-branch adaptive fusion according to claim 1, characterized in that, The Attention-Guided Feature Interaction (AGFI) module in step S4 is used to progressively fuse the two input features of the current stage during the decoding phase, where the first input feature... The second input feature is the deep input feature in the current stage. For the shallow input features in the current stage, the attention-guided feature interaction module includes: The upsampling submodule is used for feature sampling. Upsampling is performed to obtain features To make its resolution match the characteristics of the current stage match; The feature concatenation submodule is used to concatenate the upsampled features. With the second input features The splicing is performed along the channel dimension to obtain the spliced ​​features. ; The fusion attention submodule is used to process the spliced ​​features. Perform 1×1 convolutional channel dimensionality reduction and ReLU sequentially. Activation, 1×1 convolution, and sigmoid activation generate fused attention weights A, which are then combined with concatenated features. Element-wise multiplication yields the weighted features. ; The fusion convolution submodule is used to process the weighted features. The fused features are obtained by sequentially performing two-dimensional convolution, batch normalization, ReLU activation, two-dimensional convolution, and batch normalization operations. ; The residual connection submodule is used to connect features After convolution adjustment and fusion features By adding elements one by one, the output characteristics of the AGFI module at the current stage can be obtained. .

8. A lightweight polyp image segmentation method based on dual-branch adaptive fusion according to claim 5, characterized in that, The decoder in step S4 includes three cascaded attention-guided feature interaction modules, with the first stage used for fusing and enhancing features. With enhanced features The first-level output features are obtained. The second stage is used to fuse the output features of the first stage with the enhanced features. The second-level output features are obtained. The third stage is used to fuse the output features of the second stage with the enhanced features. The final fusion features are obtained. The step-by-step fusion process can be represented as: 。 9. A lightweight polyp image segmentation method based on dual-branch adaptive fusion according to claim 1, characterized in that, The segmentation output module in step S5 consists of a 3×3 convolutional layer, a normalization layer, a Dropout layer, and a 1×1 convolutional layer connected in sequence. It is used to generate a single-channel segmentation prediction map based on the final fusion features. The single-channel segmentation prediction map represents the prediction result that each pixel in the input polyp image belongs to the polyp region.

Images