Lightweight polyp segmentation method based on multi-scale differential features and spatial attention

Abstract

The application discloses a lightweight polyp segmentation method based on multi-scale differential features and spatial attention, comprising the following steps: firstly, preprocessing and dividing an intestinal polyp image dataset; then, constructing a lightweight segmentation model LightRGC-SCNet containing an encoder-decoder structure; in the encoding stage of the model, a multi-scale differential feature extraction module is adopted to enhance edges and extract multi-scale features; in the decoding stage, a joint spatial attention guiding DySample up-sampling module is used to dynamically fuse features and complete up-sampling, and on this basis, a semantic-spatial information fusion module is used to cooperatively optimize output features; finally, a combined loss function is used to train the model, and performance evaluation and segmentation result output are completed on a test set; the application combines a hierarchical processing mechanism and an encoder-decoder architecture, significantly reduces the complexity of the model, realizes accurate segmentation of intestinal polyps, and is suitable for clinical application scenarios with limited computing resources.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and medical image processing technology, specifically to a lightweight polyp segmentation method based on multi-scale differential features and spatial attention. Background Technology

[0002] Colorectal cancer, a prevalent malignant tumor worldwide, often originates from colorectal polyps. Therefore, early screening for intestinal polyps has become a crucial measure to reduce the incidence and mortality of colorectal cancer. Colonoscopy, currently the mainstream screening method, relies heavily on the operator's experience for diagnostic accuracy. Differences between operators can easily lead to missed diagnoses and inconsistent diagnostic results, directly impacting the effectiveness of early intervention. Introducing polyp segmentation algorithms into the screening process, through automated pixel-level analysis to reduce subjective interpretation bias, can effectively improve the objectivity and consistency of colonoscopy.

[0003] Existing polyp segmentation algorithms still have certain limitations. Deep learning-based methods are limited by local receptive fields, making it difficult to model the global semantic relationship between polyps and complex intestinal backgrounds, and are prone to segmentation errors in regions with blurred boundaries. While Transformer-based methods have stronger global modeling capabilities, their fixed attention weights can lead to redundant computations in complex scenes, resulting in a decrease in the ability to perceive local details and difficulty in capturing the small-scale spatial features of polyps. In addition, existing polyp segmentation algorithms have not effectively solved the problem of balancing lightweight models with segmentation accuracy. In application scenarios with diverse polyp morphologies and complex intestinal backgrounds, conventional polyp segmentation algorithms, due to their multi-layer convolution or Transformer architecture design, have excessively high parameter counts and computational costs, making them unsuitable for computationally limited clinical applications. While lightweight polyp segmentation algorithms reduce model complexity through structural compression, they weaken the ability to extract complex features and have insufficient generalization performance for polyp samples with blurred edges or varied morphologies. At the same time, the upsampling strategies of lightweight models lack dynamic adjustment mechanisms, making it difficult to cope with artifacts caused by uneven lighting, reflections, and other interferences. Summary of the Invention

[0004] The purpose of this invention is to provide a lightweight polyp segmentation method based on multi-scale differential features and spatial attention. By combining a hierarchical processing mechanism with an encoder-decoder architecture, a multi-scale differential feature extraction module is constructed in the encoder section. A center-periphery feature difference mechanism is used to achieve directional enhancement of polyp edge information. Furthermore, multi-scale receptive field parallel processing channels are used to extract polyp features at different scales. Addressing artifact interference caused by optical noise and upsampling boundary diffusion in polyp segmentation results, as well as the limitation of computational resources in clinical applications, this algorithm integrates a joint spatial attention guidance strategy and the ultra-lightweight dynamic upsampling method DySample in the decoder section. The semantic features output by the encoder and the local detail information recovered step-by-step by the decoder jointly drive the upsampling process. Learnable encoder-decoder feature hybrid weights are used for automatic optimization of local detail representation, thus solving the problems mentioned in the background art.

[0005] This invention provides the following technical solution: A lightweight polyp segmentation method based on multi-scale difference features and spatial attention includes the following steps:

[0006] Step 1: Obtain the intestinal polyp segmentation dataset, preprocess the colonoscopy images contained in the dataset to generate input image data, and simultaneously segment the dataset.

[0007] Preferably, the intestinal polyp segmentation datasets CVC-ClinicDB, Kvasir-SEG, and CVC-ColonDB are acquired. The colonoscopy images contained in the datasets are preprocessed to generate image data that meets the algorithm's input requirements. Specifically, the resolution of the colonoscopy images is unified to 352×352. Random mirroring is then performed to improve data diversity and generalization ability. Mean normalization is used to improve the stability of numerical calculations. After the above processing, the dataset is divided into training, validation, and test sets in an 8:1:1 ratio.

[0008] Step 2: Construct a lightweight polyp segmentation model based on a hierarchical feature extraction mechanism and an encoder-decoder architecture.

[0009] Preferably, the encoder-decoder architecture includes: An encoder section consisting of a four-level multi-scale differential feature extraction module, using the multi-scale differential feature extraction module as the core feature extraction unit. It simulates the center-peripheral receptive field antagonism of retinal ganglion cells through a convolutional differential structure to enhance edge contrast. It also constructs multi-scale feature extraction channels by combining depthwise separable convolution and a gradual dilation strategy. The encoder integrates a convolutional block attention module (CBAM) to calibrate the spatial information of the multi-scale differential feature extraction module's output through spatial feature supplementation and attention enhancement. A three-level decoder unit is included, employing a joint spatial attention-guided DySample upsampling module responsible for the dynamic mixing of encoder and decoder features, rapidly generating upsampled results rich in spatial detail. Feature information is exchanged between the encoder and decoder through skip connections. The output section consists of a semantic-spatial information fusion module, which uses a dual-channel parallel processing architecture to achieve synergistic enhancement of spatial and semantic features, compensating for the loss of spatial detail in the output polyp segmentation results caused by multi-level feature extraction.

[0010] Step 3: Use the multi-scale differential feature extraction module of the encoder to extract multi-scale differential features, enhance the edge of the polyp, and construct a multi-scale receptive field parallel processing channel to extract multi-scale spatial features.

[0011] Preferably, the specific implementation process includes: the convolution-dilated convolution differential structure in the head of the multi-scale differential feature extraction module extracts central and peripheral features respectively through 3×3 convolution and 3×3 dilated convolution with a dilation rate of 2, and obtains positive and negative difference results for edge-oriented enhancement through element-wise difference operation, transforming the weak gradient difference of polyp edges in low-contrast scenes into high-contrast edge features and suppressing local abnormal responses caused by optical interference. Then, the above positive and negative difference results are concatenated with the original input and fed into four branches for multi-scale spatial feature extraction. The first three branches respectively use depthwise separable dilated convolutions with dilation rates of 1, 3, and 5 to replace traditional convolutions for extracting polyp spatial features at the corresponding scales, achieving a lightweight design of the module. Based on this, channel attention enhancement is applied to the processing results of each branch to obtain the intermediate output of the three branches, and secondary feature extraction is performed through 3×3 depthwise separable convolution. The fourth branch is responsible for preserving spatial features, outputting the channel concatenation result of the intermediate output of the first three branches and the edge enhancement features, achieving the fusion of cross-scale feature information. Finally, the output of the multi-scale differential feature extraction module is multiplied element-wise with the spatial detail features provided by the convolutional block attention module (CBAM) to complete the spatial information calibration.

[0012] Step 4: Each level of the decoder unit guides the DySample upsampling module through spatial attention to fuse and upsample the input features of the encoder and decoder.

[0013] Preferably, the joint spatial attention-guided DySample upsampling module includes a joint spatial attention guidance path and a content-aware resampling path, which are responsible for generating the attention guidance map and the upsampling result, respectively. The joint spatial attention guidance path receives the output features of the peer encoder and the previous stage decoder, processes them through the spatial attention mechanism SAM to obtain the spatial attention weight map of the two, and adaptively fuses the two through learnable weight parameters to generate the joint spatial attention guidance map, realizing spatial-level information mixing. The parallel DySample content-aware resampling path achieves channel-level information mixing by splicing the output features of the peer encoder and the output features of the previous stage decoder that have been upsampled by the DySample method, and uses the spatial attention guidance map generated by the joint spatial attention guidance path to further guide the fusion result, enhancing the discriminativeness of spatial details in the upsampling result.

[0014] Step 5: Utilize the semantic-spatial information fusion module in the output section to perform collaborative enhancement of spatial and semantic information.

[0015] Preferably, the specific design and processing of the semantic-spatial information fusion module is as follows: the spatial information path uses the spatial attention mechanism (SAM) to enhance the spatial details and key regions of low-level features, compensating for the loss of spatial details caused by multi-layer feature extraction; the semantic information path uses the channel attention mechanism (CAM) to suppress redundant channel noise, focus on key semantic features, and improve the accuracy of the model's semantic expression. After parallel processing, the output results of the two paths undergo bidirectional feature interaction and channel compression to achieve synergistic optimization and deep fusion of spatial information preservation and semantic representation, enhancing the model's ability to perceive polyp spatial detail features and polyp semantic features.

[0016] Step 6: Construct a combined loss function that integrates binary cross-entropy and Dice similarity coefficient, and optimize the model parameters.

[0017] Preferably, the binary cross-entropy loss is used to perform precise parameter optimization of the model at the pixel level, assisting the model in generating clear segmentation boundaries; the Dice similarity coefficient loss is used to evaluate based on global region similarity.

[0018] Step 7: Quantitatively evaluate the model's segmentation performance on the test set and generate a visual polyp segmentation result.

[0019] Preferably, Dice coefficient (Dice), intersection-over-union ratio (IoU), accuracy (Acc), recall (R), and precision (P) are used as evaluation metrics to quantitatively assess the polyp segmentation performance of the model, and the model size and computational load are quantitatively assessed by statistically analyzing the number of model parameters and the number of floating-point operations (FLOPs).

[0020] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0021] (1) In view of the problems of difficulty in segmenting polyp boundaries and the diversity of polyp morphology in existing intestinal polyp segmentation methods, this invention proposes a multi-scale differential feature extraction module for the model encoder. By combining depthwise separable convolution with a gradual design of convolutional void rate, it simulates the spatial scale diversity of receptive fields of retinal ganglion cells and the antagonistic characteristics of the center-periphery receptive fields, thereby realizing the extraction of multi-scale contextual information and the function of spatial information enhancement.

[0022] (2) In view of the artifact interference caused by optical noise and upsampling boundary diffusion in polyp segmentation results, as well as the problem of limited computing resources in clinical application scenarios, this invention proposes a novel joint spatial attention-guided DySample upsampling module for the decoder part of the model. This module introduces information from the encoder part of the model and uses the high-level semantic features output by the encoder and the local detail information recovered by the decoder step by step to drive the upsampling process, thereby achieving accurate upsampling in lightweight scenarios and improving the detail restoration capability of polyp segmentation results.

[0023] (3) To address the problem of spatial information loss caused by deep data processing in existing polyp segmentation methods, this invention proposes a semantic-spatial information fusion module for the model output stage. This module simulates the separation and integration strategy of spatial and semantic information in the visual cortex, and achieves a balance between high semantic accuracy and spatial detail in the output feature map by constructing a dual-channel parallel processing architecture to synergistically enhance spatial and semantic information. Attached Figure Description

[0024] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0025] Figure 1 This is a schematic diagram of the implementation process provided in the embodiments of the present invention;

[0026] Figure 2 This is a schematic diagram of the overall structure of the lightweight polyp segmentation method based on multi-scale differential feature extraction and spatial attention guidance provided in the embodiments of the present invention.

[0027] Figure 3 This is a schematic diagram of the multi-scale differential feature extraction module structure provided in an embodiment of the present invention;

[0028] Figure 4 This is a schematic diagram of the structure of the joint spatial attention-guided DySample upsampling module provided in an embodiment of the present invention;

[0029] Figure 5 This is a schematic diagram of the semantic-spatial information fusion module structure provided in an embodiment of the present invention;

[0030] Figure 6 This is a polyp segmentation result diagram on the VC-ClinicDB, Kvasir-SEG, and CVC-ColonDB datasets provided in the embodiments of the present invention. Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] Example 1

[0033] Combination Figures 1-5 The present invention provides the following technical solution: a lightweight polyp segmentation method based on multi-scale difference features and spatial attention, comprising the following steps:

[0034] Step 1: Obtain the intestinal polyp segmentation datasets CVC-ClinicDB, Kvasir-SEG, and CVC-ColonDB, and preprocess and segment the colonoscopy images contained in the datasets to generate image data that meets the algorithm input requirements.

[0035] In this embodiment, the CVC-ClinicDB dataset contains 612 polyp images with a resolution of 288×384 from 29 colonoscopy video sequences; Kvasir-SEG contains 1000 polyp images and their corresponding segmentation masks from Kvasir dataset v2, with image resolutions ranging from 332×487 to 1920×1072; and CVC-ColonDB contains 380 polyp images and their corresponding annotations from 13 colonoscopy video sequences, with a resolution of 574×500. The resolution of the colonoscopy images is uniformly set to 352×352, and random mirroring is performed to improve data diversity and generalization ability. Mean normalization is used to improve the stability of numerical computation.

[0036] For example, after completing the above processing, the three intestinal polyp segmentation datasets are divided into training set, validation set and test set. Specifically, the three datasets are randomly sorted and then divided into training set, validation set and test set in a ratio of 8:1:1. The image data in the training set is randomly mirrored to improve the diversity of data during model training.

[0037] Step 2: Construct a lightweight polyp segmentation model, LightRGC-SCNet, based on a hierarchical feature extraction mechanism and an encoder-decoder structure.

[0038] In this embodiment, the encoder uses a multi-scale differential feature extraction module and a convolutional block attention module (CBAM) to form a parallel pathway for polyp feature extraction and spatial detail calibration. Feature information is exchanged between each encoder and decoder via skip connections. The decoder uses a joint spatial attention-guided DySample upsampling module to dynamically blend encoder and decoder features, rapidly generating high-quality upsampling results rich in spatial detail. Finally, the model output utilizes a semantic-spatial information fusion module with a dual-channel parallel processing architecture to achieve synergistic enhancement of spatial and semantic features, outputting accurate polyp segmentation results.

[0039] For example, the specific structure of the lightweight polyp segmentation model LightRGC-SCNet proposed in this invention is as follows: Figure 2 As shown, it includes an encoder, a decoder, and an output section.

[0040] The encoder employs a four-level multi-scale differential feature extraction module as its core feature extraction unit. It simulates the center-periphery receptive field antagonism of retinal ganglion cells through a convolutional differential structure, enhancing edge contrast. Furthermore, it combines depthwise separable convolution with a gradual dilatation strategy to construct multi-scale feature extraction channels, improving the model's ability to extract polyp morphological features at different spatial scales. To further optimize spatial details, the encoder integrates a convolutional block attention module (CBAM), which calibrates the spatial information of the multi-scale differential feature extraction module's output through spatial feature supplementation and attention enhancement.

[0041] The decoder section includes a three-level joint spatial attention-guided DySample upsampling module. It integrates the encoder-decoder dual-path attention guidance mechanism with the lightweight DySample upsampling method, and uses the outputs of the encoder and decoder to guide the detail recovery in the upsampling process. This effectively suppresses the boundary diffusion phenomenon in the traditional upsampling process and improves the ability to restore the details of polyp morphology.

[0042] The output section integrates deep semantic and shallow spatial information through a semantic-spatial information fusion module. Then, it adjusts the channels and restores the size of the output segmentation result through 1×1 convolution and upsampling operations, outputting a segmentation result with the same resolution as the polyp image. This module adopts a parallel dual-channel structure to collaboratively enhance spatial and semantic information, ultimately achieving a balance between semantic accuracy and spatial detail. Through the collaborative work of the above modules, the segmentation accuracy and detail restoration capability of complex polyps are significantly improved while maintaining lightweight design.

[0043] Step 3: Utilize the multi-scale differential feature extraction module of the encoder part to enhance the polyp edge contrast of the input image to be processed through the convolution-dilated convolution differential structure, construct a multi-scale receptive field parallel processing channel that adapts to the diversity of polyp morphology, extract polyp features at different spatial scales, output the feature extraction results, and calibrate the spatial information of the results through the spatial detail features provided by the convolutional block attention module CBAM path.

[0044] In this embodiment, the specific implementation process includes:

[0045] Step 31: The convolution-dilated convolution differential structure in the head of the multi-scale differential feature extraction module extracts the central and peripheral features through 3×3 convolution and 3×3 dilated convolution with a dilation rate of 2, respectively. The positive and negative differential results for edge orientation enhancement are obtained through element-wise differential operation, which transforms the weak gradient difference of polyp edges in low-contrast scenes into high-contrast edge features and suppresses local abnormal responses caused by optical interference.

[0046] Step 32: The positive and negative difference results are concatenated with the original input and then fed into four branches for multi-scale spatial feature extraction. The first three branches use depthwise separable dilated convolutions with dilation rates of 1, 3, and 5 to replace traditional convolutions for extracting polyp spatial features at the corresponding scales, achieving a lightweight design for the module.

[0047] Step 33: Simultaneously, channel attention enhancement is applied to the processing results of each branch to obtain the intermediate outputs of the three branches, and secondary feature extraction is performed through 3×3 depthwise separable convolution. The fourth branch is responsible for preserving spatial features and outputs the channel concatenation result of the intermediate outputs of the first three branches and the edge enhancement features, realizing the fusion of cross-scale feature information.

[0048] Step 34: Finally, the output of the multi-scale differential feature extraction module is multiplied element-wise with the spatial detail features provided by the convolutional block attention module CBAM to complete the spatial information calibration.

[0049] For example, inspired by the receptive field mechanism of retinal ganglion cells, the multi-scale differential feature extraction module simulates the response characteristics of ON-central (central excitation, peripheral inhibition) and OFF-central (central inhibition, peripheral excitation) receptive fields. A selective feature response mechanism is constructed through convolutional differential operations, transforming the weak gradient differences at polyp edges in low-contrast scenes into high-contrast edge features, while suppressing local anomalous responses caused by optical noise interference, thus enhancing the model's stability in noisy environments. Furthermore, inspired by the multi-scale feature perception mechanism of retinal ganglion cells, the multi-scale differential feature extraction module combines depthwise separable convolution with a gradual dilation design to construct a lightweight multi-scale parallel processing channel. This module replaces standard convolution with depthwise separable convolution, maintaining both lightweight design and computational efficiency despite employing a multi-path parallel structure. The gradual dilation strategy configures different dilation rates for each channel branch, enabling each channel to independently capture features at a specific scale, while also achieving cross-scale information fusion through channel splicing and compression to adapt to the diversity of polyp morphologies.

[0050] For example, the specific structure of the multi-scale differential feature extraction module included in this invention is as follows: Figure 3 As shown, for the input feature tensor The module's convolution-dilated convolution difference structure simulates the excitability difference between the center and periphery of the receptive field by separating the central path and the surrounding path. The central path is constructed using a 3×3 convolution. The simulation center receptive field extracts small-scale features, while the surrounding paths are processed by 3×3 dilated convolutions with a dilation rate of 2. Large-scale features are extracted by simulating the peripheral receptive field. Subsequently, the antagonistic mechanism between the ON and OFF central receptive fields is simulated, and element-wise differencing is performed on the results obtained from the central path and the peripheral path to obtain the outputs of the ON and OFF central receptive fields. and Then through splicing , and This provides edge-enhanced input features for subsequent multi-scale feature extraction. The specific calculation method is as follows:

[0051]

[0052]

[0053]

[0054] For input features that have undergone edge enhancement The multi-scale differential feature extraction module further extracts features from polyps through multi-scale parallel processing channels. First, it uses 1×1 convolutions... The process involves generating feature maps with fewer channels to reduce the computational and parameter requirements of subsequent operations. Then, polyp features at different spatial scales are extracted through four different information processing branches. The first three branches utilize the concept of depthwise separable convolution, employing single-channel dilated convolutions with dilation rates of 1, 3, and 5, respectively. , , Spatial morphological features of polyps at different scales in the image are extracted. Based on this, a 1×1 convolution kernel is used... Channel compression is performed, and channel attention enhancement is applied to the processing results of each branch. This allows the model to more efficiently concentrate limited computational resources on the most important channel features, resulting in the intermediate outputs of the three branches. , and The specific calculation method is as follows:

[0055]

[0056]

[0057]

[0058] in This represents the channel attention mechanism.

[0059] Furthermore, the multi-scale differential feature extraction module preserves spatial feature information through a fourth branch, which will... , , and The result after channel splicing and compression is used as the output. After the above processing , and Separable convolution with a depth of 3×3 Perform secondary feature extraction. Then extract the results. and After concatenation, channel attention enhancement and channel adjustment are performed to obtain the final output of the multi-scale differential feature extraction module. The specific calculation method is as follows:

[0060]

[0061] Final output results Spatial information calibration is completed by element-wise multiplying the spatial detail features provided by the Convolutional Block Attention (CBAM) module.

[0062] Step 4: The spatial attention-guided DySample upsampling module at each decoder unit fuses and upsamples the encoder and decoder input features. It leverages the high-level semantic features output by the encoder and the progressively recovered local detail information from the decoder to drive the upsampling process, improving the detail restoration capability of the polyp segmentation results. This module uses learnable encoder-decoder feature hybrid weights for automatic optimization of local detail representation, suppressing artifact interference caused by optical noise and upsampling boundary diffusion in the image.

[0063] In this embodiment, inspired by the Shangqiu spatial attention guidance mechanism, this invention proposes a joint spatial attention-guided DySample upsampling module, comprising a joint spatial attention guidance path and a content-aware resampling path, responsible for generating the attention guidance map and the upsampling result, respectively. The joint spatial attention guidance path receives output features from the same-level encoder and the previous-level decoder, processes them through the spatial attention mechanism SAM to obtain their spatial attention weight maps, and adaptively fuses them using learnable weight parameters to generate a joint spatial attention guidance map, achieving spatial-level information mixing. The parallel DySample content-aware resampling path achieves channel-level information mixing by splicing the output features of the same-level encoder and the output features of the previous-level decoder upsampled using the DySample method, and further guides the fusion result using the spatial attention guidance map generated by the joint spatial attention guidance path, enhancing the discriminativeness of spatial details in the upsampling result.

[0064] For example, the joint spatial attention-guided DySample upsampling module constructs an encoder-decoder feature enhancement system by fusing the lightweight upsampling method DySample with a joint spatial attention guidance strategy that simulates the attention guidance mechanism of the Upper Hill. Specifically, the joint spatial attention-guided DySample upsampling module utilizes the high-level semantic features output by the encoder and the local detail information recovered stepwise by the decoder to jointly drive the upsampling process. Encoder features provide global contextual information to enhance target boundary perception, while decoder features are dynamically upsampled using the DySample method and automatically optimized for local detail representation by combining learnable encoder-decoder feature hybrid weights. This process can quickly generate detailed feature maps without relying on complex computations, retaining the lightweight advantage of the DySample upsampling method while simulating the fast spatial attention guidance and parallel information transmission characteristics of the Upper Hill through the joint spatial attention guidance strategy. This allows the model to accurately reproduce the spatial morphological details of the polyp while maintaining computational efficiency.

[0065] For example, the specific structure of the multi-scale difference feature extraction module is as follows: Figure 4As shown, its design consists of a parallel joint spatial attention guidance path and a content-aware resampling path to extract encoder output features. With decoder output features The spatial attention weight map is generated by adaptively adding the two maps pixel-by-pixel using learnable weight parameters, thus producing a joint spatial attention guidance map. The specific calculation method is shown below, where This represents an upsampling operation, using bilinear interpolation to avoid introducing the checkerboard effect:

[0066]

[0067] in, For learnable weight parameters, This is a spatial attention mechanism.

[0068] Furthermore, the content-aware resampling path utilizes the DySample method to modify the decoder output features. Initial upsampling is performed, followed by comparing the upsampled decoder output features with the encoder output features. Information is fused by splicing and guided by a joint spatial attention map. Spatial attention enhancement is applied to the fusion results to enable the model to generate higher-precision upsampling results. This approach achieves encoder-decoder collaborative spatial attention guidance, optimizing the boundary spread problem of traditional decoders while maintaining the model's lightweight advantage through a dual-path attention mechanism. The specific calculation method is shown below:

[0069]

[0070] in Represents element-wise multiplication. Represents the DySample upsampling method. This represents a 1×1 convolution used for channel number adjustment.

[0071] Step 5: To address the issue of lost spatial details in high-level features in deep networks, this invention designs a semantic-spatial information fusion module at the model output end. This module simulates the synergistic effect of the "what-where" pathway in the visual cortex and achieves synergistic enhancement of spatial and semantic features through a dual-channel parallel processing architecture, effectively solving the problem of spatial information loss during feature fusion.

[0072] In this embodiment, the semantic-spatial information fusion module supplements the spatial information of the segmentation results after multi-level processing by introducing features from the shallow encoder. Inspired by the separation and integration strategies of spatial and semantic information in the visual cortex, the semantic-spatial information fusion module adopts a parallel processing architecture of spatial and semantic information pathways. This dual-channel parallel processing architecture achieves synergistic enhancement of spatial and semantic features. The spatial and semantic information pathways are responsible for the targeted processing of low-level spatial features and high-level semantic features, respectively. The spatial information pathway enhances the spatial details and key regions of shallow features through the spatial attention mechanism (SAM), compensating for the loss of spatial details caused by multi-level feature extraction. The semantic information pathway suppresses redundant channel noise contained in deep features through the channel attention mechanism, focusing on key semantic features and improving the accuracy of the model's semantic expression. After parallel processing, the outputs of the two pathways undergo bidirectional feature interaction and channel compression, achieving synergistic optimization and deep fusion of spatial information preservation and semantic representation. This enhances the model's ability to perceive polyp spatial details and semantic features, outputting polyp segmentation results containing accurate spatial details and semantic information, thus enhancing the model's ability to capture spatial details while maintaining semantic recognition accuracy.

[0073] For example, the specific structure of the semantic-spatial information fusion module is as follows: Figure 5 As shown, the spatial information pathway employs a 3×3 depthwise separable convolution. Extracting shallow spatial features from the input Local spatial details are captured, and spatial detail information is enhanced through a spatial attention mechanism to generate spatially enhanced features. Semantic information pathways utilize depthwise separable convolutions. Extracting deep semantic features The high-level semantic representation is obtained and processed using a channel attention mechanism to generate semantically enhanced features. The specific calculation methods for spatial information pathways and semantic information pathways are as follows:

[0074]

[0075]

[0076] Furthermore, the semantic-spatial information fusion module will and respectively with input features and The images are stitched together, and channel compression and feature integration are achieved through 1×1 convolution. Then, the results are added element-wise to achieve synergistic complementarity between spatial detail features and global semantic features, alleviating the boundary blurring problem caused by unilateral feature dominance. After feature fusion, the size and number of channels of the output features are adjusted through upsampling and 3×3 convolution to obtain a polyp segmentation result of the same size as the input image. The specific feature fusion process is as follows:

[0077]

[0078] in, It is a 3×3 convolution.

[0079] Step 6: Construct a hybrid loss function that combines binary cross-entropy (BCE) and Dice similarity coefficient (Dice) to optimize model parameters.

[0080] In this embodiment, the BCE loss performs precise parameter optimization of the model at the pixel level, assisting the model in generating clear segmentation boundaries, while the Dice loss evaluates based on global region similarity, which can alleviate the class imbalance problem between polyps and the background, prompting the model to pay more attention to the overall integrity of the polyps. The hybrid loss function used in this invention is expressed as follows:

[0081]

[0082] in For predicted values, For real labels, An index representing the location of a sample or pixel. The weights of Bce and Dice loss functions are the total sample size. and Both are 0.5.

[0083] Step 7: Train the LightRGC-SCNet model on the training set obtained in Step 1. Evaluate the model training effect in each round on the validation set, optimize the model parameters using the loss function in Step 6, evaluate the final training result of the model using the test set, and output the visualized polyp segmentation results of the model on the test set of the CVC-ClinicDB, Kvasir-SEG, and CVC-ColonDB datasets.

[0084] In this embodiment, the methods proposed in this invention are all processed on an NVIDIA GeForce GTX 4060 Ti graphics processor, using Python 3.9.13 and PyTorch 2.5.0. A poly strategy is employed for learning rate decay, with an initial learning rate set to 0.001, a power of 0.9, a maximum training epoch of 100, and a batch size of 4. The Dice coefficient, Intersection over Union (IoU), Accuracy (Acc), Recall (R), and Precision (P) are used as evaluation metrics to quantitatively assess the polyp segmentation performance of the model. The model size and computational complexity are quantitatively assessed by statistically analyzing the number of model parameters and the number of floating-point operations (FLOPs).

[0085] Example 2

[0086] Combination Figure 6 As shown, to verify the effectiveness of the lightweight polyp segmentation method based on multi-scale differential feature extraction and spatial attention guidance proposed in this invention, this invention uses seven indicators to quantitatively evaluate the polyp segmentation performance and scale of existing algorithms, and performs quantitative analysis by comparing the visualized polyp segmentation results. Tables 1, 2, and 3 show the quantitative evaluation results of each model on the CVC-ClinicDB, Kvasir-SEG, and CVC-ColonDB datasets. The method proposed in this invention performs excellently on all three datasets, achieving higher Dice coefficient, intersection-over-union ratio, and accuracy than other lightweight polyp segmentation methods, while having the smallest number of model parameters and computational cost.

[0087] Table 1: Performance Comparison of Existing Lightweight Polyp Segmentation Algorithms on the CVC-ClinicDB Dataset

[0088]

[0089] Table 2: Performance comparison of existing lightweight polyp segmentation algorithms on the Kvasir-SEG dataset

[0090]

[0091] Table 3: Performance comparison of existing lightweight polyp segmentation algorithms on the CVC-ColonDB dataset:

[0092]

[0093] To further qualitatively analyze the polyp segmentation effect of the algorithm proposed in this invention, combined with Figure 6The visualized segmentation results show that the proposed method can accurately extract spatial details from small-scale regions with low contrast in complex polyp morphology. Even in images with optical noise interference, it can accurately distinguish polyp tissue from interfering information and exhibits stronger robustness against artifact interference and boundary blurring. These results demonstrate the superiority of the proposed algorithm and the efficiency of its lightweight design.

[0094] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0095] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A lightweight polyp segmentation method based on multi-scale difference features and spatial attention, characterized in that: The following steps are included: Step 1: Obtain the intestinal polyp segmentation dataset, preprocess the colonoscopy images contained in the dataset to generate input image data, and simultaneously segment the dataset; Step 2: Construct a lightweight polyp segmentation model based on a hierarchical feature extraction mechanism and an encoder-decoder architecture; The encoder-decoder architecture includes: The encoder section consists of a four-level multi-scale differential feature extraction module. The multi-scale differential feature extraction module is used as the core feature extraction unit. It simulates the center-peripheral receptive field antagonism of retinal ganglion cells through convolutional differential structure to enhance edge contrast. It also constructs multi-scale feature extraction channels by combining depth-separable convolution and a gradual dilatation strategy. The encoder integrates a convolutional block attention module (CBAM) to perform spatial information calibration on the output of the multi-scale differential feature extraction module through spatial feature supplementation and attention enhancement. The decoder section consists of three levels of decoder units, employing a joint spatial attention-guided DySample upsampling module. These three levels of decoder units are used for progressive fusion and upsampling of input features to restore local detail information in the segmentation results. Each level of the joint spatial attention-guided DySample upsampling module uses learnable encoder-decoder feature hybrid weights to automatically optimize the representation of local details, suppressing artifact interference caused by optical noise and upsampling boundary diffusion in the image. The output section consists of a semantic-spatial information fusion module, which uses a dual-channel parallel processing architecture to achieve synergistic enhancement of spatial and semantic features, compensating for the loss of spatial details in the output polyp segmentation results caused by multi-layer feature extraction. The encoder and decoder exchange feature information through skip connections; Step 3: Use the multi-scale differential feature extraction module of the encoder to extract multi-scale differential features, enhance the edge of the polyp, and construct a multi-scale receptive field parallel processing channel to extract multi-scale spatial features; Step 4: Each decoder unit guides the DySample upsampling module through spatial attention to fuse and upsample the input features of the encoder and decoder; Step 5: Utilize the semantic-spatial information fusion module in the output section to collaboratively enhance spatial and semantic information; Step 6: Construct a combined loss function that integrates binary cross-entropy and Dice similarity coefficient, and optimize the model parameters; Step 7: Quantitatively evaluate the model's segmentation performance on the test set and generate a visual polyp segmentation result.

2. The lightweight polyp segmentation method based on multi-scale difference features and spatial attention as described in claim 1, characterized in that: Step 1 involves simultaneously dividing the dataset into training, validation, and test sets.

3. The lightweight polyp segmentation method based on multi-scale difference features and spatial attention as described in claim 2, characterized in that: The lightweight polyp segmentation model in step 2 includes: The encoder section uses a multi-scale differential feature extraction module and a convolutional block attention module (CBAM) to form a parallel path for polyp feature extraction and spatial detail calibration. The decoder section guides the DySample upsampling module through joint spatial attention to dynamically blend encoder and decoder features, generating upsampling results that include spatial details; The output section utilizes a dual-channel parallel processing architecture through a semantic-spatial information fusion module to collaboratively enhance spatial and semantic features, resulting in accurate polyp segmentation.

4. The lightweight polyp segmentation method based on multi-scale difference features and spatial attention as described in claim 3, characterized in that: The specific implementation of step 3 includes: Step 31: Use the multi-scale differential feature extraction module of the encoder to extract the center features and peripheral features respectively, and obtain the positive and negative difference results of edge orientation enhancement through element-wise difference operation; Step 32: After concatenating the positive and negative difference results with the original input, the results are fed into the four branches for multi-scale spatial feature extraction; Step 33: Perform channel attention enhancement on the processing results of each branch, and use the multi-scale differential feature extraction module for further feature extraction. Then, concatenate the extraction results of each channel with the edge enhancement features to obtain the final output result of the multi-scale differential feature extraction module. Step 34: Multiply the output of the multi-scale differential feature extraction module with the spatial detail features provided by the convolutional block attention module CBAM element-wise to complete the spatial information calibration.

5. The lightweight polyp segmentation method based on multi-scale difference features and spatial attention as described in claim 4, characterized in that: Step 33 also includes: Four information processing branches with different functions are used to extract polyp features at different spatial scales. The first three branches use channel dilated convolution to extract the spatial morphological features of polyps at different scales in the image, followed by channel compression. Channel attention enhancement is then applied to the processing results of each branch to obtain the intermediate output of the three branches. Secondary feature extraction is then performed on the intermediate output of the three branches. At the same time, the fourth branch is used to preserve the spatial feature information. Finally, the intermediate output of the first three branches is concatenated with the edge enhancement feature channel, followed by channel attention enhancement and channel adjustment to obtain the final output of the multi-scale differential feature extraction module.

6. The lightweight polyp segmentation method based on multi-scale difference features and spatial attention as described in claim 5, characterized in that: Step 4, the spatial attention-guided DySample upsampling module, includes: Joint Spatial Attention Guidance Path: By accepting the output features of the peer encoder and the previous stage decoder, the spatial attention weight map of the two is obtained through the spatial attention mechanism SAM. The two are then adaptively fused through learnable weight parameters to generate a joint spatial attention guidance map. The output features of the previous stage decoder are upsampled by the DySample method to align with the output features of the peer encoder. Content-aware resampling path: Upsampling results are generated using the DySample method. Channel-level information mixing is achieved by splicing the output features of the same level encoder and the output features of the previous level decoder that have been upsampled by DySample. The spatial attention guidance map generated by the joint spatial attention guidance path is used to further guide the fusion results and enhance the discriminativeness of spatial details in the upsampling results.

7. The lightweight polyp segmentation method based on multi-scale difference features and spatial attention as described in claim 6, characterized in that: Step 5, the semantic-spatial information fusion module, includes: A processing architecture that combines spatial information pathways and semantic information pathways is adopted. The spatial information pathway employs the spatial attention mechanism (SAM) to enhance the spatial details and key regions of low-level features, compensating for the loss of spatial details caused by multi-level feature extraction. The semantic information pathway uses the channel attention mechanism (CAM) to suppress redundant channel noise and focus on key semantic features. After parallel processing, the output results of the two paths are subjected to bidirectional feature interaction and channel compression, and the spatial information preservation and semantic representation are synergistically optimized and deeply fused.

8. The lightweight polyp segmentation method based on multi-scale difference features and spatial attention as described in claim 7, characterized in that: In step 6: Binary cross-entropy loss is used to perform precise parameter optimization of the model at the pixel level, helping the model generate clear segmentation boundaries. Dice similarity coefficient loss is used to evaluate based on global region similarity.

9. The lightweight polyp segmentation method based on multi-scale difference features and spatial attention as described in claim 8, characterized in that: Step 7 uses Dice coefficient, intersection-union ratio, accuracy, recall, and precision as evaluation metrics to quantitatively assess the polyp segmentation performance of the model, and quantitatively assesses the model size and computational load by statistically analyzing the number of model parameters and the number of floating-point operations.

Images