Polyp segmentation method based on local perception cue generation and morphological boundary refinement

Abstract

The application provides a polyp segmentation method based on local perception prompt generation and morphological boundary refinement, which comprises the following steps: pre-processing a polyp image and extracting global semantic features and local texture features; then, based on the local texture features, a local perception guide image is focused on a significant area, and multi-scale features are adaptively fused to generate task-perception sparse prompts and dense prompts; then, the generated features are input into a mask decoder to generate preliminary segmentation mask features; boundary details and image segmentation operations are performed on the preliminary segmentation mask features to obtain a prediction probability map; the probability distribution of the prediction probability map is dynamically adjusted, a loss function is constructed based on the adjustment result for training, and finally, the segmentation result and the reliability score are jointly output. The application significantly improves the boundary accuracy of segmentation, the model robustness and the cross-domain generalization ability, and provides reliable technical support for early screening of colon polyps.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and artificial intelligence, and in particular to a polyp segmentation method based on local perception cue generation and morphological boundary refinement. Background Technology

[0002] Colorectal cancer, ranking among the top three most common malignant tumors worldwide, presents a serious challenge in its prevention and treatment. Clinical medical research confirms that over 90% of colorectal cancers originate from adenomatous polyps; therefore, early detection and accurate removal of polyps are crucial for halting cancer progression and reducing mortality. Colonoscopy is currently the gold standard for diagnosing colorectal polyps, providing information on their location and appearance, enabling doctors to remove them before they become malignant. With the rapid development of artificial intelligence and computer vision technologies, utilizing deep learning algorithms to assist doctors in automated polyp segmentation has become an important direction in the digital transformation of healthcare.

[0003] However, despite the advancements in automation for polyp detection, existing technologies still face numerous unresolved technical bottlenecks in practical clinical applications. First, human factors play a significant role; endoscopists are prone to visual fatigue under prolonged, high-intensity work, and differences in clinical experience lead to a high rate of missed diagnoses of small and flat polyps, easily resulting in missed opportunities for optimal treatment. Second, the intestinal imaging environment is extremely complex, with polyps exhibiting highly heterogeneous and isomorphic characteristics. Polyps of the same type can vary greatly in size, shape, and texture, and the lesion edges often lack clear boundaries with the normal intestinal wall. Combined with noise interference such as specular highlights, intestinal residue obscuring the image, low contrast, and motion blur, conventional algorithms are highly susceptible to oversegmentation or false detection. More critically, existing deep learning models generally suffer from generalization performance bottlenecks. When faced with cross-center data collected from different hospitals and devices, performance degrades significantly due to distribution shifts. Summary of the Invention

[0004] In view of the above situation, the main objective of this invention is to propose a polyp segmentation method based on local perception cue generation and morphological boundary refinement, so as to solve the above-mentioned technical problems.

[0005] This invention proposes a polyp segmentation method based on local perceptual cue generation and morphological boundary refinement, the method comprising the following steps: Step 1: Obtain a dataset of colon polyp images, perform standardization preprocessing on the polyp images, and extract global semantic features and local texture features from the preprocessed images; Step 2: Input global semantic features and local texture features into the local perception cue generation module. Based on the local texture features, the local perception guides the image to focus on the salient region. At the same time, based on the collaborative interaction of global semantic features and local texture features, multi-scale features are adaptively fused to generate sparse and dense cuees for task perception, eliminating the segmentation model's dependence on human interaction cuees. Step 3: Input sparse and dense cues, global semantic features and local texture features into the mask decoder to perform bidirectional interactive mapping between global context and local details, generate preliminary segmentation mask features, and output the corresponding IoU score at the same time. Step 4: Introduce the preliminary segmentation mask features into the learnable morphological boundary refinement module, and optimize the blurred and irregular boundary details end-to-end through differentiable dilation and erosion operations to obtain the refined feature map; Perform a prediction operation on the refined feature map to obtain a prediction probability map; Step 5: Based on the predicted probability map, construct a confidence-aware prediction correction mechanism. Dynamically adjust the probability distribution of the predicted probability map through pixel-level and image-level uncertainty estimation to obtain the corrected predicted probability map. Based on the corrected prediction probability map, a confidence-aware regularized loss function is constructed, and end-to-end training is performed in conjunction with multi-task loss. After training, the final prediction probability map is obtained, and the final IoU score is output synchronously. A pixel-level high-precision polyp segmentation mask is generated based on the final predicted probability map, and the final IoU score is used as a quantitative evaluation index of segmentation quality to achieve the joint output of segmentation results and credibility score.

[0006] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention designs a local perception prompt generation module. By fusing the local spatial prior of the convolutional neural network with the global semantic features of the large visual model, it can achieve automated prompt generation without human interaction, while effectively solving the blind spot problem of traditional large models in the localization of small and low-contrast polyps. 2. This invention proposes a complementary feature enhancement mechanism. By establishing a bidirectional interactive mapping between the global context of ViT and the local details of CNN, it can achieve deep complementarity of multi-scale semantic information while effectively overcoming the limitation of a single architecture in taking into account both the overall morphology and subtle texture features of polyps. 3. This invention constructs a learnable morphological refinement module, which transforms traditional morphological erosion and dilation operations into differentiable neural network layers. Through a dual-path parallel architecture and spatial adaptive gating mechanism, it significantly improves the model's pixel-level characterization of blurred lesion edges and suppresses oversegmentation.

[0007] 4. This invention introduces a confidence-aware regularization loss function. By combining image-level intersection-over-union confidence and pixel-level uncertainty estimation to dynamically control the model guidance, the model actively corrects low-confidence prediction areas, effectively alleviating the problems of model overconfidence and decreased generalization performance caused by cross-center data distribution shift.

[0008] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by means of embodiments of the invention. Attached Figure Description

[0009] Figure 1 This is a flowchart of the polyp segmentation method based on local perception cue generation and morphological boundary refinement proposed in this invention. Figure 2 This is a diagram illustrating the overall framework of the polyp segmentation method based on local perception cue generation and morphological boundary refinement proposed in this invention. Figure 3 This is a structural diagram of the local perception prompt generation module proposed in this invention; Figure 4 This is a structural diagram of the complementary feature enhancement mechanism proposed in this invention; Figure 5 This is a structural diagram of the learnable morphological refinement module proposed in this invention. Detailed Implementation

[0010] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0011] These and other aspects of the embodiments of the present invention will become clear from the following description and accompanying drawings. In these descriptions and drawings, some specific embodiments of the present invention are specifically disclosed to illustrate some ways of implementing the principles of the embodiments of the present invention; however, it should be understood that the scope of the embodiments of the present invention is not limited thereto.

[0012] Please see Figure 1 and Figure 2 This embodiment provides a polyp segmentation method based on local perception cue generation and morphological boundary refinement, the method comprising the following steps: Step 1: Obtain a dataset of colon polyp images, perform standardization preprocessing on the polyp images, and extract global semantic features and local texture features from the preprocessed images; In a preferred embodiment of the present invention, a dataset of colon polyp images is obtained, and the polyp images are subjected to standardization preprocessing; and global semantic features and local texture features of the preprocessed images are extracted, specifically including the following steps: Obtain a dataset of colon polyp images, preprocess the polyp images in the dataset, and obtain preprocessed polyp images; The preprocessed polyp image is input into the ViT encoder for feature extraction to obtain the intermediate features of the third, sixth and ninth stages and the global semantic features of the twelfth stage, respectively. The preprocessed polyp image is input into a CNN encoder for feature extraction to obtain local texture features.

[0013] Step 2: Input global semantic features and local texture features into the local perception cue generation module. Based on the local texture features, the local perception guides the image to focus on the salient region. At the same time, based on the collaborative interaction of global semantic features and local texture features, multi-scale features are adaptively fused to generate sparse and dense cuees for task perception, eliminating the segmentation model's dependence on human interaction cuees. As a preferred embodiment of the present invention, please refer to Figure 3 The method guides the image to focus on salient regions based on local texture features, and adaptively fuses multi-scale features to generate task-aware sparse and dense cues based on the synergistic interaction of global semantic features and local texture features. The specific steps include the following: Extracting the frozen parameters from the mask decoder and learnable parameters ; Freeze parameters and learnable parameters A token-level cross-attention mechanism is used for information fusion, followed by linear projection and residual connection operations. The frozen parameters are updated after the residual connection is performed. and updated learnable parameters The corresponding process has the following relationship: ; ; in, and This represents the updated frozen parameters and learnable parameters. and This represents the initial frozen parameters and the learnable parameters; The formula for the cross-attention function is shown below. Indicates linear projection. Represents the formula for the cross-attention function; The calculation process is defined as follows: ; in, The learnable weight matrix representing cross-attention has the following dimensions: ; This represents a two-layer fully connected network. express Embedding dimension This represents the matrix transpose operation; Local texture features Inputting into a lightweight attention generator, dimensionality reduction, upsampling, and sigmoid activation operations are performed on local features to generate a single-channel spatial attention map, highlighting the regions with the strongest polyp response from the perspective of the convolutional neural network. The following relationship exists in the corresponding process: ; in, This represents the Sigmoid activation function. This represents the GELU activation function. and These represent the first and second convolutional layers, respectively. Indicates an upsampling operation. Represents local texture features. Single-channel spatial attention map; Spatial attention maps of single channels and global semantic features The tokens are expanded and then weighted and aggregated to obtain local perception tokens. The following relationship exists in the correspondence process: ; in, Represents a local perception token. Represents global semantic features; Local perception token Updated frozen parameters and updated learnable parameters By concatenating the sequences, a preliminary set of composite hints is obtained. ; The initial set of compound prompts Using global semantic features as query tokens and key-value tokens as key-value tokens, attention queries are performed, causing the global semantic features to recalibrate their focus based on the prompts. This results in enhanced global semantic features, enhanced frozen parameters, sparse prompts, and dense prompts. The following relationship exists in this process: ; ; ; ; in, This indicates the enhanced freeze parameters. This represents the initial set of compound prompts. Indicates sparsity. This indicates dense prompts. This represents the enhanced global semantic features. This represents the enhanced composite cue set. Indicate extraction The feature of the first position in the second dimension, Indicate extraction The second dimension indexes all features starting from 6. Indicate extraction The second dimension indexes all features from 1 to 5.

[0014] Step 3: Input sparse and dense cues, global semantic features and local texture features into the mask decoder to perform bidirectional interactive mapping between global context and local details, generate preliminary segmentation mask features, and output the corresponding IoU score at the same time. As a preferred embodiment of the present invention, please refer to Figure 4 The mask decoder includes a bidirectional Transformer module, a token-to-image attention module, and a feature fusion module. The bidirectional Transformer module includes five sub-modules: the ViT-CNN cross-attention layer, the image-to-token attention mechanism, the first multilayer perceptron, the token-to-image attention mechanism, and the self-attention mechanism. The ViT-CNN cross-attention layer, the image-to-token attention mechanism, and the token-to-image attention mechanism all adopt the cross-attention mechanism. The sparse and dense cues, global semantic features, and local texture features are input into the mask decoder to perform bidirectional interactive mapping between global context and local details, generating preliminary segmentation mask features and simultaneously outputting the corresponding IoU score. The specific steps include the following: The mask decoder includes a bidirectional Transformer module, a token-to-image attention module, and a feature fusion module. The bidirectional Transformer module includes five sub-modules: a ViT-CNN cross-attention layer, an image-to-token attention mechanism, a first multilayer perceptron, a token-to-image attention mechanism, and a self-attention mechanism. The ViT-CNN cross-attention layer, the image-to-token attention mechanism, and the token-to-image attention mechanism all adopt the cross-attention mechanism. Phase 1: Step 3.1: The combined features of the enhanced global semantic features and the dense cueing are used as the first combined feature, and the local texture features are used as the second combined feature. Both are input into the ViT-CNN cross-attention layer for cross-modal feature interaction. In the ViT-CNN cross-attention layer, the local texture features are used as the query and the first combined feature is used as the key to calculate cross-modal attention and output the combined representation of the enhanced global semantic features and the dense cueing in the first stage. Step 3.2: Input the enhanced freezing parameters and the combined features of the sparse cue into the self-attention mechanism to perform feature self-enhancement and obtain self-enhanced features; Step 3.3: The self-enhancing features are processed sequentially through the token-to-image attention mechanism and the first multilayer perceptron to obtain the combined features of the enhanced frozen parameters and sparse cueing updated in the first stage. Step 3.4: Input the combined features of the enhanced frozen parameters updated in the first stage and sparse cue, the local texture features, and the combined representation of the enhanced global semantic features and dense cue from the first stage into the token attention mechanism. In the image-to-token attention mechanism, local texture features are used as queries, the combination of enhanced global semantic features and dense cueing from the first stage is used as keys, and the combination of enhanced frozen parameters and sparse cueing updated in the first stage is used as values ​​for cross-attention calculation, outputting the enhanced global semantic features and local texture features updated in the first stage. Phase Two: Step 3.5: Take the enhanced global semantic features updated in the first stage as the first combined feature and the local texture features updated in the first stage as the second combined feature. Repeat the operation of step 3.1 again to obtain the combined representation of the enhanced global semantic features and dense cue in the second stage. Step 3.6: Repeat steps 3.2 and 3.3 with the combined features of the enhanced frozen parameters and sparse cueing updated in the first stage as input to obtain the combined features of the enhanced frozen parameters and sparse cueing updated in the second stage. Step 3.7: Input the combined features of the enhanced frozen parameters updated in the second stage and sparse cue, the local texture features, and the combined representation of the enhanced global semantic features and dense cue from the second stage into the token attention mechanism. In the image-to-token attention mechanism, local texture features are used as queries, the combined representation of the enhanced global semantic features and dense cue in the second stage is used as keys, and the combined features of the mask token and sparse cue are used as values ​​for cross-attention calculation, outputting the enhanced global semantic features and the local texture features updated in the second stage. Phase Three: Step 3.8: The enhanced global semantic features updated in the second stage, the local texture features updated in the second stage, and the combined features of the enhanced frozen parameters and sparse cue updated in the second stage are fed into the image attention module for cross-attention calculation, and then input into the second multilayer perceptron to obtain the IoU score. Step 3.9: Input the intermediate features and local texture features of the third, sixth and ninth stages of the ViT encoder into the feature fusion module; through the feature fusion module, the intermediate features of the third, sixth and ninth stages of the ViT encoder are progressively upsampled to the resolution and aligned with the local texture features to form a feature pyramid containing global-local multi-level semantic information, thus obtaining multi-level semantic features. A residual fusion strategy is used to integrate multi-level semantic features with local texture features to generate a fused feature representation; In the second stage, the enhanced frozen parameters and the combined features of sparse hints are subjected to a 2x transposed convolution to obtain upsampled combined features. The upsampled combined features and the fused feature representation are used as the initial segmentation mask features.

[0015] Step 4: Introduce the preliminary segmentation mask features into the learnable morphological boundary refinement module, and optimize the blurred and irregular boundary details end-to-end through differentiable dilation and erosion operations to obtain the refined feature map; Perform a prediction operation on the refined feature map to obtain a prediction probability map; As a preferred embodiment of the present invention, please refer to Figure 5 The initial segmentation mask features are introduced into a learnable morphological boundary refinement module. Through differentiable dilation and erosion operations, blurred and irregular boundary details are optimized end-to-end to obtain a refined feature map. The specific steps include: Perform a spatial unfolding operation on the initial segmentation mask features X to extract a tensor of size K×K for each spatial location, resulting in a neighborhood tensor. The following relationship exists in the corresponding process: ; in, This represents the initial segmentation mask features. Indicates the kernel size. Indicates the convolution kernel operation. Represents the domain tensor; By introducing channel-aware learnable bias parameters, these parameters are extended to the same spatial dimension as the domain tensor through a broadcast mechanism, and then element-wise added to the domain tensor to obtain the bias-enhanced domain representation. The following relationship exists in this process: ; in, This is the bias-enhanced domain representation. Indicates the learnable bias parameter; Parallel erosion and dilation paths are constructed to extract complementary boundary cues, and the enhanced neighborhood representations are input into the erosion and dilation paths respectively. For the erosion path, a 3×3 receptive field is used, and a minimum aggregation operation is performed on the neighborhood tensor after injection bias. For the dilation path, a 5×5 receptive field is used, and a maximum aggregation operation is performed on the neighborhood tensor after injection bias. Finally, the erosion feature map and dilation feature map are output respectively, and the following relationship exists in the corresponding process: ; ; in, and These represent minimum and maximum aggregations along the kernel dimension, respectively. Represents corrosion feature map, Represents the dilation feature map; The erosion feature map and dilation feature map are concatenated along the channel dimension and then input into a lightweight gating network. The input data passes through a 1×1 convolutional layer in the lightweight gating network, and then a spatially adaptive fusion weight map is generated by the sigmoid activation function. The following relationship exists in the corresponding process: ; in, Represents the spatial adaptive fusion weight graph. This indicates a splicing operation along the channel dimension. It is a 1×1 convolution; The erosion feature map and the dilation feature map are weighted and fused using a spatial adaptive fusion weight map to generate a thinner feature map. The following relationship exists in the corresponding process: ; in, This indicates element-wise multiplication. Represents a refined feature map; Finally, the refined feature map and the initial segmentation mask features are joined by residual connection and layer normalization to obtain the prediction probability map. The following relationship exists in the corresponding process: ; in, This represents a refined feature map. This represents a probability prediction graph. Presentation layer normalization processing.

[0016] Step 5: Based on the predicted probability map, construct a confidence-aware prediction correction mechanism. Dynamically adjust the probability distribution of the predicted probability map through pixel-level and image-level uncertainty estimation to obtain the corrected predicted probability map, and simultaneously output the final IoU score. Based on the corrected prediction probability map, a confidence-aware regularized loss function is constructed, and end-to-end training is performed in conjunction with multi-task loss. After training, the final prediction probability map is obtained. A pixel-level high-precision polyp segmentation mask is generated based on the final predicted probability map, and the final IoU score is used as a quantitative evaluation index of segmentation quality to achieve the joint output of segmentation results and credibility score.

[0017] In a preferred embodiment of the present invention, a confidence-aware prediction correction mechanism is constructed based on the prediction probability map. The probability distribution of the prediction probability map is dynamically adjusted through pixel-level and image-level uncertainty estimation to obtain the corrected prediction probability map, and the final IoU score is output synchronously. Specifically, the mechanism includes the following steps: The pixel-level uncertainty of the predicted probability map is calculated to quantify the model's hesitation regarding ambiguous regions. The following relationship exists in the corresponding process: ; in, Indicates pixel-level uncertainty; The pixel-level uncertainty is averaged over the spatial dimension to obtain the scalar uncertainty, which is then converted into pixel-level confidence. The following relationship exists in the corresponding process: ; in, Indicates the width and height of the feature. Indicates pixel-level confidence level; The average of the image-level confidence score and the pixel-level confidence score is defined as the comprehensive confidence score, which characterizes the overall reliability of the current segmentation result. The following relationship exists in the corresponding process: ; in, Indicates the IoU score. Indicates the overall confidence score; A Bernoullibote variable is introduced as a gating signal to determine whether the corrected target is used to replace the original prediction P in the loss calculation, resulting in the corrected prediction probability map, the corresponding calculation formula of which is: ; in, This represents the corrected predicted probability plot. Represents the true segmentation image. This represents a preliminary probability prediction diagram.

[0018] In a preferred embodiment of the present invention, a confidence-aware regularized loss function is constructed based on the corrected prediction probability map, and end-to-end training is performed using a multi-task loss function. After training, the final prediction probability map is obtained, specifically including the following steps: To prevent the model from minimizing the main loss function by approaching 0 confidence, a confidence-aware regularized loss function is constructed based on the comprehensive confidence score. This function suppresses model overconfidence by maximizing the comprehensive confidence score, and its corresponding relationship is as follows: ; in, This represents the confidence-aware regularized loss function; The binary cross-entropy loss function, Dice loss function, mean squared error loss function, and confidence-aware regularization loss function are combined as the total loss function, and the corresponding relationship is: ; in, These are adaptively adjusted hyperparameters used to dynamically balance the two losses. Represents the cross-entropy loss function. Represents the Dice loss function. This represents the mean squared loss error loss function; The expression for the cross-entropy loss function is as follows: ; in, This represents a probability prediction graph. These represent the weight coefficients corresponding to the binary cross-entropy loss function, where N represents the total number of classifiers. Let Y represent the cross-entropy loss function, and Y represent the true probability plot. The expression for the Dice loss function is as follows: ; Where M represents the number of pixels in the image. This represents the model's predicted probability that the i-th pixel belongs to a polyp region. This represents the true category label of the i-th pixel. To represent a very small constant; The mean squared loss error loss function expression is as follows: ; The model is trained end-to-end based on the total loss function. The corrected prediction probability map is used to participate in the main loss calculation. The confidence-aware regularized loss function guides the model to learn high-confidence predictions. After training, the final prediction probability map is obtained, and the final IoU score is output simultaneously.

[0019] To fully verify the technical effectiveness of this invention, the applicant conducted rigorous comparative experiments. Five publicly available polyp image datasets were selected as test benchmarks, and their performance was compared with five representative models in the current field: U-Net, UNet++, PraNet, Polyp-PV, and SAM. The experiments used the average Dice coefficient (mDice) and average intersection-union ratio (mIoU) as core evaluation metrics to comprehensively measure segmentation accuracy.

[0020] As shown in Table 1, this invention achieved the best mDice and mIoU metrics on all test datasets, significantly outperforming the five comparison models: U-Net, UNet++, PraNet, Polyp-PV, and SAM. Compared to current mainstream segmentation models, this invention demonstrates a clear advantage. Compared to the traditional CNN architectures U-Net and UNet++, this invention improves mDice by 9% and 11% respectively on the Kvasir dataset, and by 40% and 39% respectively on the ETIS dataset. This sufficiently demonstrates the superiority of this invention in handling complex backgrounds and polyps with varying morphologies. Furthermore, this invention significantly outperforms advanced Transformer-based models. Compared to PraNet, this invention maintains a stable performance advantage on all datasets, especially on the CVC-ColonDB dataset, where mDice is improved by 18% and mIoU by 17%. This is attributed to the morphological boundary refinement module introduced in this invention, which effectively solves the segmentation problem of traditional models in regions with blurred boundaries. Compared with directly applying the SAM basic model, the local perception prompt generation module and learnable morphological boundary optimization module introduced in this invention completely solve the problem that the SAM model requires manual prompts and has blurred boundaries in the polyp segmentation task. It significantly outperforms SAM on all datasets, which fully proves the effectiveness of this invention.

[0021] Table 1 Comparison of Indicators of the Invention

[0022] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A polyp segmentation method based on local perceptual cue generation and morphological boundary refinement, characterized in that, The method includes the following steps: Step 1: Obtain a dataset of colon polyp images, perform standardization preprocessing on the polyp images, and extract global semantic features and local texture features from the preprocessed images; Step 2: Input global semantic features and local texture features into the local perception cue generation module. Based on the local texture features, the local perception guides the image to focus on the salient region. At the same time, based on the collaborative interaction of global semantic features and local texture features, multi-scale features are adaptively fused to generate sparse and dense cuees for task perception. Step 3: Input sparse and dense cues, global semantic features and local texture features into the mask decoder to perform bidirectional interactive mapping between global context and local details, generate preliminary segmentation mask features, and output the corresponding IoU score at the same time. Step 4: Introduce the preliminary segmentation mask features into the learnable morphological boundary refinement module, and optimize the blurred and irregular boundary details end-to-end through differentiable dilation and erosion operations to obtain the refined feature map; Perform a prediction operation on the refined feature map to obtain a prediction probability map; Step 5: Based on the predicted probability map, construct a confidence-aware prediction correction mechanism. Dynamically adjust the probability distribution of the predicted probability map through pixel-level and image-level uncertainty estimation to obtain the corrected predicted probability map. Based on the corrected prediction probability map, a confidence-aware regularized loss function is constructed, and end-to-end training is performed in conjunction with multi-task loss. After training, the final prediction probability map is obtained, and the final IoU score is output synchronously. A pixel-level high-precision polyp segmentation mask is generated based on the final predicted probability map, and the final IoU score is used as a quantitative evaluation index of segmentation quality to achieve the joint output of segmentation results and credibility score.

2. The polyp segmentation method based on local perception cue generation and morphological boundary refinement according to claim 1, characterized in that, In step 1, a dataset of colon polyp images is obtained, and the polyp images are standardized and preprocessed. Global semantic features and local texture features of the preprocessed images are then extracted, specifically including the following steps: Obtain a dataset of colon polyp images, preprocess the polyp images in the dataset, and obtain preprocessed polyp images; The preprocessed polyp image is input into the ViT encoder for feature extraction to obtain the intermediate features of the third, sixth and ninth stages and the global semantic features of the twelfth stage, respectively. The preprocessed polyp image is input into a CNN encoder for feature extraction to obtain local texture features.

3. The polyp segmentation method based on local perception cue generation and morphological boundary refinement according to claim 2, characterized in that, In step 2, local perception based on local texture features guides the image to focus on salient regions. Simultaneously, based on the synergistic interaction of global semantic features and local texture features, multi-scale features are adaptively fused to generate task-aware sparse and dense cues. Specifically, this includes the following steps: Extracting the frozen parameters from the mask decoder and learnable parameters ; Freeze parameters and learnable parameters A token-level cross-attention mechanism is used for information fusion, followed by linear projection and residual connection operations, to output the updated frozen parameters. and updated learnable parameters ; Local texture features Inputting a lightweight attention generator produces a single-channel spatial attention map, and the following relationship exists in the process: ； in, This represents the Sigmoid activation function. This represents the GELU activation function. and These represent the first and second convolutional layers, respectively. Indicates an upsampling operation. Represents local texture features. Represents a spatial attention map for a single channel; Spatial attention maps of single channels and global semantic features The tokens are expanded and then weighted and aggregated to obtain local perception tokens. The following relationship exists in the correspondence process: ； in, This represents a local sensing token. Represents global semantic features. Represents a spatial attention map for a single channel; Local perception token Updated frozen parameters and updated learnable parameters By concatenating the sequences, a preliminary set of composite hints is obtained. ; The initial set of compound prompts Using global semantic features as query tokens and key-value tokens as key-value tokens, attention queries are performed, causing the global semantic features to recalibrate their focus based on the prompts, resulting in enhanced global semantic features. Enhanced freeze parameters sparse hints and dense prompts .

4. The polyp segmentation method based on local perception cue generation and morphological boundary refinement according to claim 3, characterized in that, Freeze parameters and learnable parameters A token-level cross-attention mechanism is used for information fusion, followed by linear projection and residual connection operations. The updated frozen parameters and updated learnable parameters are output, and the corresponding process has the following relationship: ； ； in, and This represents the updated frozen parameters and learnable parameters. and This represents the initial frozen parameters and the learnable parameters. Indicates linear projection. Represents the formula for the cross-attention function; The calculation process is defined as follows: ； in, The learnable weight matrix representing cross-attention has the following dimensions: , This represents a two-layer fully connected network. express Embedding dimension This represents the matrix transpose operation.

5. The polyp segmentation method based on local perception cue generation and morphological boundary refinement according to claim 4, characterized in that, Using the initial set of composite hints as query tokens and the global semantic features as key-value tokens, attention queries are performed to recalibrate the focus of the global semantic features based on the hint information, resulting in enhanced global semantic features, enhanced frozen parameters, sparse hints, and dense hints. The following relationship exists in the correspondence process: ； ； ； ； in, This indicates the enhanced freeze parameters. This represents the initial set of compound prompts. Indicates sparsity. This indicates dense prompts. This represents the enhanced global semantic features. This represents the enhanced set of composite hints. Indicate extraction The feature of the first position in the second dimension, Indicate extraction The second dimension indexes all features starting from 6. Indicate extraction The second dimension indexes all features from 1 to 5.

6. The polyp segmentation method based on local perception cue generation and morphological boundary refinement according to claim 5, characterized in that, In step 3, sparse and dense cues, global semantic features, and local texture features are input into the mask decoder to perform bidirectional interactive mapping between global context and local details, generating preliminary segmentation mask features and simultaneously outputting the corresponding IoU score. The mask decoder includes a bidirectional Transformer module, a token-to-image attention module, and a feature fusion module. The bidirectional Transformer module includes five sub-modules: a ViT-CNN cross-attention layer, an image-to-token attention mechanism, a first multilayer perceptron, a token-to-image attention mechanism, and a self-attention mechanism. ViT-CNN's cross-attention layer, image-to-token attention mechanism, and token-to-image attention mechanism all employ a cross-attention mechanism. The sparse and dense cues, global semantic features, and local texture features are input into the mask decoder to perform bidirectional interactive mapping between global context and local details, generating preliminary segmentation mask features and simultaneously outputting the corresponding IoU score. The specific steps include the following: Phase 1: Step 3.1: The combined features of the enhanced global semantic features and the dense cueing are used as the first combined feature, and the local texture features are used as the second combined feature. Both are input into the ViT-CNN cross-attention layer for cross-modal feature interaction. In the ViT-CNN cross-attention layer, the local texture features are used as the query and the first combined feature is used as the key to calculate cross-modal attention and output the combined representation of the enhanced global semantic features and the dense cueing in the first stage. Step 3.2: Input the enhanced freezing parameters and the combined features of the sparse cue into the self-attention mechanism to perform feature self-enhancement and obtain self-enhanced features; Step 3.3: The self-enhancing features are processed sequentially through the token-to-image attention mechanism and the first multilayer perceptron to obtain the combined features of the enhanced frozen parameters and sparse cueing updated in the first stage. Step 3.4: Input the combined features of the enhanced frozen parameters updated in the first stage and sparse cue, the local texture features, and the combined representation of the enhanced global semantic features and dense cue from the first stage into the token attention mechanism. In the image-to-token attention mechanism, local texture features are used as queries, the combination of enhanced global semantic features and dense cueing from the first stage is used as keys, and the combination of enhanced frozen parameters and sparse cueing updated in the first stage is used as values ​​for cross-attention calculation, outputting the enhanced global semantic features and local texture features updated in the first stage. Phase Two: Step 3.5: Take the enhanced global semantic features updated in the first stage as the first combined feature and the local texture features updated in the first stage as the second combined feature. Repeat the operation of step 3.1 again to obtain the combined representation of the enhanced global semantic features and dense cue in the second stage. Step 3.6: Repeat steps 3.2 and 3.3 with the combined features of the enhanced frozen parameters and sparse cueing updated in the first stage as input to obtain the combined features of the enhanced frozen parameters and sparse cueing updated in the second stage. Step 3.7: Input the combined features of the enhanced frozen parameters updated in the second stage and sparse cue, the local texture features, and the combined representation of the enhanced global semantic features and dense cue from the second stage into the token attention mechanism. In the image-to-token attention mechanism, local texture features are used as queries, the combined representation of the enhanced global semantic features and dense cue in the second stage is used as keys, and the combined features of the mask token and sparse cue are used as values ​​for cross-attention calculation, outputting the enhanced global semantic features and the local texture features updated in the second stage. Phase Three: Step 3.8: The enhanced global semantic features updated in the second stage, the local texture features updated in the second stage, and the combined features of the enhanced frozen parameters and sparse cue updated in the second stage are fed into the image attention module for cross-attention calculation, and then input into the second multilayer perceptron to obtain the IoU score. Step 3.9: Input the intermediate features and local texture features from the third, sixth, and ninth stages of the ViT encoder into the feature fusion module; The intermediate features of the third, sixth and ninth stages of the ViT encoder are progressively upsampled to align with the resolution and local texture features by the feature fusion module, forming a feature pyramid containing global-local multi-level semantic information, and thus obtaining multi-level semantic features. A residual fusion strategy is used to integrate multi-level semantic features with local texture features to generate a fused feature representation; In the second stage, the enhanced frozen parameters and the combined features of sparse hints are subjected to a 2x transposed convolution to obtain upsampled combined features. The upsampled combined features and the fused feature representation are used as the initial segmentation mask features.

7. The polyp segmentation method based on local perception cue generation and morphological boundary refinement according to claim 6, characterized in that, In step 4, the preliminary segmentation mask features are introduced into the learnable morphological boundary refinement module. Through differentiable dilation and erosion operations, blurred and irregular boundary details are optimized end-to-end to obtain a refined feature map. Specifically, this includes the following steps: Perform a spatial unfolding operation on the initial segmentation mask features X to extract a tensor of size K×K for each spatial location, resulting in a neighborhood tensor. The following relationship exists in the corresponding process: ； in, This represents the initial segmentation mask features. Indicates the kernel size. Indicates the convolution kernel operation. Represents the domain tensor; By introducing channel-aware learnable bias parameters, these parameters are extended to the same spatial dimension as the domain tensor through a broadcast mechanism, and then element-wise added to the domain tensor to obtain the bias-enhanced domain representation. The following relationship exists in this process: ； in, This is the bias-enhanced domain representation. Indicates the learnable bias parameter; Parallel erosion and dilation paths are constructed, and the enhanced neighborhood representation is input into the erosion and dilation paths respectively, outputting erosion feature maps and dilation feature maps respectively. The following relationship exists in the corresponding process: ； ； in, and These represent minimum and maximum aggregations along the kernel dimension, respectively. Represents corrosion feature map, Represents the dilation feature map; The erosion feature map and the dilation feature map are concatenated along the channel dimension and then input into a lightweight gating network to obtain a spatially adaptive fusion weight map. The following relationship exists in the corresponding process: ； in, Represents the spatial adaptive fusion weight graph. This indicates a splicing operation along the channel dimension. Represents a 1×1 convolution; The erosion feature map and the dilation feature map are weighted and fused using a spatial adaptive fusion weight map to generate a thinner feature map. The following relationship exists in the corresponding process: ； in, This indicates element-wise multiplication. This represents a refined feature map.

8. The polyp segmentation method based on local perception cue generation and morphological boundary refinement according to claim 7, characterized in that, In step 5, a confidence-aware prediction correction mechanism is constructed based on the prediction probability map. This mechanism dynamically adjusts the probability distribution of the prediction probability map through pixel-level and image-level uncertainty estimation to obtain the corrected prediction probability map. Specifically, this includes the following steps: The pixel-level uncertainty of the predicted probability map is calculated to quantify the model's hesitation regarding ambiguous regions. The following relationship exists in the corresponding process: ； in, Indicates pixel-level uncertainty; The pixel-level uncertainty is averaged over the spatial dimension to obtain the scalar uncertainty, which is then converted into pixel-level confidence. The following relationship exists in the corresponding process: ； in, Indicates the width and height of the feature. Indicates pixel-level confidence level; The average of the image-level confidence score and the pixel-level confidence score is defined as the comprehensive confidence score, which characterizes the overall reliability of the current segmentation result. The following relationship exists in the corresponding process: ； in, Indicates the IoU score. Indicates the overall confidence score; A Bernoullibote variable is introduced as a gating signal to determine whether to display the true segmentation map. Alternative Preliminary Prediction Probability Map The corrected prediction probability map is obtained by participating in the loss calculation, and its corresponding calculation formula is: ； in, This represents the corrected predicted probability plot. Represents the true segmentation image. This represents a preliminary probability prediction diagram.

9. The polyp segmentation method based on local perception cue generation and morphological boundary refinement according to claim 8, characterized in that, In step 5, a confidence-aware regularized loss function is constructed based on the corrected prediction probability map, and end-to-end training is performed using a multi-task loss function. After training, the final prediction probability map is obtained, which specifically includes the following steps: Based on the comprehensive confidence score, a confidence-aware regularized loss function is constructed, and its corresponding relationship is as follows: ； in, This represents the confidence-aware regularized loss function; The binary cross-entropy loss function, Dice loss function, mean squared error loss function, and confidence-aware regularization loss function are combined as the total loss function, and the corresponding relationship is: ； in, These are adaptively adjusted hyperparameters used to dynamically balance the two losses. Represents cross-entropy loss, Represents the Dice loss function. This represents the mean squared loss error loss function.

10. The polyp segmentation method based on local perception cue generation and morphological boundary refinement according to claim 9, characterized in that, The expression for the binary cross-entropy loss function is as follows: ； in, This represents a probability prediction graph. These represent the weight coefficients corresponding to the binary cross-entropy loss function, where N represents the total number of classifiers. Let Y represent the cross-entropy loss function, and Y represent the true probability plot. The expression for the Dice loss function is as follows: ； Where M represents the number of pixels in the image. This represents the model's predicted probability that the i-th pixel belongs to a polyp region. This represents the true category label of the i-th pixel. To represent a very small constant; The mean squared loss error loss function expression is as follows: ； The model is trained end-to-end based on the total loss function. The corrected prediction probability map is used in the main loss calculation. The model is guided to learn high-confidence predictions through the confidence-aware regularized loss function. After training, the final prediction probability map is obtained.

Images