Polyp segmentation method based on SAM boundary enhancement and uncertain guidance

Abstract

The application discloses a polyp segmentation method based on SAM boundary enhancement and uncertainty guidance, aiming at the problems of high pixel-level labeling cost, blurred polyp boundary, difficult cross-scale feature alignment and poor effect of SAM direct migration to a medical scene in the polyp segmentation task under enteroscopy, fusing the hierarchical feature extraction ability of Res2Net-50, the general segmentation ability of SAM, combining core strategies such as boundary enhancement, uncertainty guidance and dynamic prompt generation, and realizing high-precision polyp weakly supervised segmentation under extremely sparse scribble labeling. The application significantly improves the spatial calibration accuracy of the model in a low-contrast environment, eliminates noise interference, and effectively solves the problem of supervision imbalance caused by the large size difference of lesions in an endoscope image.

Description

Technical Field

[0001] This invention relates to a polyp segmentation method, and more particularly to a polyp segmentation method based on SAM boundary enhancement and uncertainty guidance, belonging to the field of image processing technology. Background Technology

[0002] In the clinical prevention and early diagnosis of colorectal cancer, accurate segmentation of polyps under colonoscopy images is a core prerequisite for identifying lesions, assessing the condition, and assisting in minimally invasive surgery. Efficient automated segmentation algorithms can provide crucial decision support for clinicians, effectively reducing the rate of missed diagnoses. Traditional deep learning segmentation methods mostly rely on large-scale, intensive pixel-level manual annotation. However, in the field of medical imaging, obtaining high-quality fully annotated data not only consumes a significant amount of time and effort from clinical experts, but the annotation process is also highly subjective and economically costly.

[0003] To alleviate annotation pressure, weakly supervised learning (WSL) has gradually gained attention in academia. Among them, weakly supervised methods using graffiti labels have achieved a good balance between annotation efficiency and spatial guidance performance. Nevertheless, existing weakly supervised polyp segmentation frameworks still have significant shortcomings when dealing with complex endoscopic scenarios: First, in the pseudo-label generation and evolution stages, existing weakly supervised methods often struggle to capture accurate semantic boundaries. Although the Segmentation Arbitrary Model (SAM) demonstrates powerful general segmentation capabilities, its direct application to medical images, due to a lack of medical expertise, easily leads to biases in blurred boundaries and low-contrast regions. Mainstream SAM guidance frameworks often employ static cues or offline bounding box constraints, ignoring the dynamic geometric interaction between network predictions and original annotations during training. This results in the cue boxes failing to adaptively shrink with the training process, making it difficult for pseudo-labels to accurately cover polyp targets with varying morphologies. Second, in terms of feature representation and boundary perception, polyps often have extremely high visual similarity to surrounding normal mucosal tissue, and their boundaries are extremely blurred. Existing network architectures mostly focus on simple fusion of multi-scale features, lacking active interaction mechanisms for boundary regions. This results in insufficient feature responsiveness when the model deals with small lesions or polyps with blurred edges, making it unable to achieve accurate inverse gain adjustment and edge sharpening. Finally, at the level of loss design and supervision strategy, existing weak supervision mechanisms lack explicit constraints on the consistency of semantic structure across resolutions and fail to effectively compensate for the bias caused by uneven distribution of target sizes. Traditional loss functions often lead to supervised overfitting of large targets, neglecting small polyps that are important in early clinical diagnosis. In addition, most mechanisms use fixed similarity thresholds to filter pseudo-labels, which cannot adapt to the evolution of models from "coarse learning" to "refined purification," making it difficult to fundamentally narrow the performance gap between weakly supervised and fully supervised methods.

[0004] In summary, existing weakly supervised polyp segmentation methods still have significant shortcomings in terms of high-quality dynamic pseudo-label generation, active boundary interaction enhancement, and adaptive supervision adjustment for multi-size targets. Summary of the Invention

[0005] The technical problem to be solved by this invention is to provide a polyp segmentation method based on SAM with boundary enhancement and uncertainty guidance, which overcomes the shortcomings of existing technologies in high-quality pseudo-label generation, active boundary interaction enhancement and adaptive supervision adjustment for multi-size targets.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0007] A polyp segmentation method based on SAM boundary enhancement and uncertainty guidance includes the following steps: S1. Input the preprocessed endoscope image into the Res2Net-50 encoder for feature extraction, and obtain five levels of features from low-level spatial details to high-level semantic information. S2. Use the cross-layer fusion module to perform bidirectional collaborative enhancement on adjacent layer features in the five layers, simultaneously construct multi-scale consistency branches, and perform structural consistency constraints on inputs with different resolutions. S3. In the decoding stage, an independent edge prediction head is introduced to generate an edge probability map. The spatial weight matrix generated by the edge probability map is used to perform residual inverse gain adjustment on the cross-layer aggregated features. S4. Combining the model prediction results with the extremely sparse graffiti prior, a rectangular box fusion strategy that shrinks linearly with the training cycle is executed to drive the SAM module to generate a fine mask, and unreliable noise is removed through a dynamic similarity filtering mechanism. S5. Construct an uncertainty distribution map using pixel-level entropy values ​​and introduce size-adaptive weight coefficients to perform multi-scale equalization boundary supervision; achieve closed-loop parameter updates and performance co-evolution of the segmentation network and guidance mechanism through feedback of prediction results via pseudo-labels.

[0008] Further, step S1 specifically includes: 1.1 Perform size standardization and normalization on the original endoscopic images to make them conform to the input distribution requirements of the backbone network; 1.2. Five feature maps F1 to F5 with different spatial resolutions were extracted using the bottleneck layer structure of Res2Net-50; 1.3. Extract the preliminary prediction map S1 generated by the network in real time during forward propagation. Determine the dynamic prediction box of the lesion by calculating the probability envelope range of the preliminary prediction map S1. Use the dynamic prediction box as the core constraint to drive the adaptive prompt generation mechanism to realize the online evolution of pseudo-labels.

[0009] Further, step 1.2 specifically involves: using Res2Net-50 as the feature extraction encoder, and through the multi-scale residual bottleneck layer structure of the feature extraction encoder, performing forward propagation on the preprocessed endoscopic image, sequentially outputting feature maps F1 to F5 at five levels, with feature resolutions of [missing information]. 、 Where W and H are the width and height of the input endoscopic image, respectively, and the number of channels are 64, 256, 512, 1024, and 2048, respectively; the low-level features F1 and F2 retain the spatial details of the polyp; the high-level features F4 and F5 contain the deep information of the polyp; and the middle-level feature F3 achieves the transition and fusion of details and semantics.

[0010] Further, step S2 specifically includes: 2.1. The cross-layer fusion module is used to concatenate low-level features and high-level features of adjacent resolutions, and the spatial attention mechanism is used to achieve mutual enhancement and semantic complementarity of feature maps at the pixel level. 2.2 During the training loop, the original endoscopic images are downsampled by 0.75x and 0.5x respectively, and the generated three-dimensional multi-scale images are synchronously input into the segmentation network with shared weights. 2.3 Calculate the significant structural consistency loss between the original scale prediction map and the prediction maps at each scaling scale, and force the model to maintain consistent recognition of polyp morphology and topology under different scaling ratios; 2.4. Combine the training cycle to dynamically adjust the loss weights of each scale branch, guide the model to focus on global structure alignment in the early stage of training, and shift to cross-scale alignment of lesion boundary details in the later stage of training.

[0011] Further, step 2.1 specifically involves: analyzing the adjacent layer feature pairs output by the Res2Net-50 encoder. and First, the high-level characteristics Upsampling to low-level features via bilinear interpolation With the same resolution, 3×3 convolution operations are then applied to both feature layers to unify channel dimensions and map features, resulting in pre-processed low-level features. Characteristics of high-level personnel Next, the low-level features Characteristics of high-level personnel Another set of 3×3 convolutional layers is input, and two sets of pixel-level spatial attention weight maps are generated by applying the Sigmoid activation function. The generated spatial attention weight maps are then used to apply the low-level features. Characteristics of high-level personnel Pixel-level mutual enhancement operations are performed to achieve complementary enhancement of low-level detail features and high-level semantic features. Finally, the enhanced features from the two layers are concatenated by fusing features through convolutional blocks containing convolution, batch normalization, and ReLU activation. Residual connections are introduced to preserve the original feature information, and the enhanced features after cross-layer fusion are output. Enhanced features It possesses both rich spatial details and clear semantic direction; For low-level features Characteristics of high-level personnel The formula for performing pixel-level mutual enhancement operations is as follows:

[0012]

[0013] in, For Sigmoid mutual enhancement operations, This is element-wise multiplication.

[0014] Further, step S3 specifically includes: 3.1. A convolutional layer is connected to the bottom output of the local and global decoders as an edge prediction head to transform the high-dimensional fusion features into a single-channel edge response map that reflects the probability of the existence of polyp boundaries. 3.2. The edge response is mapped to a continuous space between 0 and 1 using the normalized exponential function to obtain the spatial boundary weight matrix representing the confidence of a pixel belonging to the boundary. 3.3 Construct a boundary interaction enhancement operator, which linearly scales the spatial boundary weight matrix according to a preset scaling factor and increments it by 1 to generate an enhancement coefficient matrix; 3.4. Perform element-wise multiplication of the enhancement coefficient matrix with the deep feature map to achieve adaptive feature recalibration at the boundary pixel position. Enhance the model's perception accuracy of blurred boundaries through inverse gain adjustment.

[0015] Further, step 3.2 specifically involves: inputting the single-channel edge response map into the Sigmoid normalized exponential function, and generating a spatial boundary weight matrix by mapping the original response values ​​to the continuous interval [0,1]. The closer a pixel value is to 1 in the matrix, the higher the confidence that the pixel is a polyp boundary; the closer it is to 0, the higher the confidence that the pixel is not a boundary, thus achieving pixel-level confidence quantification of polyp boundary regions. The formula for generating the boundary weight matrix of the generated space is as follows:

[0016] in, For the Sigmoid function, This is an edge response map.

[0017] Further, step S4 specifically includes: 4.1 Extract the first bounding rectangle of the graffiti annotation and the second bounding rectangle of the model's current prediction image, respectively; 4.2 Set a redundancy radius coefficient that decays linearly with the training process. In the early stage of training, maintain a large search boundary to cover the entire lesion area. In the middle and later stages of training, perform linear shrinkage processing as the number of iterations increases. 4.3. Perform expansion processing on the first and second bounding rectangles, and calculate their overlapping intersection area. Use the overlapping intersection area as the adaptive prompt box that drives SAM. 4.4. Using the adaptive cue box as the spatial constraint prior input SAM, the visual knowledge reserves of the large model are used to infer and generate lesion candidate masks with high-fidelity edge response online; 4.5. Implement an image-level filtering mechanism, dynamically adjust the similarity threshold using a cosine annealing strategy, and automatically identify and remove unreliable noise samples below the dynamic threshold by calculating the overlap ratio between candidate masks and graffiti annotations.

[0018] Further, step 4.3 specifically involves: based on the redundancy radius coefficient determined in step 4.2, adjusting the first circumscribed rectangle... Execution extension: For the second circumscribed rectangle Perform conservative extension: Then, the overlapping intersection area of ​​the two expanded boxes is calculated, which is the adaptive tooltip. If the calculated intersection region is empty, the expanded first bounding rectangle will be used by default. This serves as a prompt to ensure that the SAM module always obtains valid space constraints.

[0019] Further, step S5 specifically includes: 5.1 Calculate the pixel-level information entropy value of the current segmentation probability distribution in real time to quantify the prediction confidence of the model in the spatial dimension and generate a dynamically updated spatial uncertainty weight distribution map; 5.2. Real-time statistics of the total number of lesion boundary pixels in the candidate mask, and calculation of the area balance correction coefficient. The correction coefficient is negatively correlated with the square root of the total number of boundary pixels. The model's supervision weight for small polyp samples is improved through a weighted compensation mechanism. 5.3 Combining weighted intersection-union loss and weighted binary cross-entropy loss, the loss value at the edge of the lesion is dynamically recalibrated using the spatial uncertainty weight distribution map; 5.4 During model training, structured consistency loss and weighted edge loss are integrated to perform online synchronous fine-tuning of the image coding layer and mask decoding layer of SAM while updating the segmentation network parameters; 5.5 When the model's evaluation metrics on the validation set reach the preset threshold, stop the iteration and output the final high-precision pixel-level binarized segmentation result image.

[0020] Compared with the prior art, the present invention has the following advantages and effects: 1. This invention utilizes an independent edge prediction head to generate a spatial weight matrix and performs residual inverse gain adjustment on the main feature stream, thereby achieving adaptive recalibration of lesion edge features and significantly improving the spatial calibration accuracy of the model in low-contrast environments. 2. This invention uses a cue box fusion strategy that shrinks linearly with the training cycle, combined with the visual knowledge of the SAM large model, to generate fine masks with high-fidelity edge response online, and effectively removes noise interference through a dynamic similarity filtering mechanism. 3. This invention achieves stable alignment of cross-scale features through multi-scale consistency branches and introduces an area balance correction coefficient to perform weighted compensation on small-sized samples, effectively solving the problem of unbalanced supervision caused by large differences in lesion size in endoscopic images. Attached Figure Description

[0021] Figure 1 This is a flowchart of the polyp segmentation method based on SAM and uncertainty guidance of the present invention.

[0022] Figure 2 This is a network architecture diagram of an embodiment of the present invention.

[0023] Figure 3 This is a schematic diagram of the edge prediction head structure of this embodiment.

[0024] Figure 4 This is a schematic diagram illustrating the adaptive prompt generation of an embodiment of the present invention.

[0025] Figure 5 This is a schematic diagram of three-way scale consistency alignment according to an embodiment of the present invention.

[0026] Figure 6 This is a comparison chart of the visualization results of the CVC-300 dataset in an embodiment of the present invention.

[0027] Figure 7 This is a comparison chart of the visualization results of the ETIS-LaribPolypDB dataset in an embodiment of the present invention.

[0028] Figure 8This is a comparison chart of the visualization results of the CVC-ColonDB dataset in an embodiment of the present invention. Detailed Implementation

[0029] To illustrate in detail the technical solutions adopted by the present invention to achieve the intended technical objectives, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Furthermore, the technical means or technical features in the embodiments of the present invention can be replaced without creative effort. The present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0030] like Figure 1 As shown, the present invention provides a boundary enhancement and uncertainty-guided polyp segmentation method based on SAM, comprising the following steps: S1. Input the preprocessed endoscope image into the Res2Net-50 encoder for feature extraction, and obtain five levels of features from low-level spatial details to high-level semantic information.

[0031] 1.1 The original endoscopic images are sized and normalized to make them conform to the input distribution requirements of the backbone network.

[0032] Clinically acquired colonoscopy polyp images were uniformly scaled to 320×320 pixels. Simultaneously, mean subtraction and variance division were performed on the three RGB channels of the images to eliminate feature distribution shifts caused by differences in lighting and equipment acquisition. This ensured that the processed image features conformed to the pre-trained distribution of the Res2Net-50 backbone network, laying the foundation for subsequent feature extraction.

[0033] 1.2. Five feature maps F1 to F5 with different spatial resolutions were extracted using the bottleneck layer structure of Res2Net-50.

[0034] Using Res2Net-50 as the feature extraction encoder, a multi-scale residual bottleneck layer structure is employed to perform forward propagation on the preprocessed endoscopic image, sequentially outputting feature maps F1 to F5 at five levels, with feature resolutions of [missing information]. 、 Where W and H are the width and height of the input endoscopic image, respectively, and the number of channels are 64, 256, 512, 1024, and 2048, respectively; the low-level features F1 and F2 retain the spatial details of the polyp; the high-level features F4 and F5 contain the deep information of the polyp; and the middle-level feature F3 achieves the transition and fusion of details and semantics.

[0035] 1.3. Extract the preliminary prediction map S1 generated by the network in real time during forward propagation. Determine the dynamic prediction box of the lesion by calculating the probability envelope range of the preliminary prediction map S1. Use the dynamic prediction box as the core constraint to drive the adaptive prompt generation mechanism to realize the online evolution of pseudo-labels.

[0036] During the network forward propagation, the preliminary prediction results of the main segmentation path are extracted in real time. The probability values ​​of polyp regions in the prediction image are thresholded (the probability threshold is set to 0.5) to obtain the binary mask of the polyp. Then, the minimum bounding rectangle of the mask is calculated through contour detection to obtain the dynamic prediction box of the lesion. This prediction box can reflect the network's prediction results of the polyp location and range in real time. It is combined with the prior box generated by the graffiti annotation as the core spatial constraint for the subsequent adaptive prompt generation of the SAM module, realizing the online dynamic evolution of pseudo-labels.

[0037] S2, such as Figure 2 and Figure 5 As shown, the cross-layer fusion module is used to perform bidirectional collaborative enhancement of adjacent layer features in the five layers, and a multi-scale consistency branch is constructed simultaneously to perform structural consistency constraints on inputs with different resolutions.

[0038] 2.1. The cross-layer fusion module is used to cascade low-level features and high-level features of adjacent resolutions, and the spatial attention mechanism is used to achieve mutual enhancement and semantic complementarity of feature maps at the pixel level.

[0039] For adjacent layer feature pairs output by the Res2Net-50 encoder and First, the high-level characteristics Upsampling to low-level features via bilinear interpolation To ensure consistent feature dimensions, a 3×3 convolution operation is then applied to both feature layers to unify channel dimensions and map features, resulting in pre-processed low-level features. Characteristics of high-level personnel Next, the low-level features Characteristics of high-level personnel Another set of 3×3 convolutional layers is input, and two sets of pixel-level spatial attention weight maps are generated using the Sigmoid activation function. These weight maps can adaptively capture discriminative semantic regions and detailed information in the two layers of features. The generated spatial attention weight maps are then used to apply the low-level features... Characteristics of high-level personnel Pixel-level mutual enhancement operations are performed to achieve complementary enhancement of low-level detail features and high-level semantic features. Finally, the enhanced features from the two layers are concatenated by fusing features through convolutional blocks containing convolution, batch normalization, and ReLU activation. Residual connections are introduced to preserve the original feature information, and the enhanced features after cross-layer fusion are output. Enhanced features It possesses both rich spatial details and clear semantic orientation.

[0040] For low-level features Characteristics of high-level personnel The formula for performing pixel-level mutual enhancement operations is as follows:

[0041]

[0042] in, For Sigmoid mutual enhancement operations, This is element-wise multiplication.

[0043] 2.2 During the training loop, the original endoscopic images are downsampled by 0.75x and 0.5x respectively, and the generated three-dimensional multi-scale images are synchronously input into the segmentation network with shared weights.

[0044] In each training iteration, using the original 320×320 pixel image as a baseline, images with resolutions of 0.75 times (240×240 pixels) and 0.5 times (160×160 pixels) are generated using bilinear interpolation. To avoid size mismatch issues during deep feature fusion, the downsampled image size is aligned to ensure that its width and height are integer multiples of 32. If the calculated size is less than 224 pixels, it is forcibly set to 256 pixels to ensure the effectiveness of feature extraction. The three sets of images—original scale, 0.75 times scale, and 0.5 times scale—are simultaneously input into a segmentation network with shared weights. The three sets of images share the same encoder, cross-layer fusion module, and decoder structure, differing only in input scale, ensuring that the network learns scale-invariant feature representations.

[0045] 2.3 Calculate the significant structural consistency loss between the original scale prediction map and the prediction maps at each scaling scale, and force the model to maintain consistent recognition of polyp morphology and topology under different scaling ratios.

[0046] After the three sets of images at different scales have completed the network forward propagation, polyp segmentation prediction maps corresponding to each scale are obtained. The prediction maps at 0.75x and 0.5x scales are upsampled to the original scale (320×320 pixels) through bilinear interpolation to maintain the same size as the prediction maps at the original scale. A saliency structural consistency loss function is defined, which quantifies the structural differences by calculating the L2 norm distance between prediction maps at different scales. This loss function can constrain the model to maintain consistency in the overall morphology, spatial topology, and identification of key structures of polyps at different scales, avoiding distortion of segmentation results or loss of structure due to scale changes.

[0047] The formula for the structural consistency loss function is as follows:

[0048] in, This is the original scale prediction map. This is the predicted image after upsampling at a scaled level. This is a scaling operation. To segment the network.

[0049] 2.4. Combine the training cycle to dynamically adjust the loss weights of each scale branch, guide the model to focus on global structure alignment in the early stage of training, and shift to cross-scale alignment of lesion boundary details in the later stage of training.

[0050] Configure a dynamic weight adjustment strategy and define the scale weight coefficients. The weights of the 0.5x scale branch and the 0.75x scale branch are linearly adjusted during the training period. In the first 60 training rounds (out of a total of 100 rounds), the weight of the 0.5x scale branch is set to 0.6, and the weight of the 0.75x scale branch is set to 0.4. This guides the model to learn the global structural alignment of polyps at large scales, ensuring the overall consistency of the segmentation results. After 60 rounds of training, the weight of the 0.5x scale branch is reduced to 0.3 through linear decay, while the weight of the 0.75x scale branch is increased to 0.7. This shifts the training focus of the model to the mesoscale, focusing on the cross-scale alignment of local details such as polyp boundaries and subtle protrusions. Through this dynamic weight adjustment mechanism, the training logic of "global first, local second" is achieved, which not only ensures the overall structural accuracy of the segmentation results but also improves the scale consistency of local details.

[0051] S3, such as Figure 2 and Figure 3 As shown, an independent edge prediction head is introduced in the decoding stage to generate an edge probability map, and the spatial weight matrix generated by the edge probability map is used to perform residual inverse gain adjustment on the cross-layer aggregated features.

[0052] 3.1. A convolutional layer is connected to the bottom output of the local and global decoders as an edge prediction head to transform the high-dimensional fusion features into a single-channel edge response map that reflects the probability of the existence of polyp boundaries.

[0053] In the low-level output stage of the local and global decoders, the low-level features after cross-layer fusion and local-global context aggregation are selected. This feature contains the richest spatial details and boundary cues of the polyp. This low-level feature is then fed into a feature mapping module consisting of a 3×3 convolutional layer, a batch normalization layer, and a ReLU activation function. First, channel compression and feature purification are performed on the high-dimensional fused feature. Then, a 1×1 convolutional layer reduces the number of feature channels from 64 dimensions to 1 dimension, generating a single-channel edge response map. Each pixel value in this response map directly reflects the original probability that the corresponding location is a polyp boundary, providing basic feature support for subsequent boundary weight construction.

[0054] 3.2. The edge response is mapped to a continuous space between 0 and 1 using the normalized exponential function to obtain the spatial boundary weight matrix representing the confidence of a pixel belonging to the boundary.

[0055] The single-channel edge response map is input into the Sigmoid normalized exponential function, and a spatial boundary weight matrix is ​​generated by mapping the original response values ​​to a continuous interval of [0,1]. In the matrix, the closer a pixel value is to 1, the higher the confidence that the pixel is a polyp boundary; the closer it is to 0, the higher the confidence that the pixel is not a boundary, thus achieving pixel-level confidence quantification of polyp boundary regions.

[0056] The formula for generating the boundary weight matrix of the generated space is as follows:

[0057] in, For the Sigmoid function, This is an edge response map.

[0058] 3.3 Construct a boundary interaction enhancement operator, which linearly scales the spatial boundary weight matrix according to a preset scaling factor and adds 1 to generate an enhancement coefficient matrix.

[0059] Design boundary interaction enhancement operators Where λ is a preset proportionality coefficient. This is the spatial boundary weight matrix; this operator adjusts the numerical range of the boundary weight matrix from [0,1] to [1,1.5] through linear scaling, generating the enhancement coefficient matrix. This design ensures the enhancement of features in the boundary region while avoiding excessive suppression of features in the non-boundary region, thus achieving adaptive enhancement and control of features in different regions.

[0060] 3.4. Perform element-wise multiplication of the enhancement coefficient matrix with the deep feature map to achieve adaptive feature recalibration at the boundary pixel position. Enhance the model's perception accuracy of blurred boundaries through inverse gain adjustment.

[0061] Enhancement coefficient matrix Deep feature maps output by local and global decoders Perform element-wise multiplication, that is... ,in This is an enhanced feature after feature recalibration; this operation achieves adaptive gain adjustment for boundary region features: for boundary pixels with high confidence, i.e. Approaching 1.5, its corresponding feature value is amplified by 1.5 times, enhancing the discriminative power of boundary features; for non-boundary pixels, i.e. When the value is close to 1, the corresponding feature value remains basically unchanged, preserving the original semantic information. Through this reverse gain adjustment mechanism, the model will automatically focus on the blurred boundary region of the polyp during the training process, significantly improving the perception and segmentation accuracy of low contrast and blurred boundaries, and effectively solving the core problem of confusion between polyps and surrounding mucosal boundaries in colonoscopy images.

[0062] S4, such as Figure 2 and Figure 4 As shown, by combining the model prediction results with the sparse graffiti prior, a rectangular box fusion strategy that shrinks linearly with the training cycle is executed to drive the SAM module to generate a fine mask, and unreliable noise is removed through a dynamic similarity filtering mechanism.

[0063] 4.1 Extract the first bounding rectangle of the graffiti annotation and the second bounding rectangle of the current prediction image of the model.

[0064] For extremely sparse graffiti annotations, they are first binarized, i.e., the graffiti annotation pixels are set to 1 and the background is set to 0. The contour information of the graffiti area is extracted by a contour detection algorithm, and then the minimum bounding rectangle is calculated based on the contour to obtain the first bounding rectangle. Its coordinate format is (in, , The coordinates of the top left corner , (The coordinates are the bottom right corner). Simultaneously, threshold segmentation is performed on the polyp prediction image currently output by the model, identifying pixels with a probability greater than 0.5 as predicted polyp regions. Similarly, contour detection and minimum bounding rectangle calculation are used to obtain the second bounding rectangle. This ensures that the coordinate system of the two bounding boxes is consistent with the input image, providing a foundation for subsequent bounding box fusion.

[0065] 4.2 Set a redundancy radius coefficient that decays linearly with the training process. In the early stage of training, maintain a large search boundary to cover the entire lesion area. In the middle and later stages of training, perform linear shrinkage processing as the number of iterations increases to improve the accuracy of the prompt information.

[0066] Define the redundancy radius coefficient The number of pixels used to expand the bounding box decreases dynamically and linearly with the training cycle: the total number of training rounds is set to 100, with the first 60 rounds of training... The pixel size is fixed at 25 pixels to ensure the prompt box covers the entire polyp area, avoiding missed lesions due to inaccurate predictions in the early stages of model training; after 60 training epochs, The pixel size is linearly reduced from 25 pixels to 10 pixels. By gradually reducing the redundancy range, the positional accuracy between the tooltip and the actual area of ​​the polyp is improved, and background noise interference is reduced. Meanwhile, to ensure a more conservative expansion of the second outer rectangle, its redundancy radius coefficient is set to [value missing]. This avoids the expanded bounding box from excessively including irrelevant background due to model prediction bias.

[0067] 4.3. Perform expansion processing on the first and second bounding rectangles, and calculate their overlapping intersection area. Use the overlapping intersection area as the adaptive prompt box that drives SAM.

[0068] Based on the redundancy radius coefficient determined in step 4.2, the first circumscribed rectangle is... Execution extension: For the second circumscribed rectangle Perform conservative extension: Then, the overlapping intersection area of ​​the two expanded boxes is calculated, which is the adaptive tooltip. If the calculated intersection region is empty, the expanded first bounding rectangle will be used by default. This serves as a prompt to ensure that the SAM module always obtains valid space constraints.

[0069] 4.4. The adaptive prompt box is used as the spatial constraint prior input SAM. The visual knowledge of the large model is used to infer and generate lesion candidate masks with high-fidelity edge response online.

[0070] A lightweight SAM model was selected as the mask generator. To adapt to the polyp segmentation scenario in colonoscopy, the SAM was fine-tuned: the cue encoder parameters were frozen to ensure the stability of the cue input; the last four layers of the image encoder and all parameters of the mask decoder were unfrozen to enable it to learn the polyp features in the medical scenario; and the adaptive cue box generated in step S43 was used... The image is converted to a format supported by SAM and used as a spatial cue input to the fine-tuned SAM model, along with the original colonoscopy image. SAM uses its pre-trained general visual knowledge and fine-tuned medical features to infer the complete outline of the polyp online and generate a single-channel lesion candidate mask. The mask has a pixel value of 0 or 1 and has high fidelity characteristics that fit the actual edge of the polyp, making up for the lack of boundary information under weak supervision annotation.

[0071] 4.5. Implement an image-level filtering mechanism, dynamically adjust the similarity threshold using a cosine annealing strategy, and automatically identify and remove unreliable noise samples below the dynamic threshold by calculating the overlap ratio between candidate masks and graffiti annotations.

[0072] Design a dynamic similarity threshold adjustment strategy based on cosine annealing, and define the dynamic threshold. Initially, the threshold is set to 0.75 to ensure that only highly reliable masks that highly overlap with the graffiti annotations are retained. After 30 rounds of training, cosine annealing is used to dynamically adjust the threshold, causing it to steadily decrease to 0.60 in the later stages of training, balancing the reliability and diversity of the masks. The overlap ratio between the candidate masks generated by SAM and the graffiti annotations is calculated, i.e., the Intersection over Union (IoU). If the calculated IoU value is lower than the current dynamic threshold, the mask is considered unsuitable. If the IoU value is higher than or equal to 1, the candidate mask is determined to be an unreliable noise sample and is discarded; if the IoU value is higher than or equal to 1, the candidate mask is discarded. If the mask is not found, it will be retained as a pseudo-label for subsequent supervised training. This filtering mechanism effectively avoids the negative interference of unreliable masks on model training and ensures the quality of the supervision signal.

[0073] S5, such as Figure 2 As shown, an uncertainty distribution map is constructed using pixel-level entropy values, and size-adaptive weight coefficients are introduced to perform multi-scale equalization boundary supervision. Through feedback of prediction results by pseudo-labels, closed-loop parameter updates and performance co-evolution of the segmentation network and guidance mechanism are achieved.

[0074] 5.1 Calculate the pixel-level information entropy value of the current segmentation probability distribution in real time to quantify the prediction confidence of the model in the spatial dimension and generate a dynamically updated spatial uncertainty weight distribution map.

[0075] After each round of forward propagation in the network, the polyp probability prediction map output by the main segmentation path is extracted. The information entropy is calculated for the predicted probability of each pixel, using the following formula: ,in For the first The probability of polyp prediction per pixel. To prevent the smoothing term with a value of 0 in the logarithmic operation, the information entropy value of each pixel is obtained through this formula. The higher the entropy value, the lower the confidence and the higher the uncertainty of the model's prediction of the class of that pixel, and vice versa. The entropy values ​​of all pixels are combined to form a spatial uncertainty distribution map. This distribution map is dynamically updated with each round of training and accurately reflects the reliability of the model's prediction of different regions in the current training stage.

[0076] 5.2. Real-time statistics of the total number of lesion boundary pixels in the candidate mask, and calculation of the area balance correction coefficient. The correction coefficient is negatively correlated with the square root of the total number of boundary pixels. The model's supervision weight for small polyp samples is improved through a weighted compensation mechanism.

[0077] Based on the reliable candidate mask generated and filtered by SAM, morphological operations are used to extract the edge pixels of the mask, and the total number S of edge pixels is counted in real time; let the area balance correction coefficient be set. Its calculation formula is ,in To avoid a smoothing term with a denominator of 0, this coefficient is negatively correlated with the square root of the total number of boundary pixels S. This means that small polyps have fewer boundary pixels, corresponding to a lower correction coefficient. Larger polyps have a higher total number of boundary pixels, corresponding to a higher correction factor. Smaller; this mechanism enables the compensation of supervision weights for small-sized polyp samples, solving the problem of insufficient supervision caused by the lack of boundary information of small polyps.

[0078] 5.3 Combining weighted cross-union loss and weighted binary cross-entropy loss, the loss value at the edge of the lesion is dynamically recalibrated using the spatial uncertainty weight distribution map.

[0079] First, we construct the basic segmentation loss, which consists of the weighted intersection-union ratio (wIoU) loss and the weighted binary cross-entropy (wBCE) loss, as shown in the formula: ;in By applying spatial weights to the crossover and union ratio loss, the monitoring of the polyp core region is enhanced. Weighted penalties are used to reduce training bias caused by class imbalance; subsequently, the spatial uncertainty distribution map generated in step S51 is used as the weight for loss recalibration, and the basic segmentation loss is weighted at the pixel level. ,in For element-wise multiplication, This is a spatial uncertainty weight distribution map; this operation allows the model to assign higher loss weights to the lesion edge region with high uncertainty during training, while maintaining normal supervision for the internal region with low uncertainty, thereby achieving dynamic recalibration of the loss value and guiding the model to prioritize optimizing the boundary segmentation accuracy.

[0080] 5.4 During model training, the structured consistency loss and weighted edge loss are integrated. While updating the segmentation network parameters, the image coding layer and mask decoding layer of SAM are simultaneously fine-tuned online.

[0081] First, the dynamic recalibration loss obtained in step S53, the structured consistency loss in step S2, and the weighted edge loss in step S3 are fused to construct a multi-task joint loss function, the formula of which is: ,in The main loss consists of recalibration segmentation loss, structured consistency loss, and weighted edge loss. For the loss of each auxiliary output, To assist in loss weighting, the gradient of the joint loss with respect to the parameters of the segmentation network and the SAM model is calculated using the backpropagation algorithm. For the segmentation network, the SGD optimizer is used to update all trainable parameters, while for the SAM model, only the parameters of the last four layers of the image encoder and the mask decoder are updated, achieving online synchronous fine-tuning of both. This collaborative training mechanism enables the segmentation network and the SAM model to promote each other, with the segmentation network providing accurate spatial cues for the SAM and the SAM providing reliable pseudo-label supervision for the segmentation network, continuously improving the overall segmentation performance of the model.

[0082] 5.5 When the model's evaluation metrics on the validation set reach the preset threshold, stop the iteration and output the final high-precision pixel-level binarized segmentation result image.

[0083] If all core evaluation metrics consistently exceed the preset thresholds in three consecutive rounds of validation without significant performance degradation, the model training is considered converged, and iterative training is stopped. After training stops, the optimal model weights from the validation set are loaded, forward propagation is performed on the test set images, and the output probability prediction map is binarized using a 0.5 threshold to obtain a pixel-level binarized segmentation result map. This result map has high-precision polyp contour and boundary representation, meeting the needs of clinical auxiliary diagnosis.

[0084] This embodiment utilizes the PyTorch deep learning framework to develop and validate the model. All training and inference processes are performed on a computing platform equipped with an NVIDIA GeForce RTX 3090 GPU. In terms of network architecture, the encoder employs a Res2Net-50 backbone network with pre-trained weights, extracting five levels of high-dimensional features through a multi-scale residual bottleneck layer. The decoder uses local and global decoders, fusing cross-layer features and capturing local and global contextual information, and is paired with an independent edge prediction head to generate boundary probability maps. The SAM module uses a lightweight ViT-B version, freezing the cue encoder and the first eight layers of the image encoder during training, and only unfreezing the last four layers of the image encoder and fine-tuning the parameters of all mask decoders. In terms of training strategy, the total number of training rounds was set to 100, and the input images were uniformly scaled to 320×320 pixels with a batch size of 12. For optimizer configuration, the segmentation network used the SGD optimizer, and the SAM fine-tuning module used the AdamW optimizer. A triangle warm-up and decay strategy was adopted for the learning rate, with the learning rate of the segmentation network backbone layer being 0.1 times the base learning rate, the head network using the base learning rate, and the SAM image encoder fine-tuning layer using a learning rate of 1e-5 and the mask decoder using 1e-4. Data augmentation only performed size standardization and normalization to preserve the original feature distribution of the medical images and avoid over-augmentation leading to segmentation bias.

[0085] This embodiment uses five publicly available colonoscopy polyp benchmark datasets—CVC-ClinicDB, Kvasir, CVC-300, ETIS-LaribPolypDB, and CVC-ColonDB—for experimental validation. These datasets cover polyp samples from different clinical scenarios. CVC-ClinicDB contains 612 high-resolution clinical colonoscopy images of 576×768 pixels, all with physician-annotated polyp segmentation masks. The samples cover polyps of various sizes and shapes, making it a core benchmark dataset for polyp segmentation. Kvasir contains 1000 gastrointestinal polyp images with resolutions ranging from [missing information - likely a range of resolutions]. Each image contains a segmentation mask and bounding box annotations. The sample sources are diverse, and the polyps exhibit significant differences in appearance, color, and texture, making it suitable for evaluating the model's generalization ability. CVC-300 contains 60 colonoscopy images of 500×574 pixels, with uniform polyp sample size and high annotation accuracy, suitable for fine-grained segmentation performance validation. ETIS-LaribPolypDB focuses on early colorectal polyps, containing 196 polyp instances with an image resolution of 966×1225 pixels. The polyp boundaries are blurred, and the contrast with the background is low, making it a key dataset for validating the model's handling of complex boundary scenarios. CVC-ColonDB contains 380 colonoscopy images of 500×574 pixels, covering complex clinical scenarios such as polyp adhesion to surrounding mucosa and occlusion. In terms of dataset partitioning and experimental design, this embodiment constructs a rigorous validation system. The training set consists of 900 images from Kvasir and 550 images from CVC-ClinicDB (a total of 1459 images), using extremely sparse doodle annotations for weakly supervised training. The validation set consists of the remaining 100 images from the Kvasir dataset, used for real-time monitoring of model performance and hyperparameter fine-tuning during training. The test set comprises one internal dataset for validation and three external datasets for validation. These consist of all images from three independent datasets: CVC-ClinicDB, CVC-300, ETIS-LaribPolypDB, and CVC-ColonDB. The aim is to leverage large-scale data for model building while simultaneously examining the model's zero-shot transfer capability and segmentation robustness under unknown distribution scenarios such as high-resolution blurred boundaries, varied clinical interference, and fine-grained accuracy requirements using completely independent external data.

[0086] To comprehensively evaluate the model's pixel classification accuracy, region overlap, boundary fit, and structural consistency in the polyp segmentation task, this embodiment selects the average Dice coefficient (Dice), average intersection-over-union ratio (IoU), and S-metric (S-metric). ), average E-measure ( ) and weighted F-metric ( These six core evaluation indicators are used as core evaluation indicators. The specific definitions and calculation formulas for these six core evaluation indicators are as follows: (1) Average Dice coefficient: measures the degree of overlap between the predicted region and the real region. It is the core evaluation index for medical image segmentation tasks, and its value range is [missing value]. The closer the value is to 1, the better the segmentation effect.

[0087]

[0088] Where TP represents true positives, FP represents false positives, and FN represents false negatives; (2) Average Intersection over Union (AUC): Calculates the ratio of the intersection of the predicted region and the actual region to the union, reflecting the accuracy of region segmentation. The range of values ​​is... .

[0089]

[0090] The definitions of TP, FP, and FN are consistent with those in the above-mentioned average Dice coefficient; (3) S-metric: A comprehensive evaluation of the region similarity and structural similarity of the segmentation results, with a value range of It balances global consistency with the matching degree of local details.

[0091]

[0092] in, For balance coefficient, For regional similarity, For structural similarity; (4) Average E-metric: An evaluation index based on edge structure similarity. It measures the segmentation quality by calculating the consistency between the predicted and the actual edge gradients of the mask. The value range is... .

[0093]

[0094] in, For edge gradient weights, To predict the mask in The pixel value of the location,

[0095] For the real mask in The pixel value of the location, To smooth out the terms and avoid denominators of 0; (5) Weighted F-metric: A comprehensive evaluation index that combines weighted precision and recall, focusing on the accurate identification of polyp regions. When β is set to 1, it degenerates into the standard F1 score.

[0096]

[0097] in, For weighted precision, This is the weighted recall rate.

[0098] To objectively verify the superiority of the proposed method in the weakly supervised segmentation task of colonoscopy polyps, this embodiment selects nine current mainstream weakly supervised semantic segmentation frontier algorithms and models designed for polyp segmentation as comparison benchmarks, specifically including: WSS, SCW, SCRE, WSL4MIS, SBANet, WSC, ME, SCNet, CFANet, and WeakPolyp-SAM are mentioned. Among them, WeakPolyp-SAM is a representative method that combines large-scale model prior knowledge and has highly competitive performance. This application further introduces active boundary interaction enhancement and multi-scale structural consistency alignment mechanisms on this basis. Tables 1-4 summarize and show the quantitative comparative experimental results of this application and the above benchmark models on the dataset. By comparing the values ​​of various evaluation indicators, the significant advantages of this application in handling complex lesions and improving segmentation accuracy and boundary quality can be intuitively reflected.

[0099] Table 1 Comparative Experiment Results of the CVC-ClinicDB Dataset

[0100] Table 1 presents the quantitative comparative experimental results on the CVC-ClinicDB dataset. As shown in Table 1, the proposed method significantly outperforms other mainstream weakly supervised algorithms in all core evaluation metrics on this dataset. Specifically, the Dice score, reflecting the overall polyp segmentation quality, reaches 0.891, an improvement of approximately 1.0% compared to the previous best method, WeakPolyp-SAM, and an improvement of over 10% compared to traditional weakly supervised methods such as SCNet. In the IoU metric, which measures the level of region overlap, the proposed method achieves an excellent score of 0.824, outperforming CFANet and SCNet, directly validating the effectiveness of the multi-scale consistency branch introduced in this application in achieving stable alignment when handling different lesion distributions. Furthermore, the proposed method also demonstrates superior performance in measuring structural similarity. The metric reached 0.915, reflecting the enhanced alignment level. The index achieved a high score of 0.959, a result that strongly supports the positive role of the edge prediction head-driven residual inverse gain adjustment in fuzzy boundary calibration in this application. Furthermore, weighted... The metric improved to 0.885, indicating that the model, driven by the synergy of adaptive prompt generation and multi-scale equalization boundary supervision, can effectively suppress noise interference in pseudo-labels and achieve a deep unification of segmentation accuracy and boundary quality.

[0101] Table 2 Comparative Experiment Results of the CVC-300 Dataset

[0102] Table 2 presents the quantitative comparative experimental results on the CVC-300 dataset. This dataset contains intestinal polyps with varying sizes and complex background interference, posing a significant challenge to the fine-grained extraction capabilities and cross-scale generalization performance of segmentation algorithms. As shown in Table 1, the proposed method significantly outperforms other mainstream weakly supervised algorithms on this dataset in all key metrics. The Dice score, reflecting the overall segmentation quality of polyps, reaches 0.902, representing an improvement of approximately 1.6% compared to the previous best method, WeakPolyp-SAM, and an improvement of over 15% compared to traditional weakly supervised methods such as WSS. In terms of the IoU metric, which measures the level of region overlap, the proposed method achieves an excellent score of 0.834, outperforming CFANet and SCNet, directly validating the effectiveness of the three-way scale-consistent alignment introduced in this application for stable alignment of multi-sized targets. Furthermore, the proposed method also demonstrates superior performance in measuring structural integrity. The metric reached 0.940, reflecting the level of enhanced alignment. The index achieved a high score of 0.977, a result that strongly supports the positive role of the edge prediction head-driven residual inverse gain adjustment in fuzzy boundary calibration in this application. Furthermore, the weighted... The metric improved to 0.886, indicating that the model, driven by the synergy of adaptive prompt generation and multi-scale supervised computation, can effectively suppress noise interference in pseudo-labels and achieve a deep unification of segmentation accuracy and boundary quality.

[0103] Table 3 Comparative experimental results of the ETIS-LaribPolypDB dataset

[0104] Table 3 presents the quantitative comparative experimental results on the ETIS-LaribPolypDB dataset. Because this dataset focuses on early colorectal polyps, the images exhibit extremely high boundary blurring and very low contrast between the lesion and the background, posing an extremely stringent challenge to the algorithm's detail capture capabilities. As shown in Table 2, the proposed method demonstrates a more significant leading advantage in this challenging scenario, achieving a Dice score of 0.730, an improvement of approximately 3.9% compared to the previously well-performing WeakPolyp-SAM, and a substantial leap in accuracy compared to SBANet. On the IoU metric, which reflects regional overlap, the proposed method achieved a score of 0.641, significantly outperforming CFANet and SCNet. This fully validates the effectiveness of the adaptive cue generation mechanism introduced in this application when handling extremely blurry targets, guiding the SAM model to extract higher-fidelity pseudo-labels in extremely low-contrast regions through a linearly shrinking redundancy radius strategy. Furthermore, the method reflecting structural similarity... The index reached 0.841, which measures the level of structural alignment. The indicator rose to 0.874, weighted. The index improved to 0.681, which strongly supports the calibration effect of the edge prediction head in complex boundary scenes in this application. It proves that under the synergistic drive of entropy weight guidance and area balance coefficient, the model can perform adaptive supervision on small and blurry polyp boundaries, thereby significantly enhancing the model's ability to capture irregular polyp contours.

[0105] Table 4 Comparative Experiment Results of the CVC-ColonDB Dataset

[0106] Table 4 presents the quantitative comparative experimental results on the CVC-ColonDB dataset. This dataset covers complex clinical challenges such as polyp adhesions and occlusions with surrounding mucosa, placing higher demands on the algorithm's robustness and detail recognition capabilities under complex interference. As shown in Table 3, the proposed method achieves state-of-the-art performance across all core metrics, with a Dice score of 0.749, representing an improvement of approximately 4.9% compared to the leading WeakPolyp-SAM, and a significant lead over the traditional weakly supervised model CFANet. On the IoU metric, reflecting the overlap between the predicted map and the ground truth label, the proposed method achieves a score of 0.667, significantly higher than SCNet and WSC. This fully validates the generalization accuracy of the three-way scale consistency alignment mechanism introduced in this application when handling irregular shapes such as mucosal occlusion. Simultaneously, the method reflecting structural similarity... The index reached 0.876, which measures the level of structural alignment. The index improved to 0.839, a result that strongly supports the technical advantage of the edge prediction head in performing precise spatial calibration of lesion boundaries through inverse gain adjustment. Furthermore, the weighted average... The index improved to 0.719, demonstrating that the model, under the combined effect of adaptive boundary supervision and online evolution mechanism of pseudo-labels, can effectively cope with contrast fluctuations in complex clinical environments and achieve stable identification and accurate segmentation of lesion areas.

[0107] like Figure 6-8 As shown in the image, a visualization comparison of the proposed method with techniques such as WeakPolyp-SAM in complex colonoscopy polyp scenarios is presented. GT represents the reference label image precisely annotated by the physician. The red outline in the image highlights the true boundaries of the polyps, emphasizing tiny polyps, mucosal obscuring areas, and blurred boundary details in the intestinal environment that are easily missed or misjudged. The comparison results show that, under complex clinical background interference, while all comparison models can identify relatively obvious polyp entities, the proposed method significantly outperforms WeakPolyp-SAM and other comparison methods in terms of the accuracy of lesion edge depiction and overall structural integrity. Observing the segmentation details corresponding to the red boundary lines in the image reveals that the WeakPolyp-SAM model often exhibits segmentation omissions and severe over-segmentation when facing small targets or polyps with colors extremely similar to the surrounding mucosa. The generated masks often significantly exceed or deviate from the true red outline range. In contrast, the proposed method, benefiting from the feature recalibration of boundary pixels by the edge prediction head and the compensation of supervisory weights for small samples by the area balance coefficient, can accurately restore the continuity of fine structures and effectively avoid missed detections. Meanwhile, due to the light reflection and mucus in the intestine, existing methods often generate false detections and spurious targets in non-lesion areas. This application's method effectively suppresses these artifacts by implementing dynamic cue enhancement and similarity filtering mechanisms, demonstrating stronger generalization ability and robustness in complex scenarios. Comparison reveals that the mask generated by this application closely matches the red outline of the ground truth (GT) at the boundary. This intuitively verifies the technical advantage of the three-way scale consistency alignment mechanism introduced in this application when handling irregular shapes, solving the common boundary overflow or local blurring problems in weakly supervised learning. In summary, the visualization results intuitively demonstrate the technical superiority of this application in maintaining the authenticity of polyp physical distribution and suppressing clinical environmental noise interference.

[0108] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent substitutions, and improvements made to the above embodiments without departing from the scope of the present invention, based on the technical essence of the present invention and within the spirit and principles of the present invention, shall still fall within the protection scope of the present invention.

Claims

1. A polyp segmentation method based on SAM (Single Aspect-Oriented Merge-Action) boundary enhancement and uncertainty-guided method, characterized in that... Includes the following steps: S1. Input the preprocessed endoscope image into the Res2Net-50 encoder for feature extraction, and obtain five levels of features from low-level spatial details to high-level semantic information. S2. Use the cross-layer fusion module to perform bidirectional collaborative enhancement on adjacent layer features in the five layers, simultaneously construct multi-scale consistency branches, and perform structural consistency constraints on inputs with different resolutions. S3. In the decoding stage, an independent edge prediction head is introduced to generate an edge probability map. The spatial weight matrix generated by the edge probability map is used to perform residual inverse gain adjustment on the cross-layer aggregated features. S4. Combining the model prediction results with the extremely sparse graffiti prior, a rectangular box fusion strategy that shrinks linearly with the training cycle is executed to drive the SAM module to generate a fine mask, and unreliable noise is removed through a dynamic similarity filtering mechanism. S5. Construct an uncertainty distribution map using pixel-level entropy values ​​and introduce size-adaptive weight coefficients to perform multi-scale equalization boundary supervision; achieve closed-loop parameter updates and performance co-evolution of the segmentation network and guidance mechanism through feedback of prediction results via pseudo-labels.

2. The polyp segmentation method based on SAM and guided by uncertainty as described in claim 1, characterized in that: Step S1 specifically involves: 1.1 Perform size standardization and normalization on the original endoscopic images to make them conform to the input distribution requirements of the backbone network; 1.

2. Five feature maps F1 to F5 with different spatial resolutions were extracted using the bottleneck layer structure of Res2Net-50; 1.

3. Extract the preliminary prediction map S1 generated by the network in real time during forward propagation. Determine the dynamic prediction box of the lesion by calculating the probability envelope range of the preliminary prediction map S1. Use the dynamic prediction box as the core constraint to drive the adaptive prompt generation mechanism to realize the online evolution of pseudo-labels.

3. The polyp segmentation method based on SAM and guided by uncertainty as described in claim 2, characterized in that: Step 1.2 specifically involves: using Res2Net-50 as the feature extraction encoder, and through the multi-scale residual bottleneck layer structure of the feature extraction encoder, performing forward propagation on the preprocessed endoscopic image, sequentially outputting feature maps F1 to F5 at five levels, with feature resolutions of [missing information]. 、 Where W and H are the width and height of the input endoscopic image, respectively, and the number of channels are 64, 256, 512, 1024, and 2048, respectively; the low-level features F1 and F2 retain the spatial details of the polyp; the high-level features F4 and F5 contain the deep information of the polyp; and the middle-level feature F3 achieves the transition and fusion of details and semantics.

4. The polyp segmentation method based on SAM and guided by uncertainty as described in claim 1, characterized in that: Step S2 specifically involves: 2.

1. The cross-layer fusion module is used to concatenate low-level features and high-level features of adjacent resolutions, and the spatial attention mechanism is used to achieve mutual enhancement and semantic complementarity of feature maps at the pixel level. 2.2 During the training loop, the original endoscopic images are downsampled by 0.75x and 0.5x respectively, and the generated three-dimensional multi-scale images are synchronously input into the segmentation network with shared weights. 2.3 Calculate the significant structural consistency loss between the original scale prediction map and the prediction maps at each scaling scale, and force the model to maintain consistent recognition of polyp morphology and topology under different scaling ratios; 2.

4. Combine the training cycle to dynamically adjust the loss weights of each scale branch, guide the model to focus on global structure alignment in the early stage of training, and shift to cross-scale alignment of lesion boundary details in the later stage of training.

5. The polyp segmentation method based on SAM and guided by uncertainty as described in claim 4, characterized in that: Step 2.1 specifically involves: analyzing the adjacent layer feature pairs output by the Res2Net-50 encoder. and First, the high-level characteristics Upsampling to low-level features via bilinear interpolation With the same resolution, 3×3 convolution operations are then applied to both feature layers to unify channel dimensions and map features, resulting in pre-processed low-level features. Characteristics of high-level personnel Next, the low-level features Characteristics of high-level personnel Another set of 3×3 convolutional layers is input, and two sets of pixel-level spatial attention weight maps are generated by applying the Sigmoid activation function. The generated spatial attention weight maps are then used to apply the low-level features. Characteristics of high-level personnel Pixel-level mutual enhancement operations are performed to achieve complementary enhancement of low-level detail features and high-level semantic features. Finally, the enhanced features from the two layers are concatenated by fusing features through convolutional blocks containing convolution, batch normalization, and ReLU activation. Residual connections are introduced to preserve the original feature information, and the enhanced features after cross-layer fusion are output. Enhanced features It possesses both rich spatial details and clear semantic direction; For low-level features Characteristics of high-level personnel The formula for performing pixel-level mutual enhancement operations is as follows: in, For Sigmoid mutual enhancement operations, This is element-wise multiplication.

6. The polyp segmentation method based on SAM and guided by uncertainty as described in claim 1, characterized in that: Step S3 specifically involves: 3.

1. A convolutional layer is connected to the bottom output of the local and global decoders as an edge prediction head to transform the high-dimensional fusion features into a single-channel edge response map that reflects the probability of the existence of polyp boundaries. 3.

2. The edge response is mapped to a continuous space between 0 and 1 using the normalized exponential function to obtain the spatial boundary weight matrix representing the confidence of a pixel belonging to the boundary. 3.3 Construct a boundary interaction enhancement operator, which linearly scales the spatial boundary weight matrix according to a preset scaling factor and increments it by 1 to generate an enhancement coefficient matrix; 3.

4. Perform element-wise multiplication of the enhancement coefficient matrix with the deep feature map to achieve adaptive feature recalibration at the boundary pixel position. Enhance the model's perception accuracy of blurred boundaries through inverse gain adjustment.

7. The polyp segmentation method based on SAM and guided by uncertainty as described in claim 6, characterized in that: Step 3.2 specifically involves: inputting the single-channel edge response map into the Sigmoid normalized exponential function, and generating a spatial boundary weight matrix by mapping the original response values ​​to the continuous interval [0,1]. ; The closer a pixel value is to 1 in the matrix, the higher the confidence that the pixel is a polyp boundary; the closer it is to 0, the higher the confidence that the pixel is not a boundary. This achieves pixel-level confidence quantification of polyp boundary regions. The formula for generating the boundary weight matrix of the generated space is as follows: in, For the Sigmoid function, This is an edge response map.

8. The polyp segmentation method based on SAM and guided by uncertainty as described in claim 1, characterized in that: Step S4 specifically involves: 4.1 Extract the first bounding rectangle of the graffiti annotation and the second bounding rectangle of the model's current prediction image, respectively; 4.2 Set a redundancy radius coefficient that decays linearly with the training process. In the early stage of training, maintain a large search boundary to cover the entire lesion area. In the middle and later stages of training, perform linear shrinkage processing as the number of iterations increases. 4.

3. Perform expansion processing on the first and second bounding rectangles, and calculate their overlapping intersection area. Use the overlapping intersection area as the adaptive prompt box that drives SAM. 4.

4. Using the adaptive cue box as the spatial constraint prior input SAM, the visual knowledge reserves of the large model are used to infer and generate lesion candidate masks with high-fidelity edge response online; 4.

5. Implement an image-level filtering mechanism, dynamically adjust the similarity threshold using a cosine annealing strategy, and automatically identify and remove unreliable noise samples below the dynamic threshold by calculating the overlap ratio between candidate masks and graffiti annotations.

9. The polyp segmentation method based on SAM and guided by uncertainty as described in claim 8, characterized in that: Step 4.3 specifically involves: based on the redundancy radius coefficient determined in step 4.2, adjusting the first circumscribed rectangle... Execution extension: For the second circumscribed rectangle Perform conservative extension: Then, the overlapping intersection area of ​​the two expanded boxes is calculated, which is the adaptive tooltip. ; If the calculated intersection region is empty, the expanded first bounding rectangle will be used by default. This serves as a prompt to ensure that the SAM module always obtains valid space constraints.

10. The polyp segmentation method based on SAM and guided by uncertainty as described in claim 1, characterized in that: Step S5 specifically involves: 5.1 Calculate the pixel-level information entropy value of the current segmentation probability distribution in real time to quantify the prediction confidence of the model in the spatial dimension and generate a dynamically updated spatial uncertainty weight distribution map; 5.

2. Real-time statistics of the total number of lesion boundary pixels in the candidate mask, and calculation of the area balance correction coefficient. The correction coefficient is negatively correlated with the square root of the total number of boundary pixels. The model's supervision weight for small polyp samples is improved through a weighted compensation mechanism. 5.3 Combining weighted intersection-union loss and weighted binary cross-entropy loss, the loss value at the edge of the lesion is dynamically recalibrated using the spatial uncertainty weight distribution map; 5.4 During model training, structured consistency loss and weighted edge loss are integrated to perform online synchronous fine-tuning of the image coding layer and mask decoding layer of SAM while updating the segmentation network parameters; 5.5 When the model's evaluation metrics on the validation set reach the preset threshold, stop the iteration and output the final high-precision pixel-level binarized segmentation result image.

Images