Large model domain self-adaption-based polyp segmentation method and system

Abstract

The invention discloses a large model domain self-adaption-based polyp segmentation method and system. The method comprises the steps of preprocessing a polyp image to obtain a to-be-segmented polyp data set; inputting the to-be-segmented polyp data set into a field adaptive feature encoder to extract an initial multi-scale feature picture; performing feature enhancement on the initial multi-scale feature picture through a SimAM adapter module to obtain a multi-scale feature picture; inputting the multi-scale feature picture into an inter-scale interactive attention module for cross-scale feature fusion to obtain a fused feature picture; and inputting the fused feature picture into a high and low frequency enhanced decoder to carry out adaptive frequency domain decomposition and Kolmogorov-Arnod channel integration to obtain a polyp segmentation output picture. According to the method, a lightweight SimAM adapter is designed, an inter-scale interactive attention module and a high and low frequency enhancement decoder are constructed, compared with various mainstream methods, the method is remarkably improved, and a segmentation result shows unique advantages in the aspects of complex background suppression and fuzzy boundary processing.

Description

Technical Field

[0001] The present invention relates to the technical field of image segmentation, and in particular to a polyp segmentation method and system based on large model domain adaptation. Background Art

[0002] Colorectal cancer is the third most common malignant tumor worldwide, with an annual incidence rate of 3.4%. Epidemiological studies indicate that approximately 90% of colorectal cancer cases originate from polyps in the colon lining. This makes early detection and accurate segmentation of polyps in endoscopic images a key clinical approach to reducing colorectal cancer mortality. However, traditional manual interpretation suffers from inherent drawbacks such as low efficiency and significant subjective variability, necessitating the development of automated segmentation technology.

[0003] Early research primarily relied on morphological feature analysis and region growing algorithms. While these methods achieved basic results in simple scenarios, their reliance on artificially designed features limited their generalization performance in complex scenarios (e.g., mucosal texture variation and uneven illumination). With the advancement of deep learning, convolutional neural networks, represented by U-Net, have been widely used in medical image segmentation tasks. They fuse multi-scale features through skip connections, significantly improving segmentation accuracy. Subsequent improvements, such as ResUNet++, enhanced the U-Net architecture by introducing residual connections, channel-spatial attention gates, and dense pyramid pooling, thereby improving boundary accuracy and noise robustness in medical image segmentation. PraNet proposed an inverse attention module that iteratively erases predicted regions and uses inverted masks to optimize edges that are ignored in colonoscopy images, thereby reducing false positives. CFA-Net utilizes a cross-layer aggregation mechanism based on deformable convolutions and a feature calibration module with dual attention, effectively addressing class imbalance and enhancing the ability to fuse multi-scale features for breast tumor and skin lesion segmentation. BMANet utilizes a cascaded partial decoder (CPD) to aggregate high-level semantic features, combined with a boundary-aware module (BAM) to generate discriminative boundary features, and a boundary-guided multi-scale attention module (BMA) to globally coordinate context and local boundary details. However, these traditional CNN methods are limited by their local receptive field and struggle to model long-range spatial dependencies. In recent years, the Visual Transformer (ViT) has emerged in the field of medical image segmentation due to its powerful ability to capture long-range dependencies. The Contrastive Transformer Network (CTNet) proposed by Xiao et al. combines a contrastive learning mechanism with the Transformer architecture, achieving high-accuracy detection in many cases. Although the Transformer excels at modeling global context, its excessive focus on global features and neglect of fine-grained spatial location information results in weak local feature extraction. To address this shortcoming, researchers have begun exploring hybrid CNN and Transformer architectures to collaboratively represent local and global information. TransUNet, proposed by Chen et al., is the first CNN-Transformer hybrid model for medical image segmentation. By injecting features from the last CNN layer into the Transformer module to strengthen local information representation, it achieves superior segmentation performance compared to traditional CNN methods. The D2polyp-Net developed by Fu et al. uses a dual-pyramid spatially guided localization framework combined with a cross-modal paired training strategy to achieve high-precision real-time detection while effectively addressing the limitations of white-light endoscopic images in polyp classification. However, the feature learning capabilities of this hybrid approach remain insufficient, limiting its generalization performance on unseen data and posing challenges for clinical application.

[0004] The recent rise of large-scale base models has provided a new paradigm for medical image analysis. SAM2 (SAM2), a milestone in general image segmentation, achieves zero-shot generalization through hint engineering. However, directly migrating SAM2 to polyp segmentation presents significant challenges. First, there is the domain drift problem: medical images and natural images exhibit significant interdomain differences, manifested in low-contrast tissue features, subtle texture variations, and noise distribution characteristics, leading to feature degradation in the model. Second, there are the issues of insufficient scale-feature interaction and boundary sensitivity: multi-layer downsampling operations can easily lead to feature loss in small polyps, while full parameter fine-tuning strategies are not only computationally expensive but can also lead to catastrophic forgetting. While existing adapter methods attempt to infuse domain knowledge through efficient parameter fine-tuning, they have yet to adequately address the multi-scale feature interaction mechanism and boundary sensitivity. Summary of the Invention

[0005] The embodiments of the present invention provide a polyp segmentation method and system based on large model domain adaptation, which are used to solve the domain drift problem, insufficient scale feature interaction and boundary sensitivity problems caused by directly migrating SAM2 to the polyp segmentation task in the prior art.

[0006] On the one hand, an embodiment of the present invention provides a polyp segmentation method based on large model domain adaptation, comprising: Preprocess the polyp image to obtain the polyp dataset to be segmented; Inputting the to-be-segmented polyp dataset into a domain adaptive feature encoder for downsampling to extract an initial multi-scale feature image of the to-be-segmented polyp dataset; Performing feature enhancement, dimension adaptation, and knowledge retention on the initial multi-scale feature image through a SimAM adapter module to obtain a multi-scale feature image; Inputting the multi-scale feature image into the inter-scale interactive attention module to perform cross-scale feature fusion to obtain a fused feature image; The fused feature image is input into a high- and low-frequency enhancement decoder for adaptive frequency domain decomposition and Kolmogorov-Arnold channel integration to obtain a polyp segmentation output image.

[0007] In one possible implementation, performing feature enhancement, dimension adaptation, and knowledge retention on the initial multi-scale feature image through the SimAM adapter module to obtain the multi-scale feature image includes: Introducing a lightweight attention mechanism through an efficient parameter fine-tuning strategy; The multi-scale feature image is obtained by performing feature enhancement, dimension adaptation and knowledge retention on the initial multi-scale feature image through fine-tuning parameters.

[0008] In one possible implementation, the SimAM adapter module is a symmetric encoding-decoding architecture, the SimAM adapter directly infers 3D attention weights by optimizing an energy function while considering both channel and spatial dimensions, and the SimAM adapter module measures the linear separability of target neurons and surrounding neurons by defining an energy function.

[0009] In a possible implementation, inputting the multi-scale feature image into the inter-scale interactive attention module to perform cross-scale feature fusion to obtain a fused feature image further includes: The inter-scale interactive attention module performs bilinear interpolation upsampling on the low-resolution image of the multi-scale feature image to enhance it; The inter-scale interactive attention module generates query, key and value projections for the high-resolution image of the multi-scale feature image through three 1×1 convolutions to preserve details.

[0010] In a possible implementation, the fused feature image is input into a high- and low-frequency enhancement decoder for adaptive frequency domain decomposition and Kolmogorov-Arnold channel integration to obtain a polyp segmentation output image, including: The high and low frequency enhancement decoder includes a frequency perception feature fusion module and a channel integration module; The high and low frequency enhancement decoder enhances the consistency and boundary clarity of the fused feature image through a frequency perception feature fusion module and a channel integration module.

[0011] On the other hand, an embodiment of the present invention provides a polyp segmentation system based on large model domain adaptation, including: The data set module is used to pre-process the polyp image to obtain the polyp data set to be segmented; A domain adaptive feature encoder module is used to downsample the polyp dataset to obtain an initial multi-scale feature image of the polyp dataset to be segmented; A SimAM adapter module is used to perform feature enhancement, dimension adaptation and knowledge retention on the initial multi-scale feature image to obtain a multi-scale feature image; an inter-scale interactive attention module, configured to perform cross-scale feature fusion on the multi-scale feature image to obtain a fused feature image; The high and low frequency enhancement decoder module is used to perform adaptive frequency domain decomposition and Kolmogorov-Arnold channel integration on the fused feature image to obtain a polyp segmentation output image.

[0012] The polyp segmentation method and system based on large model domain adaptation in the present invention have the following advantages: A lightweight SimAM adapter was designed, and an inter-scale interactive attention module and a high- and low-frequency enhancement decoder were constructed, which achieved significant improvements compared to many mainstream methods. The segmentation results also showed unique advantages in complex background suppression and blurred boundary processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0014] Figure 1 A flowchart of a polyp segmentation method based on large model domain adaptation provided in an embodiment of the present application; Figure 2 A schematic diagram of the SimAM adapter process framework for a large model domain adaptive polyp segmentation method provided in an embodiment of the present application; Figure 3 A schematic diagram of the process framework of an inter-scale interactive attention (ISIA) module of a large-model domain adaptive polyp segmentation method provided in an embodiment of the present application; Figure 4 A schematic diagram of the process framework of a high-low frequency enhancement (HLFE) decoder of a polyp segmentation method based on large model domain adaptation provided in an embodiment of the present application; Figure 5 This is a segmentation result diagram obtained by the domain-adaptive context-enhanced visual basic model SAM2-PolypNet based on a large model domain-adaptive polyp segmentation method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0015] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0016] Figure 1 A flow chart of a polyp segmentation method based on large model domain adaptation provided by an embodiment of the present invention; an embodiment of the present invention provides a polyp segmentation method based on large model domain adaptation, comprising: 1. A polyp segmentation method based on large model domain adaptation, characterized in that it comprises: Preprocess the polyp image to obtain the polyp dataset to be segmented; Inputting the to-be-segmented polyp dataset into a domain adaptive feature encoder for downsampling to extract an initial multi-scale feature image of the to-be-segmented polyp dataset; Performing feature enhancement, dimension adaptation, and knowledge retention on the initial multi-scale feature image through a SimAM adapter module to obtain a multi-scale feature image; Inputting the multi-scale feature image into the inter-scale interactive attention module to perform cross-scale feature fusion to obtain a fused feature image; The fused feature image is input into a high- and low-frequency enhancement decoder for adaptive frequency domain decomposition and Kolmogorov-Arnold channel integration to obtain a polyp segmentation output image.

[0017] The preprocessing of the polyp image to obtain the polyp dataset to be segmented includes: Adjusting the size of the polyp image to obtain a polyp image of uniform size; Normalization is performed on the polyp images of uniform size to obtain a data set of polyps to be segmented.

[0018] Inputting the to-be-segmented polyp dataset into a domain adaptive feature encoder to perform downsampling to extract an initial multi-scale feature image of the to-be-segmented polyp dataset further includes: The domain adaptive feature encoder is used as the backbone network to perform fourth-order hierarchical downsampling on the polyp dataset to be segmented to obtain an initial multi-scale feature image.

[0019] The multi-scale feature image is obtained by performing feature enhancement, dimension adaptation and knowledge retention on the initial multi-scale feature image through the SimAM adapter module, including: Introducing a lightweight attention mechanism through an efficient parameter fine-tuning strategy; The multi-scale feature image is obtained by performing feature enhancement, dimension adaptation and knowledge retention on the initial multi-scale feature image through fine-tuning parameters.

[0020] Inputting the multi-scale feature image into the inter-scale interactive attention module to perform cross-scale feature fusion to obtain a fused feature image further includes: The inter-scale interactive attention module performs bilinear interpolation upsampling on the low-resolution image of the multi-scale feature image to enhance it; The inter-scale interactive attention module generates query, key and value projections for the high-resolution image of the multi-scale feature image through three 1×1 convolutions to preserve details.

[0021] The fused feature image is input into the high and low frequency enhancement decoder for adaptive frequency domain decomposition and Kolmogorov-Arnold channel integration to obtain the polyp segmentation output image including: The high and low frequency enhancement decoder includes a frequency perception feature fusion module and a channel integration mode; The high and low frequency enhancement decoder enhances the consistency and boundary clarity of the fused feature image through a frequency perception feature fusion module and a channel integration module.

[0022] The frequency perception feature fusion module includes an adaptive low-pass filter generator, an offset generator and an adaptive high-pass filter generator.

[0023] The SimAM adapter module is a symmetric encoding-decoding architecture. The SimAM adapter directly infers 3D attention weights by optimizing an energy function while considering both channel and spatial dimensions. The SimAM adapter module measures the linear separability of target neurons and surrounding neurons by defining an energy function.

[0024] For example, the visual base model SAM2-PolypNet of this application mainly consists of a domain adaptive feature encoder, an inter-scale interactive attention module, a high- and low-frequency enhancement decoder, and a SimAM adapter. The domain adaptive feature encoder based on SAM2 is used as the backbone network, and the input image is subjected to four-stage hierarchical downsampling (step size = 4, 8, 16, 32) to extract multi-scale features ( , , , Each encoding stage is followed by a parameter-lightweight SimAM adapter (the total number of parameters accounts for only 0.8% of the base model), which alleviates the domain shift problem between medical images and natural images through dynamic channel calibration and energy attention mechanism. Secondly, the feature map output by the encoding block ( , , , ) are fed into the ISIA module. Through the interaction and fusion of cross-scale features, the model's perception of polyp regions is enhanced, enabling precise capture of polyp details and understanding of global context. Subsequently, the enhanced feature map is fed into the HLFE decoder, where adaptive frequency domain decomposition and Kolmogorov-Arnold channel integration ensure that the decoded features incorporate both global semantics and local detail information. Finally, the feature maps (Fd1, Fd2, and Fd3) output by the decoder are passed through the readout layer to generate the final segmentation output map. This entire process, through adaptive adjustment by the SimAM adapter, enhances the model's ability to capture key information and improves segmentation accuracy and robustness.

[0025] This application is a visual basic model based on domain adaptive context enhancement. When used, multiple polyp images to be detected are first input into the segmentation network of SAM2-Polyp, and then the feature map output by the decoder is obtained through the readout layer to obtain the final segmentation output map.

[0026] Inputting multiple polyp images to be detected into the segmentation network of SAM2-Polyp includes: The domain adaptive feature encoder is used as the backbone network to extract multi-scale features from the input image through four-stage hierarchical downsampling (step size = 4, 8, 16, 32) , , , ), and at the same time, a lightweight SimAM adapter module is designed. The core idea is to introduce a lightweight attention mechanism through an efficient parameter fine-tuning strategy, and achieve the three goals of feature enhancement, dimension adaptation, and knowledge retention under the premise of fine-tuning only 0.8% of the parameters.

[0027] Then, the feature map output by the encoding block ( , , , ) is fed into the scale-interactive attention module, and the high-resolution features Contains rich spatial details but limited semantic information, and low resolution The features extracted by the deep network have strong semantic representation but the spatial information is seriously lost. Perform bilinear interpolation upsampling to obtain Three 1×1 convolutions are used to generate query, key, and value projections respectively, and a learnable attention mechanism is used to guide low-resolution semantic features to enhance their content while preserving the details of the high-resolution feature space.

[0028] Then the feature map enhanced by the scale-interactive attention module ( , , ) is fed into the high and low frequency enhancement decoder through adaptive frequency domain decomposition ( ) is integrated with the Kolmogorov-Arnold channel to ensure that the decoded features have both global semantics and local detail information.

[0029] The model was optimized using the AdamW optimizer with an initial learning rate of 7.5×10 4, and a cosine decay scheduling strategy with preheating is used to stabilize training convergence. The batch size and training epochs are set to 8 and 50, respectively. All experiments are performed on a hardware environment consisting of an NVIDIA RTX 4090D GPU (24GB video memory), an AMD EPYC 9754 128-core CPU, and 80GB of system memory. The code is implemented on an Ubuntu 22.04 LTS system, using the PyTorch 2.1.0 framework, accelerated by CUDA 12.1, and enabling PyTorch mixed precision training. The network weights are initialized with the SAM2 pre-trained model, and end-to-end training is performed using a hybrid loss function that combines Dice loss and cross entropy loss to jointly optimize region consistency and boundary localization accuracy.

[0030] When the size of the polyp image is adjusted to obtain a uniform size, the size of each set of input features is ensured to be the same.

[0031] The SimAM adapter of this application adopts a symmetric encoding-decoding architecture, which is mathematically expressed as:

[0032] in The input features are represented by Down, GeLU, SimAM, and UP. The Down layer uses 1×1 convolution to achieve channel compression, reducing the number of parameters and computation required for fine-tuning. The Up layer uses 1×1 convolution to restore the dimension. The GeLU activation function uses a continuously differentiable nonlinear transformation:

[0033] SimAM directly infers 3D attention weights (considering both channel and spatial dimensions) by optimizing an energy function without introducing additional parameters. SimAM measures the linear separability of the target neuron from its surrounding neurons by defining an energy function. For each neuron in the input feature map, the energy function is defined as:

[0034] By optimizing the function through closed-form solution, the minimum energy value is obtained: Energy value The smaller it is, the more significant the neuron is. The inverse of the energy is normalized to generate 3D attention weights for feature recalibration:

[0035]

[0036] The inter-scale interactive attention module uses a learnable attention mechanism to guide low-resolution semantic features to enhance their content while preserving the details of the high-resolution feature space. This effectively solves the problem of mis-segmentation of blurred areas around polyp edges. The specific steps are as follows: Given input high-resolution features and low-resolution features , first of all Perform bilinear interpolation upsampling to obtain The query, key, and value projections are generated by three 1×1 convolutions respectively:

[0037] in 、 、 is the learnable parameter matrix, is the channel compression ratio. Then the attention weight matrix is ​​generated by normalizing along the feature dimension through the softmax function:

[0038] The final output features are enhanced by attention weighting and residual connection. :

[0039] The inter-scale interactive attention module achieves effective matching of heterogeneous features through resolution alignment and channel compression, and uses low-resolution semantic features to guide the generation of attention weights, thereby enhancing the representation ability of fuzzy boundaries.

[0040] The high- and low-frequency enhancement decoder enhances feature consistency and boundary clarity through a frequency-aware feature fusion module and a channel integration module. The core idea of ​​the frequency-aware feature fusion module is to optimize the feature fusion process through high- and low-frequency analysis and processing. It includes three key components: an adaptive low-pass filter generator, an offset generator, and an adaptive high-pass filter generator.

[0041] in represents the high-resolution features enhanced by the inter-scale interactive attention module, represents the low-resolution decoding features, Represents the features after fusion through the frequency-aware feature fusion module.

[0042] The Kolmogorov-Arnold theory is used to design a lightweight channel integration module to model channel interactions in a more efficient way while enhancing nonlinear expression capabilities. A global average pooling operation is performed along the spatial dimensions (i.e., height and width). Then, the weight matrix is ​​converted through the Kolmogorov-Arnold layer and the Sigmoid function.

[0043] The embodiment of the present invention further provides a polyp segmentation system based on large model domain adaptation, comprising: a data set module for preprocessing polyp images to obtain a data set of polyps to be segmented; A domain adaptive feature encoder module is used to downsample the polyp dataset to obtain an initial multi-scale feature image of the polyp dataset to be segmented; A SimAM adapter module is used to perform feature enhancement, dimension adaptation and knowledge retention on the initial multi-scale feature image to obtain a multi-scale feature image; an inter-scale interactive attention module, configured to perform cross-scale feature fusion on the multi-scale feature image to obtain a fused feature image; The high and low frequency enhancement decoder module is used to perform adaptive frequency domain decomposition and Kolmogorov-Arnold channel integration on the fused feature image to obtain a polyp segmentation output image.

[0044] For example, this network first uses the SimAM adapter module to perform feature enhancement, dimension alignment, and knowledge preservation on the initial multi-scale feature map, resulting in a multi-scale feature map with a uniform number of channels. The inter-scale interactive attention module then effectively fuses information across scales to generate a fused feature map. Finally, the high- and low-frequency enhancement decoder module performs frequency domain decomposition and channel integration on the fused feature map, enhancing edge and structural information to generate an accurate polyp segmentation map. This achieves end-to-end optimization from multi-scale feature extraction to high-quality segmentation.

[0045] In a possible embodiment, the domain-adaptive context-enhanced visual base model of the present application specifically includes the following steps: Step 1: Construct polyp segmentation training samples, including training set and test set.

[0046] Step 2: Select AdamW optimizer, and all images are uniformly scaled to 512×512×3.

[0047] Step 3: Build a visual base model based on domain-adaptive contextual enhancement, including a domain-adaptive feature encoder, an inter-scale interactive attention module, and a high- and low-frequency enhancement decoder. The corresponding processing of the domain-adaptive feature encoder, the inter-scale interactive attention module, and the high- and low-frequency enhancement decoder is as follows: Evaluation was performed on three publicly available polyp datasets: Kvasir-SEG, ETIS-LaribPolypDB, and CVC-ColonDB. Following previous work, the training set consisted of 900 images from Kvasir-SEG and 550 images from CVC-ClinicDB, for a total of 1,450 samples. The test set consisted of the remaining 100 images from Kvasir-SEG, the complete CVC-ColonDB dataset (380 samples), and the complete ETIS-LaribPolypDB dataset (196 samples). The Kvasir-SEG dataset contains approximately 1,000 colonoscopic images covering a wide range of polyp morphologies. The ETIS-LaribPolypDB dataset contains approximately 196 polyp images from colonoscopic images and endoscopic video frames.

[0048] Constructing a domain-adaptive feature encoder. SAM2 performs well in general segmentation tasks, but directly fine-tuning its large pre-trained parameters leads to excessive computational cost and the risk of catastrophic forgetting. To this end, this application designs a lightweight SimAM adapter module. The core idea is to introduce a lightweight attention mechanism through an efficient parameter fine-tuning strategy, achieving three goals while fine-tuning only 0.8% of the parameters: (1) Feature enhancement: Dynamically strengthen the feature response of the lesion area through SimAM attention; (2) Dimensionality adaptation: Match the low signal-to-noise ratio characteristics of medical images through dimensionality reduction / increase operations; (3) Knowledge preservation: Freeze the main model parameters to avoid destroying the pre-trained knowledge representation.

[0049] like Figure 2 As shown in Figure 1, the SimAM adapter of this application utilizes a symmetric encoder-decoder architecture. SimAM is a lightweight attention module based on neuroscience theory designed to enhance the feature representation capabilities of convolutional neural networks. Unlike traditional attention mechanisms (such as channel or spatial attention), SimAM directly infers 3D attention weights (considering both channel and spatial dimensions) by optimizing an energy function without introducing additional parameters. SimAM defines an energy function that measures the linear separability of a target neuron from its surrounding neurons.

[0050] For each neuron in the input feature map, the energy function is defined as:

[0051] in, is the number of neurons, is the regularization factor. The function is optimized by closed-form solution to obtain the minimum energy value: The smaller it is, the more significant the neuron is. The inverse of the energy is normalized to generate 3D attention weights for feature recalibration:

[0052] and Represent the mean and variance of the channel respectively. A smaller energy value indicates that the neuron has a higher significance. Finally, the energy The inverse of is normalized by the Sigmoid function to generate three-dimensional attention weights for feature recalibration.

[0053]

[0054] in, is across all channels and spatial dimensions Aggregate the results. The purpose of the Sigmoid function is to constrain excessively large values.

[0055] Constructing an inter-scale interactive attention module: This application proposes an inter-scale interactive attention module. In the polyp segmentation task, the multi-scale characteristics of polyp tissue (such as large size differences, blurred boundaries, etc.) put forward special requirements on feature expression capabilities. Traditional multi-scale methods usually use simple feature superposition or concatenation, which makes it difficult to establish cross-scale semantic associations. This application proposes an inter-scale interactive attention module, the core idea of ​​which is to achieve dynamic fusion of multi-scale features by establishing a bidirectional association between high-resolution detail features and low-resolution semantic features. The inter-scale interactive attention module uses a learnable attention mechanism to guide the low-resolution semantic features to enhance their content while maintaining the details of the high-resolution feature space, effectively solving the problem of mis-segmentation of the blurred areas of the polyp edges.

[0056] like Figure 3 As shown, the operation steps of the difference adaptive enhancement module are as follows: Given input high-resolution features and low-resolution features , first of all Perform bilinear interpolation upsampling to obtain The query (Q i ), key (K i ) and value projection (V i ):

[0057]

[0058] in, 、 and is the learnable parameter matrix, represents the convolution operation, is the channel compression ratio. Then, by and Normalize the scaled dot product on the feature dimension and calculate the attention weight matrix :

[0059] The final output features are enhanced by attention weighting and residual connection. :

[0060] The inter-scale interactive attention module achieves effective matching of heterogeneous features through resolution alignment and channel compression, and uses low-resolution semantic features to guide the generation of attention weights, thereby enhancing the representation ability of fuzzy boundaries.

[0061] Build a high and low frequency enhancement decoder: The high- and low-frequency enhancement decoder enhances feature consistency and boundary clarity through a frequency-aware feature fusion module and a channel integration module. The core idea of ​​the frequency-aware feature fusion module is to optimize the feature fusion process through high- and low-frequency analysis and processing. It includes three key components: an adaptive low-pass filter generator, an offset generator, and an adaptive high-pass filter generator.

[0062] in represents the high-resolution features enhanced by the inter-scale interactive attention module, represents the low-resolution decoding features, Represents the features after fusion through the frequency-aware feature fusion module.

[0063] The training process is as follows: The model is optimized using the AdamW optimizer with an initial learning rate of 7.5×10⁻ 4 , and a cosine decay scheduling strategy with preheating is used to stabilize training convergence. The batch size and training epochs are set to 8 and 50, respectively. All experiments are performed on a hardware environment consisting of an NVIDIA RTX 4090D GPU (24GB video memory), an AMD EPYC 9754 128-core CPU, and 80GB of system memory. The code is implemented on an Ubuntu 22.04 LTS system, using the PyTorch 2.1.0 framework, accelerated by CUDA 12.1, and enabling PyTorch mixed precision training. The network weights are initialized with the SAM2 pre-trained model, and end-to-end training is performed using a hybrid loss function that combines Dice loss and cross entropy loss to jointly optimize region consistency and boundary localization accuracy.

[0064] This application uses a series of evaluation indicators to measure the performance of the model in the polyp segmentation task. These indicators include MAE, Max-FM, Mean-FM, Max-Emeasure, Mean-Emeasure, S-measure, AP, and AUC. Among them, MAE is used to evaluate the absolute error between the predicted value and the true value, F-Measure and E-measure combine the performance of segmentation accuracy and boundary matching, and Max-FM and Max-Emeasure further evaluate the best performance under different thresholds. S-measure takes into account the consistency between the segmented area and the boundary. AP and AUC are used to evaluate the accuracy and overall performance of the model at different recall rates, the current polyp segmentation task. Through these comprehensive indicators, the accuracy, robustness and generalization ability of the proposed model in polyp segmentation can be comprehensively evaluated.

[0065] Table 1 below presents experimental results on the Kvasir-SEG dataset, a dataset containing high-resolution colonoscopy images characterized by significant polyp morphological diversity and complex background interference. The proposed SAM2-PolypNet achieves a MAE of 0.0229, a 15.1% improvement over the state-of-the-art baseline, NPDNet (0.0269). This is attributed to the SimAM adapter's effective mitigation of the domain shift between natural and medical images through dynamic channel calibration. Significant improvements in the boundary-sensitive metrics Max-FM (0.945 vs. 0.9149) and S-measure (0.9332 vs. 0.9156) validate the enhancement of edge details achieved by the mid-frequency decomposition strategy within the high- and low-frequency enhancement decoder. Notably, the Max-Emeasure improved by 3.1% (0.9659 vs. 0.9538), demonstrating that the cross-resolution feature alignment mechanism of the inter-scale mutual attention module effectively maintains semantic consistency among multi-scale features.

[0066] Table 1 Performance comparison of each model on the Kvasir dataset

[0067] The above experimental results show that the performance of this application is the best compared with other change detection networks, verifying that the detection method provided by this application is the most effective.

[0068] In order to further verify the superiority of the polyp segmentation method and system based on large model domain adaptation in this application, the following Figure 4The visualization analysis shown shows that most methods suffer from boundary shifts and intra-class inconsistencies when segmenting polyps. However, the polyp segmentation method and system based on large-scale model domain adaptation provided by this application demonstrates internal integrity and smooth boundaries. In areas with a high concentration of polyps, the polyp segmentation method and system based on large-scale model domain adaptation provided by this application achieves more complete results than other methods, with virtually no missed or mis-segmented segments.

[0069] The domain adaptive feature encoder of this application overcomes the high computational cost and catastrophic forgetting risk of SAM2. By designing a lightweight SimAM adapter module, it achieves the three goals of feature enhancement, dimensionality adaptation, and knowledge retention with only 0.8% fine-tuning of the parameters.

[0070] The inter-scale interactive attention module of this application uses a learnable attention mechanism to guide low-resolution semantic features to enhance their content while maintaining the details of the high-resolution feature space, effectively solving the problem of mis-segmentation of blurred areas at the edges of polyps.

[0071] The high and low frequency enhancement decoder of the present application enhances the consistency of features and the clarity of boundaries through a frequency perception feature fusion module and a channel integration module.

[0072] like Figure 5 As shown, the experimental results show that SAM2-PolypNet not only has high segmentation accuracy, but also has good robustness and generalization ability, and can effectively cope with various complex scenarios. Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including all changes and modifications that fall within the scope of the present invention and the preferred embodiments.

[0073] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A polyp segmentation method based on large model domain adaptation, characterized in that: include: Preprocess the polyp image to obtain the polyp dataset to be segmented; Inputting the to-be-segmented polyp dataset into a domain adaptive feature encoder for downsampling to extract an initial multi-scale feature image of the to-be-segmented polyp dataset; Performing feature enhancement, dimension adaptation, and knowledge retention on the initial multi-scale feature image through a SimAM adapter module to obtain a multi-scale feature image; Inputting the multi-scale feature image into the inter-scale interactive attention module to perform cross-scale feature fusion to obtain a fused feature image; The fused feature image is input into a high- and low-frequency enhancement decoder for adaptive frequency domain decomposition and Kolmogorov-Arnold channel integration to obtain a polyp segmentation output image.

2. The polyp segmentation method based on large model domain adaptation according to claim 1, characterized in that: The multi-scale feature image is obtained by performing feature enhancement, dimension adaptation and knowledge retention on the initial multi-scale feature image through the SimAM adapter module, including: Introducing a lightweight attention mechanism through an efficient parameter fine-tuning strategy; The multi-scale feature image is obtained by performing feature enhancement, dimension adaptation and knowledge retention on the initial multi-scale feature image through fine-tuning parameters.

3. The polyp segmentation method based on large model domain adaptation according to claim 1, characterized in that: The SimAM adapter module is a symmetric encoding-decoding architecture. The SimAM adapter directly infers 3D attention weights by optimizing an energy function while considering both channel and spatial dimensions. The SimAM adapter module measures the linear separability of target neurons and surrounding neurons by defining an energy function.

4. The polyp segmentation method based on large model domain adaptation according to claim 1, characterized in that: Inputting the multi-scale feature image into the inter-scale interactive attention module to perform cross-scale feature fusion to obtain a fused feature image further includes: The inter-scale interactive attention module performs bilinear interpolation upsampling on the low-resolution image of the multi-scale feature image to enhance it; The inter-scale interactive attention module generates query, key and value projections for the high-resolution image of the multi-scale feature image through three 1×1 convolutions to preserve details.

5. The polyp segmentation method based on large model domain adaptation according to claim 1, characterized in that: The fused feature image is input into the high and low frequency enhancement decoder for adaptive frequency domain decomposition and Kolmogorov-Arnold channel integration to obtain the polyp segmentation output image including: The high and low frequency enhancement decoder includes a frequency perception feature fusion module and a channel integration module; The high and low frequency enhancement decoder enhances the consistency and boundary clarity of the fused feature image through a frequency perception feature fusion module and a channel integration module.

6. A polyp segmentation system based on large model domain adaptation, characterized in that: include: The data set module is used to pre-process the polyp image to obtain the polyp data set to be segmented; A domain adaptive feature encoder module is used to downsample the polyp dataset to obtain an initial multi-scale feature image of the polyp dataset to be segmented; A SimAM adapter module is used to perform feature enhancement, dimension adaptation and knowledge retention on the initial multi-scale feature image to obtain a multi-scale feature image; an inter-scale interactive attention module, configured to perform cross-scale feature fusion on the multi-scale feature image to obtain a fused feature image; The high and low frequency enhancement decoder module is used to perform adaptive frequency domain decomposition and Kolmogorov-Arnold channel integration on the fused feature image to obtain a polyp segmentation output image.

Images