Adapter-based interactive camouflage target segmentation method, electronic device and storage medium

By employing an adapter-based interactive camouflage target segmentation method, which utilizes high- and low-frequency feature fusion and dynamic feature attention adjustment, the problem of insufficient model stability and accuracy in camouflage target segmentation is solved, achieving high-precision and high-compatibility segmentation results.

CN121353310BActive Publication Date: 2026-05-01XIAMEN UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
XIAMEN UNIV
Filing Date
2025-12-19
Publication Date
2026-05-01

AI Technical Summary

Technical Problem

Existing interactive segmentation methods suffer from insufficient model stability and mask accuracy when dealing with camouflaged targets, especially when the foreground and background textures are similar, making it difficult to accurately segment camouflaged targets. Furthermore, existing methods do not perform well in complex scenes.

Method used

An adapter-based interactive camouflage target segmentation method is adopted. High and low frequency features are extracted through the frozen parameters ViT-Base backbone and Adapter-ViT branch in the feature extraction module. Combined with the click enhancement module and feature fusion module, the feature attention is dynamically adjusted to realize multi-round interaction and mask generation.

Benefits of technology

It significantly improves segmentation mask accuracy, enhances model robustness and generalization ability, improves model segmentation performance in camouflaged target scenarios, and is highly compatible with existing models.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121353310B_ABST
    Figure CN121353310B_ABST
Patent Text Reader

Abstract

The application discloses an interactive camouflage target segmentation method based on an adapter, dynamically adjusts the attention degree of a model to different frequency domain characteristics, refines the originally repeated segmentation process into a process of sequentially and gradually understanding the segmentation target according to 'overall first and then details', and strengthens the role of user prompting in accurately understanding the segmentation target through a click reinforcement mechanism. The application mainly includes a feature extraction module and an interactive segmentation module. In the feature extraction module, an image will extract different frequency features through two paths, a pre-trained ViT framework will pay more attention to high-frequency features of the image, and a branch added with an adapter fine-tuning training will pay more attention to low-frequency features due to the characteristics of the convolutional layer. In the interactive segmentation module, the features extracted by the branch added with the adapter fine-tuning will perform cross-attention operation with click embedding, and the expression ability of the features is increased through a linear layer to obtain reinforced features. Finally, the two branches are fused to obtain a predicted mask through a decoder.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, and mainly relates to an interactive segmentation framework for solving the problems of accuracy and comprehensiveness in image segmentation under camouflaged target environments. In particular, it refers to an adapter-based interactive camouflaged target segmentation method, electronic device and storage medium. Background Technology

[0002] Interactive image segmentation (IIS) aims to accurately segment objects in images using user-provided interactive prompts. IIS has demonstrated significant application value in various fields, including image editing, autonomous driving, and medical image processing. Furthermore, interactive segmentation methods have greatly promoted the expansion and improvement of image segmentation datasets.

[0003] In recent years, the SAM model has garnered significant attention for its outstanding real-time, high-quality interactive segmentation capabilities, greatly aiding segmentation annotation and demonstrating broad application potential in various visual tasks. However, despite SAM's strong interactive segmentation performance in natural images, its performance on camouflaged target segmentation datasets still has room for improvement. Furthermore, interactive segmentation often faces ambiguity issues when handling camouflaged targets. For example, when the texture of a target object is highly similar to the background, the model easily confuses foreground objects with the background environment. This error is prevalent in various interaction methods, such as clicks, bounding boxes, and graffiti, which not only increases the complexity of the interaction but also significantly reduces segmentation accuracy. Therefore, a click-based segmentation method specifically designed for camouflaged targets is urgently needed to accurately identify them, thereby improving the performance of interactive image segmentation (IIS) tasks.

[0004] Existing interactive segmentation methods still have significant shortcomings when dealing with the problem of segmenting camouflaged targets:

[0005] Methods based on fine-tuning pre-trained SAM models still require further improvement in both model stability and mask precision when dealing with camouflaged targets in complex scenes. For example, the SAM-adapter method often exhibits slow performance improvement when optimizing the mask multiple times using continuous interaction information. Although using MLP as an adapter alleviates the problem of lack of expertise in pre-trained models to some extent, the issue of coarse masks remains unresolved in practical applications because the fine-tuning of the model blurs the boundary information between the foreground and background. Furthermore, existing convolutional neural network-based methods also face the dual challenges of model stability and accuracy when dealing with camouflaged targets in complex scenes. Summary of the Invention

[0006] The main objective of this invention is to provide an adapter-based interactive camouflaged target segmentation method, electronic device, and storage medium, which solves the problems existing in the prior art, significantly improves the segmentation mask accuracy, enhances model robustness, improves model generalization ability, strengthens interactive prompting ability, and has strong compatibility.

[0007] To achieve the above objectives, one solution of the present invention is:

[0008] An adapter-based interactive camouflage target segmentation method includes:

[0009] Step 1. Feature Extraction Stage

[0010] Given an input image Cut it into The patch, in which Represents the real number field. Indicates the height of the input image. Indicates the width of the input image. Indicates the dimensional parameters of a local area;

[0011] The feature extraction module consists of two branches:

[0012] (1) The first branch is the ViT-Base trunk with frozen parameters. The output of the first branch is a high-frequency feature. ,in Indicates the number of patches. Indicates feature dimension;

[0013] (2) The second branch is Adapter-ViT, which consists of ViT, a layer-specific MLP, and a stage-shared MLP; the output of the second branch is a low-frequency feature. ,in Indicates the Adapter-ViT backbone;

[0014] Step 2. Interactive Segmentation Phase

[0015] The click enhancement module and feature fusion module are used to realize the use of two features: click information and features extracted in the feature extraction stage.

[0016] Step 2.1: The embedding encoding obtained from the user-input click prompt coordinates is obtained through embedding. ,in This represents the length of the click embedding sequence. In the click enhancement module, cross-attention is first performed using embedding encoding and low-frequency features, as expressed below:

[0017] ,in This represents image features that have been injected with click information. This indicates a normalization operation. express transpose, The dimension representing a single attention head;

[0018] Subsequently, an MLP layer is used to enhance the expressive power of the model, and cross-attention from image to click is performed to feed back the image context to the pixel level. The expression is as follows:

[0019] ,in This represents image features that are globally modulated based on click location and semantics.

[0020] Step 2.2: In the feature fusion module, dynamic weights are used in each round of interaction. Will and Fusion, its expression is as follows:

[0021] , ,in This represents the final image features after this round of fusion. Indicates the first The weight value of the wheel, This represents the weight value for the first round. This indicates the preset parameters used to control the decay rate;

[0022] Step 3. Training Phase

[0023] During the training phase, all attention layers of the pre-trained ViT model are frozen, and only the parameters of each MLP layer in the Adapter-ViT branch, the interactive attention component in the click enhancement module, the feature fusion module, and the final decoder are updated.

[0024] Step 4. Reasoning Stage

[0025] During the inference phase, multiple rounds of click and mask generation operations are performed according to the interactive segmentation process.

[0026] In the second branch of the feature extraction module, the layer-specific MLP is independently owned by each Transformer layer, denoted as... ,in This indicates a hierarchy-specific MLP. This indicates the MLP layer that maps the number of dimensions upwards. This represents the activation function. This represents the MLP layer that maps the number of dimensions downwards. This represents the image features input to this layer; the stage-shared MLP shares parameters once every four layers for information fusion in the pre-, mid-, and post-stages, denoted as... ,in Indicates that the phases share the MLP. This indicates the stage of the MLP layer that maps the number of dimensions upwards. This represents the stage of the MLP layer that maps the dimension number down, the first... The output of the layer is ,in This represents the Transformer layer.

[0027] In step 3, a number of clicks is randomly sampled in each training iteration. And based on this, generate the corresponding one containing The embedded vector of each point information is used as the click input to simulate the multi-point prompts given by the user during the segmentation process.

[0028] Step 4 specifically involves:

[0029] Initially, the first click point is given based on prior information, and the model uses this click point to generate the first round of mask prediction. Subsequently, based on the segmentation results output from the previous round, the system automatically selects a new click position and embeds the new click into the click enhancement module. Then, the feature fusion module uses decaying dynamic weights... The newly generated click enhancement features are then fused back with the general features, and the decoder generates the next round of masks based on this. This process is repeated until the preset accuracy requirements are met or the maximum number of interaction rounds is reached.

[0030] The second solution of the present invention is:

[0031] An electronic device is characterized by comprising a processor, a memory, and an application program; the application program is stored in the memory and configured to be executed by the processor using the adapter-based interactive camouflage target segmentation method.

[0032] The third solution of the present invention is:

[0033] A computer-readable storage medium, characterized in that the storage medium stores a computer program; when the computer program is executed in the computer, it causes the computer to perform the adapter-based interactive camouflage target segmentation method.

[0034] After adopting the above technical solution, the present invention has the following technical effects:

[0035] (1) Significantly improves the accuracy of segmentation mask: With the help of feature fusion mechanism and attention adaptive adjustment mechanism, the present invention can accurately extract and understand multi-frequency domain image features in various scenarios, greatly improving the accuracy of segmentation mask; especially in target segmentation tasks where the foreground and background textures are highly similar, its performance is particularly outstanding.

[0036] (2) Enhance model robustness: Through feature fusion mechanism and attention adaptive adjustment mechanism, the model can more accurately adapt to the goals of each stage in the interaction process, thereby significantly improving the robustness of the model and effectively enhancing the reliability of the model in practical applications.

[0037] (3) Improve the model's generalization ability: Through the interval shared adapter mechanism, the model can more effectively adapt to targets from different angles, thereby significantly enhancing its generalization performance in diverse scenarios. This allows the model to maintain high segmentation accuracy when facing different tasks and scenarios.

[0038] (4) Enhanced interactive prompt function: By introducing a click enhancement module, the area of ​​interest of the user in the image features is highlighted, so that the model can understand the object that the user needs to segment more quickly, ensuring that the present invention exhibits high efficiency in practical applications.

[0039] (5) Excellent compatibility: The framework of this invention is theoretically highly compatible with most existing interactive segmentation models, has excellent scalability, and can be easily integrated into existing interactive segmentation systems, providing innovative technical means for research and application in related fields. Attached Figure Description

[0040] Figure 1 This is a diagram illustrating the interactive segmentation effect of the present invention in various camouflaged target scenarios.

[0041] Figure 2 This is a schematic diagram of the training and reasoning structure of the present invention.

[0042] Figure 3 The results of this invention and other methods on the CAMO dataset are shown. Detailed Implementation

[0043] To further explain the technical solution of the present invention, the present invention will be described in detail below through specific embodiments.

[0044] In interactive segmentation models, the input image primarily extracts image features through a pre-trained ViT backbone. The coordinates of user clicks, after embedding, are directly concatenated to these image features to provide location information. However, for targets in camouflaged data, the foreground and background textures are extremely similar, making it difficult for the model to effectively distinguish between them due to texture differences inside and outside the object's boundaries. This characteristic causes segmentation models to struggle to accurately identify the boundaries of camouflaged targets, often misclassifying parts of the object or similar background elements as targets. Furthermore, the pre-trained backbone network lacks expertise in camouflaged targets, and current segmentation models underutilize click information, making it difficult to effectively guide the model in segmenting camouflaged targets. These problems significantly exacerbate the performance degradation of conventional segmentation methods when processing camouflaged data.

[0045] To address this problem, this invention discloses an adapter-based interactive camouflage target segmentation method. Its core idea is to dynamically adjust the model's focus on different frequency domain features, refining the originally repetitive segmentation process into a gradual understanding of the segmentation target, proceeding "from the whole to the details." A click-enhancement mechanism further strengthens the role of user prompts in accurately understanding the segmentation target. The framework used in this invention mainly consists of two modules: a feature extraction module and an interactive segmentation module. In the feature extraction module, the image is processed through two paths to extract features of different frequencies. The pre-trained ViT framework focuses more on high-frequency features, while the branch with adapter fine-tuning focuses more on low-frequency features due to the characteristics of convolutional layers. In the interactive segmentation module, the features extracted by the adapter-fine-tuned branch undergo cross-attention operations with the click embedding, and their expressive power is enhanced through linear layers to obtain enhanced features. Finally, the features obtained from the two branches are fused to obtain the predicted mask through the final decoder.

[0046] Specifically, refer to Figure 1 , Figure 2 As shown, the above method includes:

[0047] Step 1. Feature Extraction Stage

[0048] Given an input image Cut it into The patch (local region), where Represents the real number field. Indicates the height of the input image. Indicates the width of the input image. This represents the size parameters of a local area.

[0049] The feature extraction module consists of two branches:

[0050] (1) The first branch is the ViT-Base trunk with frozen parameters. The output of the first branch is a high-frequency feature. ,in Indicates the number of patches. Indicates the feature dimension (e.g., 256 dimensions).

[0051] (2) The second branch is Adapter-ViT, which consists of ViT, layer-specific MLP, and stage-shared MLP; the layer-specific MLP is independently owned by each Transformer layer, denoted as ,in This indicates a hierarchy-specific MLP. This represents the MLP layer that maps the number of dimensions upwards (e.g., ...). ), This represents the activation function. This represents the hierarchical MLP layer that maps the number of dimensions downwards (e.g., ...). ), This represents the image features input to this layer; the stage-shared MLP shares parameters once every four layers for information fusion in the pre-, mid-, and post-stages, denoted as... ,in Indicates that the phases share the MLP. This indicates the stage of the MLP layer that maps the number of dimensions upwards (e.g., ), This indicates the stage of the MLP layer that maps the number of dimensions down (e.g., ), then the first The output of the layer is ,in This represents a Transformer layer, therefore the output of the second branch is a low-frequency feature. ,in This indicates the Adapter-ViT backbone.

[0052] thus, As high-frequency features extracted by ViT, they have a significant advantage in capturing boundary information, mainly because ViT, through its global self-attention mechanism, can perceive the relationships between distant pixels in an image, thus being more sensitive to high-frequency changing areas such as texture and contours; this makes... Preserving clearer and more accurate features at object boundaries helps improve boundary localization accuracy in segmentation tasks. As low-frequency features extracted by convolutional layers, they have advantages in terms of image detail texture and semantic expression. Convolutional layers excel at capturing local spatial structure information, which helps model cross-channel feature relationships. They can more effectively extract stable and continuous low-frequency components in images, thereby enhancing the ability to express the texture, color distribution, and semantic consistency within regions, and helping to improve overall understanding and region discrimination capabilities.

[0053] Step 2. Interactive Segmentation Phase

[0054] The click enhancement module and feature fusion module are used to realize the use of two features: click information and features extracted in the feature extraction stage.

[0055] Step 2.1: The embedding encoding obtained from the user-input click prompt coordinates is obtained through embedding. ,in This represents the length of the click embedding sequence. In the click enhancement module, cross-attention is first performed using embedding encoding and low-frequency features, as expressed below:

[0056] ,in This represents image features that have been injected with click information. This indicates a normalization operation. express transpose, The dimension representing a single attention head;

[0057] Subsequently, an MLP layer is used to enhance the expressive power of the model, and cross-attention from image to click is performed to feed back the image context to the pixel level. The expression is as follows:

[0058] ,in This represents image features that are globally modulated based on click location and semantics.

[0059] Step 2.2: In the feature fusion module, dynamic weights are used in each round of interaction. Will and Fusion, its expression is as follows:

[0060] , ,in This represents the final image features after this round of fusion. Indicates the first The weight value of the wheel, This represents the weight value for the first round. This represents the preset parameters used to control the decay rate. Therefore, As learnable initial weights, they can guarantee the success of the first round of interaction. Dominated, and gradually shifted to Transition. Through the aforementioned dual-branch feature extraction, cross-attention interaction, and adaptive fusion strategy, this invention can quickly locate the target in the first round of interaction and maintain its refinement capability in subsequent iterations, thereby producing high-quality segmentation results more quickly.

[0061] Step 3. Training Phase

[0062] During the training phase, all attention layers of the pre-trained ViT model are frozen, and only the parameters of each layer of the MLP in the Adapter-ViT branch, the interactive attention component in the click enhancement module, the feature fusion module, and the final decoder are updated.

[0063] Specifically, in each training iteration, a number of clicks is randomly sampled. And based on this, generate the corresponding one containing The embedded vectors of point information are used as click inputs to simulate multi-point prompts given by users during segmentation. This design ensures the stability of general visual features while allowing a small number of trainable parameters to quickly adapt to different annotation requirements, thus completing the fusion learning of multi-point interaction information in a single forward propagation.

[0064] Step 4. Reasoning Stage

[0065] During the inference phase, multiple rounds of click and mask generation operations are performed according to the interactive segmentation process:

[0066] Initially, the first click point is given based on prior information, and the model uses this click point to generate the first round of mask prediction. Subsequently, based on the segmentation results of the previous round, the system automatically selects new click locations (usually concentrated in areas with inaccurate segmentation or blurred boundaries) and embeds the new clicks into the click enhancement module. Then, the feature fusion module uses decaying dynamic weights... The newly generated click-enhanced features are then fused with general features, and the decoder generates the next round of masks based on this. This process is iterated until the preset accuracy requirement is met or the maximum number of interaction rounds is reached. This multi-round click-mask iteration scheme enables the model to dynamically focus on hard-to-segment regions and gradually optimize the segmentation boundaries, significantly improving the precision and robustness of the final result.

[0067] In the above solution, the key points of the present invention are as follows:

[0068] 1. Inter-area shared adapter

[0069] In existing technologies, pre-trained segmentation models generally suffer from insufficient expertise when handling special scene datasets, making it difficult for the model to accurately capture user intent during inference and thus affecting overall detection accuracy. This invention introduces a region-sharing adapter into the image encoder, effectively integrating expertise from special scene data and combining the general image understanding capabilities of the pre-trained model with the adapter's specialized image understanding capabilities. Furthermore, the region-sharing mechanism in the ViT encoder enables information sharing between layers of similar depth, significantly improving the encoder's image understanding capabilities. This allows it to accurately identify foreground objects in camouflaged scenes, thereby greatly improving the model's performance in camouflaged target scenes.

[0070] 2. Cross-frequency domain feature fusion

[0071] In existing technologies, when image encoders extract features from camouflaged target images, the fine-tuned MLP layer tends to focus on low-frequency features, leading to blurred image boundary information and affecting the model's segmentation accuracy and interactive performance. This invention introduces a cross-frequency domain feature fusion mechanism, encoding the image into high-frequency and low-frequency features and effectively fusing these two types of features through a cross-attention mechanism. Through this feature fusion, the model can comprehensively utilize semantic features from different frequency domains, preserving the semantic understanding ability of low-frequency features while enhancing the boundary refinement ability of high-frequency features, thus significantly improving the accuracy of mask segmentation. Furthermore, the cross-frequency domain feature fusion mechanism also enables the model to better adapt to the needs of interactive segmentation, significantly enhancing the model's interactive segmentation capabilities in camouflaged target scenarios.

[0072] 3. Adaptive adjustment of feature attention

[0073] In existing technologies, click-based camouflage target segmentation models suffer from varying feature requirements at different stages of the interaction process, making it difficult for the model to consistently improve mask accuracy during inference. This invention introduces a feature attention adaptive adjustment mechanism to dynamically regulate the fusion process of high-frequency and low-frequency features, enabling the model to flexibly switch between features in different frequency domains. This effectively alleviates the problem of unstable performance improvement and enhances the flexibility of the segmentation model. Furthermore, the introduction of the feature attention adjustment mechanism only requires adding one extra parameter during inference, resulting in low computational overhead and ease of implementation.

[0074] Through the above solution, the present invention can achieve the following effects:

[0075] (1) Significantly improves the accuracy of segmentation mask: With the help of feature fusion mechanism and attention adaptive adjustment mechanism, the present invention can accurately extract and understand multi-frequency domain image features in various scenarios, greatly improving the accuracy of segmentation mask; especially in target segmentation tasks where the foreground and background textures are highly similar, its performance is particularly outstanding.

[0076] (2) Enhance model robustness: Through feature fusion mechanism and attention adaptive adjustment mechanism, the model can more accurately adapt to the goals of each stage in the interaction process, thereby significantly improving the robustness of the model and effectively enhancing the reliability of the model in practical applications.

[0077] (3) Improve the model's generalization ability: Through the interval shared adapter mechanism, the model can more effectively adapt to targets from different angles, thereby significantly enhancing its generalization performance in diverse scenarios. This allows the model to maintain high segmentation accuracy when facing different tasks and scenarios.

[0078] (4) Enhanced interactive prompt function: By introducing a click enhancement module, the area of ​​interest of the user in the image features is highlighted, so that the model can understand the object that the user needs to segment more quickly, ensuring that the present invention exhibits high efficiency in practical applications.

[0079] (5) Excellent compatibility: The framework of this invention is theoretically highly compatible with most existing interactive segmentation models, has excellent scalability, and can be easily integrated into existing interactive segmentation systems, providing innovative technical means for research and application in related fields.

[0080] The technical effects of the present invention are further illustrated by the following simulation experiments.

[0081] (I) Experimental conditions

[0082] This invention was developed on the Linux platform, and the developed deep learning framework is based on PyTorch, with Python as the primary language.

[0083] (II) Experiment Content

[0084] The network was trained using CAMO, a dataset commonly used in the field of camouflaged target segmentation, following the steps outlined above, and then validated using a test set. Figure 3 The results of this invention and other methods on the CAMO dataset are presented. It can be seen that this invention consistently improves the performance of the baseline algorithm, and achieves the best results compared to other methods. Here, FuseSAM represents the results of this invention, NoC represents the average number of clicks when the segmentation accuracy reaches the target IoU, and Click represents the average segmentation performance after a certain number of clicks. The segmentation performance of this method on the CAMO dataset reaches NoC90=9.80, which is superior to other methods, fully demonstrating the better effect of this invention in camouflaged target segmentation.

[0085] Furthermore, the present invention also discloses an electronic device including a processor, a memory, and an application program; the application program is stored in the memory and configured to be executed by the processor using the methods described above.

[0086] The present invention also discloses a computer-readable storage medium storing a computer program; when the computer program is executed in a computer, it causes the computer to perform the above-described method. The storage medium may include various media capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0087] The above embodiments and figures are not intended to limit the product form and style of the present invention. Any appropriate changes or modifications made by those skilled in the art should be considered as not departing from the patent scope of the present invention.

Claims

1. An adapter-based interactive camouflage target segmentation method, characterized in that, include: Step 1. Feature Extraction Stage Given an input image Cut it into The patch, in which Represents the real number field. Indicates the height of the input image. Indicates the width of the input image. Indicates the dimensional parameters of a local area; The feature extraction module consists of two branches: (1) The first branch is the ViT-Base trunk with frozen parameters. The output of the first branch is a high-frequency feature. ,in Indicates the number of patches. Indicates the feature dimension; (2) The second branch is Adapter-ViT, which consists of ViT, a layer-specific MLP, and a stage-shared MLP; the output of the second branch is a low-frequency feature. ,in Indicates the Adapter-ViT backbone; Step 2. Interactive Segmentation Phase The click enhancement module and feature fusion module are used to realize the use of two features: click information and features extracted in the feature extraction stage. Step 2.1: The embedding encoding obtained from the user-input click prompt coordinates is then processed. ,in This indicates the length of the click embedding sequence. In the click enhancement module, cross-attention is first performed using embedding encoding and low-frequency features, as expressed below: ,in This represents image features that have been injected with click information. This indicates a normalization operation. express transpose, The dimension representing a single attention head; Subsequently, an MLP layer is used to enhance the expressive power of the model, and cross-attention from image to click is performed to feed back the image context to the pixel level. The expression is as follows: ,in This represents image features that are globally modulated based on click location and semantics. Step 2.2: In the feature fusion module, dynamic weights are used in each round of interaction. Will and Fusion, its expression is as follows: , ,in This represents the final image features after this round of fusion. Indicates the first The weight value of the wheel, This represents the weight value for the first round. This indicates the preset parameters used to control the decay rate; Step 3. Training Phase During the training phase, all attention layers of the pre-trained ViT model are frozen, and only the parameters of each layer of MLP in the Adapter-ViT branch, the interactive attention component in the click enhancement module, the feature fusion module, and the final decoder are updated. Step 4. Reasoning Stage During the inference phase, multiple rounds of click and mask generation operations are performed according to the interactive segmentation process.

2. The adapter-based interactive camouflage target segmentation method as described in claim 1, characterized in that: In the second branch of the feature extraction module, the layer-specific MLP is independently owned by each Transformer layer, denoted as... ,in This indicates a hierarchy-specific MLP. This indicates the MLP layer that maps the number of dimensions upwards. This represents the activation function. This represents the MLP layer that maps the number of dimensions downwards. This represents the image features input to this layer; The phase-shared MLP shares parameters every four layers for information fusion in the pre-, mid-, and post-stages, denoted as... ,in Indicates that the phases share the MLP. This indicates the stage of the MLP layer that maps the number of dimensions upwards. This represents the stage of the MLP layer that maps the dimension number down, the first... The output of the layer is ,in This represents the Transformer layer.

3. The adapter-based interactive camouflage target segmentation method as described in claim 1, characterized in that: In step 3, a number of clicks is randomly sampled in each training iteration. And based on this, generate the corresponding one containing The embedded vector of each point information is used as the click input to simulate the multi-point prompts given by the user during the segmentation process.

4. The adapter-based interactive camouflage target segmentation method as described in claim 1, characterized in that, Step 4 specifically involves: Initially, the first click point is given based on prior information, and the model uses this click point to generate the first round of mask prediction. Subsequently, based on the segmentation results output from the previous round, the system automatically selects a new click position and embeds the new click into the click enhancement module. Then, the feature fusion module uses decaying dynamic weights... The newly generated click enhancement features are then fused back with the general features, and the decoder generates the next round of masks based on this. This process is repeated until the preset accuracy requirements are met or the maximum number of interaction rounds is reached.

5. An electronic device, characterized in that... It includes a processor, memory, and an application; the application is stored in the memory and configured to be executed by the processor as described in any one of claims 1 to 4, an adapter-based interactive camouflage target segmentation method.

6. A computer-readable storage medium, characterized in that... The storage medium stores a computer program; when the computer program is executed in the computer, it causes the computer to perform the adapter-based interactive camouflage target segmentation method as described in any one of claims 1 to 4.

Citation Information

Patent Citations

  • Deleted modal brain tumor image segmentation method based on multi-axis shift MLP modal mask

    CN120147637A

  • Night semantic segmentation method and device based on wavelet transform detail enhancement and text prompt

    CN120236080A