A method for detecting camouflaged targets

Abstract

This invention discloses a method for camouflaged target detection, belonging to the field of camouflaged target detection. This method constructs a BiSAM based on the SAM network. The SAM encoder, CNN encoder, and dual-domain cascaded attention module BCAM work collaboratively to enrich multi-scale feature information and extract multi-source features from the input image. Specifically, BCAM operates collaboratively in the spatial and spectral domains, selectively recovering and enhancing high-frequency object boundaries while suppressing irrelevant background noise. The multi-source features are fused using AKM to guide the SAM decoder in decoding the ViT image embedding. The method replaces the original position encoding of the cross-attention key vector in the SAM decoder, injecting task-specific knowledge into the frozen SAM decoder. By replacing the original position encoding, the general attention key vector is transformed into a task-aware attention key vector, specifically adapted to camouflaged target detection scenarios, thereby improving the accuracy of camouflaged target detection while maintaining generalization ability.

Description

Technical Field

[0001] This invention belongs to the field of camouflage target detection, and more specifically, relates to a method for camouflage target detection. Background Technology

[0002] The Segment Anything Model (SAM) is a foundational model developed specifically for image segmentation tasks. Built on the Transformer architecture and trained on over 11 million images, it demonstrates strong zero-shot generalization capabilities across various fields, including medical imaging. SAM's robust adaptability has naturally inspired researchers to explore its potential in camouflaged object detection (COD). In these scenarios, objects exhibit visual similarities to their surroundings (e.g., color, texture, or optical flow similarities). However, SAM's performance in COD is less than ideal, primarily due to the inherent similarity between camouflaged targets and the background, contrasting sharply with the clear and salient targets in the commonly used salient object dataset (SA-1B training dataset) for training SAM.

[0003] Recent studies have proposed various strategies to overcome this limitation and apply the general segmentation capabilities of SAM to the field of camouflage target detection. For example, by integrating additional depth information to optimize RGB features, the challenge posed by color and texture similarity is effectively alleviated; by introducing interactive cues as guidance, the image encoder and mask decoder of SAM are fine-tuned, thereby achieving significant improvements in the field of camouflage target detection.

[0004] While these methods have achieved some breakthroughs, they often rely on auxiliary inputs such as depth maps or explicit user prompts, which limits their practical applications. More importantly, their performance remains unsatisfactory in complex scenes, especially those involving multi-scale targets, multiple targets, occlusion, and dense edges. Summary of the Invention

[0005] In view of the above-mentioned defects or improvement needs of the existing technology, the present invention provides a camouflage target detection method, which can improve the detection accuracy of camouflage targets in complex scenes.

[0006] To achieve the above objectives, according to a first aspect of the present invention, a camouflage target detection system is provided, comprising:

[0007] The SAM encoder is used to encode the input image to obtain the ViT image embedding. ;

[0008] A CNN encoder is used to encode the input image to obtain local fine features. ;

[0009] The dual-cascaded attention module (BCAM) includes a spatial attention module and a spectral attention module.

[0010] The spatial attention module includes a first multi-head self-attention module SA1 and a first linear transformation layer, used to perform multi-head self-attention calculation on the input feature X to obtain the first multi-head self-attention feature. , The intermediate feature is obtained by adding it to X after processing by the first linear transformation layer. X represents the feature output of the first Transformer block in a series of cascaded Transformer blocks in the SAM encoder; the first FFN network with residual connections is used to... FFN processing yields spatially optimized feature maps. ;

[0011] The spectrum attention module includes: a discrete wavelet transform module DWT, first and second convolutional layers, second to fifth linear transform layers, a second multi-head self-attention module SA2, and a second FFN network with residual connections; After processing by the second linear transformation layer, it serves as the query vector for the j-th attention head in SA2. ; After being processed sequentially through the first convolutional layer, DWT, the second convolutional layer, and the third linear transformation layer, it becomes the key vector of the j-th attention head in SA2. ; After being processed sequentially through the first convolutional layer, DWT, the second convolutional layer, and the fourth linear transformation layer, it becomes the value vector of the j-th attention head in SA2. j=1,2,…,J, where J is the number of attention heads in the second multi-head self-attention module; the second multi-head self-attention module is used to... , and The second multi-head self-attention feature is obtained by performing multi-head self-attention calculation. , and After splicing, the frequency domain optimized features are obtained through the fifth linear transform layer. The second FFN network with residual connections is used for... and After addition and fusion, FFN processing is performed to obtain spatial-frequency enhanced features. ;

[0012] The adaptive key modulation module AKM is used for... , and The fusion is performed to obtain fusion features. ;

[0013] An improved SAM decoder, which replaces the original positional encoding of the SAM decoder's cross-attention key vector with... Obtain, for use in Under the guidance of [the method], the ViT image embedding is decoded to obtain the camouflage target detection result of the input image.

[0014] According to a second aspect of the present invention, a training method for a camouflaged target detection system as described in the first aspect is provided, comprising:

[0015] The Neck layer of the SAM encoder, the CNN encoder, BCAM, and AKM were trained under supervision using the training dataset.

[0016] According to a third aspect of the present invention, a method for detecting camouflaged targets is provided, comprising: inputting an image to be detected into a system as described in the first aspect, and obtaining a camouflaged target detection result of the image to be detected.

[0017] According to a fourth aspect of the present invention, an electronic device is provided, comprising: a computer-readable storage medium and a processor;

[0018] The computer-readable storage medium is used to store executable instructions;

[0019] The processor is configured to read executable instructions stored in the computer-readable storage medium and execute the method described in the second or third aspect.

[0020] According to a fifth aspect of the invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to perform the method as described in the second or third aspect.

[0021] According to a sixth aspect of the invention, a computer program product is provided, comprising a computer program or instructions that, when executed by a processor, implement the method as described in the second or third aspect.

[0022] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects:

[0023] The method provided by this invention constructs a BiSAM based on the SAM network. The SAM encoder, CNN encoder, and dual-domain cascaded attention module BCAM work together to enrich multi-scale feature information and extract multi-source features from the input image. Specifically, BCAM operates collaboratively in the spatial and spectral domains, selectively recovering and enhancing high-frequency object boundaries while suppressing irrelevant background noise. The multi-source features are then fused using AKM to obtain... This is used to guide SAM decoding for decoding ViT image embeddings; By replacing the original positional encoding of the cross-attention key vector in the SAM decoder, task-specific knowledge is injected into the frozen SAM decoder. By replacing the original positional encoding, the general attention key vector is transformed into a task-aware attention key vector, which is specifically adapted to camouflage target detection scenarios, thereby improving the accuracy of camouflage target detection tasks while maintaining generalization ability.

[0024] As a further preferred embodiment, the method provided by the present invention adjusts the SAM network to a fully automatic segmentation mode by removing sparse cue markers and dense cue embeddings from all cross-attentions of the SAM decoder, so that the SAM network can still maintain excellent performance without relying on external cues. Attached Figure Description

[0025] Figure 1 This is a schematic diagram illustrating the low-pass filtering effect of the ViT backbone network in SAM.

[0026] Figure 2 This is a schematic diagram of the camouflaged target detection system provided in an embodiment of the present invention; wherein, the snowflake icon represents the frozen parameter and the spark icon represents the learnable parameter.

[0027] Figure 3 This is a schematic diagram of the structure of the BCAM module provided in an embodiment of the present invention.

[0028] Figure 4 (a), (b), (c), (d), and (e) in the figure are schematic diagrams of the camouflage target detection results of the camouflage target detection method (BiSAM) provided in the embodiments of the present invention and the existing advanced COD method DSAM based on SAM in five complex scenarios: occlusion, large target, small target, multiple targets, and dense edges; where GT is the real label. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0030] Recent studies have proposed various strategies to apply the zero-shot generalization ability of Visual Amplifier (SAM) to camouflaged target detection. However, these methods often underperform in complex scenes involving multi-scale targets, multiple targets, occlusion, and dense edges. Research has revealed a fundamental but overlooked reason for this limitation: the Visual Transformer (ViT) backbone network in SAM acts as a low-pass filter, suppressing key high-frequency details needed to distinguish camouflaged targets, such as... Figure 1 As shown. Based on this, embodiments of the present invention provide a camouflaged target detection system, such as... Figure 2 As shown, it includes:

[0031] The SAM encoder is used to encode the input image to obtain the ViT image embedding. ;

[0032] A CNN encoder is used to encode the input image to obtain local fine features. ;

[0033] The dual-cascaded attention module (BCAM) includes a spatial attention module and a spectral attention module.

[0034] The spatial attention module includes a first multi-head self-attention module SA1 and a first linear transformation layer, used to perform multi-head self-attention calculation on the input feature X to obtain the first multi-head self-attention feature. , The intermediate feature is obtained by adding it to X after processing by the first linear transformation layer. X represents the feature output of the first Transformer block in a series of cascaded Transformer blocks in the SAM encoder; the first FFN network with residual connections is used to... FFN processing yields spatially optimized feature maps. ;

[0035] The spectrum attention module includes: a discrete wavelet transform module DWT, first and second convolutional layers, second to fifth linear transform layers, a second multi-head self-attention module SA2, and a second FFN network with residual connections; After processing by the second linear transformation layer, it serves as the query vector for the j-th attention head in SA2. ; After being processed sequentially through the first convolutional layer, DWT, the second convolutional layer, and the third linear transformation layer, it becomes the key vector of the j-th attention head in SA2. ; After being processed sequentially through the first convolutional layer, DWT, the second convolutional layer, and the fourth linear transformation layer, it becomes the value vector of the j-th attention head in SA2. j=1,2,…,J, where J is the number of attention heads in the second multi-head self-attention module; the second multi-head self-attention module is used to... , and The second multi-head self-attention feature is obtained by performing multi-head self-attention calculation. , and After splicing, the frequency domain optimized features are obtained through the fifth linear transform layer. The second FFN network with residual connections is used for... and After addition and fusion, FFN processing is performed to obtain spatial-frequency enhanced features. ;

[0036] The adaptive key modulation module AKM is used for... , and The fusion is performed to obtain fusion features. ;

[0037] An improved SAM decoder, which replaces the original positional encoding of the SAM decoder's cross-attention key vector with... Obtain, for use in Under the guidance of [the method], the ViT image embedding is decoded to obtain the camouflage target detection result of the input image.

[0038] The camouflaged target detection system provided in this embodiment of the invention is based on the BiSAM dual-domain segmentation model, which includes SAM, BCAM module, CNN encoder and AKM module. Each module will be further described below.

[0039] 1. SAM encoder

[0040] In the SAM encoder, the input image first passes through the embedding layer and the first Transformer block sequentially to obtain the basic feature map X, preparing for deep feature extraction. Then, the basic feature map X passes through multiple cascaded Transformer blocks to obtain the deep feature map. The deep feature map is then processed by multiple cascaded Neck layers to obtain the ViT image embedding. .

[0041] 2. CNN encoder

[0042] The CNN encoder, as a branch of CNN, is used to extract local fine features of the input image. .

[0043] 3. Dual-Domain Cascaded Attention Module (BCAM)

[0044] This invention designs a dual-domain cascaded attention module (BCAM). The BCAM module uses a cascaded spatial and spectral domain strategy to achieve accurate localization and edge enhancement. The basic feature map X is processed by BCAM to obtain spatial-frequency domain enhanced features.

[0045] BCAM aims to resolve the conflict between high-frequency detail enhancement and background noise amplification. For example... Figure 3 As shown, the module adopts a two-step cascaded processing flow: first, the spatial domain attention module identifies high-confidence target areas, and then the spatial domain guiding signal drives the spectrum attention module to selectively enhance multi-directional boundary cues (extracted by discrete wavelet transform) only in these key areas.

[0046] Specifically, BCAM includes cascaded spatial attention modules and spectral attention modules.

[0047] The spatial attention module is used to process input features. Where H, W, and C represent the input features, respectively. The height, width, and channel dimensions. This stage employs a standard multi-head self-attention (MHSA) architecture. In the first MHSA module, for each attention head, the query vector... Key vector Sum value vector All from input features Extracted from [the data]. Its core operation is the scaled dot product attention mechanism, defined as:

[0048]

[0049] in Represents the dimension of the key vector; i = 1, 2, ..., I, where I is the number of attention heads in the first multi-head self-attention module.

[0050] The outputs of all head units are concatenated, linearly projected, and then residually concatenated with X. Finally, a spatially optimized feature map is generated through a first standard feedforward network (FFN) with residual connections. Subsequently This serves as input for subsequent spectral attention modules. This design ensures that frequency domain attention is guided by the spatial domain, allowing it to focus on key features already captured in the spatial domain, rather than searching from scratch.

[0051] The spectral domain attention module includes a discrete wavelet transform module (DWT), first and second convolutional layers, second to fifth linear transform layers, a second multi-head self-attention module, and a second FFN network with residual connections.

[0052] The second multi-head self-attention module includes multiple self-attention heads, and the query vector of the j-th attention head in the second multi-head self-attention module is... Through the second linear transformation layer It is obtained by performing a linear transformation.

[0053] The key vector of the j-th attention head in the second multi-head self-attention module Sum value vector Through wavelet transform and convolution operations (e.g.) This is derived from the convolution operation. Specifically, Through the first convolutional layer (e.g.) The convolutional layer is reduced in dimensionality (e.g., to C / 4 channels) to obtain Discrete wavelet transform (DWT) will Decomposed into four sub-bands:

[0054]

[0055] in, This represents the low-frequency subband containing global structure and lighting information, while , and These correspond to the high-frequency components that encode the texture details of the horizontal, vertical, and diagonal edges, respectively.

[0056] The Discrete Wavelet Transform (DWT) process described above is lossless. DWT can use any basis function. Experiments have shown that the classic Haar wavelet basis function yields the best results.

[0057] Subsequently, the four sub-bands are spliced ​​together along the channel dimension to form The key vector is then generated through convolution operations in the second convolutional layer (e.g., a 3×3 convolutional layer) and linear transformations in the third linear transformation layer. ; Value vectors are generated through convolution operations in the second convolutional layer (e.g., a 3×3 convolutional layer) and linear transformations in the fourth linear transformation layer. .

[0058] Subsequently, the second multi-head self-attention module utilizes , and The second multi-head self-attention feature is obtained by executing a multi-head self-attention mechanism. Then Spatial domain attention results The features are then concatenated and subsequently transformed using a linear transformation layer (the fifth linear transformation layer) to generate optimized frequency domain features.

[0059]

[0060] in, For frequency domain optimization features, Represents a linear transformation. This indicates a splicing operation.

[0061] Finally, through a second FFN network with residual connections, As a guide, with After addition and fusion, the final output of the dual-domain cascaded attention module is obtained after layer normalization and multilayer perceptron processing: spatial-frequency domain enhanced features. .

[0062] 4. Adaptive Key Modulation Module AKM

[0063] When applying SAM to camouflaged target detection, the core challenge lies in effectively injecting carefully optimized, task-specific features into this powerful and general-purpose decoder. Crucially, this process should ideally be accomplished without altering the pre-trained structure of the SAM decoder to maintain its superior generalization capabilities. Introducing new modules with a large number of parameters is not suitable for this goal, as it may disrupt the carefully learned feature space of the decoder on large-scale datasets.

[0064] To address this challenge, this invention proposes an Adaptive Keying Modulation (AKM) module, which enables task-specific adaptive adjustments while completely freezing the SAM decoder.

[0065] Specifically, the first step is to process multi-source features from ViT image embeddings, BCAM, and CNN branches (i.e. , and To align their spatial resolution and unify the channel dimensions (e.g., to 32) for subsequent fusion. This process can be described as follows:

[0066]

[0067]

[0068]

[0069] in It contains a transposed convolutional layer, followed by a two-dimensional normalization layer and the GELU activation function, k=1,2,…,4. This represents a transposed convolutional layer used to upsample the feature map and unify the channel dimension to 32, r=1,2,3.

[0070] Then, these multi-source features are fused into a unified and compact representation. This is used to guide the attention of the decoder subsequently.

[0071] 5. Improved SAM decoder

[0072] AKM will enhance its features (i.e. , and Encoding as a compact embedding The embedded To capture key positional information of camouflaged objects, the default positional encoding of the cross-attention key vector in the SAM decoder is replaced, resulting in an improved SAM decoder. This improved SAM decoder adjusts the cross-attention key of the SAM decoder, injecting accurate prior knowledge while maintaining compatibility with the original SAM structure. The improved SAM decoder requires no fine-tuning. These original positional encodings based on random Fourier features are content-independent. By replacing the original positional encodings, the general attention key vector is transformed into a task-aware attention key vector, specifically adapted for camouflaged target detection scenarios.

[0073] To save computing resources, the SAM network (including the SAM codec) mentioned above preferably uses Efficient SAM.

[0074] Preferably, by removing input parameters such as sparse cue markers and dense cue embeddings from all cross-attention pairs in the SAM decoder, the SAM network is adjusted to a fully automatic segmentation mode, allowing it to maintain excellent performance without relying on external cues. This demonstrates that the method provided by this invention can effectively guide a powerful SAM decoder, maintaining high-quality segmentation results even without manual cues.

[0075] The AKM module is designed with excellent scalability, providing a flexible architecture that can inspire other SAM-based vision tasks to incorporate additional cues, such as depth or text information, as location cues to further improve task-specific performance.

[0076] The training method for the aforementioned camouflage target detection system can employ conventional training methods, such as end-to-end training, phased training, pre-training + fine-tuning, etc. Taking the end-to-end training method as an example, this embodiment of the invention provides a training method for a camouflage target detection system based on a dual-domain segmentation model as described in any of the above embodiments, comprising:

[0077] The Neck layer of the SAM encoder, the CNN encoder, BCAM, and AKM were trained under supervision using the training dataset.

[0078] SAM was originally designed for multi-class target segmentation, employing a linear combination strategy of focal loss, Dice loss, and MSE loss (20:1:1) specifically for challenging samples such as small targets. However, this method has limitations in camouflaged target detection tasks: SAM often prioritizes segmenting small targets while ignoring large targets, occluded targets, and multiple targets.

[0079] Therefore, BiSAM employs a linear combination of BCE loss, Dice loss, and MSE loss; where BCE loss measures pixel-level differences, MSE minimizes the squared error, and Dice loss optimizes structural similarity. The total loss function is expressed as follows:

[0080]

[0081] in, For BCE loss, For the loss of dice, For MSE loss; The weights for each loss can be set according to actual needs. Pred represents the segmentation result generated by the BiSAM model, and GT is the true value of the segmentation result.

[0082] This invention provides a method for detecting camouflaged targets, comprising: inputting an image to be detected into a system as described in any of the above embodiments, and obtaining a camouflaged target detection result of the image to be detected.

[0083] The camouflaged target detection method provided in this embodiment of the invention (illustrated as BiSAM in the figure) and the existing advanced COD method DSAM based on SAM are used to detect camouflaged targets in five complex scenarios: occlusion, large target, small target, multiple targets, and dense edges. The detection results are as follows: Figure 4 As shown in (a), (b), (c), (d), and (e), it can be seen that the method provided by the present invention can better cope with complex and challenging scenarios, including multi-scale, occlusion, multiple objects, and dense edges, while effectively suppressing noise interference and focusing only on the true target.

[0084] This invention provides an electronic device, including: a computer-readable storage medium and a processor;

[0085] The computer-readable storage medium is used to store executable instructions;

[0086] The processor is used to read executable instructions stored in the computer-readable storage medium and execute the training method or camouflage target detection method as described in any of the above embodiments.

[0087] This invention provides a computer-readable storage medium storing computer instructions for causing a processor to execute a training method or a camouflage target detection method as described in any of the above embodiments.

[0088] This invention provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement the training method or camouflage target detection method as described in any of the above embodiments.

[0089] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A camouflage target detection system, characterized by Comprising: A SAM encoder for encoding an input image to obtain a ViT image embedding ; CNN encoder, configured to encode an input image to obtain local fine features ; a bi-cascade attention module (BCAM) comprising a spatial attention module and a spectral attention module; The spatial domain attention module comprises a first multi-head self-attention module SA1 and a first linear transformation layer, which are used for performing multi-head self-attention calculation on the input feature X to obtain a first multi-head self-attention feature , After being processed by the first linear transformation layer, the intermediate feature is obtained by adding X , X is a feature output by a first Transformer block in a plurality of cascaded Transformer blocks in the SAM encoder; a first FFN network with a residual connection, which is used for performing FFN processing on to obtain a spatial domain optimization feature map ; The spectrum attention module comprises a discrete wavelet transform module DWT, a first and a second convolution layer, a second to a fifth linear transformation layer, a second multi-head self-attention module SA2, and a second FFN network with a residual connection. a query vector after being processed by the second linear transformation layer as the jth attention head in the SA2 . a key vector after being processed by the first convolution layer, the DWT, the second convolution layer, and the third linear transformation layer in sequence as the jth attention head in the SA2 . a value vector after being processed by the first convolution layer, the DWT, the second convolution layer, and the fourth linear transformation layer in sequence as the jth attention head in the SA2 ; j = 1, 2, …, J, J being the number of attention heads of the second multi-head self-attention module; the second multi-head self-attention module is used for performing multi-head self-attention calculation on 、 and to obtain second multi-head self-attention features , , after being spliced, and performing processing by a fifth linear transformation layer to obtain frequency domain optimization features ; the second FFN network with a residual connection is used for performing FFN processing on and after being added and fused to obtain a spatial-frequency domain enhanced feature ; An adaptive key modulation module AKM is configured to modulate the fused feature , and to obtain a fused feature ; An improved SAM decoder that replaces the original positional encoding of the cross-attention key vectors of the SAM decoder with obtained, for decoding the ViT image embedding under the guidance of obtained, for decoding the ViT image embedding under the guidance of 2. The system of claim 1, wherein, Sparse prompt tokens and dense prompt embeddings are removed from all cross-attention in the improved SAM decoder.

3. A method of training a camouflage target detection system as claimed in claim 1 or 2, characterized in that, Comprising: The Neck layer, CNN encoder, BCAM and AKM of the SAM encoder are supervised trained with a training dataset.

4. The method of claim 3, wherein, The loss function employed for training is: wherein, is the BCE loss, is the dice loss, is the MSE loss; are the weights of each loss, Pred denotes the segmentation result generated by the BiSAM model, and GT is the true value of the segmentation result.

5. A method of detecting a camouflaged target, the method comprising: Comprising: inputting an image to be detected into the system of claim 1 or 2 to obtain a camouflage target detection result of the image to be detected.

6. An electronic device, comprising: Comprising: a computer readable storage medium and a processor; the computer readable storage medium is configured to store executable instructions; the processor is configured to read the executable instructions stored in the computer readable storage medium and execute the method of any one of claims 3-5.

7. A computer-readable storage medium, characterized in that, the computer readable storage medium stores computer instructions configured to cause the processor to execute the method of any one of claims 3-5.

8. A computer program product comprising computer programs or instructions, characterized in that, the computer program or instructions, when executed by the processor, implement the method of any one of claims 3-5.

Images