Reference camouflage target detection method and system based on cross-modal semantic guidance

Abstract

The application discloses a reference camouflage target detection method and system based on cross-modal semantic guidance, comprising: acquiring a query image and a reference image, the query image being a main image to be detected; performing preliminary feature extraction on the query image based on a multi-scale feature encoding and preliminary decoding method to obtain high-resolution detail features and low-resolution semantic features; extracting target category semantic prior from the reference image, and generating a reference semantic gating weight map that is completely matched with the visual feature dimension of the query image after multi-image aggregation, dimension and space double alignment; performing bidirectional cross-scale interaction on the high-resolution detail features and the low-resolution semantic features to obtain the features after interaction; and performing element-by-element modulation and feature splicing on the features after interaction by using the reference semantic gating weight map to obtain cross-modal fusion features; and generating a final camouflage target prediction map based on a detection head according to the cross-modal fusion features. The application improves the accuracy and robustness of camouflage target detection.

Description

Technical Field

[0001] This invention relates to the field of reference camouflage target detection technology, and specifically to a reference camouflage target detection method and system based on cross-modal semantic guidance. Background Technology

[0002] In fields such as agricultural pest and disease control (e.g., detecting armyworms camouflaged in green leaves), military and civilian target identification (e.g., locating snipers lurking in the grass), and ecological monitoring (e.g., tracking concealed wild animals in dense vegetation), the demand for reference-guided camouflaged object detection (RefCOD) is increasingly urgent. These tasks require the use of visual cues from reference targets (such as descriptions like "the stripes on the back of an armyworm" or "the texture of a sniper's ghillie suit") to accurately locate specific categories of camouflaged targets in scenes that are highly integrated with the background and visually confusing.

[0003] However, traditional detection methods face significant challenges in this task: general-purpose object detection algorithms (such as the YOLO series) struggle to effectively distinguish targets highly similar to the background; existing camouflaged object detection (COD) algorithms, while capable of identifying camouflaged objects, lack "reference cue guidance" capabilities and cannot perform targeted detection based on user-provided target examples; and the emerging RefCOD method is still in its early stages of development, with considerable room for improvement in detection accuracy and practicality. Specifically:

[0004] (i) General target detection methods cannot effectively segment highly camouflaged targets.

[0005] General object detection algorithms, such as YOLO and Faster R-CNN, mainly rely on the salient edges, color contrast, or texture differences of objects for identification. However, camouflaged targets have characteristics such as high similarity to the background, blurred edges, and blended textures, making it difficult for general detectors to extract effective discriminative features. This results in a very high false negative rate in complex camouflaged scenes, failing to meet the basic requirements of the RefCOD task.

[0006] (ii) Existing COD methods lack the ability to be guided by reference clues.

[0007] Existing camouflage target detection (COD) algorithms (such as SINet and PraNet) are specifically designed for camouflage characteristics and can detect camouflaged objects from a single image. However, their detection process lacks a "user-specified target" guidance mechanism. When it is necessary to detect specific types of camouflaged targets in a scene (such as detecting only "sticky bugs" instead of all insects), COD methods cannot filter targets based on reference examples, making it difficult to meet the refined detection needs of scenarios such as agricultural plant protection and military / civilian reconnaissance.

[0008] (iii) Existing RefCOD methods have limited detection effectiveness and are difficult to apply in practice.

[0009] Currently, the mainstream RefCOD methods mainly include R2CNet and UAT:

[0010] Limitations of R2CNet: As a pioneering work in the RefCOD field, R2CNet was the first to attempt to introduce reference images into the camouflage target detection task. However, its reference cue fusion method is relatively simple, mainly relying on feature concatenation and global average pooling. This coarse fusion fails to fully exploit the semantic complementarity between multiple reference images. When there are differences in viewpoint, lighting changes, or partial occlusion in the reference images, noise interference is easily introduced, resulting in an impure and unstable semantic guidance signal, and the detection effect is relatively average.

[0011] Limitations of UAT: While UAT introduces an uncertainty-aware mechanism on top of R2CNet, improving detection performance, its semantic fusion remains at a shallow interaction level, lacking dedicated feature interaction design for the characteristics of camouflaged targets. Specifically:

[0012] Weak multi-scale feature correlation: Using a simple top-down fusion strategy, it is difficult to establish semantic correlations between features of different scales at the region level. When the target is partially occluded or highly similar to the background texture, the feature response is scattered, which can easily lead to holes or incomplete edges in the detection results.

[0013] Rigid anchor point mechanism: Using fixed grid anchor points or global pooling operations cannot adaptively fit the irregular geometry of the camouflaged target. For camouflaged targets with variable shapes and non-rigidity, it is difficult to accurately capture their component-level features.

[0014] The semantic guidance is coarse-grained: it lacks a fine-tuning mechanism for visual features on an element-by-element basis, making it difficult to accurately enhance the disguised target area and suppress the background at the pixel level.

[0015] In summary, existing technologies have significant shortcomings in reference clue fusion, multi-scale feature interaction, and semantic guidance granularity: cross-modal semantic fusion is coarse and difficult to accurately guide visual features; multi-scale feature correlation is weak and it is difficult to cope with the scale variability and feature dispersion of camouflaged targets; and there is a lack of dedicated feature interaction mechanisms for the characteristics of camouflaged targets. These factors result in low accuracy and robustness of reference camouflaged target detection, making it difficult to meet the practical application needs in fields such as agricultural pest and disease control, military and civilian target identification, and ecological monitoring.

[0016] In view of the above, this application is hereby submitted. Summary of the Invention

[0017] The technical problem this invention aims to solve is that existing reference camouflage target detection methods suffer from several issues: crude cross-modal semantic fusion methods, making it difficult to accurately guide visual features; weak multi-scale feature correlation, making it difficult to cope with the scale variability and feature dispersion of camouflage targets; and a lack of dedicated feature interaction mechanisms tailored to the characteristics of camouflage targets. These problems result in low accuracy and robustness in reference camouflage target detection, making it difficult to meet the practical application needs in fields such as agricultural pest and disease control, military and civilian target identification, and ecological monitoring. The purpose of this invention is to provide a reference camouflage target detection method and system based on cross-modal semantic guidance. This method utilizes cross-modal semantic guidance and multi-scale feature interaction to improve the accuracy and robustness of reference camouflage target detection, enhance the regional semantic correlation of cross-scale features, and improve adaptability to changes in the geometry and structure of camouflage targets.

[0018] This invention is achieved through the following technical solution:

[0019] In a first aspect, the present invention provides a reference camouflage target detection method based on cross-modal semantic guidance, the method comprising:

[0020] Obtain the query image and the reference image, where the query image is the main image to be detected;

[0021] The query image is preliminarily feature extracted based on a multi-scale feature encoding and preliminary decoding method to obtain the visual features of the query image. The visual features of the query image include high-resolution detail features and low-resolution semantic features. The multi-scale feature encoding and preliminary decoding method includes input feature enhancement, multi-scale feature encoding and demodulation enhancement, cross-scale feature reshaping, and hierarchical grouping fusion decoding.

[0022] Extract semantic priors containing the target category from the reference image, and generate a reference semantic gating weight map that perfectly matches the visual feature dimensions of the query image after multi-image aggregation and dimensional and spatial alignment.

[0023] A bidirectional cross-scale interaction is performed on high-resolution detail features and low-resolution semantic features to obtain the interacted features; then, the interacted features are modulated element-wise and concatenated using a reference semantic gating weight map to obtain cross-modal fusion features; and

[0024] Based on cross-modal fusion features, a final camouflaged target prediction map is generated using the detection head.

[0025] Furthermore, based on multi-scale feature encoding and preliminary decoding methods, preliminary feature extraction is performed on the query image to obtain its visual features, including:

[0026] Lightweight dual-attention modulation is applied to the query image to obtain an enhanced feature map; lightweight dual-attention modulation is achieved by cascading channel attention and spatial attention.

[0027] The enhanced feature map is input into a multi-scale feature encoder for deep feature extraction to obtain feature maps at multiple scales; and each scale feature map is demodulated and enhanced to obtain a demodulated and enhanced feature map.

[0028] The spatial structure of the demodulated and enhanced feature maps is restored by dimensional transformation, resulting in a set of two-dimensional feature maps with a uniform format.

[0029] Based on the single-level fusion rule, a top-down multi-level progressive fusion is performed on a set of two-dimensional feature maps with a uniform format. The temporary features obtained from the single-level fusion of the previous level are used as the input of the next level of fusion. After multiple rounds of iteration, an intermediate feature map is obtained. A channel attention mechanism is introduced to adaptively recalibrate the intermediate feature map to obtain an updated fused intermediate feature map.

[0030] The updated fused intermediate feature maps are grouped and aggregated according to their resolution to generate high-resolution detail features and low-resolution semantic features respectively.

[0031] Furthermore, the multi-scale feature encoder adopts a hierarchical pyramid structure, which consists of multiple encoding stages. Each encoding stage includes downsampling embedding and multiple Transformer modules, and achieves the interaction and fusion of multi-scale information through cross-stage connections.

[0032] Furthermore, a reference semantic gating weight map containing the target category semantic prior is extracted from the reference image and generated after multi-image aggregation and dimensional and spatial alignment, which perfectly matches the visual feature dimensions of the query image. This includes:

[0033] Extract semantic priors containing the target category from the reference image, use a pre-trained multimodal visual language model as a semantic extractor, and perform global semantic feature extraction on the reference image based on the semantic extractor to obtain a global semantic feature map of common categories;

[0034] The global semantic feature map is reduced in channel dimension by convolutional layers to obtain the reference features after channel dimension reduction.

[0035] Bilinear interpolation upsampling is used to enlarge the reference features after channel dimensionality reduction to the same spatial size as the query image features;

[0036] The Sigmoid activation function is then used to map each element value of the dimensionally and spatially aligned reference feature to... The interval generates a reference semantic gating weight map that perfectly matches the visual feature dimensions of the query image.

[0037] Furthermore, based on the semantic extractor, global semantic features are extracted from the reference image to obtain semantic feature maps of common categories, including:

[0038] The reference image is reshaped into a batch and image quantity combined form to obtain a flattened reference image;

[0039] The flattened reference image is fed into the semantic extractor to obtain the global semantic feature vectors corresponding to all reference images; the semantic extractor uses a pre-trained multimodal visual language model.

[0040] The global semantic feature vector is reshaped back to the batch dimension and the image quantity dimension, and a multi-head cross-attention mechanism is introduced to calculate the aggregated semantic features;

[0041] The aggregated semantic features are then reshaped in terms of feature dimensions to obtain a semantic feature map of common categories.

[0042] Furthermore, bidirectional cross-scale interaction is performed on high-resolution detail features and low-resolution semantic features to obtain the interacted features, including:

[0043] The interaction process is performed using high-resolution detail features as the primary feature and low-resolution semantic features as the secondary feature to obtain the first interaction feature. ;

[0044] The interaction process is performed using low-resolution semantic features as the main features and high-resolution detail features as auxiliary features to obtain the second interaction feature. ;

[0045] The interaction process is as follows:

[0046] Based on auxiliary features, a spatial attention weight map is generated;

[0047] Channel dimensionality reduction is performed on the main features to obtain the dimensionality-reduced features; the dimensionality-reduced features are then weighted using a spatial attention weight map to generate mask features;

[0048] Dynamically deformable anchors are generated from mask features using a lightweight subnetwork;

[0049] Calculate the association weight matrix between each anchor point and the global semantic features;

[0050] Based on the association weight matrix, the main features are projected onto the anchor space to obtain the anchor state features;

[0051] The Transformer self-attention module is used to interact with the anchor point state features to obtain the features after Transformer interaction;

[0052] The features after the Transformer interaction are projected back to the original spatial dimensions through the association weight matrix, and then... After adjusting the number of channels through convolution, residual connections are made with the main features to obtain the interactively enhanced features.

[0053] Furthermore, the Transformer self-attention module is used to interact with the anchor point state features, as expressed in the following expression:

[0054] ;

[0055] in, ;

[0056] In the formula, Features resulting from Transformer interaction; The Transformer self-attention module consists of a multi-head self-attention layer and a feedforward network. It allows each anchor point to exchange information with all other anchor points and dynamically models long-distance dependencies between anchor points. Anchor point state characteristics; For flattening operation; The features are dimensionality reduced; W is the correlation weight matrix. This represents the transpose of the correlation weight matrix; For real number field identifier, indicating It is a three-dimensional tensor whose elements are composed of real numbers, and its dimension is . B stands for Batch size. The number of feature channels after projection; K is the number of anchor points.

[0057] Furthermore, the interactive features are modulated element-wise and concatenated using a reference semantic gating weight graph to obtain cross-modal fusion features, including:

[0058] Using the reference semantic gated weight map as a spatial modulation signal, the interactive features are weighted element by element to obtain two modulated features.

[0059] The two modulated features are concatenated along the channel dimension to obtain the concatenated features.

[0060] By using a multi-level convolutional structure to deeply refine the concatenated features, cross-modal fusion features are obtained.

[0061] Secondly, the present invention provides a reference camouflage target detection system based on cross-modal semantic guidance, the system comprising:

[0062] The acquisition module is used to acquire the query image and the reference image, where the query image is the main image to be detected.

[0063] The multi-scale feature encoding and preliminary decoding module is used to perform preliminary feature extraction on the query image based on the multi-scale feature encoding and preliminary decoding method to obtain the visual features of the query image. The visual features of the query image include high-resolution detail features and low-resolution semantic features. The multi-scale feature encoding and preliminary decoding method includes input feature enhancement, multi-scale feature encoding and demodulation enhancement, cross-scale feature reshaping, and hierarchical group fusion decoding.

[0064] The reference image semantic feature extraction and adaptation module is used to extract semantic priors of the target category from the reference image, and generate a reference semantic gating weight map that perfectly matches the visual feature dimensions of the query image after multi-image aggregation and dimensional and spatial alignment.

[0065] The cross-modal feature interaction and semantic-guided modulation module is used to perform bidirectional cross-scale interaction between high-resolution detail features and low-resolution semantic features to obtain the interacted features; and to use a reference semantic gating weight map to perform element-wise modulation and feature concatenation on the interacted features to obtain cross-modal fusion features.

[0066] The camouflage target detection module is used to generate a final camouflage target prediction map based on the detection head, according to cross-modal fusion features.

[0067] Thirdly, the present invention provides a computer-readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the above-described reference camouflage target detection method based on cross-modal semantic guidance.

[0068] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0069] This invention relates to a reference camouflage target detection method and system based on cross-modal semantic guidance. It improves the accuracy and robustness of the detection by using cross-modal semantic guidance and multi-scale feature interaction, enhances the regional semantic association of cross-scale features, and improves the adaptability to changes in the geometry and structure of the camouflage target.

[0070] (1) Improve the accuracy and robustness of semantic guidance

[0071] This invention introduces a multi-head cross-attention mechanism in step S3 to adaptively aggregate multiple reference images. The network can dynamically focus on common key semantic information in different reference images, effectively suppressing noise interference from individual images and generating a purer and more discriminative semantic prior. Subsequently, this semantic prior is transformed into a gated weight map, and visual features are modulated element-wise in module three, achieving precise pixel-level guidance of "semantic matching region enhancement and background region suppression," significantly improving detection accuracy.

[0072] (2) Enhance regional semantic associations of cross-scale features

[0073] In step S4, this invention designs a bidirectional cross-scale interaction mechanism and innovatively introduces dynamic deformable anchor points and Transformer self-attention interaction. First, the deformable anchor points can adaptively distribute themselves across key parts of the camouflaged target, rather than being limited to a fixed grid, thereby accurately capturing the target's component-level features. Second, by using Transformer to interact with the anchor point features, long-range dependencies between different parts of the target can be dynamically modeled. Even when occluded or highly similar to the background, the complete target region can be inferred through strongly correlated components, effectively solving the feature dispersion problem and improving the completeness of detection.

[0074] (3) Enhance the ability to adapt to changes in the geometry and structure of camouflaged targets.

[0075] Dynamically deformable anchors enable the network to automatically adjust its receptive field and sampling position according to the specific shape of the target, exhibiting stronger geometric adaptability to non-rigid, shape-variable camouflage targets (such as animals and plants). This adaptive anchor generation method, combined with bidirectional cross-scale interaction, allows the network to capture fine edge textures using high-resolution features while also grasping the global context using low-resolution features, thereby achieving accurate segmentation of camouflage target boundaries in complex backgrounds.

[0076] (4) Achieve end-to-end overall optimization and multi-level supervision

[0077] The multi-level structure-aware loss function designed in this invention not only focuses on pixel-level classification accuracy but also strengthens the constraint on the overall structure of the target through weighted intersection-over-union loss. Simultaneously, auxiliary supervision is applied to intermediate features, enabling gradients to be effectively backpropagated to the shallow layers of the network, alleviating the gradient vanishing problem and ensuring that each stage of the network learns feature representations with target-aware capabilities, thereby improving the convergence speed and final accuracy of the overall model. Attached Figure Description

[0078] The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and form part of this application, do not constitute a limitation thereof. In the drawings:

[0079] Figure 1 This is a flowchart illustrating the overall process of the cross-modal semantic-guided reference camouflage target detection method of the present invention.

[0080] Figure 2 This is a detailed flowchart of the reference camouflage target detection method based on cross-modal semantic guidance of the present invention;

[0081] Figure 3 This is a flowchart of step S2 in Embodiment 1 of the present invention;

[0082] Figure 4 This is a flowchart of step S3 in Embodiment 1 of the present invention;

[0083] Figure 5 This is a flowchart of step S4 in Embodiment 1 of the present invention;

[0084] Figure 6 This is a block diagram of the reference camouflage target detection system based on cross-modal semantic guidance according to the present invention. Detailed Implementation

[0085] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0086] Existing reference camouflage target detection technologies mainly suffer from the following technical problems:

[0087] (i) Cross-modal semantic fusion methods are crude and difficult to accurately guide visual features.

[0088] Existing methods often employ simple feature concatenation or global average pooling when processing reference images, failing to fully consider the semantic complementarity and differences between multiple reference images. This coarse-grained fusion approach easily introduces noise interference from individual images, resulting in impure and unstable semantic guidance signals. Furthermore, directly concatenating semantic and visual features lacks a refined element-wise modulation mechanism, making it difficult to accurately enhance and suppress the background of the camouflaged target region at the pixel level.

[0089] (ii) The weak correlation of multi-scale features makes it difficult to cope with the scale variability and feature dispersion of camouflaged targets.

[0090] Camouflaged targets often exhibit drastic scale variations, and their edges, textures, and other detailed features are highly indistinguishable from the background. Existing Feature Pyramid Fusion (FPN) techniques are mostly simple top-down summations or stitching, lacking region-level semantic association modeling. When a target is partially occluded or has a texture similar to the background, its feature responses at different scales are often scattered, making it difficult to form a coherent target representation, resulting in gaps or incomplete edges in the detection results.

[0091] (iii) Lack of dedicated feature interaction mechanisms for camouflaged target characteristics

[0092] General cross-scale interaction methods (such as simple attention mechanisms) struggle to effectively capture long-range dependencies between different parts of a camouflaged target (such as the head, torso, and limbs of an animal). Fixed-grid anchor points or pooling operations cannot adaptively fit the irregular shape of the camouflaged target, resulting in extracted anchor point features that cannot accurately represent the various semantic components of the target, thus limiting the effectiveness of subsequent feature interactions.

[0093] Therefore, there is an urgent need for a detection method and system that can deeply integrate reference target cues and accurately address the characteristics of camouflaged targets such as "fuzzy features, variable scales, and easy confusion with the background". The method of this invention improves the detection rate and reliability of RefCOD in complex camouflage scenarios through innovative cross-modal semantic guidance and multi-scale feature interaction mechanism, and provides technical support for practical applications in fields such as agricultural plant protection, national defense security, and ecological protection.

[0094] The system of this invention includes an acquisition module, a multi-scale feature encoding and preliminary decoding module, a reference image semantic feature extraction and adaptation module, a cross-modal feature interaction and semantic-guided modulation module, and a camouflaged target detection module, such as... Figure 1 As shown, the functions and processes of the main modules of the system of this invention are as follows:

[0095] The multi-scale feature encoding and preliminary decoding module is the visual feature base of the detection network. Through four-level progressive processing, it transforms the main image to be detected (i.e. the query image) into high / low resolution core visual features (i.e. high-resolution detail features and low-resolution semantic features) that have both fine-grained details and global semantics, providing high-quality visual representations for subsequent cross-modal fusion.

[0096] The reference image semantic feature extraction and adaptation module, as a semantic guidance unit, extracts category semantic priors from the reference image. After multi-image aggregation and dimensional and spatial alignment, it generates a reference semantic gating weight map that perfectly matches the visual feature dimensions of the main image, providing accurate semantic guidance for the selection of target regions.

[0097] The cross-modal feature interaction and semantic-guided modulation module is the core decision-making unit of the detection network. It takes the visual features (i.e., high-resolution detail features and low-resolution semantic features) output by the multi-scale feature encoding and preliminary decoding module and the reference semantic gating weight map output by the reference image semantic feature extraction and adaptation module as dual inputs. It strengthens the cross-scale feature correlation of the target through bidirectional cross-scale feature interaction, and then uses semantic gating weights to selectively enhance and suppress the interaction features. Finally, it outputs cross-modal features that fuse visual details and semantic priors, which directly support the generation of subsequent camouflage target prediction maps.

[0098] Example 1

[0099] like Figure 2As shown, the present invention provides a reference camouflage target detection method based on cross-modal semantic guidance, which includes:

[0100] S1, obtain the query image and reference image;

[0101] In this embodiment, the query image is the main image to be detected. For example, the query image is a soldier wearing camouflage, and the corresponding reference image is any explicit human image. Or, for example, the query image is a camouflaged agricultural pest, and the corresponding reference image can be an explicit image of the pest taken under any circumstances, such as a laboratory image.

[0102] S2, based on the multi-scale feature encoding and preliminary decoding method, preliminary feature extraction is performed on the query image to obtain the visual features of the query image. The visual features of the query image include high-resolution detail features and low-resolution semantic features. The multi-scale feature encoding and preliminary decoding method includes input feature enhancement, multi-scale feature encoding and demodulation enhancement, cross-scale feature reshaping and hierarchical group fusion decoding.

[0103] Step S2 serves as the visual feature extraction foundation for the entire detection network. Addressing the core technical challenges of camouflaged targets in complex backgrounds—characteristic blurring, scale variation, and easy indistinguishability from the background—it constructs a four-level progressive processing architecture: "input feature enhancement → multi-scale feature encoding and demodulation enhancement → cross-scale feature reshaping → hierarchical grouping fusion decoding." This transforms the original input image (i.e., the query image) into high- and low-resolution core features (i.e., high-resolution detail features and low-resolution semantic features) that simultaneously contain fine-grained edge details and global contextual semantics, providing a high-quality visual representation foundation for subsequent cross-modal semantic guidance. The flowchart of step S2 is as follows: Figure 3 As shown.

[0104] In this embodiment, step S2 specifically includes:

[0105] (a) Input

[0106] The input is the main image to be detected (query image), denoted as a tensor. Its dimensions are ,in:

[0107] Indicates the batch size.

[0108] Indicates the number of RGB color channels;

[0109] and These represent the height and width of the image being queried, respectively.

[0110] (II) Core Processing Flow

[0111] S21, Input Feature Enhancement

[0112] To suppress interference from complex backgrounds and initially highlight the camouflaged target, the query image is... Lightweight dual-attention modulation is performed to obtain the enhanced feature map; lightweight dual-attention modulation is achieved through the cascading of channel attention and spatial attention; specifically as follows:

[0113] First, calculate the channel attention weights. :

[0114]

[0115] in:

[0116] This indicates global average pooling, which converts the input image... spatial dimensions compressed to This yields the channel descriptor;

[0117] and For two consecutive Convolution is used to reduce and increase the dimensionality of channels, respectively, to form a bottleneck structure.

[0118] To activate the linear rectification function, a nonlinearity is introduced;

[0119] The sigmoid activation function maps the output value to... The interval is used to obtain the channel attention weights. The weight of each channel indicates the importance of that channel's features to the detection task.

[0120] Subsequently, channel attention is applied to the input query image. And further extract spatial attention weights. :

[0121]

[0122] in:

[0123] This indicates the channel attention weights. Multiply the input image channel by channel This enables feature recalibration at the channel dimension.

[0124] and Also Convolution, used to extract spatial attention weights The weight of each spatial location represents the degree of contribution of that location to the detection task.

[0125] Finally, the query image will be... Residual fusion is performed with the attention-enhanced features to obtain the enhanced feature map. :

[0126]

[0127] in:

[0128] This represents the feature map after undergoing both channel and spatial attention weighting;

[0129] These are the modulation coefficients, used to balance the contributions of the original features and the enhanced features, avoiding over-modulation that could lead to the loss of original information.

[0130] The enhanced output feature map retains the original structure while highlighting potential target regions.

[0131] S22, Multi-scale Feature Coding and Demodulation Enhancement

[0132] S221, the enhanced feature map Deep feature extraction is performed on the input multi-scale feature encoder to obtain feature maps at multiple scales;

[0133] In this embodiment, the multi-scale feature encoder adopts a hierarchical pyramid structure, which consists of multiple encoding stages. Each encoding stage includes downsampling embedding and multiple Transformer modules, and the interaction and fusion of multi-scale information are realized through cross-stage connections.

[0134] Specifically, step S221 is as follows:

[0135] Let the first The feature map of the level output is Then the first The encoding process at each level can be abstracted as follows:

[0136]

[0137] in:

[0138] Indicates the first A level of embedding layer is used to downsample the input feature map and convert it into a token sequence;

[0139] This represents a module composed of multiple Transformer Blocks, where the computation of each Block can be represented as:

[0140]

[0141]

[0142] in, For layer normalization, For the sake of the bulls' self-attention, It is a multilayer perceptron;

[0143] For the first The token characteristics of the level output; For the first The token characteristics of the level output; For the first In the Transformer Block, the intermediate transition token features, after being processed by the multi-head self-attention module and fused with the input features using residuals, serve as the input for the subsequent multilayer perceptron.

[0144] After all encoding stages, a set of multi-scale token features is obtained, which is then reshaped into a two-dimensional feature map, denoted as . ,in For the number of scales (n=4 in this implementation scheme), each A feature map corresponding to a specific resolution, .

[0145] S222, In order to further enhance the discriminativeness of the camouflaged target in the feature map, demodulation enhancement is performed on the feature map at each scale to obtain the demodulated and enhanced feature map;

[0146] Specifically, step S222 is as follows:

[0147] First of all, Multiple convolutional operations with different dilation rates are applied to extract multi-receptive field features in parallel:

[0148]

[0149] in:

[0150] Indicates the void ratio Convolutional operations are used to capture contextual information at different scales;

[0151] Features extracted corresponding to the void ratio.

[0152] Subsequently, features from different receptive fields are adaptively fused using learnable attention weights:

[0153]

[0154] in:

[0155] For learnable attention weights, satisfying This is used to dynamically adjust the contribution of different receptive field features;

[0156] The demodulated and enhanced feature map is further enhanced by improving the response of the camouflaged target region and suppressing background noise.

[0157] S23, Cross-scale Feature Reconstruction

[0158] In this embodiment, because the multi-scale feature maps output by the multi-scale feature encoder differ in the number of channels and spatial size, they cannot be directly fused and need to be uniformly converted into a format that facilitates subsequent processing. For each demodulated feature map If the current feature map is in token sequence format, then the spatial structure of the demodulated and enhanced feature map is restored through dimensionality transformation, resulting in a set of two-dimensional feature maps with a uniform format. ;

[0159] S24, Layered Group Fusion Decoding

[0160] In this embodiment, step S24 unifies the two-dimensional feature map set according to the format of the output of step S23. As input, to fully utilize the complementary advantages of multi-scale features, a top-down grouping and fusion strategy is adopted, passing deep semantic features layer by layer to shallow detail features. Taking one level of fusion as an example, let the two feature maps to be fused be... (From shallower layers, higher resolution) and (From deeper layers, lower resolution), the fusion process is as follows:

[0161] First, low-resolution features Through upsampling operation Magnify to Same space dimensions:

[0162]

[0163] in Bilinear interpolation upsampling is used to preserve the continuity of semantic information.

[0164] Then, Compared with upsampling By concatenating along the channel dimension, the fused features are obtained:

[0165]

[0166] in This indicates a concatenation operation along the channel dimension. The number of channels is the sum of the two.

[0167] Finally, through the convolution refinement module The splicing features are fused and compressed:

[0168]

[0169] in Composed of convolutional layers, batch normalization, and ReLU activation functions in series, this method performs non-linear fusion of the concatenated features and compresses the number of channels to a preset dimension, resulting in a single-level fused temporary feature. .

[0170] Based on the above single-level fusion rules, a set of two-dimensional feature maps with a unified format is processed. Perform a top-down, multi-level progressive fusion, combining the single-level fused temporary features obtained from the previous level of fusion. As input for the next level of fusion, after multiple iterations, an intermediate feature map is obtained, denoted as... (The first subscript indicates the 3rd fusion stage of this step, and the second subscript indicates the resolution level from 1 to 4, with the resolution decreasing sequentially as the number increases), corresponding to the fusion results of different resolutions and semantic levels. In the final fusion stage, the feature map after fusion at resolution level 3 is taken. To further highlight the target region, a channel attention mechanism is introduced to adaptively recalibrate the intermediate feature maps:

[0171]

[0172] in, This is a channel attention module that learns the importance weights of each channel and weights the feature maps accordingly, so that channels that contribute more to the detection task receive higher responses. The recalibrated... Replacement of the original No. 3 resolution fusion feature The updated fusion intermediate feature map is obtained. .

[0173] S25, Feature Aggregation and Refinement Output

[0174] In this embodiment, step S25 groups and aggregates the updated fused intermediate feature maps according to their resolution, generating high-resolution detail features and low-resolution semantic features respectively.

[0175] High-resolution detail feature aggregation:

[0176]

[0177] in This is a high-resolution refining module, consisting of multiple convolutional layers, batch normalization, and ReLU activation functions. It is used to perform non-linear mapping and channel compression on the aggregated features, thereby enhancing detailed features.

[0178] Low-resolution semantic feature aggregation:

[0179]

[0180] in For low-resolution refining modules, the structure and The same applies, used to enhance semantic features.

[0181] In a preferred embodiment, to further improve feature quality, it is possible to... and Then, the concatenated feature refinement module :

[0182]

[0183] in By combining depthwise separable convolution with channel attention, features are fine-tuned using residual connections to further enhance their expressive power.

[0184] (III) Output

[0185] High-resolution detail features It preserves the fine spatial structure of the input image, focusing on the edge and texture details of the camouflaged target; low-resolution semantic features During the encoding process, progressive downsampling is performed, resulting in rich global contextual semantic information. To enable subsequent cross-scale interaction and semantic gating modulation, step S2 performs bilinear interpolation upsampling on the low-resolution features before output, enlarging their spatial size to a scale similar to... The same This allows for the acquisition of high- and low-resolution feature combinations with consistent dimensions. Both sets of feature channels have the same number of channels. Ensure dimensional alignment.

[0186] The final output consists of two sets of core visual features:

[0187] High-resolution detail features This feature map maintains a large spatial size and focuses on fine-grained information such as the edges and textures of the camouflaged target, providing detailed support for subsequent accurate localization.

[0188] Low-resolution semantic features This feature map has the same channel dimension and spatial size as high-resolution features, but contains richer global context and semantic information, providing semantic guidance for subsequent target confirmation.

[0189] The two sets of features together constitute a complete visual representation of the original input image, which is passed as the core input to the cross-modal fusion in the subsequent step S4.

[0190] S3 extracts semantic priors containing the target category from the reference image, and generates a reference semantic gating weight map that perfectly matches the visual feature dimensions of the query image after multi-image aggregation and dimensional and spatial alignment.

[0191] Step S3, serving as the semantic guidance unit of the entire detection network, addresses the modal differences (semantic and visual features) and dimensionality / spatial mismatch between the reference and main images. It employs a progressive processing flow: "global semantic extraction → robust multi-image aggregation → channel dimensionality reduction → spatial upsampling → gating weight generation." This transforms the target category semantic priors contained in the reference image into adaptive gating weights that perfectly match the core features of the main image, providing precise semantic guidance for subsequent cross-modal feature modulation. The flowchart for step S3 is as follows: Figure 4 As shown.

[0192] In this embodiment, step S3 specifically includes:

[0193] (a) Input

[0194] The input is a set of reference images containing semantic information about the target category, denoted as a tensor. ,in:

[0195] Batch size;

[0196] The number of reference images corresponding to each sample. ;

[0197] This refers to the number of RGB color channels;

[0198] and These are the height and width of the reference image, respectively, and can be any size. Step S3 has no special requirements for the input size.

[0199] (II) Core Processing Flow

[0200] S31, Global Semantic Feature Extraction and Robust Multi-Graph Aggregation

[0201] In this embodiment, in order to extract stable and generalizable semantic representations from the reference images, step S31 extracts semantic priors containing the target category from the reference images, uses a pre-trained multimodal visual language model (such as CLIP) as a semantic extractor, performs global semantic feature extraction on the reference images based on the semantic extractor, and replaces simple feature averaging with a multi-head attention aggregation strategy to fuse the semantic information of multiple reference images in a more refined way to obtain a global semantic feature map of category commonality.

[0202] Specifically, step S31 includes:

[0203] S311, Batch flattening and feature extraction of reference images

[0204] First, the reference image The image is reshaped into a batch and image quantity combination form so that features of all images can be extracted at once to obtain a flattened reference image;

[0205]

[0206] in This indicates a dimension reshaping operation that flattens the original five-dimensional tensor into four dimensions and merges the batch dimension and the number of images.

[0207] Next, the flattened reference image is fed into the semantic extractor. (Parameter freezing) yields the global semantic feature vectors corresponding to all reference images;

[0208]

[0209] in, This represents the encoding function of a pre-trained multimodal visual language model (such as CLIP), which outputs D-dimensional global semantic features.

[0210] In addition, pre-trained multimodal visual language models can also employ more advanced multimodal models (such as Florence and BLIP-2) to not only extract visual features, but also combine category text prompts (such as "camouflagedanimal") to perform cross-modal contrastive learning and generate more discriminative semantic priors.

[0211] In addition, the target region in the reference image can be finely segmented or detected, and features of multiple local regions (such as head and torso) can be extracted. These local features can then be used as a set of semantic tokens to input into subsequent steps to achieve more granular semantic guidance.

[0212] S312, Robust aggregation of multi-graph features

[0213] The global semantic feature vector obtained in step S311 is reconstructed back to the batch dimension and the image number dimension, resulting in... To adaptively fuse information from multiple reference images, a multi-head cross-attention mechanism is introduced. A learnable query vector is defined. ,Will Using the key and value as keys, calculate the aggregated semantic features:

[0214]

[0215] This operation allows the network to dynamically focus on key information in different reference images according to task requirements, making it more discriminative than simple average pooling. The results are then compressed back... .

[0216] S313, Feature Dimension Reshaping

[0217] To adapt to the two-dimensional format required for subsequent convolution operations, the aggregated semantic features are converted from one-dimensional vectors into two-dimensional feature maps (size: ):

[0218]

[0219] in This means adding a dimension to the last dimension, and after two consecutive operations, the resulting shape is... The semantic feature map of the commonalities of categories, with a spatial size of The number of channels is .

[0220] S32, Channel Dimensionality Reduction Adaptation

[0221] In this embodiment, the core features of the output main image and The number of channels is (In this embodiment) ), while the current reference semantic features are The number of channels in the two systems does not match. Therefore, a solution is found... Convolutional layers perform channel dimensionality reduction on the global semantic feature map, supplemented by batch normalization and non-linear activation, to enhance the expressive power of the features and obtain reference features after channel dimensionality reduction.

[0222]

[0223] in:

[0224] Indicates the core size is Convolution, input channels Output channel ;

[0225] Batch normalization is used to stabilize the feature distribution;

[0226] To activate the linear rectification function, a nonlinear transformation is introduced;

[0227] The reference feature after dimensionality reduction of the channel still has a spatial size of The number of channels has been aligned with the core features of the main image.

[0228] S33, space-size adaptation

[0229] The spatial size of the core features of the main image is (In this embodiment) ), while the spatial size of the current reference feature is only Element-by-element operations cannot be performed directly. This step uses bilinear interpolation upsampling to enlarge the channel-reduced reference features to the same spatial size as the query image features.

[0230]

[0231] in, This is a bilinear interpolation upsampling operation. Specify the target spatial size. This operation expands the reference features in spatial dimensions to the same size as the main image features, laying the foundation for subsequent element-wise modulation.

[0232] S34, Semantic Gated Weight Generation

[0233] To transform the reference semantic prior into quantifiable feature modulation weights, a sigmoid activation function is used to map each element value of the dimensionally and spatially aligned reference feature to... For each interval, a reference semantically gated weighted graph is generated that perfectly matches the visual feature dimensions of the query image.

[0234]

[0235] in The Sigmoid function is defined as follows: Output Each element takes a value between 0 and 1. Its physical meaning is as follows: the closer the weight value is to 1, the higher the matching degree between the corresponding spatial location and the semantics of the reference target. The main image features at this location will be enhanced in the future. The closer the weight value is to 0, the more the location belongs to the background area. The feature response at this location will be suppressed in the future.

[0236] (III) Output

[0237] Output reference semantic gating weight graph Its dimensions are the same as the core features of the main image output in step S2. , Completely identical, that is:

[0238] Number of channels (Consistent with the core features of the main image);

[0239] Space Dimensions (Consistent with the core features of the main image);

[0240] Numerical range .

[0241] This reference semantic gating weight graph As the core carrier of cross-modal semantic guidance, it will be sent to step S4 to selectively enhance and suppress the visual features of the main image, so as to achieve deep fusion of reference semantic priors and visual features.

[0242] S4 performs bidirectional cross-scale interaction between high-resolution detail features and low-resolution semantic features to obtain the interactive features; and uses a reference semantic gating weight map to perform element-wise modulation and feature concatenation on the interactive features to obtain cross-modal fusion features.

[0243] Step S4 is the core decision unit of the entire detection network, responsible for converting the high and low resolution visual features (i.e., high-resolution detail features) of the main image output from step S2. Low-resolution semantic features The reference semantic gating weight graph output from step S3 Deep fusion is performed. Addressing the core challenges of "highly confused features with the background and weak cross-scale feature correlation" in camouflaged targets, this step designs a three-level progressive processing architecture: "cross-scale feature interaction → semantic gating modulation → multi-level feature refinement (i.e., feature splicing and deep refinement)." A dynamic region-level graph structure is used to enhance the cross-scale continuity of the target region, and semantic weights are used to selectively enhance and suppress visual features. The final output is a cross-modal fusion feature that combines fine details with strong semantic discriminativeness, providing high-quality feature representation for the accurate localization of camouflaged targets. The flowchart for step S4 is as follows: Figure 5 As shown.

[0244] In this embodiment, step S4 specifically includes:

[0245] (a) Input

[0246] Receive three sets of input tensors, all from the outputs of previous steps S2 and S3:

[0247] High-resolution visual features The output of step S2 focuses on the fine-grained edge and texture information of the camouflaged target.

[0248] Low-resolution visual features The output of step S2 contains richer global context and semantic information, with greater dimensionality and... same.

[0249] Reference semantic gating weights The output of step S3 shows that each element takes the value of... The degree of matching between the corresponding spatial location and the semantics of the reference target is represented.

[0250] (II) Core Processing Flow

[0251] S41, Cross-scale feature interaction

[0252] To enhance the feature correlation of camouflaged targets at different scales, this step fuses high- and low-resolution features through a two-way interactive mechanism. The interactive process uses one set of features as the main features (retaining the main information) and another set of features as auxiliary features (providing supplementary information). Through operations such as dynamic deformable anchor point generation and Transformer self-attention aggregation, complementary enhancement of cross-scale information is achieved.

[0253] When high-resolution detail features are the primary focus:

[0254] = (Main characteristics);

[0255] = (Auxiliary features);

[0256] When low-resolution semantic features are the primary focus:

[0257] = (Main characteristics);

[0258] = (Auxiliary features);

[0259] In this embodiment, step S41 specifically includes:

[0260] S411, Spatial Attention Weight Generation

[0261] With auxiliary features Based on this, a spatial attention weight map is generated for filtering main features. The key area in. First, through a Convolution will assist features The number of channels is compressed to 1, resulting in a single-channel attention map. Subsequently, the attention map is Softmax normalized in the spatial dimension to generate a spatial attention weight map:

[0262]

[0263] The value of each spatial location in the spatial attention weight map represents the relative importance of that region in the auxiliary features, which is used for subsequent anchor point extraction.

[0264] S412, Dynamic Deformable Anchor Point Generation

[0265] For main features Perform channel dimensionality reduction to obtain the reduced features. (In this embodiment) Subsequently, spatial attention weights were combined. Weight the dimensionality-reduced features to generate mask features:

[0266]

[0267] To more flexibly represent the target region, instead of using a fixed number of anchor points and global pooling, a deformable anchor point generation strategy is adopted. This is achieved through a lightweight subnetwork (e.g., two...). (convolutional layers) from mask features direct regression Offset of each anchor point and corresponding anchor features That is, dynamically deformable anchor points are generated from mask features through a lightweight subnetwork; this The anchor points can be adaptively distributed at any location on the target, rather than being limited to fixed grid points, thereby more accurately capturing the geometric and semantic structure of the target (to balance computational overhead and representation capability, in this embodiment...). ).

[0268] S413, Anchor Point-Global Feature Association

[0269] Calculate the association weight matrix between each anchor point and the global semantic features:

[0270]

[0271] The correlation weight matrix This represents the probability that each spatial location belongs to each anchor point region.

[0272] S414, Transformer Self-Attention Feature Interaction

[0273] Projecting the main features onto the anchor point space yields the anchor point state features:

[0274]

[0275] in, Anchor point state characteristics; For flattening operation; The features are dimensionality reduced; W is the correlation weight matrix. This represents the transpose of the correlation weight matrix; For real number field identifier, indicating It is a three-dimensional tensor whose elements are composed of real numbers, and its dimension is . B stands for Batch size. The number of feature channels after projection; K is the number of anchor points.

[0276] To enhance the semantic connections between anchor points, the traditional graph convolutional network (GCN) was abandoned in favor of the more expressive Transformer self-attention module. Interact with the anchor features to obtain the features after Transformer interaction:

[0277]

[0278] in, Features resulting from Transformer interaction; Composed of a multi-head self-attention layer and a feedforward network, it allows each anchor point to exchange information with all other anchor points, dynamically modeling long-distance dependencies between anchor points, making it more flexible than GCNs with predefined graph structures.

[0279] S415, Residual Fusion and Feature Reconstruction

[0280] Features after Transformer interaction Through the correlation weight matrix Project back to the original spatial dimensions, and through After adjusting the number of channels through convolution, a residual connection is made with the main feature to obtain the interactively enhanced feature. :

[0281]

[0282] in, Main features; Used to reduce the number of channels from the projected feature channel count Restore to the original number of channels , ( ) represents a dimension reshaping operation.

[0283] S416, bidirectional interactive execution

[0284] Each with high-resolution detail features Main features, low-resolution semantic features To assist in performing the above interaction process, the first interaction feature is obtained. ;

[0285] Then use low-resolution semantic features Main features, high-resolution detail features To perform the interaction for the auxiliary features, the second interaction feature is obtained. .

[0286] Both feature dimensions are 1. .

[0287] The above approach uses dynamic deformable anchors combined with Transformer for feature interaction. Alternatively, deformable convolution can be used to directly transform the feature map spatially, allowing the network to adaptively adjust the sampling position of the convolution kernel to fit the target shape. Feature fusion can then be achieved through multi-scale deformable convolution modules, similarly enabling geometric adaptation to irregular targets.

[0288] S42, semantic gated modulation

[0289] Reference semantic gating weight graph As a spatially modulated signal, the two interactive features are weighted element-wise to achieve semantically guided feature selection, resulting in the two modulated features. :

[0290]

[0291] in, This represents the element-wise Hadamard product. This operation allows spatial locations with a high degree of semantic matching with the reference (i.e., The feature responses of locations with high median values ​​are preserved and enhanced, while the feature responses of locations with low matching degree (background or irrelevant areas) are suppressed, thereby effectively highlighting the camouflaged target area.

[0292] The above approach uses a reference semantic gating weight map as a spatial modulation signal to modulate the two interactive features element-wise. Alternatively, semantic features can be transformed into dynamic convolution kernels to directly convolve the main image features. This method enables semantically guided feature recalibration within a larger receptive field and may be more effective for targets with fixed texture patterns.

[0293] S43, Feature Concatenation and Deep Refinement

[0294] The two modulated features are concatenated along the channel dimension to integrate complementary information from different primary and secondary interaction paths, resulting in the concatenated features. ;

[0295]

[0296] The number of feature channels after splicing becomes .

[0297] Subsequently, the concatenated features are deeply refined using a multi-level convolutional structure to obtain cross-modal fusion features. This enhances the distinctiveness and expressive power of features.

[0298]

[0299] in:

[0300] for Convolution is used to integrate cross-channel information, where both input and output channels are... ;

[0301] and For two consecutive Convolution (padding with 1s) is used to refine features in the spatial dimension and enhance local structure;

[0302] To normalize the batch and stabilize the training process;

[0303] For the linear rectification activation function, a nonlinear transformation is introduced.

[0304] After the above refinement, the final cross-modal fusion features are output. This feature integrates multi-scale visual information from the main image with semantic priors from the reference image, resulting in stronger target discrimination capabilities.

[0305] (III) Output

[0306] Output cross-modal fusion features Its dimensions are In this embodiment, , .

[0307] S5, based on cross-modal fusion features, generates the final camouflage target prediction map based on the detection head.

[0308] In this embodiment, the cross-modal fusion features output in step S4 are... The data can be directly input into subsequent detection heads (such as convolutional classifiers) to generate the final camouflaged target prediction map, or it can be used as an advanced feature for multi-task learning.

[0309] As a further implementation, the loss function of the present invention is designed as follows:

[0310] To achieve accurate segmentation of camouflaged targets, this method employs a structure-aware loss function for end-to-end network optimization and introduces a multi-level supervision strategy. Auxiliary constraints are applied to the intermediate features in Module 3 to further guide the network in learning more discriminative feature representations. The overall loss function consists of a weighted sum of the main loss and two auxiliary losses.

[0311] (a) Prediction map generation

[0312] Final prediction map: The cross-modal fusion features obtained in step S4 The final predicted logits graph is obtained through a sigmoid function. ,in The feature map space size (in this embodiment, it is...) ).

[0313] Auxiliary prediction map: the two-path features after semantic gating modulation in step S4 and (Right now and Apply independent sigmoid functions to generate auxiliary prediction logits plots. .

[0314] All predicted images (including final and auxiliary predictions) must be upsampled to the same size as the input image using bilinear interpolation to match the ground truth labels. Perform loss calculation.

[0315] (ii) Adaptive Difficult Example Weight Calculation

[0316] First, calculate the adaptive weight graph. To avoid gradient vanishing due to zero weights, a constant offset is introduced:

[0317]

[0318] in This is a weighting coefficient used to balance the contributions of difficult and easy examples. In this embodiment, it is taken as... This operation assigns a larger weight to regions where the label changes drastically (such as the edge of the target), while flat regions have a weight close to 1, ensuring that all pixels have effective gradient backpropagation.

[0319] in Indicates the kernel size as Average pooling (step size 1, padding size 15). The weights are absolute values ​​for each element. The weights are larger in regions where the labels change drastically, causing the loss function to focus on the difficult examples.

[0320] (III) Structure-aware loss

[0321] For any predictive logits graph and real labels Define structure-aware loss It consists of weighted binary cross-entropy loss and weighted cross-union ratio loss.

[0322] Weighted binary cross-entropy loss :

[0323]

[0324] in , This is the Sigmoid function.

[0325] Weighted intersection and union ratio loss:

[0326] First calculate the prediction probability Then calculate the weighted intersection and weighted union:

[0327]

[0328] Weighted intersection and combination loss Defined as:

[0329]

[0330] Ultimately, structure-aware loss The sum of the two:

[0331]

[0332] (iv) Total losses from multi-level supervision

[0333] For the final prediction map And two auxiliary prediction graphs Calculate the structure-aware loss separately, and then sum the three to obtain the total loss:

[0334]

[0335] Here, all loss weights are set to 1, but can be adjusted according to task requirements in practical applications. This multi-level supervision mechanism enables gradients to be simultaneously backpropagated to different stages of step S4, effectively alleviating the gradient vanishing problem in deep networks and promoting good target perception capabilities in intermediate features, thereby improving overall detection accuracy.

[0336] Experiments have shown that the multi-level structure perception loss function, in synergy with the aforementioned cross-modal feature interaction and semantic gating modulation steps, can significantly improve the accuracy and robustness of reference camouflage target detection.

[0337] The multi-level structure-aware loss function of this invention calculates the sum of weighted binary cross-entropy loss and weighted cross-union ratio loss for the final prediction result and the intermediate feature map, respectively, to achieve end-to-end multi-level supervised optimization.

[0338] Key technical points of this invention:

[0339] (1) A robust semantic aggregation method for multi-reference graphs based on multi-head attention mechanism:

[0340] By using features from multiple reference images as values ​​and employing learnable query vectors for multi-head cross-attention computation, adaptive, non-linear multi-image semantic fusion is achieved, resulting in stable and pure class commonality representations. This forms the foundation for subsequent precise semantic guidance.

[0341] (2) Method for generating and extracting dynamically deformable anchor points:

[0342] Unlike traditional fixed grids or global pooling anchors, this method uses a lightweight sub-network to directly regress the spatial offset of anchors from the weighted feature map, enabling anchors to be adaptively distributed across key parts of the camouflaged target and accurately capture component-level features.

[0343] (3) Anchor self-attention interaction mechanism based on Transformer:

[0344] By treating dynamic anchors as a set of tokens and using the self-attention mechanism of Transformer to model their interactions, long-range semantic dependencies between anchors are dynamically captured. This replaces the traditional Graph Convolutional Network (GCN) that requires a predefined adjacency matrix, enhancing the flexibility and expressive power of regional feature associations.

[0345] (4) A fusion architecture of bidirectional cross-scale interaction and semantic gating modulation:

[0346] The system employs bidirectional interaction, primarily from the perspectives of high-resolution and low-resolution features, to ensure sufficient complementarity of multi-scale information. The interacting features are then combined with reference semantic gating weights through an element-wise Hadamard product to achieve semantically guided feature selection. This combined architecture is the core decision-making unit of this invention.

[0347] Example 2

[0348] like Figure 6As shown, the difference between this embodiment and Embodiment 1 is that this embodiment provides a reference camouflage target detection system based on cross-modal semantic guidance, which corresponds one-to-one with the reference camouflage target detection method based on cross-modal semantic guidance in Embodiment 1; the system includes:

[0349] The acquisition module is used to acquire the query image and the reference image, where the query image is the main image to be detected.

[0350] The multi-scale feature encoding and preliminary decoding module is used to perform preliminary feature extraction on the query image based on the multi-scale feature encoding and preliminary decoding method to obtain the visual features of the query image. The visual features of the query image include high-resolution detail features and low-resolution semantic features. The multi-scale feature encoding and preliminary decoding method includes input feature enhancement, multi-scale feature encoding and demodulation enhancement, cross-scale feature reshaping, and hierarchical group fusion decoding.

[0351] The reference image semantic feature extraction and adaptation module is used to extract semantic priors of the target category from the reference image, and generate a reference semantic gating weight map that perfectly matches the visual feature dimensions of the query image after multi-image aggregation and dimensional and spatial alignment.

[0352] The cross-modal feature interaction and semantic-guided modulation module is used to perform bidirectional cross-scale interaction between high-resolution detail features and low-resolution semantic features to obtain the interacted features; and to use a reference semantic gating weight map to perform element-wise modulation and feature concatenation on the interacted features to obtain cross-modal fusion features.

[0353] The camouflage target detection module is used to generate a final camouflage target prediction map based on the detection head, according to cross-modal fusion features.

[0354] The reference image semantic feature extraction and adaptation module further includes a multi-head cross-attention unit, which takes the semantic features of multiple reference images as values ​​and adaptively aggregates them through a learnable query vector to obtain a robust semantic representation.

[0355] The cross-modal feature interaction and semantic guidance modulation module further includes: a dynamic deformable anchor point generation unit, which adaptively regresses the spatial offset of the anchor point based on the spatial attention-weighted feature map to generate anchor point features representing key parts of the target.

[0356] The cross-modal feature interaction and semantic guidance modulation module further includes: a Transformer anchor interaction unit, which uses the anchor features as a token sequence to perform feature interaction and aggregation through a self-attention mechanism to enhance the semantic association between anchors.

[0357] The cross-modal feature interaction and semantic-guided modulation module adopts a two-way interaction strategy: high-resolution detail features and low-resolution semantic features are used as the main features respectively, and the other is used as an auxiliary feature to perform the interaction process of the dynamic deformable anchor point generation unit and the Transformer anchor point interaction unit, resulting in two interactive features.

[0358] The execution process of each module can be carried out according to the steps of the reference camouflage target detection method based on cross-modal semantic guidance in Example 1, and will not be described in detail in this example.

[0359] Meanwhile, the present invention also provides a computer-readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the above-described reference camouflage target detection method based on cross-modal semantic guidance.

[0360] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0361] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0362] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0363] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0364] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A reference camouflage target detection method based on cross-modal semantic guidance, characterized in that, The method includes: Obtain a query image and a reference image, wherein the query image is the main image to be detected; The query image is preliminarily feature extracted based on a multi-scale feature encoding and preliminary decoding method to obtain the visual features of the query image. The visual features of the query image include high-resolution detail features and low-resolution semantic features. The multi-scale feature encoding and preliminary decoding method includes input feature enhancement, multi-scale feature encoding and demodulation enhancement, cross-scale feature reshaping, and hierarchical grouping fusion decoding. Extract the semantic prior of the target category from the reference image, and generate a reference semantic gating weight map that perfectly matches the visual feature dimensions of the query image after multi-image aggregation and dimensional and spatial alignment. The high-resolution detail features and the low-resolution semantic features are subjected to bidirectional cross-scale interaction to obtain the interacted features; and the interacted features are then modulated element-wise and concatenated using the reference semantic gating weight map to obtain cross-modal fusion features; and Based on the cross-modal fusion features, a final camouflage target prediction map is generated using the detection head.

2. The reference camouflage target detection method based on cross-modal semantic guidance according to claim 1, characterized in that, Preliminary feature extraction is performed on the query image based on multi-scale feature encoding and preliminary decoding to obtain the visual features of the query image, including: The query image is subjected to lightweight dual-attention modulation to obtain an enhanced feature map; the lightweight dual-attention modulation is achieved by cascading channel attention and spatial attention. The enhanced feature map is input into a multi-scale feature encoder for deep feature extraction to obtain feature maps at multiple scales; and each scale feature map is demodulated and enhanced to obtain a demodulated and enhanced feature map. The spatial structure of the demodulated and enhanced feature map is restored by dimensional transformation to obtain a set of two-dimensional feature maps with a uniform format; Based on the single-level fusion rule, a top-down multi-level progressive fusion is performed on the uniform two-dimensional feature map set. The temporary features obtained from the single-level fusion of the previous level are used as the input of the next level of fusion. After multiple rounds of iteration, an intermediate feature map is obtained. A channel attention mechanism is introduced to adaptively recalibrate the intermediate feature map to obtain an updated fused intermediate feature map. The updated fused intermediate feature maps are grouped and aggregated according to their resolution to generate high-resolution detail features and low-resolution semantic features respectively.

3. The reference camouflage target detection method based on cross-modal semantic guidance according to claim 2, characterized in that, The multi-scale feature encoder adopts a hierarchical pyramid structure, which consists of multiple encoding stages. Each encoding stage includes downsampling embedding and multiple Transformer modules, and achieves the interaction and fusion of multi-scale information through cross-stage connections.

4. The reference camouflage target detection method based on cross-modal semantic guidance according to claim 1, characterized in that, Extracting semantic priors containing the target category from the reference image, and generating a reference semantic gating weight map that perfectly matches the visual feature dimensions of the query image after multi-image aggregation and dimensional and spatial alignment, includes: Extract semantic priors containing the target category from the reference image, use a pre-trained multimodal visual language model as a semantic extractor, and perform global semantic feature extraction on the reference image based on the semantic extractor to obtain a global semantic feature map of category commonality; The global semantic feature map is reduced in channel dimension by convolutional layers to obtain the reference features after channel dimension reduction. Bilinear interpolation upsampling is used to enlarge the reference features after channel dimensionality reduction to the same spatial size as the query image features; The Sigmoid activation function is then used to map each element value of the dimensionally and spatially aligned reference feature to... The interval is used to generate a reference semantic gating weight map that perfectly matches the visual feature dimensions of the query image.

5. The reference camouflage target detection method based on cross-modal semantic guidance according to claim 4, characterized in that, Based on the semantic extractor, global semantic features are extracted from the reference image to obtain a semantic feature map of common categories, including: The reference image is reshaped into a batch and image quantity combined form to obtain a flattened reference image; The flattened reference image is fed into a semantic extractor to obtain global semantic feature vectors corresponding to all reference images; the semantic extractor uses a pre-trained multimodal visual language model. The global semantic feature vector is reshaped back into the batch dimension and the image quantity dimension, and a multi-head cross-attention mechanism is introduced to calculate the aggregated semantic features; The aggregated semantic features are then reshaped in terms of feature dimensions to obtain a semantic feature map of common categories.

6. The reference camouflage target detection method based on cross-modal semantic guidance according to claim 1, characterized in that, The high-resolution detail features and the low-resolution semantic features are subjected to bidirectional cross-scale interaction to obtain the interacted features, including: The interaction process is performed using the high-resolution detail features as the main features and the low-resolution semantic features as the auxiliary features to obtain the first interaction feature. The interaction process is performed using the low-resolution semantic features as the main features and the high-resolution detail features as the auxiliary features to obtain the second interaction feature. The interaction process is as follows: Based on auxiliary features, a spatial attention weight map is generated; Channel dimensionality reduction is performed on the main features to obtain the dimensionality-reduced features; the dimensionality-reduced features are then weighted using the spatial attention weight map to generate mask features; Dynamically deformable anchor points are generated from the mask features using a lightweight subnetwork; Calculate the association weight matrix between each anchor point and the global semantic features; Based on the association weight matrix, the main features are projected onto the anchor point space to obtain the anchor point state features; The anchor point state features are interacted with using the Transformer self-attention module to obtain the features after Transformer interaction; The features after Transformer interaction are projected back to the original spatial dimensions through the association weight matrix, and then... After adjusting the number of channels through convolution, residual connections are made with the main features to obtain the interactively enhanced features.

7. The reference camouflage target detection method based on cross-modal semantic guidance according to claim 6, characterized in that, The Transformer self-attention module is used to interact with the anchor point state features, and the expression is: ； in, ; In the formula, Features resulting from Transformer interaction; The Transformer self-attention module consists of a multi-head self-attention layer and a feedforward network. It allows each anchor point to exchange information with all other anchor points and dynamically models long-distance dependencies between anchor points. Anchor point state characteristics; For flattening operation; The features are dimensionality reduced; W is the correlation weight matrix. This represents the transpose of the correlation weight matrix; For real number field identifier, indicating It is a three-dimensional tensor whose elements are composed of real numbers, and its dimension is . B represents the batch size. The number of feature channels after projection; K is the number of anchor points.

8. The reference camouflage target detection method based on cross-modal semantic guidance according to claim 1, characterized in that, The interactive features are modulated element-wise and concatenated using the reference semantic gating weight graph to obtain cross-modal fusion features, including: The reference semantic gated weight map is used as a spatial modulation signal, and the interactive features are weighted element by element to obtain two modulated features. The two modulated features are concatenated along the channel dimension to obtain the concatenated features. By using a multi-level convolutional structure to deeply refine the concatenated features, cross-modal fusion features are obtained.

9. A reference camouflage target detection system based on cross-modal semantic guidance, characterized in that, The system includes: The acquisition module is used to acquire a query image and a reference image, wherein the query image is the main image to be detected; The multi-scale feature encoding and preliminary decoding module is used to perform preliminary feature extraction on the query image based on the multi-scale feature encoding and preliminary decoding method to obtain the visual features of the query image. The visual features of the query image include high-resolution detail features and low-resolution semantic features. The multi-scale feature encoding and preliminary decoding method includes input feature enhancement, multi-scale feature encoding and demodulation enhancement, cross-scale feature reshaping, and hierarchical group fusion decoding. The reference image semantic feature extraction and adaptation module is used to extract semantic priors containing the target category from the reference image, and generate a reference semantic gating weight map that perfectly matches the visual feature dimensions of the query image after multi-image aggregation and dimensional and spatial alignment. The cross-modal feature interaction and semantic-guided modulation module is used to perform bidirectional cross-scale interaction between the high-resolution detail features and the low-resolution semantic features to obtain the interacted features; and to use the reference semantic gating weight map to perform element-wise modulation and feature splicing on the interacted features to obtain cross-modal fusion features. The camouflage target detection module is used to generate a final camouflage target prediction map based on the detection head according to the cross-modal fusion features.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the reference camouflage target detection method based on cross-modal semantic guidance as described in any one of claims 1 to 8.

Images