A scene-aware adversarial camouflage generation method and system for camouflage target detection

Abstract

The application discloses a scene-aware camouflage generation method and system for a camouflage target detector, an electronic device, a storage medium and a computer program product. The method repairs a target area in an original image according to a target mask to obtain a scene reference image without target pixel interference; extracts a main color and a proportion, a texture frequency, a main direction and an anisotropy parameter from the scene reference image, generates a multi-scale directional digital camouflage pattern and fuses the pattern with an original image; uses the pattern as an initial texture, constructs a joint attack loss using responses of multiple camouflage target detectors, and iteratively updates target area pixels in combination with a perceptual consistency loss and a total variation regularization to obtain an anti-camouflage image. The scheme takes into account detection response suppression and background color, texture and direction coordination, and can be used for digital camouflage generation and detection model robustness evaluation.

Description

Technical Field

[0001] This application relates to the fields of computer vision, image processing, digital camouflage and adversarial sample generation, specifically to a scene-aware adversarial camouflage generation method that combines background restoration, scene statistical analysis and camouflage target detection model output feedback. Background Technology

[0002] Digital camouflage typically reduces the visual separability between the target and the background through mechanisms such as pixelated color blocks, multi-scale pattern structures, color similarity, and outline fragmentation. It has been widely used in tasks such as target concealment, scene camouflage, and visual adversarial evaluation.

[0003] With the development of deep learning visual models, camouflage target detection technology can utilize semantic context, boundary details, and multi-scale features to locate camouflaged targets from highly similar backgrounds. Traditional digital camouflage is mainly designed for the human visual system, and when faced with camouflage target detectors based on deep neural networks, it is prone to problems such as still having a high detection response and the target outline still being captured by the model.

[0004] Existing adversarial camouflage methods are mostly geared towards general object detection models, or primarily reduce detector response through pixel-level perturbations. While these methods can weaken the model output to some extent, they often neglect the color harmony, texture scale consistency, and orientation structure consistency between the target region and the real background, easily generating high-frequency noise, abrupt color blocks, or local artifacts, thereby reducing visual naturalness.

[0005] On the other hand, if the background dominant color, texture frequency, and orientation structure are directly statistically analyzed on the original image containing the target, the target's own color and texture will participate in the statistical calculation, causing the generated camouflage pattern to deviate from the true background distribution. Therefore, how to extract reliable scene statistical information while eliminating target pixel interference, and combine this scene statistical prior with multi-detector joint adversarial optimization, is a key technical problem for improving the naturalness of adversarial camouflage and the effectiveness of attacks. Summary of the Invention

[0006] The purpose of this application is to provide a scene-aware adversarial camouflage generation method, system and storage medium for camouflage target detectors, so as to solve the problems in the prior art such as lack of scene adaptability of camouflage patterns, easy generation of visual artifacts in adversarial optimization, insufficient generalization ability of single detector optimization and lack of transfer evaluation of detectors not involved in optimization.

[0007] To achieve the above objectives, this application provides a scene-aware adversarial camouflage generation method. This method takes the original image and target mask as input and unfolds in four stages: scene reference construction, scene feature analysis, digital camouflage pattern generation, and multi-detector joint adversarial optimization. This enables the camouflage pattern to adapt to the color and texture statistics of the real background while also being optimized for the response features of the camouflage target detector.

[0008] In the scene reference construction stage, this application performs background inpainting on the target region in the original image based on the target mask, resulting in a scene reference map free from target pixel interference. Compared with directly extracting background statistics from the original image, this processing can avoid the target's own color, texture, and contour from entering the background feature estimation process, thereby improving the reliability of subsequent main color, texture scale, and orientation structure analysis from the source.

[0009] In the scene feature analysis stage, this application extracts color features, texture frequency features, and directional structure features from the scene reference image. Specifically, color features can be obtained through primary color clustering in the CIELAB color space, texture frequency features can be obtained through Gram energy distribution of shallow visual features, and directional structure features can be obtained through multi-directional Gabor filter responses. These features collectively constitute the scene feature parameters used to control the generation of camouflage patterns.

[0010] In the digital camouflage pattern generation stage, this application controls the color distribution of color blocks based on the main color center and color ratio, controls the size of the basic color blocks based on the texture frequency index, and controls the arrangement direction and major-minor axis ratio of the color blocks based on the main texture direction and anisotropy ratio, thereby generating a multi-scale directional digital camouflage pattern. Furthermore, CIELAB color alignment can be performed on the generated pattern to make the target area pattern closer to the surrounding background in brightness and hue.

[0011] In the joint adversarial optimization phase, this application uses the digital camouflage pattern as the initial texture to construct an input image obtained by fusing the original image, target mask, and the current optimized texture. This input image is then fed into multiple camouflage target detectors. During the optimization process, only the texture pixels within the target area are updated, while non-target background areas remain unchanged to minimize damage to the original scene content.

[0012] As an optional implementation, the loss function of joint adversarial optimization includes a multi-detector joint attack loss, a perceptual consistency loss, and a total variation regularization term. The multi-detector joint attack loss is used to suppress the output responses of different camouflage target detectors within the target region; the perceptual consistency loss is used to constrain the deep feature shift of the optimized texture relative to the initial digital camouflage pattern; and the total variation regularization term is used to suppress local isolated noise and discontinuous textures.

[0013] This application also provides a scene-aware adversarial camouflage generation system, which includes a data acquisition module, a background restoration module, a scene feature analysis module, a camouflage pattern generation module, a fusion module, and a joint adversarial optimization module. Each module can be deployed via software programs in a computer, server, image processing device, or other electronic device with a processor and memory to execute the above methods.

[0014] This application also provides an electronic device, a computer-readable storage medium, and a computer program product. The electronic device includes a processor and a memory, wherein the memory stores a computer program; when the computer program is executed by the processor, it implements the above-mentioned method from acquiring the original image and target mask, generating a scene reference map, extracting scene features, generating an initial camouflage pattern, to jointly optimizing the texture of the target region.

[0015] Compared with existing technologies, this application has at least the following beneficial effects: First, by eliminating the interference of target pixels on scene statistics through background inpainting, the estimation of main color, texture scale, and orientation structure is closer to the real background; Second, by generating an initial digital camouflage pattern through joint constraints of color, texture, and orientation, the target area is visually more coordinated with the background; Third, by improving the response suppression capability of different camouflage target detection models through multi-detector joint adversarial optimization; Fourth, by limiting the degradation of texture into high-frequency noise through perceptual consistency loss and total variation regularization, the visual naturalness of the final camouflage result is improved; Fifth, by updating only the region corresponding to the target mask, unnecessary changes to the non-target background are avoided. Attached Figure Description

[0016] Figure 1 The overall flowchart of the scene-aware adversarial camouflage generation method provided in the embodiments of this application is shown.

[0017] Figure 2 A flowchart illustrating the first-stage scene-aware digital camouflage pattern generation process provided in this application embodiment.

[0018] Figure 3 This is a schematic diagram of mask-aware Transformer background restoration provided in an embodiment of this application.

[0019] Figure 4 This is a schematic diagram of texture frequency estimation and principal direction analysis provided in an embodiment of this application.

[0020] Figure 5 A schematic diagram of multi-scale directional digital camouflage pattern synthesis provided in an embodiment of this application.

[0021] Figure 6 A flowchart of multi-detector joint adversarial optimization and optional migration evaluation provided for embodiments of this application.

[0022] Figure 7 A comparison chart of the output responses of a white-box camouflage target detector at different processing stages provided in the embodiments of this application. Detailed Implementation

[0023] The technical solution of this application will be described below with reference to embodiments. It should be understood that the embodiments are only used to explain this application and are not intended to limit the scope of protection of this application. Where there is no conflict, the technical features in the various embodiments can be combined with each other.

[0024] Please see Figure 1 This embodiment provides a scene-aware adversarial camouflage generation method for camouflage target detectors. The method takes an original image I and a binary target mask M corresponding to the pixel-by-pixel region of the target to be camouflaged as input, adjusting the original image and the target mask to the same spatial size required by the camouflage target detector; this embodiment uses 512×512 pixels, but in other embodiments, it can be adjusted proportionally according to the detector input size. The method includes a first-stage scene-aware digital camouflage pattern generation and a second-stage multi-detector joint adversarial optimization, outputting a final adversarial camouflage image I with the same spatial size as the original image. adv .

[0025] Please see Figure 2 and Figure 3 In the first stage, the original image I and the target mask M are input into a Mask-Aware Transformer (MAT). Pixels with a value of 1 in the target mask M indicate the area to be repaired. MAT uses known background areas with a value of 0 to establish long-range contextual relationships, performs structural and texture completion on the area to be repaired, and preserves the content of non-target areas, outputting a repaired background map Imat with the same spatial dimensions as the original image. Thus, Imat constitutes a scene reference map free from target pixel interference.

[0026]

[0027] In other implementations, the background inpainting network may employ a convolutional inpainting network, a Fourier convolutional inpainting network, or a diffusion inpainting network that has mask conditional input and is able to preserve the content of non-target areas. Regardless of the network used, its input includes at least the original image and the target mask, and the output is a scene reference map that completes the background of the target area while preserving the content of non-target areas.

[0028] Repair background image I mat Used for all subsequent scene analysis processes, including dominant color clustering, texture frequency estimation, principal orientation and anisotropy analysis. Because I matThe interference of target region pixels on background statistics has been eliminated, thus reducing the impact of the target's own color and texture on camouflage pattern generation.

[0029] Continue reading Figure 2 During the primary color clustering process, the background image I will be repaired. mat The color space was converted from RGB to CIELAB, and K-Means++ was used to cluster the sampled pixels. The number of clusters, K... c The value can be selected from 2 to 8; in this embodiment, it is set to K. c =4, thus obtaining the set of primary color centers C and the color ratio r corresponding to each primary color center. k .

[0030]

[0031] Among them, c k Ω represents the k-th dominant color center. k Let r represent the set of pixels corresponding to the k-th cluster. k This represents the area proportion of the primary color in the scene's sampled pixels. The primary color center C is used to construct the color palette for the digital camouflage pattern, and the color proportion r... k Used to constrain the distribution ratio of different color blocks in a pattern.

[0032] Please see Figure 4 (a) In the texture frequency estimation process, one or more shallow feature layers of the pre-trained VGG-19 are used to extract and repair the background image I. mat The characteristic response F. The spatial response of each characteristic channel is expanded, and the Gram energy G is calculated based on the mean of the autocorrelation term of that channel. c The channels are divided into low-frequency and high-frequency channels according to their channel energy or preset frequency band, and the low-frequency energy E is obtained by summing the values ​​of each channel. low and high-frequency energy E high Therefore, the texture frequency index F is defined. idx .

[0033]

[0034] Where ε is a minimal constant to prevent the denominator from being zero. In this embodiment, the size range of the camouflage blocks is set to S. min =12、S max =32, in pixels. When the background texture is fine, F idx The size is relatively large, and the generated color block size is close to S. min When the background texture is relatively smooth, F idx Smaller, the generated color block size is close to S max .

[0035] Please see Figure 4 (b) In the analysis of principal direction and anisotropy, a multi-directional Gabor filter is used to perform directional response analysis on the repaired background image Imat. The Gabor filter direction set can be set according to the scene texture resolution. In this embodiment, it is set to Θ={0°, 45°, 90°, 135°}. The average energy R(θ) of the absolute value or square value of the filter response is calculated for each direction θ, and the direction with the largest average energy is selected as the main texture direction θ of the background. main .

[0036]

[0037] Here, AR represents the anisotropy ratio. A larger AR indicates a more significant difference in response between the principal direction and its perpendicular direction, resulting in stronger background texture directionality; when AR is close to 1, it indicates weaker background texture directionality. Based on θ main With AR, the base color block can be adjusted from an approximate square to a directional rectangle, and along θ main Directional arrangement.

[0038] Please see Figure 2 and Figure 5 Based on the aforementioned scene feature parameters, an initial digital camouflage pattern P that adapts to the scene is generated. camo Specifically, the primary color set C and its color proportion r k As a color constraint, with texture frequency index F idx Main texture direction θ main Using anisotropy ratio (AR) as a structural constraint, multi-scale directional digital camouflage patterns are constructed.

[0039]

[0040] Among them, H pat G(·) represents the pattern size, and G(·) represents the multi-scale camouflage pattern generation function. During the generation process, a large-scale directional color block layer is generated based on the basic color block size, main direction, and anisotropic parameters. Then, the color block size is gradually reduced to generate medium-scale and fine-scale pattern layers. Finally, the layers of different scales are merged through random overlay, and the colors are requantized to the main color palette.

[0041] To further improve color harmony, CIELAB full-channel alignment was performed on the target area after pattern synthesis. Specifically, the L channel used mean-standard deviation matching to align the brightness distribution, while the a and b channels used mean shifting to align the hue, preserving the color differences between different color blocks. The color-aligned pattern is still denoted as P. camo .

[0042] Then, P camoThe target mask M is fused into the original image I to obtain the initial fused image I of the first stage. camo .

[0043]

[0044] Where ⊙ denotes element-wise multiplication. Non-target background areas retain their original image content, while the target area is covered by a scene-adaptive digital camouflage pattern, resulting in I... camo This serves as the initial input for the second-stage multi-detector joint adversarial optimization.

[0045] Please see Figure 6 In the multi-detector joint adversarial optimization stage, Pcamo generated in the first stage is used as the initial texture P(0), and the complete image obtained by fusing the original image I, the target mask M, and the current texture P(t) is input into at least two camouflaged target detectors. This embodiment combines three models—SINet, CamoFormer, and ZoomNeXt—for optimization. These three models represent three typical camouflaged target detection modeling paradigms: search-recognition, Transformer global modeling, and multi-scale detail enhancement, respectively.

[0046] Let P be the optimized texture in the t-th iteration. (t) Then the complete image I is sent to the camouflage target detection model. (t) Represented as:

[0047]

[0048] During the optimization process, the gradient is fed back to P from the response of the camouflaged target detection model. (t) The texture of the target region is updated only, while the background pixels remain unchanged. The total loss function is defined as:

[0049]

[0050] Among them, the multi-detector joint attack loss is used to suppress the detection response of the COD model in the target area; the perceptual consistency loss is used to maintain the perceptual consistency between the optimized texture and the initial camouflage pattern; and the total variation regularization term is used to suppress local isolated noise points by constraining the differences between adjacent pixels.

[0051] For the i-th camouflaged target detection model D i , will complete image I (t) The model input is used as input, and the model output is adjusted to the same spatial size as the target mask M before Sigmoid normalization is performed to obtain the pixel-wise output response S. i =D i (I (t)The desired response within the target region is set to 0, and a binary cross-entropy is used to reduce the detection response in that region. To make the optimization process focus more on high-response pixels, an adaptive pixel weight A related to the response intensity is introduced. i (x,y). The loss from a multi-detector joint attack is defined as:

[0052]

[0053]

[0054] Where, α i σ represents the attack weight of the i-th COD model, and the sum of the attack weights of all models is 1; σ(·) is the Sigmoid function; ω min and ω max These represent the minimum and maximum pixel weights, respectively. In this embodiment, the weights of SINet, CamoFormer, and ZoomNeXt are set to 0.4, 0.3, and 0.3, respectively.

[0055] To maintain visual structural consistency of the texture before and after optimization, the current texture P... (t) Compared with the initial camouflage pattern P (0) Input the pre-trained VGG-19 into each layer, select at least one feature layer, and calculate the L1 feature difference between the two layers using a target mask scaled to the resolution of that feature layer. This yields the perceptual consistency loss L. nat Simultaneously, the sum of the absolute differences between horizontally and vertically adjacent texture pixels within the target region is calculated to obtain the total variation regularization term L. tv This reduces isolated noise and discontinuous textures introduced by the adversarial gradient.

[0056] To balance aggressiveness and visual naturalness, this embodiment employs a progressive weighting strategy. In the initial optimization phase, naturalness constraints are increased to preserve the overall structure of the scene-adaptive camouflage, subsequently gradually increasing the attack loss weight. At the t-th iteration, the attack loss weight λ... adv (t) and the perceived consistency loss weight λ nat (t) is set as follows:

[0057]

[0058] Where T represents the total number of iterations. The final texture P is obtained after optimization. adv =P (T) And synthesize the final adversarial camouflage image I according to the target mask M. adv .

[0059]

[0060] This embodiment uses the Adam optimizer, setting only the texture P within the target region as the variable to be optimized, with a learning rate of 0.01, a total number of iterations T of 10, and a total variation regularization weight λ. tv Set to 0.05; after each update, the pixel value of P is cropped to the valid range of the normalized image. The above parameters can be adjusted according to image resolution, target area, and number of detectors.

[0061] Continue reading Figure 6 Ultimately, combating camouflaged images I adv It can maintain a color and texture appearance that is consistent with the background in the target region and reduce the output response of the camouflaged target detection model participating in the joint optimization within the target region. Optionally, I can adv The input detectors that do not participate in or do not fully participate in the joint optimization are used to perform auxiliary evaluation of mobility under gray-box or black-box conditions only through forward inference; this evaluation does not participate in the target region texture update.

[0062] Please see Figure 7 , Figure 7 The output responses of two sets of samples at different processing stages are shown. Each set of samples, from left to right, includes the input image, the ground truth target mask (GT), and the output response maps of SINet, CamoFormer, and ZoomNeXt; from top to bottom, it includes the original image, the image processed only with scene-aware digital camouflage, the image directly optimized from the original target texture for adversarial purposes, and the final adversarial camouflage image obtained using the complete method of this application. White areas in the output response maps represent high-response locations for the corresponding detectors.

[0063] Depend on Figure 7 It is evident that when using only scene-aware digital camouflage, different detectors may still retain continuous target responses. When performing adversarial optimization directly from the original target texture, the detection response can be weakened, but the prior knowledge of color, texture scale, and orientation structure provided by the targetless scene reference map is not utilized. After adopting the complete method of this application, the target responses of the three white-box detectors are further reduced, discretized, or disappear, while the target region maintains a multi-scale digital camouflage appearance. Figure 7 This is used to illustrate the technical effects that can be achieved in this embodiment and does not constitute a limitation on the target category, scene type, number of detectors or output format.

[0064] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A scene-aware adversarial camouflage generation method for camouflage target detectors, characterized in that, include: Acquire the original image and the target mask used to indicate the target area to be disguised; Background restoration is performed on the target region in the original image based on the target mask to obtain a scene reference image free from target pixel interference. The primary color and its color ratio, texture frequency parameters, primary texture direction, and anisotropy parameters are extracted from the scene reference image. A multi-scale directional digital camouflage pattern is generated based on the primary color and its color ratio, the texture frequency parameters, the primary texture direction, and the anisotropy parameters. The multi-scale directional digital camouflage pattern is fused with the original image according to the target mask to obtain an initial fused image. Using the multi-scale directional digital camouflage pattern as the initial texture, the current texture is fused with the original image according to the target mask to form an iterative image. The iterative image is input to at least two camouflage target detectors, and the current texture is jointly updated based on the output responses of the at least two camouflage target detectors. The texture update is restricted to the target region indicated by the target mask to obtain an anti-camouflage image.

2. The scene-aware adversarial camouflage generation method according to claim 1, characterized in that, The background restoration includes: inputting the original image and the binarized target mask into a mask-aware Transformer background restoration network, using the contextual features around the target region to perform structural and texture restoration on the target region, while keeping the pixels of non-target regions unchanged, and outputting a scene reference image with the same spatial size as the original image.

3. The scene-aware adversarial camouflage generation method according to claim 1, characterized in that, Extracting the primary color and its color ratio includes: converting the scene reference image from RGB color space to CIELAB color space; using the K-Means++ algorithm to cluster the sampled pixels in the scene reference image to obtain multiple primary color centers; and calculating the proportion of the number of pixels in each cluster to the total number of sampled pixels to obtain the color ratio corresponding to each primary color center.

4. The scene-aware adversarial camouflage generation method according to claim 1, characterized in that, Extracting the texture frequency parameters includes: extracting the feature response of the scene reference image using at least one shallow feature layer of a pre-trained convolutional neural network; statistically analyzing low-frequency energy and high-frequency energy based on the Gram energy of different feature channels; determining the texture frequency index based on the proportion of the high-frequency energy to the sum of the low-frequency energy and the high-frequency energy; and determining the basic color block size between a preset minimum size and a preset maximum size based on the texture frequency index.

5. The scene-aware adversarial camouflage generation method according to claim 1, characterized in that, Extracting the main texture direction and anisotropy parameters includes: constructing a Gabor filter bank in multiple preset directions; calculating the average response energy of the scene reference map in each preset direction; determining the preset direction with the largest average response energy as the main texture direction; and determining the anisotropy parameters based on the ratio between the average response energy of the main texture direction and the average response energy perpendicular to the main texture direction.

6. The scene-aware adversarial camouflage generation method according to claim 1, characterized in that, Generating the multi-scale directional digital camouflage pattern includes: using the primary color and its color ratio as color constraints, and using the basic color block size determined according to the texture frequency parameter, the primary texture direction, and the anisotropy parameter as structural constraints, generating large-scale, medium-scale, and fine-scale directional color block layers respectively; fusing the large-scale, medium-scale, and fine-scale directional color block layers, and requantizing the fused color to the primary color to obtain the multi-scale directional digital camouflage pattern.

7. The scene-aware adversarial camouflage generation method according to claim 6, characterized in that, Before fusing the multi-scale directional digital camouflage pattern with the original image, the method further includes: performing CIELAB full-channel color alignment on the multi-scale directional digital camouflage pattern within the target area, wherein the L channel uses mean and standard deviation matching to align the brightness distribution, and the a and b channels use mean shifting to align the hue distribution.

8. The scene-aware adversarial camouflage generation method according to claim 1, characterized in that, The total loss function used for jointly updating the current texture includes a multi-detector joint attack loss, a perceptual consistency loss, and a total variation regularization term; the multi-detector joint attack loss is used to reduce the output response of the at least two camouflaged target detectors in the target region; the perceptual consistency loss is used to limit the perceptual feature offset of the current texture relative to the initial texture; and the total variation regularization term is used to limit the difference between adjacent texture pixels in the target region.

9. The scene-aware adversarial camouflage generation method according to claim 8, characterized in that: The multi-detector joint attack loss sets model weights for different camouflaged target detectors, and for the output response of each camouflaged target detector, it sets adaptive pixel weights for pixels in the target region that increase with the output response, so that the joint update prioritizes reducing the positions with higher output responses in the target region; when jointly updating the current texture, a progressive weight scheduling is adopted, so that the weight of the multi-detector joint attack loss increases with the number of iterations, and the weight of the perceptual consistency loss decreases with the number of iterations; after each joint update, the pixel values ​​of the current texture are restricted to a preset image value range.

10. A scene-aware adversarial camouflage generation system for camouflage target detectors, characterized in that, include: The data acquisition module is used to acquire the original image and the target mask used to indicate the target area to be disguised. The background restoration module is used to restore the background of the target region in the original image according to the target mask to obtain a scene reference image. The scene feature analysis module is used to extract the main color and its color ratio, texture frequency parameters, main texture direction and anisotropy parameters from the scene reference image; the camouflage pattern generation module is used to generate a multi-scale directional digital camouflage pattern based on the main color and its color ratio, the texture frequency parameters, the main texture direction and the anisotropy parameters. The fusion module is used to fuse the multi-scale directional digital camouflage pattern with the original image according to the target mask; the joint adversarial optimization module is used to fuse the current texture with the original image according to the target mask, and update the current texture only within the target area indicated by the target mask according to the output response of at least two camouflage target detectors, so as to obtain an adversarial camouflage image.

Images