Camouflage confrontation generation method and device based on environmental style migration

Abstract

The invention discloses a camouflage confrontation generation method and device based on environment style migration. The method comprises the following steps: step 1, constructing an environment scene simulation platform; 2, collecting a multi-environment simulation data set; step 3, performing differential neural renderer mapping; 4, environment style migration fusion; 5, optimizing a composite loss function; step 6, performing end-to-end texture optimization; 7, performing secondary optimization based on physically realizable camouflage textures: performing reverse optimization on the camouflage texture generation model by introducing constraint functions related to printing, pasting and materials; and 8, outputting the camouflage texture. According to the method, by optimizing the resistance loss, the recognition performance of the detector is remarkably reduced, and the camouflage deception effect is better.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of artificial intelligence and target camouflage technology, in particular to a camouflage adversarial generation method and device based on environmental style transfer. BACKGROUND

[0002] Traditional camouflage techniques and methods such as physical barriers and false demonstration techniques have limited effect in the face of AI-driven intelligent target detection systems. Although the wide application of current AI target perception algorithms can improve the accuracy of target detection and recognition, there are limitations and vulnerabilities in the face of adversarial attacks.

[0003] Recent achievements in adversarial camouflage technology include: an adversarial sticker for deceiving traffic sign recognition systems, which uses local two-dimensional perturbation and is obviously ineffective under changes in viewing angle. The Camou method learns vehicle overall camouflage patterns, but is still based on two-dimensional image space optimization and lacks three-dimensional geometric adaptation capability. An infrared adversarial patch that can learn shape and position, but is limited to local areas. The ACTIVE framework uses 3D neural rendering to generate physically transferable camouflage, verifying the advantages of three-dimensional modeling in multi-view robustness.

[0004] Existing two-dimensional or three-dimensional camouflage methods focus more on the effectiveness of adversarial texture images, but still lack the naturalness of camouflage texture in vision and environmental integration, which easily arouses the suspicion of human observers to the concealed target; in addition, most methods do not fully eliminate the influence of complex environments (such as different geomorphic environments, weather, and lighting) on the camouflage effect, resulting in insufficient generalization ability.

[0005] Therefore, a new camouflage generation method is needed that can balance the adversarial ability of interfering AI detectors, environmental scene integration, multi-view robustness, and physical realizability. The present application introduces an environmental style transfer mechanism, uses an improved adversarial GAN network with an attention module to realize the deep integration of environmental style and equipment texture, proposes a new equipment cover camouflage adversarial generation method, effectively counteracts the recognition and detection of AI target detectors, and achieves the purpose of concealing targets or misleading reconnaissance systems. SUMMARY

[0006] The present application aims to solve the problems raised in the background art and proposes a camouflage adversarial generation method and device based on environmental style transfer.

[0007] To achieve the purpose of the present application, the present application provides a camouflage adversarial generation method based on environmental style transfer, which comprises:

[0008] Step 1: Build an environmental scene simulation platform:

[0009] Based on the Unreal Engine 5 platform, various typical simulation environments are constructed, covering landform environments such as cities, jungles, and deserts. Each landform environment simulates multiple weather conditions, including sunny, rainy, and foggy days, and different lighting conditions, including daytime, dusk, and night, to fully reproduce the complex environmental scene changes that the real environment may face.

[0010] Step 2: Collecting multi-environment simulation dataset:

[0011] Using virtual drones in the environmental scene simulation platform, set the flight cruising path and field of view, collect high-resolution image data of the target equipment model in various environmental backgrounds under multiple azimuth angles (e.g. 0°, 45°), multiple elevation angles (e.g. 22.5°, 67.5°), and multiple distances (e.g. 5m, 20m), and establish a dataset for adversarial model training; during the collection of the simulation dataset, control any two of the azimuth angle, distance, and elevation angle to remain constant, and only change the last one of the three, and collect data under multiple values, then use the same method to control the three quantities in turn under the condition that the other quantities remain constant, and collect multiple values.

[0012] Step 3: Differentiable neural renderer mapping:

[0013] Using the Differentiable Neural Renderer (Differentiable Neural Renderer), the three-dimensional OBJ model of the target equipment and its UV camouflage texture map are mapped to the two-dimensional image space, achieving smooth rendering from three dimensions to two dimensions, and maintaining the matching of the geometric structure. This process supports gradient backpropagation, providing a foundation for subsequent texture optimization.

[0014] Step 4: Environment style transfer fusion:

[0015] Using the pre-trained ResNet backbone network to extract high-level environmental style features of the background image, and fusing them with the features of the equipment surface area, an initial camouflage texture is generated that is approximately consistent with the color style and texture of the surrounding environment of the equipment, enhancing the visual naturalness of the camouflage texture.

[0016] Step 5: Composite loss function optimization: To enable the adversarial camouflage algorithm to achieve optimization in multiple dimensions, the adversarial loss, visual smoothness loss, and style transfer loss are combined to construct a composite loss function.

[0017] Adversarial loss Based on the output confidence of the AI detector (such as YOLOv8, Faster R-CNN), the recognition error of the target being missed or misclassified is maximized to reduce the ability to detect and recognize the camouflage target, and the formula is: H d(x) is the confidence of the detector to the target category x, the negative logarithm function converts the missed detection probability into a loss value, when the missed detection probability is larger, the loss value tends to 0, so as to more effectively generate the counterfeiting sample.

[0018] Style transfer loss The style difference between the input image and the camouflage texture is measured based on the Gram matrix, which is used to extract the style features of the camouflage texture from the environmental background image and match with the texture of the camouflage target, so that the generated camouflage texture minimizes the difference with the environmental background style features, and the formula is:

[0019]

[0020] Wherein And The Gram matrix of the generated camouflage texture x and the background image y at the convolution layer φ l , C represents the channel, and H, W represent the height and width of the feature map;

[0021] Visual smoothness loss It is used to maintain the visual smoothness of the camouflage texture, constrain the gradient change in texture space, prevent sharp mutations, and maintain the natural transition of the generated camouflage texture in vision, and the formula is:

[0022] For the adversarial loss, the style transfer loss and the visual smoothness loss, the comprehensive weighted sum is carried out, and the total composite loss function formula is constructed:

[0023] Wherein, α, β and γ are weight parameters, which respectively control the contribution of the adversarial loss, the style transfer loss and the visual smoothness loss to the total loss, and realize the multi-objective collaborative optimization.

[0024] In the model training process, the composite loss function is iteratively optimized, so that the camouflage generation algorithm can adapt to multi-angle, multi-distance and complex environment scenes, and ensure that the generated camouflage texture has good visual naturalness and environmental scene adaptability.

[0025] Step 6: end-to-end texture optimization:

[0026] Taking the initial camouflage texture as the input, the composite loss function reversely propagates the total composite loss through the Adam optimizer, dynamically updates the generation parameters of the UV patch camouflage texture map, and stops iteration calculation until the composite loss function reaches the expected convergence interval, and combines CUDA parallel acceleration in the optimization process to improve the efficiency of calculation optimization.

[0027] Step 7: Secondary optimization based on physically achievable camouflage texture: By introducing constraint functions related to printing, coating, and material, the camouflage texture generation model is optimized in reverse, ensuring that the final output texture is both adversarial and meets actual process conditions.

[0028] Printable color gamut constraint: Limit the color range of the texture output to ensure that it can be realized by the printing device and coating material, avoiding the simulation of colors that cannot be reproduced in reality;

[0029]

[0030] where TUV(i) is the color value of each pixel of the generated camouflage texture;

[0031] Coating deformation constraint: According to the Jacobian matrix of UV coordinates, the degree of geometric deformation of the texture during coating is constrained to prevent local areas from being stretched or compressed, causing pattern distortion; the determination method is as follows:

[0032]

[0033] where, is the coating deformation constraint loss; J u , J v is the Jacobian matrix of UV mapping, reflecting the scaling and stretching of the texture when projected onto the model surface; I is the identity matrix, used to measure the ideal state without deformation;

[0034] Reflection consistency constraint: The surface reflection characteristics of the texture are constrained to be consistent with the real material, avoiding unnatural highlights or brightness abnormalities under light; the determination method is as follows:

[0035]

[0036] where, is the reflection consistency loss; is the surface reflection intensity function calculated according to the generated texture, representing the incident angle and observation angle, is the reflection distribution function BRDF of the real material, obtained through measurement or calibration;

[0037] Joint optimization objective function: Combine the physical constraint terms into the overall loss function, and perform gradient backpropagation optimization on the camouflage generation model; the joint optimization objective function is as follows:

[0038]

[0039] is the joint optimization objective function, α g , α j , α bA weight coefficient of each constraint term is used to balance the influence degree of different constraints.

[0040] The generated camouflage texture image is scaled according to the size of the target model, and image color channel conversion is performed and output and saved.

[0041] Step 8: Camouflage texture output: The finally generated UV patch camouflage texture map is output after being scaled according to the geometric size of the equipment, and then printed into a patchable camouflage texture map.

[0042] The application also provides a camouflage adversarial generation device based on environmental style transfer, the device is used to implement the method provided by the application, and the device comprises:

[0043] The construction module is configured to construct an environmental scene simulation platform.

[0044] The acquisition module is configured to acquire a multi-environment simulation data set.

[0045] The mapping module is configured to map the differential neural renderer.

[0046] The fusion module is configured to perform environmental style transfer fusion.

[0047] The optimization module is configured to optimize the composite loss function: combine the adversarial loss, the visual smoothness loss and the style transfer loss to construct a composite loss function, and perform optimization; and perform end-to-end texture optimization; and perform secondary optimization based on the physically realizable camouflage texture: perform reverse optimization on the camouflage texture generation model by introducing constraint functions related to printing, coating and material.

[0048] The output module is configured to output the camouflage texture: the finally generated UV patch camouflage texture map is output after being scaled according to the geometric size of the equipment, and then printed into a patchable camouflage texture map.

[0049] Compared with the prior art, the application has the following advantages:

[0050] 1) The detector has high adversarial performance: the method significantly reduces the recognition performance of the AI detector through adversarial loss optimization, and the camouflage deception effect is better, for example, in the case of target model camouflage, the AP@0.5 in the urban environment is 0.045, and the AP@0.5 in the jungle environment is 0.177.

[0051] 2) Wide environmental adaptability: through experiments, it is verified that the method can maintain stable camouflage concealment performance in various weather conditions such as sunny, rainy and foggy days, and different light conditions.

[0052] 3) Background environment fusion nature: this method can keep the camouflage texture visually smooth fusion with the background environment, with the dual effect of deceiving detection and recognition system and confusing the human eye observation;

[0053] 4) Solve the environment domain offset and manufacturing physical constraint mismatch problem between the simulation domain and the installation domain, the invention further introduces a physical realizability joint constraint module to realize closed-loop optimization from virtual generation to physical deployment.

[0054] To more clearly illustrate the functional characteristics and structural parameters of the present application, the following further describes the present application in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 is the overall flowchart provided by the embodiment of the present application;

[0056] Figure 2 is the urban environment style target camouflage texture UV map provided by the embodiment of the present application.

[0057] Figure 3 is the jungle environment style target camouflage texture UV map provided by the embodiment of the present application. DETAILED DESCRIPTION

[0058] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of the present application.

[0059] The present application will be further described below in conjunction with specific embodiments.

[0060] The implemented processing process is as follows Figure 1 .

[0061] Embodiment: target model camouflage generation in urban rainy environment

[0062] A virtual simulation environment scene is established using Unreal Engine 5, and a city block scene is built.

[0063] Environment scene configuration: the environment type is city, including street, roadside building, green trees, etc.; the weather mode is set to rainy day, raindrop density 0.8mm / s, raindrop size 1.5mm; the lighting condition is dusk, sun elevation angle 15°, color temperature 4500K, light intensity 3000lux.

[0064] Virtual UAV parameter settings, azimuth angle 0° start, horizontal rotation angle every 45° set a point, elevation angle height angle 22.5° and 67.5°, to the target center distance is set 5m, 10m, 15m, 20m in turn.

[0065] Under the above conditions, collect images of the target model in the environment background at multiple different block positions, with a resolution of 1280x720 or 1920x1080.

[0066] Import the OBJ model of the target, use Blender to configure UV expansion, cover the entire surface of the 3D model of the target, and normalize the UV coordinates to the range of [0, 1]x[0, 1] to maintain texture mapping continuity and reduce rendering distortion.

[0067] Perform rendering with the differential neural renderer, set the depth buffer smoothing parameter sigma = 1e-4, which will cause the texture edge to be blurred if the parameter is too large, and the texture will appear jagged if the parameter is too small; set the depth buffer quantization granularity bin_size = 100, the larger the parameter, the higher the depth resolution, which will cause the calculation to increase, the smaller the calculation speed will be faster, but the depth accuracy will be reduced. After rendering, a 2D rendered image matching the background is generated, with an output resolution of 720x720, which can support gradient backpropagation to the UV texture.

[0068] Extract the high-level environmental style features of the background image using the pre-trained backbone network, and represent the Gram matrix of the background style features Where F is the φ l layer convolutional feature, i, j is the channel index. Generate an initial camouflage texture through the style transfer network, and the style loss , measures the difference between the two matrices at each element position, and through the accumulation of the square of the difference value, it can comprehensively and quantitatively evaluate the overall difference between the generated camouflage texture and the background texture in terms of style features, and then divide by C 2 H 2 W 2 for normalization to get a relatively stable and comparable loss value to guide the style adjustment of the generated texture in the camouflage optimization process.

[0069] The composite loss function is calculated, Each loss function component is:

[0070] Adversarial loss:

[0071] Style transfer loss:

[0072] Visual smoothing loss:

[0073] a, b and g are weight parameters, and the weight values are set as a = 0.5 to dominate the detection effect of the detector, b = 0.1 to smooth the texture of the environment style, and g = 10 after experimental verification -2 Maintain the visual smoothness of the generated texture.

[0074] In the iterative optimization process, the Adam optimizer is used, the number of iterations is set to 100 rounds, and the convergence threshold Until the composite loss reaches the convergence range, terminate the iteration loop.

[0075] Based on the physically achievable camouflage texture secondary optimization. By introducing multiple constraints related to printing, coating and material, the camouflage texture generation model is optimized in reverse, so that the final output texture can not only maintain the adversarial nature, but also meet the actual process conditions.

[0076] (1) Print color gamut constraint

[0077] Limit the color range of the texture output to ensure that it can be realized by the printing device and coating material, and avoid the simulation generated color from being unable to reproduce in reality.

[0078]

[0079] where T UV (i) is the color value of each pixel of the generated camouflage texture

[0080] (2) Coating deformation constraint formula

[0081] According to the Jacobian matrix of the UV coordinates, the geometric deformation degree of the texture when coating is constrained to prevent local areas from being stretched or compressed to cause pattern distortion.

[0082]

[0083] where, is the coating deformation constraint loss. J u , J v is the Jacobian matrix of UV mapping, reflecting the scaling and stretching of the texture when projected on the model surface. I is the identity matrix, used to measure the ideal state without deformation.

[0084] (5) Reflection consistency constraint,

[0085] Constraint the surface reflection characteristics of the texture to be consistent with the real material, avoiding unnatural highlights or brightness abnormalities under light.

[0086]

[0087] where, is the reflection consistency loss. The surface reflection intensity function, calculated based on the generated texture, represents the incident angle and the viewing angle. It is the reflectance distribution function (BRDF) of the actual material, obtained through actual measurement or calibration.

[0088] (6) Jointly optimize the objective function

[0089] The aforementioned physical constraints are combined into a single overall loss function, and gradient backpropagation optimization is performed on the camouflage generation model.

[0090]

[0091] This combines the aforementioned physical constraint terms into a single overall loss function, α. g α j α b The weighting coefficients of each constraint are used to balance the degree of influence of different constraints.

[0092] Generate physically achievable camouflage textures, scale the generated camouflage texture image to the size of the target model, perform image color channel conversion, and then output and save the result. See the example for the generated target model camouflage texture map. Figure 2 After the generated camouflage texture is rendered onto the surface of the target 3D model, the output shows the camouflage effect of the target model in the urban environment from different perspectives.

[0093] Following the above implementation process, camouflage textures for equipment in other environmental scenarios can be generated. Figure 3 Apply camouflage textures to target models in jungle environments.

[0094] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for generating adversarial camouflage based on environmental style transfer, characterized in that, The method includes: Step 1: Build an environmental scenario simulation platform; Step 2: Collect multi-environment simulation datasets; Step 3: Differential Neural Renderer Mapping; Step 4: Environmental style transfer and integration; Step 5: Composite Loss Function Optimization: Combine adversarial loss, visual smoothing loss, and style transfer loss to construct a composite loss function and optimize it; Step 6: End-to-end texture optimization; Step 7: Secondary optimization of physically realizable camouflage texture: By introducing constraint functions related to printing, overlaying, and materials, the camouflage texture generation model is optimized in reverse. Step 8: Camouflage Texture Output: The final generated UV patch camouflage texture map is enlarged according to the geometric size ratio of the equipment and then printed into a camouflage texture map that can be patched and applied.

2. The method according to claim 1, characterized in that, Step 1: Construct an environmental scenario simulation platform, including: Based on the construction of various typical simulation environments, covering landforms such as cities, jungles, and deserts, each landform environment simulation is set with various meteorological conditions including sunny days, rainy days, and foggy days, and different lighting conditions including daytime, dusk, and nighttime, in order to fully reproduce the complex environmental scene changes that real environments may face.

3. The method according to claim 1, characterized in that, Step 2: Collect multi-environment simulation datasets; including: Using a virtual drone in an environmental scenario simulation platform, a flight path and field of view were set. Under conditions of multiple azimuth angles, multiple altitude angles, and multiple distances, high-resolution image data of a 3D target equipment model in various environmental backgrounds were collected to establish a dataset for training the adversarial model. During the collection of the simulation dataset, any two of the azimuth angle and distance were kept constant, and only the last of the three quantities was changed. Data was collected under multiple values. Then, in the same way, the three quantities were controlled in turn to collect multiple values ​​while keeping the other quantities constant.

4. The method according to claim 1, characterized in that, Step 3: Differential Neural Renderer Mapping; including: The Differentiable Neural Renderer is used to map the 3D OBJ model of the target equipment and its UV camouflage texture map to the 2D image space, achieving smooth rendering from 3D to 2D.

5. The method according to claim 1, characterized in that, Step 4: Environmental style transfer and integration; including: High-level environmental style features of the background image are extracted using a pre-trained ResNet backbone network and fused with features of the equipment surface area to generate an initial camouflage texture that is approximately consistent with the surrounding environment of the equipment in terms of color style and texture.

6. The method according to claim 1, characterized in that, Step 5: Optimize the composite loss function, including: Adversarial loss The output confidence score of the AI ​​detector is used to maximize the recognition error of missed or misclassified targets, thereby reducing the ability to detect and identify camouflaged targets. The formula is as follows: H d (x) is the detector's confidence in the target category x. The negative logarithmic function converts the false negative probability into a loss value. The larger the false negative probability, the closer the loss value is to 0. Style transfer loss The Gram matrix is ​​used to measure the style difference between the input image and the camouflage texture. Style features of the camouflage texture are extracted from the background image and matched with the texture of the camouflage target to minimize the difference between the generated camouflage texture and the background style features. The formula is as follows: in and These are the generated camouflage texture x and the background image y in the convolutional layer φ. l The Gram matrix at the location, where C represents the channel, and H and W represent the height and width of the feature map; Visual smoothing loss To maintain the visual smoothness of the camouflage texture, constrain the changes in texture space gradient, prevent drastic abrupt changes, and maintain a natural visual transition of the generated camouflage texture, the formula is: The adversarial loss, style transfer loss, and visual smoothing loss are comprehensively and weightedly summed to form the formula for the total composite loss function: Wherein, α, β and γ are weight parameters that control the contribution of adversarial loss, style transfer loss and visual smoothing loss to the total loss, respectively. Iterative optimization of the composite loss function during model training.

7. The method according to claim 1, characterized in that, Step 6: End-to-end texture optimization, including: Using the initial camouflage texture as input, the composite loss function is backpropagated through the Adam optimizer to dynamically update the generation parameters of the UV patch camouflage texture map until the composite loss function reaches the expected convergence interval and stops iterative calculation. The optimization process is accelerated in parallel by combining CUDA.

8. The method according to claim 1, characterized in that, Step 7: Secondary optimization of physically realizable camouflage textures, including: Printing gamut constraint: Limits the range of colors for texture output to ensure that it can be achieved by printing equipment and coating materials; Where T UV (i) is the color value of each pixel in the generated camouflage texture; Overlay deformation constraint: Based on the Jacobian matrix of UV coordinates, the degree of geometric deformation of the texture during overlay is constrained to prevent local areas from being stretched or compressed, which would cause pattern distortion; the determination method is as follows: in, It is the loss due to deformation constraint; J u J v I is the Jacobian matrix of UV mapping, reflecting the scaling and stretching of the texture when projected onto the model surface; I is the identity matrix, used to measure the invariance under ideal conditions. Reflection consistency constraint: Constrains the surface reflective properties of the texture to be consistent with those of the real material, avoiding unnatural highlights or abnormal brightness under lighting; the determination method is as follows: in, It is a reflection consistency loss; The surface reflection intensity function, calculated based on the generated texture, represents the incident angle and the viewing angle. It is the BRDF (Brake-Rich-Diffuse Reflectance Function) of the actual material, obtained through actual measurement or calibration; Joint optimization objective function: The physical constraint terms are merged into the overall loss function, and gradient backpropagation optimization is performed on the camouflage generation model; the joint optimization objective function is as follows: It is a joint optimization objective function, α g α j α b The weighting coefficients of each constraint are used to balance the degree of influence of different constraints; Generate physically achievable camouflage textures, scale the generated camouflage texture images to the size of the target model, perform image color channel conversion, and then output and save them. After rendering the generated camouflage textures onto the surface of the target 3D model, the output shows the camouflage effect of the target model in an urban environment from different perspectives.

9. A camouflage adversarial generation device based on environmental style transfer, the device being used to implement any one of the methods of claims 1 to 8, characterized in that, The device includes: Build modules are used to construct environmental scenario simulation platforms; The data acquisition module is used to collect multi-environment simulation datasets; The mapping module is used for differential neural renderer mapping; The blending module is used for environmental style transfer and blending. The optimization module is used for composite loss function optimization: combining adversarial loss, visual smoothing loss and style transfer loss to construct a composite loss function and optimize it; as well as end-to-end texture optimization; and physically realizable camouflage texture secondary optimization: by introducing constraint functions related to printing, overlaying and material, the camouflage texture generation model is optimized in reverse. Output module, used for camouflage texture output: The final generated UV patch camouflage texture map is enlarged according to the geometric size ratio of the equipment and then printed into a camouflage texture map that can be patched and applied.

Images