Camouflage confrontation coating generation and physical enhancement method

Abstract

The invention discloses a camouflage confrontation coating generation and physical enhancement method, which comprises the following steps: generating a uniform Thiessen polygon based on an RGB-D image, and calculating control point distribution regularization loss; fusing the multi-scale RGB and depth features; the color distribution probability is determined by combining the pixel point distance and depth information, and preliminary coating is generated; the camouflage concealment is optimized through saliency detection and environment dominant hue fusion; spatial domain affine transformation and RGB-CMYK transformation are introduced to enhance physical robustness, and visual consistency loss is calculated; and optimizing a comprehensive loss function by combining control point distribution regularization loss, multi-scale fusion saliency loss, visual consistency loss and target detection confrontation loss, and iteratively generating final coating. According to the method, the geometric camouflage precision is improved, the visual saliency of physical coating is reduced, the environmental adaptability and the anti-robustness are enhanced, and the method is suitable for hiding and anti-attack of a complex surface target.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence security, and particularly to a method for generating and physically enhancing camouflage countermeasure coatings. Background Art

[0002] With the continuous progress of deep learning technology, object detection and recognition technology has been widely applied in various fields, including maritime security, environmental monitoring, traffic management, etc. In these fields, deep learning-based object detection models, such as YOLO and Faster R-CNN, have demonstrated excellent detection accuracy and real-time performance, providing important information support for practical applications. These technologies can quickly and accurately detect targets in complex natural environments, greatly enhancing the automated monitoring capabilities of various industries for the environment, objects, and security events.

[0003] However, with the improvement of the performance of these detection systems, people have gradually realized that in some civilian and military fields and applications, it has become particularly important to avoid these advanced detection systems. Traditional camouflage means, such as color coatings and structural deformation, although they have improved in visual concealment, not only require careful design based on experience but also often have limited effects when dealing with deep learning-based multi-scale and multi-level detection systems. Intelligent detection systems can not only recognize simple shape camouflage but also maintain efficient recognition of targets from multiple angles and in multiple environments, making traditional camouflage technologies appear inadequate in certain application scenarios.

[0004] To address this challenge, researchers have begun to explore adversarial sample technology, hoping to interfere with the normal judgment of detection systems by adding tiny perturbations to input images. Szegedy et al. first proposed the concept of adversarial samples in 2014. They found that although these perturbations are almost invisible to the human eye, they are sufficient to cause deep neural network models to misclassify. Subsequently, Goodfellow et al. proposed the Fast Gradient Sign Method (FGSM), further optimizing the method for generating adversarial samples; Madry et al. proposed an attack method based on Projected Gradient Descent (PGD) in 2017, making the attack more robust through multiple iterations and random initialization; Tramèr et al. proposed an adaptive adversarial attack in 2020, which can dynamically adjust the perturbation direction according to the defense strategy, greatly improving the success rate of the attack. While these methods improve the effect of adversarial sample attacks, they also promote the continuous development of adversarial attack technology.

[0005] The extensive research on adversarial examples has also prompted their gradual entry into the application scenarios of the physical world. For example, Eykholt et al. applied adversarial coatings to traffic signs, making it impossible for the autonomous driving system to correctly identify the signs. Although these coatings look no different from ordinary signs to the naked eye, they can significantly affect the recognition effect of the detection system. However, the adversarial example technology applied in the physical world still faces challenges. Due to the interference of multiple real-world factors such as light, material, and perspective, the performance of adversarial examples migrated to the physical world is often less stable than in the digital environment. For example, Thys et al. tried to interfere with the YOLO detection system by generating adversarial patches in the physical world, but their effects were weakened under different environmental conditions, indicating that there are still limitations in the existing technology when migrating to practical applications.

[0006] In addition to the problem of insufficient effectiveness of adversarial examples in the physical world, RGB to CMYK color conversion and geometric deformation are also key challenges in adversarial coating design. Traditional RGB to CMYK color conversion methods, such as the standardized conversion of ICC profiles, have been widely used in printing devices, but this method still has significant deficiencies in the face of complex color gradients and color consistency, resulting in inconsistent digital design effects of the printed coatings in the physical world. This color distortion problem significantly affects the effect of physical adversarial examples, especially in scenarios with high precision requirements. In addition, the geometric deformation problem has not been effectively solved either. The current adversarial example technology mostly focuses on two-dimensional plane design, such as applications in traffic signs or body coatings. However, when these designs are applied to physical objects, due to the deformation of the three-dimensional geometric shape of the object in the real environment, the model often fails to correctly identify the adversarial examples, weakening their attack ability.

[0007] Therefore, although the existing technology has made certain progress in adversarial coating design and the application of digital adversarial examples migrated to the physical world, there are still many problems to be solved in how to ensure the physical adaptability and robustness of adversarial examples and in the physical coating design of large target objects. Summary of the Invention

[0008] The purpose of the present invention is to provide a method for generating and physically enhancing camouflage adversarial coatings to achieve the robustness and consistency of adversarial coatings in the digital and physical worlds, and significantly improve their concealment and adversarial effects.

[0009] To solve the above technical problems, the present invention provides a method for generating and physically enhancing camouflage adversarial coatings, including: S1. Based on the acquired RGB image and depth image, arrange multiple control points on the target surface, generate uniformly distributed Thiessen polygons, and calculate the regularization loss of the control point distribution; S2. Extract features from RGB images and depth images of different scales respectively, and then perform feature fusion to obtain a multi-scale feature map; S3. Obtain the color distribution probability of each pixel point according to the distance between the pixel point and the control point, as well as the depth information of the pixel point; the depth information of the pixel point is obtained through the depth image; S4. Determine the color of each pixel point according to the color distribution probability of each pixel point and the camouflage color within the area formed by each preset Thiessen polygon, and then obtain the target image with painting; S5. Input the target image with painting after downsampling at different scales into a pre-trained saliency detection model to obtain saliency maps at different scales; S6. Obtain the feature mean and variance within the area formed by each Thiessen polygon based on the multi-scale feature map; S7. Obtain the fusion weight coefficient according to the feature mean and variance, as well as the saliency map; S8. Perform weighted fusion on the camouflage color and the main environmental color according to the fusion weight coefficient to obtain the fused camouflage color; S9. Obtain the target image with painting after fusion according to the fused camouflage color; S10. Optimize the target image with painting after fusion to obtain a preliminary camouflage painting, and calculate the multi-scale fusion saliency loss; S11. Perform an affine transformation in the spatial domain on the preliminary camouflage painting; S12. Convert the color space of the preliminary camouflage painting after the spatial domain affine transformation from RGB to CMYK, and calculate the visual consistency loss before and after the color space conversion; S13. Input the target image with the preliminary camouflage painting after the color space conversion into a trained target detection model to obtain the target detection adversarial loss; S14. Obtain a comprehensive loss function according to the control point distribution regularization loss, the multi-scale fusion saliency loss, the visual consistency loss, and the target detection adversarial loss; S15. Iteratively optimize the foregoing steps according to the comprehensive loss function until a final camouflage painting that meets the requirements is obtained.

[0010] According to the above solution, the step S2 includes: S201. Extract features of RGB images and depth images of different scales; S202. Perform feature fusion on the features of the RGB image and the depth image of the same scale based on the set weight coefficient; S203. Integrate the feature fusion results of different scales into a multi-scale feature map.

[0011] According to the above solution, step S3 includes: constructing a color distribution model in the form of a Gaussian kernel, and obtaining the color distribution probability of each pixel point according to the set smoothing parameter.

[0012] According to the above solution, in step S5, the pre-trained saliency detection model is U²-Net, and the pre-trained saliency detection model is trained based on the public datasets DUTS or SALICON.

[0013] According to the above solution, step S7 includes: S701. Obtain the saliency scores within the regions formed by each Thiessen polygon of the saliency maps at different scales through mean aggregation; S702. Obtain the fusion weight coefficients according to the set global optimizable parameters, saliency scores, feature means, and variances.

[0014] According to the above solution, step S10 includes: S1001. Obtain the saliency maps at different scales corresponding to the painted target image after fusion, and normalize the pixel values within the saliency maps; S1002. Weightedly fuse the saliency maps at each scale according to the preset weights corresponding to different scales, and calculate the multi-scale fusion saliency loss.

[0015] According to the above solution, step S11 includes: S1101. Input the preliminary camouflage coating into the spatial transformation network to predict the affine transformation matrix; S1102. Perform geometric transformation on the preliminary camouflage coating according to the affine transformation matrix.

[0016] According to the above solution, step S12 includes: S1201. Input the preliminary camouflage coating after affine transformation in the spatial domain into the trained multi-layer perceptron mapping model for color space conversion; S1202. Obtain the brightness difference, contrast difference, and structure difference before and after color space conversion; S1203. Fuse the brightness difference, contrast difference, and structure difference through the set brightness importance index parameter, contrast importance index parameter, and structure importance index parameter to obtain the structural similarity index; S1204. Calculate the visual consistency loss according to the structural similarity index.

[0017] The present invention also provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of the above-mentioned camouflage confrontation coating generation and physical enhancement method are implemented.

[0018] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the camouflage confrontation painting generation and physical enhancement method described above are implemented.

[0019] Beneficial effects The present invention restricts the generation of control points through the set control point distribution regularization loss, thereby making the Voronoi polygons generated based on the control points more uniform and improving the geometric stability of the camouflage painting; by performing color space conversion on the camouflage painting, color distortion generated during the printing process is avoided; by performing spatial domain affine transformation on the camouflage painting, the camouflage painting can adapt to different viewing angles and lighting conditions, enhancing its adaptability; through the set comprehensive loss function and repeated iterative optimization, the comprehensive performance of the final camouflage painting is improved. The specific beneficial effects of the present invention will be described in detail in the specific implementation manner. Description of the drawings

[0020] Figure 1 is the flowchart of the camouflage confrontation painting generation and physical enhancement method of this embodiment; Figure 2 is the color mapping schematic diagram of printing from the digital world to the physical world in this embodiment. Specific implementation manner

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are some, but not all, of the embodiments of the present disclosure. All other embodiments obtained by those of ordinary skill in the art based on the described embodiments of the present disclosure without creative efforts shall fall within the scope of protection of the present disclosure.

[0022] Embodiment 1: A camouflage confrontation painting generation and physical enhancement method includes: S1. Based on the acquired RGB image and depth image, arrange a plurality of control points on the target surface, generate evenly distributed Voronoi polygons, and calculate the control point distribution regularization loss; S2. Extract features from the RGB images and depth images of different scales respectively and perform feature fusion to obtain a multi-scale feature map; S3. According to the distance between the pixel points and the control points, and the depth information of the pixel points, obtain the color distribution probability of each pixel point; the depth information of the pixel points is obtained through the depth image; S4. According to the color distribution probability of each pixel point and the preset camouflage colors within the regions formed by the respective Voronoi polygons, determine the color of each pixel point, and thus obtain the target image with the painting. S5. Downsample the painted target image at different scales and input it into a pre-trained saliency detection model to obtain saliency maps at different scales; S6. Obtain the feature mean and variance within the region formed by each Thiessen polygon based on the multi-scale feature maps; S7. Obtain the fusion weight coefficients according to the feature mean and variance, and the saliency map; S8. Perform weighted fusion of the camouflage color and the main environmental color according to the fusion weight coefficients to obtain the fused camouflage color; S9. Obtain the painted target image after fusion according to the fused camouflage color; S10. Optimize the painted target image after fusion to obtain a preliminary camouflage coating, and calculate the multi-scale fusion saliency loss; S11. Perform an affine transformation in the spatial domain on the preliminary camouflage coating; S12. Convert the color space of the preliminary camouflage coating after the spatial domain affine transformation from RGB to CMYK, and calculate the visual consistency loss before and after the color space conversion; S13. Input the target image with the preliminary camouflage coating after the color space conversion superimposed into the trained target detection model to obtain the target detection adversarial loss; S14. Obtain the comprehensive loss function according to the control point distribution regularization loss, the multi-scale fusion saliency loss, the visual consistency loss, and the target detection adversarial loss; S15. Iteratively optimize the foregoing steps according to the comprehensive loss function until the final camouflage coating that meets the requirements is obtained.

[0023] Further, in step S1, in this embodiment, an RGB-D camera is used to obtain an RGB image and a depth image (i.e., an RGB-D map). Optionally, common RGB-D cameras such as Kinect and RealSense can be selected; optionally, the target is a ship; In this embodiment, the control points are represented as , and Thiessen polygons are generated using the distances between the control points. The boundaries of the Thiessen polygons are defined by the nearest neighbor distances; when arranging the control points, the two-dimensional position and depth information on the target surface are used simultaneously, so that the division of the generated Thiessen polygons conforms to the three-dimensional shape of the target surface. The division of the Thiessen polygons satisfies the following formula:

[0024] where, is the th Thiessen polygon region, is any point x on the target surface and the control point The distance between; to ensure the uniform distribution of the control points on the target surface, a control point distribution regularization loss is adopted , to encourage a certain distance to be maintained between the control points, prevent the generated Thiessen polygons from being too small, and affect the adversarial nature of the generated painting. Its form can be defined as:

[0025] wherein, is the scale parameter of the distance between the control points, which can be set according to the application scenario and actual generation requirements; the optimization goal is to make this control point distribution regularization loss as small as possible, so as to avoid the over-concentration of the control points; through this optimization, the distribution of the Thiessen polygons is made more uniform, ensuring that the camouflage painting can cover the entire target surface and be evenly distributed.

[0026] Furthermore, the step S2 includes: S201. Extract the features of RGB images and depth images at different scales; S202. Perform feature fusion on the features of the RGB image and the depth image at the same scale based on the set weight coefficients; S203. Integrate the feature fusion results at different scales into a multi-scale feature map; Specifically, the purpose of step S2 is to improve the effectiveness of the adversarial painting at different scales; first, specifically through the Feature Pyramid Network (FPN), multi-scale features are extracted from RGB images at different scales (resolutions) ; then the features at different scales are weighted and fused. In particular, to combine depth information, features are extracted from depth images at different scales , and are fused with the features of the RGB image to obtain a multi-scale feature map. The formula is as follows:

[0027] wherein, is the multi-scale feature map, representing the feature map extracted and fused at different scales (resolutions); L is the number of scales, indicating how many different resolution levels the feature map is extracted from; is the weight of the feature maps at each scale, and the robustness of the painting in a multi-scale environment is improved through feature fusion. The Fusion() function performs feature fusion by weighted summation. The specific formula is:

[0028] wherein, and is the weight coefficient, which is used to balance the contributions of the features of the RGB image and the depth image. These weights can be learned from the training dataset or manually adjusted according to the specific application scenario.

[0029] Further, the step S3 includes: constructing a color distribution model in the form of a Gaussian kernel and obtaining the color distribution probability of each pixel point according to the set smoothing parameter; Specifically, since it is necessary to model the color distribution for each region formed by the Thiessen polygons respectively, in this study, the form of the Gaussian kernel is selected to construct this model; in this embodiment, if it is assumed that there are kinds of camouflage colors , then the color distribution probability of each pixel point is expressed as:

[0030] where is the squared distance between the pixel and the control point , is the smoothing parameter, which is used to control the expansion radius of the color distribution. It is usually determined by experimental adjustment, and its value should reflect the typical distance scale between the expected control points; is the depth information of the pixel x in the depth image. When the depth is large, the same planar distance will cause faster weight decay. This design enables the model to better reflect the actual object surface situation in the three-dimensional scene.

[0031] Further, in the step S4, the color of the pixel point x is expressed as:

[0032] where is the camouflage color within the region formed by each preset Thiessen polygon; Based on the above color distribution modeling method, the rationality and effectiveness of the color allocation within the regions formed by different Thiessen polygons are ensured. At the region boundary, due to the similar influence of multiple control points, a smooth transition appears at the edges of each region, forming an irregular soft boundary effect instead of a traditional hard boundary, making the entire camouflage pattern look more natural, reducing the obtrusiveness of large color blocks, and at the same time well conforming to the key characteristics of "block unity and boundary fragmentation" in traditional camouflage design, which is more helpful to "confuse" the system visually or in front of the detection model.

[0033] Further, in the step S5, the pre-trained saliency detection model is U²-Net, and the pre-trained saliency detection model is trained based on the public datasets DUTS or SALICON; Specifically, the purpose of adopting the feedback mechanism based on the visual saliency detection model in this embodiment is to optimize the selection of camouflage colors and give full play to the visual concealment of camouflage as much as possible. The saliency detection model adopted in this embodiment can generate a saliency map with the same resolution as the original image or after upsampling for the input image, and its output value is usually normalized to the range of [0, 1], indicating the probability or importance degree of each pixel being noticed by the human eye. The painted target image is expressed as , and after downsampling at different scales , a series of multi-scale images are obtained , and then they are respectively input into the pre-trained saliency monitoring model to obtain the saliency maps at each scale:

[0034] In the above formula, is the pre-trained saliency detection model.

[0035] Furthermore, in step S6, the feature mean and variance within the region formed by each Voronoi polygon are respectively expressed as and , then there is:

[0036] In the above formula, represents the region formed by the k-th Voronoi polygon.

[0037] Furthermore, step S7 includes: S701. Obtain the saliency scores within the regions formed by each Voronoi polygon of the saliency maps at different scales through mean aggregation; Specifically, the saliency score is expressed as , that is, the saliency score after mean aggregation within the region at multiple scales; S702. Obtain the fusion weight coefficients according to the set global optimizable parameters, saliency scores, feature means, and variances; Specifically, the fusion weight coefficient is expressed as , then there is:

[0038] In the above formula, where and are global optimizable parameters (i.e., adjustable parameters in the iterative optimization process) and are updated together with other parameters through gradient descent; represents the feature mean, represents the variance.

[0039] Furthermore, in step S8, the fused camouflage color is expressed as , then there is:

[0040] Among them, represents the main color tone in the environment, which is obtained by performing color histogram analysis on the environmental image and acquiring.

[0041] Furthermore, the step S10 includes: S1001. Obtain the saliency maps at different scales corresponding to the painted target image after fusion, and normalize the pixel values in the saliency maps; Specifically, the saliency maps at different scales corresponding to the painted target image after fusion are expressed as , where each pixel value is normalized to [0, 1]; S1002. According to the weights corresponding to the preset different scales, perform weighted fusion on the saliency maps at each scale, and calculate the multi-scale fusion saliency loss; Specifically, the multi-scale fusion saliency loss is expressed as , then there is:

[0042] In the above formula, is the weight of the l-th scale when performing weighted fusion on the saliency maps at each scale, and it can be empirically selected according to the specific application scenario and the characteristics of the target object; is the total number of pixels in the entire painted area.

[0043] After completing the above environmental color tone fusion and optimization, the color of each pixel point can be regenerated based on the fused control point colors and color distribution probabilities, and a camouflage coating with both concealment and preliminary adversarial properties in the digital domain (i.e., the preliminary camouflage coating) can be generated.

[0044] Furthermore, the step S11 includes: S1101. Input the preliminary camouflage coating into the Spatial Transformer Network (STN) to predict the affine transformation matrix; Specifically, to solve the visual effect problem caused by geometric deformation and perspective change when the adversarial coating is covered on a large target object, in this embodiment, the Spatial Transformer Network is adopted, and its core is to learn the affine transformation matrix , and combine the depth information to dynamically adjust the geometric transformation of the input image, so as to maintain the adversarial effect of the coating under different observation angles, illuminations, and perspective conditions; In the Spatial Transformer Network, a convolutional sub-network is used to predict the affine transformation matrix , and the affine transformation matrix is expressed as:

[0045] In the above formula, the elements within the matrix are used to describe the scaling and rotation parameters of the image, while and represent the translation parameters. The elements are automatically predicted by the spatial transformation network based on the input image features and are used to perform spatial domain affine transformation on the generated camouflage coating, so as to simulate the geometric deformations that may occur during the printing or installation process; S1102. Perform geometric transformation on the preliminary camouflage coating according to the affine transformation matrix; Specifically, apply the aforementioned matrix to perform geometric transformation on the input camouflage coating. For any pixel within the input camouflage coating, its corresponding pixel after spatial domain affine transformation satisfies:

[0046] It should be understood that since the image transformed by the spatial transformation network is used as the input for calculating the subsequent visual consistency loss and the object detection adversarial loss, and all these losses are differentiable functions, their gradients will backpropagate through the spatial transformation network, thereby updating the parameters of the spatial transformation network to make the spatial domain affine transformation more conform to the real deformations in the physical environment.

[0047] Further, the step S12 includes: S1201. Input the preliminary camouflage coating after spatial domain affine transformation into the trained multi-layer perceptron mapping model for color space conversion; Specifically, since the coating effect in the real world needs to be achieved through printing devices, and most printing devices are based on CMYK colors and cannot directly process RGB colors, it is necessary to consider the RGB to CMYK color conversion problem. Based on this, in this embodiment, a multi-layer perceptron mapping model from RGB to CMYK is adopted. This model is trained based on the publicly available RGB-CYKM pairing dataset to ensure the consistency of color conversion and avoid the coating effect deviation caused by color space conversion. The color space conversion process is expressed as:

[0048] In the above formula, represents the trained multi-layer perceptron mapping model; S1202. Obtain the brightness difference, contrast difference, and structure difference before and after color space conversion; Specifically, to ensure the visual consistency of colors in different environments, in this embodiment, the images before and after the spatial domain affine transformation and color space operation conversion are evaluated in terms of brightness, contrast, and structure to ensure that the visual effects before and after the color space conversion are consistent in the digital world and the physical world (see Figure 2 , Figure 2 ; the left side is the digital world image, and the right side is the physical world image); therefore, the obtained brightness difference, contrast difference, and structure difference are respectively expressed as , , , where a and b respectively represent the camouflage coatings before and after the color space conversion; in this embodiment, the calculation methods of the above three differences are respectively expressed as:

[0049]

[0050]

[0051] Among them, , are respectively the local standard deviations of image a and image b, , are respectively the local means of image a and image b; is a stability constant, which is to avoid the situation of the denominator being 0. Usually, the values are as follows:

[0052]

[0053]

[0054] Empirically, usually take = 0.01 and = 0.03; here, L represents the dynamic value range of pixels, and the calculation method is: , for example, for an 8-bit image, the value range of each pixel is 0 to 255, so L = 255; S1203. By setting the brightness importance index parameter, contrast importance index parameter, and structure importance index parameter, fuse the brightness difference, contrast difference, and structure difference to obtain the structural similarity index; Specifically, in this embodiment, considering the differences in different aspects before and after the color space conversion, an evaluation method based on the structural similarity index (SSIM) is proposed, which is expressed as:

[0055] In the above formula, They are exponential parameters for adjusting the relative importance of brightness, contrast, and structure. Usually, the values of these three parameters are 1, indicating that the three parts have equal importance. S1204. Calculate the visual consistency loss according to the structural similarity index. Specifically, the visual consistency loss is expressed as , which satisfies: .

[0056] Furthermore, after the steps of generating the camouflage confrontation coating and physically robust enhancement, direct adversarial verification needs to be carried out on the camouflage confrontation coating; in step S13, the target image with the preliminary camouflage coating after color space conversion is expressed as . The trained target detection model can be YOLO or Faster R-CNN, and the target detection adversarial loss is expressed as , then there is:

[0057] Among them, is the detection confidence of the target detection model for the input image; the optimization goal of the target detection adversarial loss is to make this value as accurate as possible, which can prompt the system to generate a more attack-resistant coating and reduce the risk of the target being detected and recognized.

[0058] Furthermore, the comprehensive loss function in step S14 is expressed as , and there is:

[0059] Among them, are the weight hyperparameters corresponding to the control point distribution regularization loss, multi-scale fusion saliency loss, visual consistency loss, and target detection adversarial loss respectively. Usually, they can be set through prior experience and experimental verification to achieve the best balance of the overall system in terms of generating geometric, environmental adaptability, visual consistency, and adversarial properties of the camouflage confrontation coating.

[0060] Furthermore, the iterative optimization process in step S15 is as follows: Using the chain rule, calculate the gradient of the comprehensive loss function with respect to all trainable parameters (including control points , the fusion weights in the environmental fusion step, and the global parameter , as well as the parameters of the indirect spatial transformation network, etc.), and update each trainable parameter through backpropagation to generate an ideal camouflage confrontation coating with high concealment and strong aggressiveness. For example:

[0061] Among them, is the learning rate, and the setting of this hyperparameter can be based on prior experience and experimental verification; in particular, since the affine transformation is placed before the RGB-CMYK conversion and the SSIM loss calculation, the predicted affine transformation parameters directly affect the subsequent visual consistency loss and the object detection adversarial loss, and thus its gradient is also updated through the backpropagation of the overall loss.

[0062] Figure 1 is the flowchart of the camouflage adversarial coating generation and physical enhancement method described in this embodiment. The functions of each step in the figure are as follows: ① Generation of optimized adversarial camouflage coating design based on Voronoi polygons Acquisition and input of the RGB-D image of the target object: For the target object, its corresponding RGB-D image can be obtained through common RGB-D cameras such as Kinect and RealSense as the input of the subsequent model.

[0063] Voronoi polygon division of the target surface: According to the input RGB-D image, multiple control points are arranged to divide the target surface into multiple Voronoi polygon regions, so that the subsequent coating design can be more evenly distributed and adapted to the object shape.

[0064] Geometric optimization of Voronoi polygons: Geometric optimization is performed on the divided Voronoi polygon regions. By combining the gradient descent algorithm with depth information, the positions of the control points are adjusted to minimize the control point distribution regularization loss, ensuring the uniform distribution and geometric accuracy of the camouflage coating.

[0065] Multi-scale feature extraction: The Feature Pyramid Network (FPN) is used to extract multi-scale features from RGB images and depth layers with different resolutions, laying the foundation for subsequent environmental color fusion.

[0066] Voronoi polygon color distribution modeling: Based on the distance between the control points and the pixel points and the depth information, color modeling is performed for each Voronoi polygon region. Through this model, it is ensured that the camouflage coating can reasonably cover the entire target surface and effectively improve its camouflage effect in the three-dimensional form.

[0067] Visual saliency feedback: The camouflage coating is evaluated through a visual saliency analysis model, and the coating design is adjusted according to the feedback to optimize the selection of camouflage colors, ensuring its good concealment effect in complex environments.

[0068] Background environment color extraction and camouflage color fusion: The main background color is extracted from the environmental image, and based on the multi-scale fusion features obtained previously, the environmental color and the camouflage color are weighted and fused to reduce the visual saliency of the target in the actual environment and improve the camouflage effect.

[0069] ② Domain adaptation optimization and physical world adversarial coating generation and enhancement Geometric Robustness Enhancement Module Based on STN Network: The STN network is used to adjust the geometric shape of the coating in combination with depth information, adapt to deformations and perspective changes in the physical world, and ensure that the coating maintains an adversarial effect under different angles and lighting conditions.

[0070] Color Depth Neural Network Model Conversion (RGB to CMYK): The RGB color model in the digital domain is converted into the CMYK model through a multi-layer perceptron mapping model to meet the requirements of printing devices and ensure color consistency of the printed coating.

[0071] Optimizing the Visual Consistency of the Coating Based on the Structural Similarity Index: The brightness, contrast, and structural consistency of the coating are optimized through SSIM to ensure that the coating in the physical world is consistent with the design effect in the digital domain.

[0072] Optimization and Update of Camouflage Coating Based on Comprehensive Loss Function: An adversarial loss function is established based on an object detector, and at the same time, the control point distribution regularization loss, multi-scale fusion saliency loss, and visual consistency loss designed in the previous steps are combined as the comprehensive loss function. The camouflage coating is optimized and updated through gradient backpropagation, and finally, the best balance of the camouflage adversarial coating in terms of geometry, environmental adaptability, visual consistency, and adversariality is achieved.

[0073] Generation of Camouflage Physical Adversarial Coating for Target Objects: The finally generated optimized Thiessen polygon camouflage coating design for domain adaptability is printed and applied to the surface of the object to ensure its excellent concealment and adversariality in the physical world and adapt to complex environments and multi-angle observations. Through the mutual cooperation and continuous iteration of all the steps and modules in the foregoing steps, this embodiment ensures the effectiveness and consistency of the adversarial coating from the digital to the physical domain, solves problems such as color conversion and geometric deformation, and improves the concealment, robustness, and adversarial camouflage effect of the adversarial coating. The beneficial effects of applying this embodiment include: (1) Improving the accuracy of geometric camouflage effect: The present invention simulates the effect of camouflage coating through a soft boundary optimization strategy based on Thiessen polygons, effectively reducing the dependence of large target objects on traditional manually designed coatings, achieving a simpler and more effective geometric camouflage distribution, and being applicable to various complex surfaces.

[0074] (2) Reducing the visual saliency of the adversarial coating in the physical world: Through a camouflage color extraction method based on environmental perception, the present invention can dynamically adjust the camouflage color according to the hue in the physical environment of the target object, and at the same time, introduces the calculation and optimization of the visual saliency function, effectively reducing the visual saliency of the coated target object in the real world.

[0075] (3) The robustness of the coating is improved: Through the geometric adaptability enhancement module based on SSIM and STN, the coating generated by the present invention can still maintain the concealment effect under complex geometric shapes and multi-view observation conditions in the physical domain, significantly improving the robustness of the coating.

[0076] (4) The adaptability of the coating on complex surfaces is improved: By combining the depth information of RGB-D images (i.e., composite images formed by combining RGB images and depth images), the present invention can achieve more accurate coating on complex surfaces, reducing the dependence on high-precision 3D modeling while ensuring the camouflage effect in the physical environment.

[0077] It should be noted that according to the needs of implementation, each step / component described in this application can be split into more steps / components, or two or more steps / components or partial operations of steps / components can be combined into new steps / components to achieve the purpose of the present invention.

[0078] Those skilled in the art can easily understand that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for generating camo confrontation painting and physical enhancement, characterized in that, Including: S1. Based on the acquired RGB image and depth image, arrange multiple control points on the target surface, generate a uniformly distributed Thiessen polygon, and calculate the control point distribution regularization loss; S2. Extract features from RGB images and depth images of different scales respectively, and then perform feature fusion to obtain a multi-scale feature map; S3. According to the distance between the pixel point and the control point, and the depth information of the pixel point, obtain the color distribution probability of each pixel point; the depth information of the pixel point is obtained through the depth image; S4. According to the color distribution probability of each pixel point and the camouflage colors within the regions formed by the preset Thiessen polygons, determine the color of each pixel point, and then obtain the target image with painting; S5. After downsampling the target image with painting at different scales, input it into a pre-trained saliency detection model to obtain saliency maps at different scales; S6. Based on the multi-scale feature map, obtain the feature mean and variance within the regions formed by each Thiessen polygon; S7. According to the feature mean and variance, and the saliency map, obtain the fusion weight coefficient; S8. According to the fusion weight coefficient, perform weighted fusion on the camouflage color and the main environmental color to obtain the fused camouflage color; S9. According to the fused camouflage color, obtain the fused target image with painting; S10. Optimize the fused target image with painting to obtain the preliminary camouflage painting, and calculate the multi-scale fusion saliency loss; S11. Perform a spatial domain affine transformation on the preliminary camouflage painting; S12. Convert the color space of the preliminary camouflage painting after the spatial domain affine transformation from RGB to CMYK, and calculate the visual consistency loss before and after the color space conversion; S13. Input the target image with the preliminary camouflage painting after the color space conversion into a trained target detection model to obtain the target detection adversarial loss; S14. According to the control point distribution regularization loss, the multi-scale fusion saliency loss, the visual consistency loss, and the target detection adversarial loss, obtain the comprehensive loss function; S15. Iteratively optimize the foregoing steps according to the comprehensive loss function until the final camouflage painting that meets the requirements is obtained.

2. The camouflage confrontation painting generation and physical enhancement method according to claim 1, wherein The step S2 includes: S201. Extract the features of RGB images and depth images of different scales; S202. Based on the set weight coefficients, perform feature fusion on the features of RGB images and depth images of the same scale; S203. Integrate the feature fusion results of different scales into a multi-scale feature map.

3. The camouflage confrontation painting generation and physical enhancement method according to claim 1, characterized in that The step S3 includes: constructing a color distribution model in the form of a Gaussian kernel, and obtaining the color distribution probability of each pixel point according to the set smoothing parameter.

4. The camouflage confrontation painting generation and physical enhancement method according to claim 1, characterized in that In the step S5, the pre-trained saliency detection model is U²-Net, and the pre-trained saliency detection model is trained based on the public dataset DUTS or SALICON.

5. The camouflage confrontation painting generation and physical enhancement method according to claim 1, characterized in that The step S7 includes: S701. Through mean aggregation, obtain the saliency scores within the regions formed by each Thiessen polygon of the saliency maps at different scales; S702. According to the set globally optimizable parameter, the saliency score, the feature mean and variance, obtain the fusion weight coefficient.

6. The method for generating camo confrontation painting and physical enhancement according to claim 1, characterized in that The step S10 includes: S1001. Obtain the saliency maps of the fused target image with camouflage coating at different scales, and normalize the pixel values in the saliency maps; S1002. According to the preset weights corresponding to different scales, perform weighted fusion on the saliency maps at each scale, and calculate the multi-scale fusion saliency loss.

7. The camouflage confrontation painting generation and physical enhancement method according to claim 1, characterized in that The step S11 includes: S1101. Input the preliminary camouflage coating into the spatial transformation network to predict the affine transformation matrix; S1102. Perform geometric transformation on the preliminary camouflage coating according to the affine transformation matrix.

8. The camouflage confrontation painting generation and physical enhancement method according to claim 1, characterized in that The step S12 includes: S1201. Input the preliminary camouflage coating after affine transformation in the spatial domain into the trained multi-layer perceptron mapping model for color space conversion; S1202. Obtain the brightness difference, contrast difference, and structure difference before and after color space conversion; S1203. Through the set brightness importance index parameter, contrast importance index parameter, and structure importance index parameter, fuse the brightness difference, contrast difference, and structure difference to obtain the structural similarity index; S1204. Calculate the visual consistency loss according to the structural similarity index.

9. A computer device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the camouflage countermeasure coating generation and physical enhancement method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the camouflage countermeasure coating generation and physical enhancement method according to any one of claims 1 to 8.

Images