A visually natural adversarial texture generation method and an adversarial detection method

Abstract

This invention discloses a visually natural adversarial texture generation method and an adversarial detection method, belonging to the field of computer vision and image processing technology, including: acquiring a real-world image and generating its texture tensor T. con Randomly initialize the adversarial texture tensor T adv Obtain the environmental dataset; optimize T according to the following steps. adv (S1) For each sampled image x in the environmental dataset, T is respectively... con and T adv Rendering the mesh tensor M onto the target to be camouflaged yields I. con and I adv and I con and I adv Transform them into the physical scene where x is located to obtain I′ con and I′ adv ; Calculate I′ con and I′ adv Content and style loss between, and I′ adv Perform target detection and calculate the adversarial loss using the true label of x; (S2) with T adv To optimize the objective, the gradient information is obtained by backpropagation after weighted summation of the three loss terms, and T is updated accordingly. adv (S3) Repeat (S1) to (S2) until training is complete. This invention can generate adversarial textures that have the ability to detect attack targets and appear visually natural.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and image processing technology, and more specifically, relates to a method for generating adversarial textures that produce visually natural textures and a method for adversarial detection. Background Technology

[0002] Deep neural networks possess powerful expressive capabilities and have seen rapid development in object detection over the past few years, improving the ability of object detectors to detect images. However, this has also increased the difficulty of object camouflage. An object detector performs object detection on an input image, specifically outputting bounding boxes (BBoxes) representing the location of each object, along with the corresponding object type (Class), and a confidence score for each box. Currently, conventional camouflage techniques utilize deep learning to generate adversarial textures, which are then applied to the surface of the target to evade detector detection.

[0003] Adversarial textures trained using deep learning in different environments have good environmental adaptability. When applied to the target surface, they can effectively camouflage the target in diverse backgrounds. However, these adversarial textures are brightly colored, irregular in texture, and visually unnatural. Although they can deceive the detector, they increase visual attention and still fail to achieve an effective camouflage effect.

[0004] Therefore, how to generate adversarial textures that are visually natural and capable of camouflage in diverse environments for specific camouflage targets is a problem that urgently needs to be solved to achieve effective camouflage. Summary of the Invention

[0005] To address the shortcomings and improvement needs of existing technologies, this invention provides a visually natural adversarial texture generation method and an adversarial detection method. The purpose is to generate visually natural adversarial textures that can attack target detectors for specific targets to be disguised, so that the target has the dual camouflage ability to deceive machine vision and biological vision, thereby protecting the target from malicious attacks or tracking.

[0006] To achieve the above objectives, according to one aspect of the present invention, a visually natural adversarial texture generation method is provided, comprising:

[0007] Initialization steps: Obtain the mesh tensor M of the 3D model of the target to be disguised; obtain the image from the real world and generate its texture tensor, which serves as the content texture tensor T. con Randomly initialize a texture tensor as the adversarial texture tensor T. adv ;

[0008] The steps for constructing the environmental dataset are as follows: a series of sampled images from the real environment acquired under different camera sampling parameters are preprocessed to obtain the environmental dataset; the preprocessing includes: labeling the target bounding box information and target category information in the sampled images as the ground truth labels, and segmenting the target locations in the sampled images to obtain the target mask;

[0009] And training optimization steps, including:

[0010] (S1) For each sampled image x in the environment dataset, convert the content texture tensor T to... con and adversarial texture tensor T adv Rendering to a mesh tensor M yields the target image I. con and target image I adv and the target image I con and target image I adv Each image is transformed to the physical scene containing the sampled image x to generate the content image I′. con and adversarial image I′ adv ; Calculate the content image I′ con and adversarial image I′ adv Content and style loss between images, and adversarial image I′ adv Perform target detection and calculate the adversarial loss using the real label corresponding to the sampled image x;

[0011] (S2) to counter the texture tensor T adv To optimize the objective, the content loss, style loss, and adversarial loss are weighted and summed, and then backpropagated to obtain gradient information. The adversarial texture tensor T is then updated based on the gradient information. adv ;

[0012] (S3) Repeat steps (S1) to (S2) until the preset training termination condition is met, and the current adversarial texture tensor T adv This refers to the generated visually natural anti-texture.

[0013] Furthermore, the formula for calculating the counter-loss is as follows:

[0014]

[0015] Among them, L adv K represents the adversarial loss; K represents the loss used to adversarially attack the image I′. adv The set of target feature layers in a target detector for target detection, where k represents the feature layer in the target detector; D k (Ι′ adv ;θ f ) represents the adversarial image I′ adv The k-th layer features output by the target detector after inputting into it. D represents the feature of the k-th layer. k (Ι′ adv ;θ f The degree of influence of the target detection result output by the target detector is represented by ⊙, which indicates the multiplication of corresponding elements.

[0016] Furthermore,

[0017] Where O represents the target detection result output by the target detector, y represents the real label corresponding to the sampled image x, and P(O,y) represents the result obtained after performing post-processing operations on the target detection result O; the post-processing operations include: non-maximum suppression, and filtering the target objects in the target detection result O according to the real label y.

[0018] Further, in step (S1), the target image I con and target image I adv The generation method is as follows:

[0019] The content texture tensor T con The mesh tensor M, along with the camera sampling parameters e corresponding to the sampled image x, is input to the neural renderer, which then renders the content texture tensor T. con Rendering to a mesh tensor M yields the target image I. con ;

[0020] Adversarial texture tensor T adv The mesh tensor M, along with the camera sampling parameters e corresponding to the sampled image x, is input to the neural renderer, which then converts the adversarial texture tensor T into a texture tensor T. adv Rendering to a mesh tensor M yields the target image I. adv .

[0021] Furthermore, the camera sampling parameters include imaging distance and imaging angle.

[0022] Furthermore, in the environmental dataset construction step, different sampling parameters include:

[0023] Four imaging distances: 10m, 15m, 20m, and 25m;

[0024] Four pitch angles: 22.5°, 45°, 67.5°, 90°; eight yaw angles: 0°, 45°, 90°, 135°, 180°, 225°, 270°, 315°; one roll angle: 0°.

[0025] Further, in step (S1), the target image I... con and target image I adv Each image is transformed to the physical scene containing the sampled image x to generate the content image I′. con and adversarial image I′adv The transformation function Φ on which it is based is as follows:

[0026] Ι′ con =m·I con +(1-m)·x

[0027] Ι′ adv =m·I adv +(1-m)·x

[0028] Where m represents the target mask corresponding to the sampled image x.

[0029] Furthermore, content image I′ con and adversarial image I′ adv The content loss and style loss are calculated as follows:

[0030] Content image I′ con Input a pre-trained feature extraction network F to obtain the activation matrices of F in each feature layer; the activation matrix of any l-th layer is denoted as F. l [Ι′ con ];

[0031] Adversarial image I′ adv Input the feature extraction network F to obtain the activation matrix of the feature extraction network F in each feature layer; the activation matrix of any l-th layer is denoted as F. l [Ι′ adv ];

[0032] Calculate the content image I′ using the following formula. con and adversarial image I′ adv Content loss between L con :

[0033]

[0034] Calculate the content image I′ using the following formula. con and adversarial image I′ adv Style loss between L style :

[0035]

[0036] Where L and H represent the number of feature layers involved in calculating the loss, and N l and N h M represents the number of active channels. l G represents the number of flat activation values ​​in each channel; h [·]=F h [·]F h [·] T Let represent the Gram matrix of the activation matrix of the h-th layer.

[0037] According to another aspect of the present invention, a method for resisting detection is provided, comprising:

[0038] After obtaining the mesh tensor M of the 3D model of the target to be disguised, the visually natural adversarial texture generated by the above-mentioned visually natural adversarial texture generation method provided by the present invention is rendered onto the mesh tensor M.

[0039] According to another aspect of the present invention, a computer-readable storage medium is provided, including a stored computer program; when the computer program is executed by a processor, it controls the device where the computer-readable storage medium is located to execute the above-described visually natural adversarial texture generation method and / or the above-described adversarial detection method provided by the present invention.

[0040] In summary, the above-described technical solutions conceived in this invention can achieve the following beneficial effects:

[0041] (1) This invention utilizes images from the real world to construct a content texture tensor and randomly initializes an adversarial texture tensor. By rendering the content texture tensor and the adversarial texture tensor onto the mesh tensor of the target to be disguised, a target image is obtained that is covered with the content texture tensor and the adversarial texture tensor, respectively. By constructing content loss and style loss between the two target images, the content texture can be used to guide the generation direction of the content and style of the adversarial texture, making the adversarial texture similar to the content texture in terms of content and style, achieving a visually natural effect. At the same time, by constructing the adversarial loss of the target image covered with the adversarial texture tensor, it can be ensured that the generated adversarial texture has the ability to counter target detection. Overall, the adversarial texture constructed by this invention has the ability to attack target detectors and is visually natural, giving the target the dual camouflage ability to deceive machine vision and biological vision, thereby protecting the target from malicious attacks or tracking.

[0042] (2) The present invention will collect images in the real environment under different camera sampling parameters, and before calculating content loss, style loss and adversarial loss, the target image covered with content texture tensor and adversarial texture tensor will be converted to the corresponding physical scene. Since the environmental dataset corresponds to a variety of different imaging distances and imaging angles, the conversion can obtain multi-view and multi-imaging distance images of the target, which is beneficial to improving the robustness and effectiveness of adversarial texture patterns.

[0043] (3) This invention proposes to use gradients to attribute features to the output when calculating adversarial loss. For each feature layer, the corresponding attribution score is calculated to measure the importance of each feature in the target detection. The attribution score of the feature layer is multiplied with the corresponding feature element by element and then summed as the adversarial loss. This calculation method realizes multi-scale feature perception when calculating adversarial loss and destroys the target features, which can effectively improve the ability to detect adversarial targets and transfer capabilities. Attached Figure Description

[0044] Figure 1 A schematic diagram of a visually natural adversarial texture generation method provided in an embodiment of the present invention;

[0045] Figure 2 This is a schematic diagram of an adversarial texture generated based on the visually natural adversarial texture generation method provided by the present invention. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0047] In this invention, the terms "first," "second," etc. (if present) in the invention and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0048] To address the problem that existing camouflage methods often result in targets that are easily detected or appear visually unnatural, this invention provides a visually natural adversarial texture generation and detection method. The overall approach involves constructing a 3D mesh tensor model of the target to be camouflaged, rendering adversarial textures or content textures generated from real-world images onto this 3D mesh tensor model, and imaging from different distances and angles. The rendered target images are then embedded into different backgrounds. A deep neural network is used to train the adversarial textures, enabling them to exhibit attack capabilities against different backgrounds while maintaining content and style similar to pre-selected content textures, thus achieving visual naturalness. This gives the target a dual camouflage capability, deceiving both machine and biological vision, thereby protecting it from malicious attacks or tracking and solving the problem of easily detected or visually unnatural camouflaged targets in existing technologies.

[0049] It's easy to understand that, in order for the generated adversarial textures to appear visually natural, the pre-selected real-world images should also be visually natural and consistent with the target being disguised (i.e., semantically related to the target object within its context), such as Pikachu patterns. The natural texture patterns used should have a decorative effect, serving as the direction for generating the content and style of the adversarial textures. The trained pattern textures, when sprayed onto the target surface, can achieve a visually natural effect and are less likely to attract the attention of the reconnaissance party.

[0050] Without loss of generality, in the following embodiments, vehicles are used as the targets to be disguised. The following are examples.

[0051] Example 1:

[0052] A visually natural adversarial texture generation method, such as Figure 1 As shown, it includes: initialization steps, environment dataset construction steps, and training optimization steps.

[0053] In this embodiment, the initialization step is mainly used to initialize the adversarial texture tensor, which is the optimization target, and to determine the generation direction of its content and style. The initialization step specifically includes:

[0054] Obtain the mesh tensor M of the 3D model of the target to be disguised; obtain the image from the real world and generate its texture tensor, which serves as the content texture tensor T. con Randomly initialize a texture tensor as the adversarial texture tensor T. adv .

[0055] Optionally, in the initialization step of this embodiment, a 3D model (M, T) of the target is constructed using modeling software (such as 3ds Max), where M represents the mesh tensor of the 3D model and T represents the texture tensor of the 3D model. Simultaneously, the area that the 3D model is allowed to render, i.e., the area where adversarial textures can be sprayed, is set so that content textures or adversarial textures can be rendered to appropriate locations on the target surface subsequently.

[0056] Optionally, in this embodiment, a Pikachu pattern is selected to generate the content texture tensor T. con The content texture tensor T con Used to guide the generation direction of content and style of the adversarial texture tensor to ensure the visual naturalness of the adversarial texture tensor;

[0057] In this embodiment, the adversarial texture tensor is randomly initialized as the target to be optimized; optionally, in this embodiment, a normal distribution N(0,1) is specifically selected to randomly initialize the adversarial texture tensor.

[0058] In this embodiment, the environmental dataset construction step is mainly used to obtain multi-view, multi-imaging distance images of the target, in order to improve the robustness and effectiveness against texture patterns. The environmental dataset construction step specifically includes:

[0059] A series of sampled images acquired from the real environment under different camera sampling parameters are preprocessed to obtain an environmental dataset. The preprocessing includes: labeling the target bounding box information and target category information in the sampled images as ground truth labels, and segmenting the target positions in the sampled images to obtain target masks.

[0060] Optionally, in the environmental dataset construction step of this embodiment, a realistic simulation environment can be used to simulate the real environment. Specifically, this realistic simulation environment is implemented using the CARLA simulation environment. The camera sampling parameters mainly include imaging distance and imaging angle. Four sampling distance settings are provided (10m, 15m, 20m, and 25m). The target's angle is fixed at a single value. Then, the camera angles are set: four pitch angles (22.5°, 45°, 67.5°, and 90°), eight yaw angles (0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315°), and one roll angle (0°). By setting different sampling distance and angle information, multi-view and multi-imaging distance images of the target can be obtained, which helps improve the robustness and effectiveness against texture patterns.

[0061] Optionally, the sampled image is input into a pre-trained semantic segmentation network to segment the target from the sampled image. After binarizing the output of the semantic segmentation network, the target mask of the sampled image can be obtained. In the target mask, pixels with a value of 1 represent the target and pixels with a value of 0 represent the background.

[0062] For each sampled image x, its corresponding camera sampling parameters e, ground truth label y, and target mask m are taken as a set of environmental data. All environmental data constitute the environmental dataset. In the environmental dataset, the set of all sampled images is denoted as X, the set of all camera sampling parameter sets is denoted as E, and the set of all ground truth label sets is denoted as Y.

[0063] In this embodiment, the training optimization step utilizes the constructed environment dataset to optimize and update the adversarial texture tensor, gradually bringing it closer to the content texture tensor in terms of content and style, thus ensuring its visual naturalness and enabling adversarial object detection under different backgrounds. The training optimization step specifically includes:

[0064] (S1) For each sampled image x in the environment dataset, convert the content texture tensor T to... con and adversarial texture tensor T adv Rendering to a mesh tensor M yields the target image I.con and target image I adv and the target image I con and target image I adv Each image is transformed to the physical scene containing the sampled image x to generate the content image I′. con and adversarial image I′ adv ; Calculate the content image I′ con and adversarial image I′ adv Content and style loss between images, and adversarial image I′ adv Perform target detection and calculate the adversarial loss using the real label corresponding to the sampled image x;

[0065] Optionally, in this embodiment, a neural renderer is used to render the texture tensor, specifically a Neural 3DMesh Renderer, which can approximate the gradient of the rasterization process and thus apply it to the backpropagation process of the neural network. During the rendering process, this neural renderer simultaneously takes the texture tensor, the mesh tensor, and the corresponding camera sampling parameters as input to render the target image I. con and target image I adv These are target images covered with content texture tensors and target images covered with adversarial texture tensors, respectively.

[0066] In this embodiment, the imaging distance and angle of the target in the sampled image x are consistent with the imaging distance and angle of the rendered target. Accordingly, the target image I is... con and target image I adv The transformations are respectively applied to the physical scene where the sampled image x is located. The specific transformation function Φ is as follows:

[0067] Ι′ con =Φ(I con )=m·I con +(1-m)·x

[0068] Ι′ adv =Φ(I adv )=m·I adv +(1-m)·x

[0069] In the formula, m·I con Used to extract target image I based on target mask m con The target image I is (1-m)·x, which is used to extract the background from the sampled image x. The two are added together to make the target image I. con The target in the sampled image x is embedded into the background, thus completing the embedding of the target image I. con Transform to the physical scene where the sampled image x is located; target image I adv The conversion process is similar and will not be repeated here;

[0070] (S2) to counter the texture tensor T adv To optimize the objective, the content loss, style loss, and adversarial loss are weighted and summed, and then backpropagated to obtain gradient information. The adversarial texture tensor T is then updated based on the gradient information. adv ;

[0071] With L con L style and L adv Let L represent content loss, style loss, and adversarial loss, respectively. Then, the total loss obtained by weighted summation is L. total It can be represented as:

[0072] L total =L adv +λL con +βL style

[0073] Wherein, λ and β are weighting coefficients; the selection of λ and β reflects the degree of influence of content loss and style loss on adversarial texture during backpropagation. The higher λ and β are, the closer the content and style of the generated adversarial texture are to the selected content texture pattern (such as the Pikachu pattern). However, λ and β should not be set too high, otherwise it may affect the ability to detect adversarial targets. In this embodiment, after comprehensively considering visual naturalness and the ability to detect adversarial targets, λ is set to 0.005 and β is set to 0.0001.

[0074] Optionally, in this embodiment, after obtaining the gradient information through backpropagation, the optimizer is specifically used to update the adversarial texture tensor T. adv The optimizer specifically uses the Adam optimizer with an initial learning rate of 0.01;

[0075] (S3) Repeat steps (S1) to (S2) until the preset training termination condition is met, and the current adversarial texture tensor T adv This refers to the generated visually natural contrast texture;

[0076] During the repeated execution of steps (S1) to (S2), the adversarial texture tensor will gradually approach the content texture tensor in terms of content and style, and its ability to resist object detection will also be gradually enhanced.

[0077] Normally, a specified number of iterations can be set as the training termination condition. That is, the training ends after the number of repetitions of steps (S1) to (S2) reaches the specified number of iterations.

[0078] The calculation of adversarial loss depends on the adversarial image I′ advConsidering the widespread use of multi-scale features in mainstream object detectors, disrupting these features can improve adversarial target detection capabilities. Furthermore, changes in the object detector can alter the detection results and consequently, the adversarial loss. Therefore, to enhance adversarial target detection capabilities and transferability, this embodiment proposes a multi-scale feature attack loss calculation method. This method utilizes gradients to attribute features to the output, calculating an attribution score for each feature layer to measure the importance of each feature in the target detection. Intuitively, for the attribution score, important features will produce a relatively large intensity response, indicating the effort of the corrected features to approach the true label, while the sign provides the direction of correction. In the object detection task, positive features will be corrected in the positive direction, and negative features will be corrected in the negative direction, thus achieving correct target detection. The correction direction for generating transferable adversarial examples is exactly opposite to the correction direction of the object detection task. Therefore, the objective function of the adversarial attack should be designed to manipulate features in the opposite direction to the correction direction of the object detection task. Specifically, in this embodiment, the formula for calculating the adversarial loss is as follows:

[0079]

[0080] Among them, L adv K represents the adversarial loss; K represents the loss used to adversarially attack the image I′. adv The set of target feature layers in a target detector for target detection, where k represents the feature layer in the target detector; D represents the target detector, D k (Ι′ adv ;θ f ) represents the adversarial image I′ adv The k-th layer features output by the target detector after inputting into it. D represents the feature of the k-th layer. k (Ι′ adv ;θ f The degree of influence of the target detection result output by the target detector is represented by ⊙, which indicates element-wise multiplication. Optionally, in this embodiment, the target detector D adopts YOLOv3, and the parameter θ f This was obtained by training YOLOv3 on the COCO dataset.

[0081] In this embodiment, for any k-th layer feature in the target detector, the attribution score is used to represent the degree of influence of that feature on the output of the target detector. The calculation formula is as follows:

[0082]

[0083] Where O represents the target detection result output by the target detector, y represents the true label corresponding to the sampled image x, and P(O,y) represents the result obtained after post-processing the target detection result O. The post-processing operations include: non-maximum suppression and filtering the target objects in the target detection result O according to the true label y. The non-maximum suppression operation in the post-processing can eliminate redundancy and reduce the amount of computation. Filtering the target objects in the target detection result O according to the true label y can improve the accuracy of the calculation and further reduce the amount of computation.

[0084] In this embodiment, the content loss L con Specifically, it calculates the content image I′ con and adversarial image I′ adv The difference in content (in feature representation) causes the adversarial texture to approximate the content texture during training. Considering the computational cost, this embodiment first processes the content image I′. con and adversarial image I′ adv The input feature extraction network F, specifically employing the VGG16 architecture, is used. Then, the mean squared error (MSE) of the activation matrices of the content image and the adversarial image at the feature layer is calculated; the style loss L... style Specifically, the mean squared error loss of the Gram matrix at the feature layer is calculated for both the content image and the adversarial image. In summary, in this implementation, the content image I′ con and adversarial image I′ adv The content loss and style loss are calculated as follows:

[0085] Content image I′ con Input a pre-trained feature extraction network F to obtain the activation matrices of F in each feature layer; the activation matrix of any l-th layer is denoted as F. l [Ι′ con ];

[0086] Adversarial image I′ adv Input the feature extraction network F to obtain the activation matrix of the feature extraction network F in each feature layer; the activation matrix of any l-th layer is denoted as F. l [Ι′ adv ];

[0087] Calculate the content image I′ using the following formula. con and adversarial image I′ adv Content loss between L con :

[0088]

[0089] Calculate the content image I′ using the following formula. con and adversarial image I′adv Style loss between L style :

[0090]

[0091] Where L and H represent the number of feature layers involved in calculating the loss, and N l and N h M represents the number of active channels. l G represents the number of flat activation values ​​in each channel; h [·]=F h [·]F h [·] T Let H represent the Gram matrix of the activation matrix of layer h. Specifically, when calculating the style loss, H is chosen to be 4, representing the ReLU1_2, ReLU2_2, ReLU3_3, and ReLU4_3 layers of the VGG16 architecture's feature modules, respectively; when calculating the content loss, L is chosen to be 1, representing the ReLU2_2 layer of the VGG16 architecture's feature modules. The difference between the content texture and the adversarial texture at the feature layers is calculated using the mean squared error loss.

[0092] In summary, this embodiment utilizes pre-selected content textures to guide the generation direction of adversarial textures, making the adversarial textures similar to the content textures in content and style, achieving a visually natural effect and possessing the ability to attack target detectors. A neural renderer is used to render the content textures and adversarial textures onto a 3D target model, and then, based on the target mask m, the rendered target is embedded into various backgrounds to obtain the content image and the adversarial image. The adversarial image is then input into the target detector to calculate the multi-scale feature attack loss L. adv This approach enhances adversarial capabilities and transferability by targeting feature differences; it inputs content images and adversarial images into a feature extraction network to calculate the content loss L. con and style loss L style The calculated loss is weighted and summed, then backpropagated to obtain gradient information. The adversarial texture is updated based on this gradient information. Through multiple iterations of training, a visually natural and offensive adversarial texture can be obtained. By fully considering the target's shape and imaging distance and angle through 3D modeling, the adversarial texture generated in this embodiment can effectively protect the target from detection.

[0093] Figure 2 The images shown depict adversarial textures generated using the method provided in this embodiment. Three images illustrate the adversarial textures from top, front, and side views, respectively. The adversarial textures primarily feature selectable Pikachu patterns and maintain a consistent style. Overall, the adversarial textures generated in this embodiment not only possess the capabilities of an attack detector but also appear visually natural.

[0094] Example 2:

[0095] One method for counter-detection includes:

[0096] After obtaining the mesh tensor M of the 3D model of the target to be disguised, the visually natural adversarial texture generated by the visually natural adversarial texture generation method provided in Embodiment 1 above is rendered onto the mesh tensor M.

[0097] Example 3:

[0098] A computer-readable storage medium includes a stored computer program; when the computer program is executed by a processor, it controls the device where the computer-readable storage medium is located to execute the visually natural adversarial texture generation method provided in Embodiment 1 above, and / or the adversarial detection method provided in Embodiment 2 above.

[0099] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for generating visually natural adversarial textures, characterized in that, include: Initialization steps: Obtain the mesh tensor M of the 3D model of the target to be camouflaged; Obtain an image from the real world and generate its texture tensor, which is then used as the content texture tensor T. con Randomly initialize a texture tensor as the adversarial texture tensor T. adv ; The steps for constructing the environmental dataset are as follows: a series of sampled images acquired from the real environment under different camera sampling parameters are preprocessed to obtain the environmental dataset; the preprocessing includes: labeling the target bounding box information and target category information in the sampled images as real labels, and segmenting the target positions in the sampled images to obtain the target mask; And training optimization steps, including: (S1) For each sampled image x in the environmental dataset, the content texture tensor T is respectively... con and the adversarial texture tensor T adv Rendering to the mesh tensor M yields the target image I. con and target image I adv and the target image I con and the target image I adv The samples are respectively transformed to the physical scene where the sampled image x is located to generate content image I′. con and adversarial image I′ adv ; Calculate the content image I′ con and adversarial image I′ adv Content and style loss between the two, and the adversarial image I′ adv Perform target detection and calculate the adversarial loss using the real label corresponding to the sampled image x; (S2) with the adversarial texture tensor T adv To optimize the objective, the content loss, style loss, and adversarial loss are weighted and summed, then backpropagated to obtain gradient information. The adversarial texture tensor T is then updated based on this gradient information. adv ; (S3) Repeat steps (S1) to (S2) until the preset training termination condition is met, and the current adversarial texture tensor T adv This refers to the generated visually natural anti-texture.

2. The visually natural adversarial texture generation method as described in claim 1, characterized in that, The formula for calculating the adversarial loss is as follows: Among them, L adv K represents the adversarial loss; K represents the loss used to adversarially attack the image I′. adv The set of target feature layers in the target detector used for target detection, where k represents the feature layer in the target detector; D k (Ι′ adv ;θ f ) represents the adversarial image I′ adv The k-th layer features output by the target detector after input. D represents the feature of the k-th layer. k (Ι′ adv ;θ f The degree of influence of the target detection result output by the target detector is represented by ⊙, which indicates the multiplication of corresponding elements.

3. The visually natural adversarial texture generation method as described in claim 2, characterized in that, Where O represents the target detection result output by the target detector, y represents the real label corresponding to the sampled image x, and P(O,y) represents the result obtained after performing post-processing operations on the target detection result O; the post-processing operations include: non-maximum suppression, and filtering the target objects in the target detection result O according to the real label y.

4. The visually natural adversarial texture generation method as described in any one of claims 1 to 3, characterized in that, In step (S1), the target image I con and target image I adv The generation method is as follows: The content texture tensor T con The mesh tensor M, along with the camera sampling parameters e corresponding to the sampled image x, is input to the neural renderer, which then renders the content texture tensor T. con Rendering to the mesh tensor M yields the target image I. con ; The adversarial texture tensor T adv The mesh tensor M, along with the camera sampling parameters e corresponding to the sampled image x, are input to the neural renderer, which then processes the adversarial texture tensor T. adv Rendering to the mesh tensor M yields the target image I. adv .

5. The visually natural adversarial texture generation method as described in any one of claims 1 to 3, characterized in that, The camera sampling parameters include imaging distance and imaging angle.

6. The visually natural adversarial texture generation method as described in claim 5, characterized in that, In the environmental dataset construction step, different sampling parameters include: Four imaging distances: 10m, 15m, 20m, and 25m; Four pitch angles: 22.5°, 45°, 67.5°, 90°; eight yaw angles: 0°, 45°, 90°, 135°, 180°, 225°, 270°, 315°; one roll angle: 0°.

7. The visually natural adversarial texture generation method as described in any one of claims 1 to 3, characterized in that, In step (S1), the target image I is... con and the target image I adv The samples are respectively transformed to the physical scene where the sampled image x is located to generate content image I′. con and adversarial image I′ adv The transformation function Φ on which it is based is as follows: Ι′ con =m·I con +(1-m)·x Ι′ adv =m·I adv +(1-m)·x Where m represents the target mask corresponding to the sampled image x.

8. The visually natural adversarial texture generation method as described in any one of claims 1 to 3, characterized in that, The content image I′ con and adversarial image I′ adv The content loss and style loss are calculated as follows: The content image I′ con Input the trained feature extraction network F to obtain the activation matrix of the feature extraction network F in each feature layer; The activation matrix of any l-th layer is denoted as F. l [Ι′ con ]; The adversarial image I′ adv Input the feature extraction network F to obtain the activation matrix of the feature extraction network F in each feature layer; The activation matrix of any l-th layer is denoted as F. l [Ι′ adv ]; The content image I′ is calculated according to the following formula. con and adversarial image I′ adv Content loss between L con : The content image I′ is calculated according to the following formula. con and adversarial image I′ adv Style loss between L style : Where L and H represent the number of feature layers involved in calculating the loss, and N l and N h M represents the number of active channels. l G represents the number of flat activation values ​​in each channel; h [·]=F h [·]F h [·] T Let represent the Gram matrix of the activation matrix of the h-th layer.

9. A method for counter-detection, characterized in that, include: After obtaining the mesh tensor M of the 3D model of the target to be disguised, the visually natural adversarial texture generated by the visually natural adversarial texture generation method according to any one of claims 1 to 8 is rendered onto the mesh tensor M.

10. A computer-readable storage medium, characterized in that, Includes a stored computer program; when executed by a processor, the computer program controls the device containing the computer-readable storage medium to perform the visually natural adversarial texture generation method according to any one of claims 1 to 8, and / or the adversarial detection method according to claim 9.

Images