A full-view-angle adversarial camouflage generation method and device for image classification

Abstract

The application provides a full-view-angle countermeasure camouflage generation method and device for image classification, which can generate full-view-angle countermeasure camouflage and can deceive a classification model at any view angle, and comprises the following steps: step 1: obtaining a three-dimensional model file of a target object and corresponding texture; step 2: performing three-dimensional rendering on the target object to be camouflaged according to the corresponding texture, and obtaining target object images at different view angles, distances, brightnesses, backgrounds and environmental noises in real time to form training samples; and step 3: integrating an attack source classification model according to the training samples to update the countermeasure camouflage texture and obtain three-dimensional full-view-angle countermeasure camouflage. Experimental results show that the integrated countermeasure camouflage generated by the method can deceive an intelligent image classifier at full view angles, and the target attack success rate in a three-dimensional simulation scene is greatly improved compared with a multi-patch splicing texture.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence computer vision confrontation, and proposes a full-view robust adversarial camouflage generation method for image classification. BACKGROUND

[0002] Deep neural networks have achieved advanced levels in many computer vision tasks and have been deployed in real-world applications such as autonomous driving systems. However, research has shown that adding adversarial noise to input samples that does not hinder human recognition can cause intelligent recognition models to malfunction in unexpected ways. Such carefully perturbed samples are called adversarial samples, and the potential dangers they pose have made them a current research hotspot.

[0003] Adversarial sample attacks can be divided into digital domain attacks and physical domain attacks. Digital domain attacks usually use imperceptible adversarial noise to deceive deep models, but are easily destroyed by environmental noise and other factors, making it difficult for their performance to be effectively used in the physical domain. Physical domain attacks use visual noise that can maintain a certain degree of robustness in changing environments, and related research on physical domain adversarial samples for image classification models has made some very meaningful achievements, such as local adversarial patches, stop sign attacks, and small adversarial objects such as turtles. However, these works still have some shortcomings:

[0004] 1) Attacks based on local adversarial patches, since the adversarial patches are generated in two-dimensional space, i.e. the patches do not undergo corresponding spatial transformations with the objects in the image during the training process, but only perform two-dimensional image transformations on the image, resulting in a sharp decline in attack effectiveness when the adversarial patches change their poses and viewing angles in three-dimensional space.

[0005] 2) Adversarial noise is only used for planar objects such as stop signs, and attacks on objects with arbitrary viewing angles are easily ineffective. SUMMARY

[0006] To improve the shortcomings of current work, the present application proposes a full-view adversarial camouflage generation method and device for image classification, which can generate full-view adversarial camouflage and deceive classification models at any viewing angle.

[0007] To achieve the above purpose, the present application adopts the following technical solutions:

[0008] In a first aspect, the present application provides a full-view adversarial camouflage generation method for image classification, comprising the following steps:

[0009] Step 1: Obtain a three-dimensional model file of a target object and the corresponding texture;

[0010] Step 2: according to the corresponding texture, the target object to be camouflaged is rendered in three dimensions, and real-time target object images of different perspectives, distances, brightness, backgrounds and environmental noise are obtained to form training samples;

[0011] Step 3: according to the training samples, an attack source classification model is integrated to update the adversarial camouflage texture, and a three-dimensional full-view adversarial camouflage is obtained.

[0012] Further, the picture of the object to be camouflaged is rendered in three dimensions according to the corresponding texture, including the following steps:

[0013] The target object three-dimensional model file and the corresponding texture picture file are taken as input, the texture is mapped to the rendering picture of the target object, and according to the given three-dimensional environmental condition parameter distribution, geometric transformation and environmental transformation are introduced to approximately simulate the distribution of the real environment, and different training samples are obtained in real time to provide diversified data for training the generated adversarial texture.

[0014] Further, the geometric transformation includes rotation, translation and distance change.

[0015] The environmental transformation includes brightness, noise and background change.

[0016] Further, the environmental parameter distribution is shown in Table 1:

[0017] Table 1 Physical domain environmental condition parameter distribution

[0018]

[0019] Further, the target object three-dimensional model file and the corresponding texture picture file are taken as input, the texture is mapped to the rendering picture of the target object, and according to the given three-dimensional environmental condition parameter distribution, geometric transformation and environmental transformation are introduced to approximately simulate the distribution of the real environment, and different training samples are obtained in real time, including:

[0020] 3D target object m covers texture tensor c, denoted as (m, c);

[0021] Let e represent the environmental condition parameter, E be the environmental condition parameter distribution, including perspective, camera distance, light, noise, background change, and the parameters are all independent continuous random variables, conforming to uniform distribution;

[0022] The real scene light change and printing error are modeled and approximated by adding and multiplying the brightness and channel pixels; under the environmental condition e, the image rendering operation The sample x is obtained as:

[0023] x = ((m, c), e) (1) By adding adversarial noise to the texture c, the adversarial camouflage texture c is obtained adv, the generated adversarial sample is:

[0024]

[0025] where (m, c adv ) is the obtained 3D adversarial target object.

[0026] Further, according to the training sample, an attack source classification model is integrated to update the adversarial camouflage texture, and a three-dimensional full-view angle adversarial camouflage is obtained, including:

[0027] determining a 3D target object (m, c), an environmental condition parameter e E, a neural renderer target class label y / , K classification models f1, f2…, f K , the corresponding logits values are l1, l2…, l K , the integrated weights are w1, w2…, w K , and the maximum iteration step N.

[0028] assigning the texture tensor c of the input 3D object to the initial adversarial camouflage texture

[0029] performing a loop of t from 0 to N-1:

[0030] generating training data:

[0031] from the input calculate the logits values of the K models and integrate them:

[0032]

[0033] update the camouflage texture by attacking all classifications

[0034]

[0035]

[0036] where the cross-entropy loss function J(l(x adv ), y * ) can be represented as where is the one-hot encoding of y * ; X is an image training set of arbitrary view and changing environmental conditions generated by the renderer in real time, E is the environmental condition distribution simulated by the renderer, and the "true" input x adv obtained by the classifier f during the training process is the input target object (m, c adv); the finally generated camouflage pixel value is limited in the range of valid image pixels [0, 1];

[0037] After the loop ends, the adversarial camouflage texture of the 3D target object is obtained through iterative updating

[0038] Further, in the training process, the adversarial texture c is not constrained adv The distance between the original texture c.

[0039] Further, the method further comprises:

[0040] A virtual scene is established by using a three-dimensional engine to perform a changing scene simulation experiment, a Clarifai commercial image classification system is attacked for testing, and the attack performance of the 3D printed adversarial object in the real world is verified and evaluated.

[0041] In a second aspect, the present application provides a full-view adversarial camouflage generation device for image classification, comprising:

[0042] An input module is used to acquire a three-dimensional model file of a target object and a corresponding texture;

[0043] A generation module is used to perform three-dimensional rendering on the target object to be camouflaged according to the corresponding texture, to obtain target object images in different view angles, distances, brightnesses, backgrounds and environmental noises in real time, and to form training samples;

[0044] A camouflage updating module is used to update the adversarial camouflage texture by integrating an attack source classification model according to the training samples, to obtain a three-dimensional full-view adversarial camouflage.

[0045] In a third aspect, the present application provides a full-view adversarial camouflage generation device for image classification, comprising a processor and a storage medium;

[0046] The storage medium is used to store instructions;

[0047] The processor is used to operate according to the instructions to perform the steps of the method according to the first aspect.

[0048] Compared with the prior art, the present application has the following beneficial effects:

[0049] 1) The present method can generate a three-dimensional full-view integrated adversarial camouflage to deceive an image classifier, and the attack performance is significantly better than that of a multi-adversarial patch splicing texture;

[0050] 2) The present method can be used for adversarial camouflage texture generation of any object, and the result can be 3D printed or the adversarial camouflage texture can be pasted on a target object, which shows good generalization in the real world.

[0051] 3) The one-piece adversarial camouflage generated by this method can deceive intelligent image classifiers in full view, and the average target attack success rate on three-dimensional simulation scenes is 25% higher than that of multi-patch spliced texture, the target attack success rate on Clarifai commercial classification system black box is 42%, and the average attack success rate in the real world is about 66% in 3D printing model experiments, which shows advanced attack performance. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 is the process of generating adversarial camouflage.

[0053] Figure 2 is a car with different textures;

[0054] Figure 3 is a Unity city scene;

[0055] Figure 4 is a free-view schematic diagram;

[0056] Figure 5 is a car with different textures in a bright scene under multiple perspectives;

[0057] Figure 6 is a car with different textures in a dark scene under multiple perspectives;

[0058] Figure 7 is another category with tractor features;

[0059] Figure 8 is an example of different occlusion results of adversarial texture cars;

[0060] Figure 9 is the Clarifai classification result of the adversarial vehicle in a bright environment;

[0061] Figure 10 is the Clarifai classification result of the adversarial vehicle in a dark environment;

[0062] Figure 11 is an example of model classification experiment results of adversarial camouflage cars;

[0063] Figure 12 is an example of adversarial camouflage attack results on other target objects. DETAILED DESCRIPTION

[0064] The application will be further described below in conjunction with the drawings. The following examples are only used to more clearly illustrate the technical solutions of the application, and cannot be used to limit the protection scope of the application.

[0065] Example 1:

[0066] The embodiment proposes an all-view angle adversarial camouflage generation (Adv-Camou) attack method for image classification. Firstly, a spatial combination distribution transformation is used to render the target object geometry and environmental changes in real time, eliminate the adversarial view angle blind spot, and minimize the prediction class and target class cross-entropy loss, so that the model output specifies the wrong class, and an all-view angle adversarial camouflage is obtained. In addition, a virtual scene is established by using a three-dimensional engine to perform change scene simulation experiments, and an attack test is performed on the Clarifai commercial image classification system, and further, the attack performance of the 3D printed adversarial object in the real world is verified and evaluated.

[0067] Overall, the main contributions of the present application are:

[0068] 1) The Adv-Camou can generate a three-dimensional all-view integrated adversarial camouflage to deceive the image classifier, and the attack performance is significantly better than that of the multi-adversarial patch splicing texture;

[0069] 2) Adv-Camou can be used for adversarial camouflage texture generation of any object, and the result can be 3D printed or the adversarial camouflage texture can be pasted on the target object, which shows good generalization in the real world.

[0070] White box and black box attacks

[0071] According to the amount of available target model knowledge, the adversarial sample attack can be divided into white box and black box attacks. The white box attack method assumes that the attacker can access the internal structure and parameters of the model, and can obtain the adversarial sample by updating the adversarial noise through gradient backpropagation. However, in practice, the attacker usually cannot access the internal structure and parameters of the target model, which belongs to a black box model, so a black box method needs to be used for attack. There are mainly two kinds of black box attack methods, which are based on query and based on migration. The query-based black box attack method requires a large number of queries, and the calculation cost is high, and in reality, the deep learning system is difficult to meet the condition of free query, which objectively causes the low feasibility of the query-based method. Research shows that adversarial samples have cross-model migration, that is, the adversarial samples generated by attacking a model in a white box way can deceive another black box model of the same task, which provides an important way for realizing black box attack.

[0072] Digital domain adversarial samples

[0073] Digital domain adversarial samples are usually obtained by adding imperceptible adversarial noise to the original input image. Since Szegedy et al. first discovered adversarial samples, a variety of image adversarial sample methods have been proposed, such as the fast gradient method, which generates a large number of adversarial samples at very low computational cost by calculating a single gradient, but shows a lower attack success rate. Kurakin et al. extended the single-step fast gradient method to an iterative method, which significantly improved the white-box attack success rate, but was prone to overfitting to the parameters of the white-box model being attacked, showing weak cross-model migration attack. A series of enhancement methods were proposed to alleviate the overfitting problem, avoid falling into local extrema, and improve the migration of adversarial samples, such as introducing momentum factors, multiple input enhancement adversarial attacks, translation-independent method to enhance adversarial noise, and feature-level perturbation of the network. However, since the adversarial noise of digital domain adversarial samples is weak, it is easily destroyed by changes such as viewing angle, environmental noise, and brightness in the physical domain, resulting in attack failure.

[0074] Physical domain adversarial samples

[0075] Physical domain adversarial samples usually add strong adversarial noise to the target object, but the noise is also more noticeable to the human eye. Such noise is called visible adversarial noise. Eykholt et al. found that pasting stickers at different positions of a stop sign would have different attack effects on the recognition task of traffic signs. By applying sticker attacks to sensitive and vulnerable areas, they successfully caused traffic signs to be misclassified. Brown et al. generated scene-independent local adversarial patches that could cause a classification model to output a specified class label at any position in a two-dimensional image. However, since the adversarial patches were trained and generated in two-dimensional space, the adversarial patches did not undergo corresponding spatial transformations with the objects in the image during the training process, resulting in a sharp decline in patch attack effectiveness as the viewing angle increased when tested in three-dimensional space. Wang et al. generated patches that simultaneously suppressed the attention of the model and the human eye. Athalye et al. successfully extended two-dimensional planar adversarial samples to three-dimensional space and generated adversarial turtles and baseballs. However, the generated adversarial objects could only deceive image classifiers at a small camera distance, making it difficult to maintain robust attackability.

[0076] Image classification adversarial camouflage generation method

[0077] The Adv-Camou adversarial camouflage generation process mainly includes real-time generation of training samples and generation and updating of adversarial camouflage textures. First, the adversarial camouflage generation process framework is introduced, and then the three-dimensional environmental condition parameters and the camouflage generation process are described in detail.

[0078] Attack process

[0079] There are many challenges in generating robust physical domain adversarial samples. First, the adversarial noise of the physical domain adversarial sample will be lost in the production process (such as printing). Second, the adversarial noise is easily destroyed by environmental noise and light. Third, there is information loss when the adversarial noise is collected by the camera, because the camera cannot perfectly capture the image. Fourth, the angle and distance of the camera also affect the effect of adversarial attack.

[0080] In order to make the generated adversarial samples robust and adversarial in complex physical domains, a series of image transformations such as rotation, translation and distance changes are introduced in the training process. Geometric transformations and brightness, noise and background changes and other environmental transformations are used to approximate the distribution of the real environment.

[0081] Since autonomous driving is a safety-critical research hotspot, vehicles are the focus of attention, so cars are selected as experimental target objects, such as Figure 1 As shown in the figure, the adversarial camouflage process mainly includes two steps:

[0082] (1) The first step is to generate training samples in real time. The three-dimensional model file and the corresponding texture picture file of the target object are taken as input, the texture is mapped to the rendering image of the target object, and a series of image transformations such as rotation, translation and distance changes are introduced. Geometric transformations and brightness, noise and background changes and other environmental transformations are used to approximate the distribution of the real environment, so as to obtain training samples with different angles, brightness and different environmental backgrounds in real time, and provide diversified data for training adversarial textures.

[0083] (2) The second step is to attack the classification model (source model) to update the adversarial camouflage texture. The classification model makes a class prediction on the camouflage texture car in the adversarial texture training process, updates the adversarial noise by minimizing the cross-entropy loss of the predicted class and the specified target class, reduces the inter-class distance between the target vehicle and the specified target class (such as tractor), and makes the model identify the target vehicle as the specified wrong class. The adversarial camouflage changes the space transformation with the vehicle posture in the training process, eliminating the adversarial angle blind spot. The adversarial camouflage after updating the adversarial noise is input as a new iteration step until the preset iteration step is reached, and the three-dimensional full-view adversarial camouflage that can maintain robust adversarial in the physical domain is output.

[0084] Three-dimensional rendering

[0085] A 3D object is generated by a mesh tensor m covering a texture tensor c, denoted as (m, c). Let e represent the environmental condition parameter, and E be the environmental condition parameter distribution, including view angle, camera distance, light, noise, background change, etc. The parameter distribution is shown in Table 1, all of which are independent continuous random variables, conforming to uniform distribution (except Gaussian noise). The brightness and channel pixel addition and multiplication are used to model and approximate the real scene light changes and printing errors. Under the environmental condition e, the image rendering operation gets the sample x as:

[0086] x = ((m, c), e) (1)

[0087] By adding adversarial noise to the texture c, an adversarial camouflage texture c adv is obtained, and the generated adversarial sample is:

[0088]

[0089] where (m, c adv ) is the obtained 3D adversarial object.

[0090] Table 1 Physical domain environmental condition parameter distribution

[0091]

[0092]

[0093] Adversarial camouflage texture generation

[0094] For a clean input image x, it can be correctly classified as the real label y by the image classification model f. After adding adversarial noise to x, the adversarial sample x adv is obtained. The adversarial sample can be divided into untargeted adversarial sample and targeted adversarial sample according to the attacker's expected output. The untargeted adversarial sample satisfies f(x adv ) ≠ y, that is, the adversarial sample is not classified as the label y, which means that the attack is successful. The targeted adversarial sample satisfies f(x adv ) = y * and y * ≠ y, that is, it requires the adversarial sample to be misclassified as the specified target class y * .

[0095] Since the classes of cars in ImageNet dataset are too fine-grained, i.e., the distance between different classes of cars is small, a car is easily misclassified as another car, so the untargeted attack without specifying the wrong classification is relatively uninteresting. In addition, there is a domain difference between the real picture data used to train the classifier and the three-dimensional simulation data, which may lead to random misclassification. The targeted attack can exclude the influence of this factor on the results, so this paper studies the more challenging targeted attack, which makes the target car misclassified as a specified target class with a large difference from the original class, such as "tractor" and "traffic light".

[0096] Studies have shown that the integrated model attack method can obtain stronger adversarial samples than single model attack, and has stronger cross-model black-box migration attack, because if adversarial samples can deceive multiple models, they are more likely to successfully attack other unknown models, and it is found that logits integration is better than loss integration and softmax integration scheme. K classification model logits integration can be expressed as:

[0097]

[0098] In the formula, l k (x adv ) is the logits output of the kth model, w k is the weight coefficient of the kth model, w k ≥ 0 and

[0099] For targeted attack, the cross-entropy loss function J(l(x * ), y adv ) between the predicted value and the target class label y * needs to be minimized. Unlike two-dimensional objects, human eyes recognize three-dimensional objects mainly based on the shape contour of the object, and texture hardly affects the recognition result of human eyes. In view of this, the experiment does not constrain the distance between the adversarial texture c adv and the original texture c. Combined with formula (2), the adversarial camouflage texture c adv is updated by minimizing the cross-entropy loss function. The attack optimization problem for K model integration can be expressed as:

[0100]

[0101] The cross-entropy loss function J(l(x adv ), y * ) can be expressed as where y * is the one-hot encoding of y adv .

[0102] To generate robust 3D adversarial camouflage, the overall loss under arbitrary angles, varying distances, and other environmental conditions needs to be optimized, i.e., to train the following objective function:

[0103]

[0104] where X is the training set of images with arbitrary viewing angles and varying environmental conditions generated by the renderer in real time, E is the distribution of environmental conditions simulated by the renderer, is the expected of transformation (EOT) technique, which simulates adversarial perturbations during the optimization process. The "true" input x adv obtained by the classifier f during the training process adv is the input object (m, c ) with environmental conditions e after the rendering operation. The generated camouflage pixel values are limited to the valid pixel range [0, 1] of the image. The attack process of the integrated model is shown in Algorithm 1, and the adversarial camouflage texture c adv of the 3D target object is obtained through iterative updates.

[0105] Algorithm 1: Integrated model Adv-Camou camouflage generation algorithm

[0106] Input: 3D target object (m, c), environmental condition parameter e E, neural renderer Target class label y * , K classification models f1, f2…, f K , corresponding logits values l1, l2…, l K , integrated weights w1, w2…, w K , maximum iteration step N

[0107] Output: Adversarial camouflage texture tensor c adv

[0108]

[0109] Experimental analysis

[0110] Experimental setup

[0111] Source models. Four classification models pre-trained on the ImageNet dataset were selected for integration to generate adversarial samples: Inception-v3 (Inc-v3), Inception-v4 (Inc-v4), Inception-ResNet-v2 (IncRes-v2), and ResNet-v2-152 (Res-152).

[0112] Target models. To comprehensively evaluate the attack effect of the generated adversarial camouflage, in addition to the source model as the target model for white-box attack testing, three models are selected to evaluate the cross-model black-box attack effect: Densenet-169 (Dense-169), Xception-71 (Xcep-71), and NASNetLarge (NASNet).

[0113] Evaluation index. For the test data, the targeted attack success rate is taken as the evaluation index, that is, when the original input x is correctly classified as the label y, and the adversarial sample x adv is misclassified as the specified target class label y * , it is considered as a successful attack, which is more difficult than the untargeted attack, and can be expressed as:

[0114]

[0115] Baseline method. The physical domain adversarial samples generated by existing research work mainly use local adversarial patches attached to the surface of objects or in the vicinity of objects. The original vehicle texture (Original), target class physical map texture (Natural), double attention suppression (DAS) patch texture, and RP2 adversarial patch texture, and Adv-Patch adversarial patch texture are selected as baseline comparisons. As shown in Figure 2 , in order to compare fairly, the adversarial patch patterns are distributed on the roof, hood, and side of the vehicle, etc. Adv-Camou adversarial camouflage texture covers the entire vehicle body, while the tire, window, and lamp are not easy or suitable for adversarial noise, and the original texture is maintained unchanged. Figure 2 The target class of the first row in the right column is a tractor, and the target class of the second row is a traffic light. DAS is an untargeted attack.

[0116] Physical simulation environment

[0117] The experiment uses the Unity engine to build a realistic 3D simulation scene, which can set different experimental environment parameters to evaluate different physical domain attacks fairly and repeatedly.

[0118] Virtual scene. As shown in Figure 3 , the virtual scene established is a typical urban environment, including buildings, streets, road signs, and green plants, etc. The simulation engine enables the experiment to be conducted under different environmental conditions, such as different viewing angles, camera distances, lighting, backgrounds, and occlusions, etc.

[0119] Camera settings. As shown in Figure 4 , the camera viewing angle and distance are freely arranged within a 360° semi-ellipsoid around the vehicle, which is more in line with the randomness of real-world camera distribution compared to the local adversarial patch restriction of viewing angle, thus more closely evaluating the physical domain attack effect.

[0120] Illumination settings. Previous studies have mostly evaluated attack effectiveness in bright environments, but many real-world scenarios are relatively dark, such as shadowy environments. Therefore, to more comprehensively evaluate attack effectiveness, two brightness levels were obtained in the simulation environment using area lighting and directional light sources, simulating daylight brightness (bright) and shadow brightness (dark), respectively. Figure 3 There are no shaded areas or shadowed areas.

[0121] Simulation scenario experiment

[0122] The results of the anti-camouflage attack are compared with the original texture, the physical texture, and the anti-patch texture under any viewpoint and different lighting conditions. Figure 2 The simulation displays DAS targetless textures as well as target-type textures such as tractors and traffic lights. At different locations in the simulated scene, 240 (120×2) free-viewpoint images with two brightness levels were rendered for each textured vehicle. The original textured vehicles were all correctly identified.

[0123] Table 2 presents the attack success rates under arbitrary viewpoints and different lighting conditions in white-box and cross-model black-box attack scenarios. Since the internal structure and parameters of the target model are usually inaccessible in real-world environments (considered black-box scenarios), focusing on black-box transfer attack results is more meaningful; that is, the cross-model attack results of adversarial examples generated based on the white-box source model against other target models. Due to page space limitations, Table 2 selects the Inc-v3 model as the representative model for white-box attack testing. It can be seen that the average targeted attack success rate of the integrated adversarial camouflage texture obtained by the Adv-Camou method is more than 25% higher than the baseline method and also superior to the DAS non-targeted attack method, maintaining strong robustness in 3D scenes. The higher black-box attack success rate achieved by Adv-Camou indicates that the generated adversarial camouflage better learns the decision boundary information of multiple models, thus exhibiting good cross-model attack performance and verifying the effectiveness of the proposed method.

[0124] Table 2 shows the target attack results. Under different brightness levels, the attack success rate for targets of the tractor class is higher in bright environments, while the attack success rate for targets of the traffic light class is higher in dark environments. Analysis of the tractor and traffic light data in the ImageNet training set reveals that the vast majority of tractor training data was captured in bright environments, while a significant portion of traffic light data was captured in dark environments or at night. Traffic lights have more prominent features in dark environments, which may explain why the attack success rate for traffic lights as a target class is higher in dark environments than in bright environments. Furthermore, the overall attack success rate for tractors as a target class is significantly higher than that for traffic lights. This is because the wheels of jeeps are also an important feature of tractors, and the inter-class distance between jeeps and tractors is smaller, making them easier to attack successfully.

[0125] Attack success rate (%) at arbitrary view and different illumination. * represents white-box attack

[0126]

[0127] Figure 5 and Figure 6 The classification results of Inc-v3 model on various view and texture samples under bright and dark scenes are shown respectively. As can be seen from the figures, the generalization of two-dimensional adversarial patches to three-dimensional scenes is weak, mainly because the deformation of the patch caused by the large view angle reduces the attack effect, and the attack effect of multiple local patches is not as good as that of an integrated adversarial texture. In addition, the adversarial camouflage texture car obtained by Adv-Camou can maintain strong adversariality at varying distances. Figure 5 and Figure 6 The top-2 classification results and confidence scores of Inc-v3 model on Adv-Camou texture car are shown in Table 3.

[0128] Table 3 Top-2 results and confidence scores of targeted attack on Inc-v3 model by adversarial camouflage texture car

[0129]

[0130] An interesting phenomenon was found in the experiment that the adversarial texture car of the tractor target class can be classified as a lawn mower, a plow or a harvester. By analyzing the data of these three classes in the ImageNet training set, it was found that some pictures of these three classes have very similar features to tractors, such as Figure 7 , and even contain tractors. Therefore, the adversarial texture car with the target class of tractor is also easily misclassified as these three classes. For the traffic light target class, it is easy to be misclassified as a barrel and a tricycle, and the reason may be that the circular traffic light abstract pattern is similar to the barrel or wheel.

[0131] Further, to verify the attack performance of the adversarial camouflage texture of the target object under partial occlusion, occlusions are set for different parts of the jeep, and 60 free-view images are taken for the original texture car, the tractor target class and the traffic light target class camouflage texture car respectively. The experimental results are shown in Table 4, which shows that the attack effect is improved under the occlusion condition compared with the non-occlusion condition, and the reason may be that although part of the adversarial texture is occluded, the occlusion may also cause the loss of important features of the jeep (such as the original texture car wheel), making the model's discrimination result relatively far from the original class space, and making the model pay more attention to the remaining adversarial texture area, thereby improving the deception effect. Figure 8 Examples of results of different part occlusions.

[0132] Table 4 Attack success rate (%) of adversarial texture car under arbitrary view and occlusion. * represents white-box attack

[0133]

[0134] Commercial classification system experiment

[0135] To test the attack effect of the generated adversarial camouflage on real-world classification applications, an experiment was conducted on a black-box attack test of Clarifai's general image classification system to verify the effectiveness and advancement of the proposed method. Clarifai is an internationally leading commercial deep learning artificial intelligence platform that provides common picture and video recognition services.

[0136] The category labels of Clarifai are different from the ImageNet dataset category labels, and the confidence score is not a softmax classification result, that is, the sum is not 1. Since the model structure and parameters of the Clarifai system cannot be known, it also belongs to a black-box model, and the model used has a large difference from the deep model trained based on ImageNet, the attack has greater challenges.

[0137] In the experiment, 100 adversarial camouflage vehicle samples with the target class of tractors were input into the Clarifai system, and 42% of the images could be identified as tractors with high confidence scores, further proving the effectiveness of the proposed Adv-Camou method. Figure 9 and Figure 10 Examples of attack results under different viewing angles in bright and dark environments are given.

[0138] Real scene experiment

[0139] To further verify the adversarial attack effect, a 3D printed tractor target class adversarial vehicle was used as the research object to conduct experimental analysis, and the robustness of the adversarial camouflage texture in the real world was evaluated. The experiment used a HONOR20 mobile phone to take pictures and record videos. Similar to the simulation scene experiment, the 3D printed vehicle was photographed at a free viewing angle. The adversarial camouflage texture vehicle was placed on a rotating turntable to evaluate the adversarial attack under a 360° surround viewing angle.

[0140] The total number of frames of the 5 videos was 2126 frames, which were 405, 409, 427, 435 and 446 frames, respectively. The light intensity had different changes, making the test data more diverse and representative, so as to comprehensively evaluate the effectiveness of the adversarial camouflage. The target attack success rates for the Inc-v3 model were 71% (289 / 405), 65% (266 / 409), 72% (308 / 427), 66% (288 / 435) and 58% (259 / 446), respectively. The experimental results show that the adversarial camouflage obtained by the Adv-Camou method can achieve a high target attack success rate in real scenes, proving the generalization ability of the obtained adversarial samples to the real world. Figure 11100 image frames with fixed interval sampling are displayed, and the classification results of the Inc-v3 model are given.

[0141] It is found in experiments that higher target attack success rates can be achieved in bright real-world environments than in dark environments, which may be because the camera is difficult to capture the details of the vehicle against the texture in low light conditions, resulting in a decrease in attack effect.

[0142] Other target experiments

[0143] To further verify the Adv-Camou method, a container and a barrel are selected as representatives of square and circular objects, and 120 images of different perspectives are generated in Unity. Table 5 shows the attack success rates of the Adv-Camou adversarial texture container and barrel at any perspective, which shows that the adversarial camouflage texture container and barrel are misclassified as the specified target class with high success rates, indicating that Adv-Camou can be used for other objects in addition to vehicles. According to the experimental results, compared with the jeep target, the attack success rate of the container and the barrel on the tractor target class decreases, and the analysis reason is that the container and the barrel do not have the feature of "wheels" possessed by the tractor, resulting in a relatively far distance from the tractor class space, making the attack more difficult; however, the attack success rate of the container and the barrel on the traffic light target class is significantly higher than that of the jeep target, and the analysis is also because the container and the barrel do not have the "wheels" feature of the vehicle, and the distance between the container and the barrel and the traffic light class is closer than that of the jeep, making the attack easier. Figure 12 Container and barrel adversarial camouflage sample examples.

[0144] Table 5 Attack success rate (%) of adversarial texture container and barrel at any perspective.* represents white-box attack

[0145]

[0146] The present application studies the generation of adversarial camouflage for three-dimensional objects, which can deceive intelligent image classifiers to output specified target classes at any perspective. The attack effect under different perspectives and brightness conditions is evaluated in the established realistic three-dimensional simulation scene, and the results show that the obtained adversarial camouflage has a significantly higher attack success rate than the baseline method. Further, the attack test on the Clarifai commercial image classification system takes a high black-box attack success rate. Finally, a full-color adversarial camouflage vehicle is made through 3D printing technology, which can deceive image classifiers at any perspective and under different lighting conditions in the real world. Adv-Camou adversarial camouflage can also be generalized to attack any other object. The research results are helpful for coping with physical domain adversarial attacks to more targetedly maintain the security of intelligent recognition systems, and on the other hand, it is helpful for the development of privacy protection technology for intelligent recognition systems. In the future, optimization research will be further carried out on the naturalness of the camouflage texture.

[0147] Embodiment Two:

[0148] The embodiment provides a full-view angle adversarial camouflage generation device for image classification, and the device comprises:

[0149] An input module is configured to acquire a three-dimensional model file of a target object and a corresponding texture;

[0150] A generation module is configured to perform three-dimensional rendering on the target object to be camouflaged according to the corresponding texture, and obtain target object images of different view angles, distances, brightnesses, backgrounds and environmental noises in real time to form training samples;

[0151] A camouflage updating module is configured to integrate an attack source model according to the training samples to update adversarial camouflage texture and obtain three-dimensional full-view angle adversarial camouflage.

[0152] The device of the embodiment can be used to implement the method of the embodiment one.

[0153] Embodiment Three:

[0154] The embodiment provides a full-view angle adversarial camouflage generation device for image classification, and the device comprises a processor and a storage medium.

[0155] The storage medium is configured to store instructions.

[0156] The processor is configured to operate according to the instructions to perform the steps of the method according to the embodiment one.

[0157] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can adopt a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt a computer program product in the form of one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program codes.

[0158] The present application is described with reference to flowcharts and / or block diagrams according to the method, device (system), and computer program product of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the computer or other programmable data processing devices produce a device for implementing the functions described in the flowcharts and / or block diagrams. Figure 1 Each flow or multiple flows and / or blocks Figure 1means for performing the function specified by the block or blocks.

[0159] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the flow Figure 1 one or more flowcharts and / or blocks Figure 1 one or more blocks or means for performing the function specified by the block or blocks.

[0160] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flow Figure 1 one or more flowcharts and / or blocks Figure 1 one or more blocks or steps for performing the function specified by the block or blocks.

[0161] The above description is merely that of the preferred embodiments of the application and modifications and alterations are possible without departing from the principles of the application as set forth in the claims.

Claims

1. A full-view-angle adversarial camouflage generation method for image classification, characterized in that, The method comprises the following steps: obtaining a three-dimensional model file of a target object and corresponding texture; mapping the texture to a rendering image of the target object using the three-dimensional model file of the target object and the picture file of the corresponding texture as input, and performing geometric transformation and environmental transformation according to a given three-dimensional environmental condition parameter distribution to approximately simulate the distribution of a real environment, thereby obtaining training samples in real time under different viewing angles, distances, brightness, backgrounds and environmental noise; integrating an attack source classification model to update the adversarial camouflage texture according to the training samples, thereby obtaining a three-dimensional full-view adversarial camouflage; mapping the texture to a rendering image of the target object using the three-dimensional model file of the target object and the picture file of the corresponding texture as input, and introducing geometric transformation and environmental transformation according to a given three-dimensional environmental condition parameter distribution to approximately simulate the distribution of a real environment, thereby obtaining different training samples in real time, comprising the following steps: 3D target from grid tensor Cover texture tensor Generated, denoted ; Let denote the environmental condition parameters; all parameters are independent continuous random variables, following uniform distribution; modeling the real scene light changes and printing errors by adding and multiplying the brightness and channel pixels; At ambient conditions The image rendering operation The sample Is: (1) By applying the texture Adding adversarial noise, getting adversarial camouflage texture The generated adversarial sample is: (2) wherein is the resulting 3D adversarial object; integrating an attack source classification model to update the adversarial camouflage texture according to the training samples, thereby obtaining a three-dimensional full-view adversarial camouflage, comprising: Determining 3D objects , environmental condition parameters , image rendering operations , target class labels , classification models , corresponding logits values , integration weights , maximum iteration steps ; assigning to the initial adversarial camouflage texture the input 3D object's texture tensor ; Perform From 0 to -1 loop: Generating training data: ; from the input compute logits values for the individual models and integrate: ； By attacking all the classification update camouflage textures : ； ； where the cross-entropy loss function can be expressed as where is the one-hot encoding of ; is a training set of images of arbitrary viewpoints and varying environmental conditions generated in real-time by a renderer, is the range of environmental condition distribution simulated by the renderer, is the transformation expectation, represents a sample distributed in the set X, represents an environmental condition distributed in E, the input obtained by the classifier during the training process is , is the input target object with environmental conditions after the rendering operation ; the generated camouflage pixel value is limited to the valid pixel range of the image [0, 1]; After the loop ends, the 3D target object is obtained through the iterative update of the adversarial camouflage texture .

2. The omnidirectional adversarial camouflage generation method for image classification according to claim 1, characterized in that, the geometric transformation comprises rotation, translation and camera distance change; the environmental transformation comprises brightness, noise and background change.

3. The omnidirectional adversarial camouflage generation method for image classification according to claim 2, characterized in that, The environmental parameter distribution is shown in Table 1: Table 1: Physical domain environmental condition parameter distribution 。 4. The omnidirectional adversarial camouflage generation method for image classification according to claim 1, characterized in that, During the training process, the unconstrained adversarial camouflage texture is generated based on the distance between the original texture and the target texture.

5. The omnidirectional adversarial camouflage generation method for image classification according to claim 1, characterized in that, The method further comprises: establishing a virtual scene using a three-dimensional engine to perform a change scene simulation experiment, and testing the Clarifai commercial image classification system to verify and evaluate the attackability of the 3D printed adversarial object in the real world.

6. An all-view adversarial camouflage generation device for image classification for performing the method of claim 1. The device comprises: an input module for obtaining a three-dimensional model file of a target object and corresponding texture; a generation module for performing three-dimensional rendering on the target object to be camouflaged according to the corresponding texture, thereby obtaining target object images under different viewing angles, distances, brightness, backgrounds and environmental noise in real time, and forming training samples; a camouflage update module for integrating an attack source classification model to update the adversarial camouflage texture according to the training samples, thereby obtaining a three-dimensional full-view adversarial camouflage.

7. An all-view adversarial camouflage generation device for image classification, characterized in that, comprising a processor and a storage medium; the storage medium is used to store instructions; the processor is used to operate according to the instructions to perform the steps of the method according to any one of claims 1-5.

Images