Three-dimensional confrontation sample generation method based on model attention distraction

Abstract

The invention discloses a three-dimensional confrontation sample generation method based on model attention distraction, and relates to the technical field of artificial intelligence confrontation attacks, and the method comprises the steps: collecting data sets of a three-dimensional target model in different virtual scenes, carrying out the neural rendering of the same group of data in the data sets, and generating a scene image containing a target; analyzing the scene image by using a white box model, calculating model attention distraction loss, and calculating texture constraint loss and smooth loss of an adversarial sample by generating target foreground images of different samples; integrating the attention distraction loss, the texture constraint loss and the smooth loss of the model, and calculating the total loss of the model; and performing iterative training by using the data set to obtain a three-dimensional confrontation sample generated by optimization. The three-dimensional adversarial sample has good attack performance and attack migration capability, can generate an adversarial effect on an image classifier and a target detector which are constructed by using different network models, and considers the robustness, the adversarial property and the appearance constraint of the adversarial sample.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence adversarial attack technology, and in particular to a three-dimensional adversarial sample generation method based on model distraction. Background Art

[0002] Traditional research on adversarial sample generation methods focuses on the two-dimensional field and adversarial performance, such as adding perturbations to images. However, two-dimensional adversarial samples lack robustness. When they are rotated, offset, scaled, etc., they will be correctly re-identified by the neural network model and it is difficult to maintain high adversarial performance, which limits the relevance of two-dimensional adversarial samples to the real world.

[0003] In recent years, the generation of three-dimensional adversarial samples has gradually become a hot topic based on the practical application significance of adversarial samples. Three-dimensional adversarial sample generation methods can be divided into patch method and full coverage method in terms of style. The patch method is to attach adversarial patches to the surface of three-dimensional objects to make the objects adversarial. The patch covers a small area and will not change the appearance of the original object to a great extent. However, at some specific angles, the adversarial patch will be completely or partially occluded, resulting in a significant reduction in the adversarial performance of three-dimensional adversarial samples. In order to make the adversarial performance unaffected by viewing angle factors, researchers have proposed a full coverage method to generate three-dimensional adversarial samples by changing the appearance texture of the three-dimensional model. At present, the full coverage method pays too much attention to the adversarial performance and robustness of the sample, but changes the original appearance of the object to a large extent, making the adversarial sample have poor visual features, such as large areas of suspicious colors and distortions, which is contrary to the original design of the adversarial sample. Summary of the invention

[0004] Based on this, it is necessary to provide a three-dimensional adversarial sample generation method based on model attention distraction to address the above technical problems.

[0005] The present invention provides a three-dimensional adversarial sample generation method based on model distraction, comprising:

[0006] S1, collecting data sets of 3D target models in different virtual scenes, and performing neural rendering on the same group of data in the data sets to generate scene images containing the target;

[0007] S2. Use the white box model to analyze the scene image, calculate the model's attention distraction loss, and calculate the texture constraint loss and smoothness loss of the adversarial sample by generating target foreground images of different samples;

[0008] S3, integrate the model's attention distraction loss, texture constraint loss and smoothness loss, calculate the total model loss, set the optimization object and optimization goal, and use the optimization algorithm for optimization update;

[0009] S4. Use the dataset for iterative training to obtain optimized three-dimensional adversarial samples.

[0010] Furthermore, data sets of the three-dimensional target model in different virtual scenes are collected, and the same group of data in the data set is neurally rendered to generate scene images containing the target, including:

[0011] S11, placing a three-dimensional target model of a vehicle in a simulator environment, setting different virtual scenes, using a camera sensor to collect scene images of the three-dimensional target model under multiple perspectives, and reading the position information of the three-dimensional target model and the camera sensor as rendering parameters;

[0012] S12, merging each group of corresponding scene images and rendering parameters to form a data set;

[0013] S13, reading the vertex, face and texture data of the three-dimensional target model, replacing the original texture data with the optimized texture data, and reading the scene image and rendering parameters of the same group in the data set, rendering with a neural renderer, and generating a foreground image and a mask image of the target;

[0014] S14, loading the scene image of the same group of data in the data set, combining the foreground image and the mask image of the target, and generating a scene image containing the target.

[0015] Furthermore, the white box model is used to analyze the scene image, calculate the model attention distraction loss, and calculate the texture constraint loss and smoothness loss of the adversarial sample by generating target foreground images of different samples, including:

[0016] S21. Select the ResNet-50 network as the white box model, use the white box model to analyze the attention distribution of the scene image, and obtain the model attention heat map;

[0017] S22. Calculate the model attention distraction loss based on the model attention heat map;

[0018] S23, extracting a set of rendering parameters from the data set, inputting the 3D target model and texture through neural rendering, and generating the target foreground image of the clean sample and the adversarial sample under the same rendering parameters;

[0019] S24. Based on the target foreground images of clean samples and adversarial samples, measure the visual gap between adversarial samples and clean samples, and calculate the texture constraint loss and smoothness loss of adversarial samples.

[0020] Furthermore, the ResNet-50 network is selected as the white box model, and the attention distribution of the scene image is analyzed using the white box model. The model attention heat map includes:

[0021] S211, extracting the feature map output by the convolution block of the white box model during the forward propagation process;

[0022] S212, using the back propagation of the white box model to obtain all channel weights of the target category in the feature map;

[0023] S213. Combine the feature maps of each channel with the corresponding weights to obtain the attention distribution map of the white box model for the input scene image as the model attention heat map.

[0024] Furthermore, the formula for calculating the model's attention distraction loss is:

[0025]

[0026] Where, L a is the attention loss of the model; K is the number of attention regions in the model’s attention heat map; V k is the sum of the pixel values ​​of the kth attention area in the model attention heat map; W is the width of the model attention heat map; H is the height of the model attention heat map; S k is the area size of the kth attention region in the model's attention heat map.

[0027] Furthermore, the formulas for calculating the texture constraint loss and smoothness loss of adversarial samples include:

[0028]

[0029] Where, L s is the texture constraint loss; I obj is the target foreground image of the adversarial sample; I obj0 is the target foreground image of the clean sample; L m is the smoothing loss; x i，j is the pixel value at coordinate (i, j) in the target foreground image of the adversarial sample; x i+1，j is the pixel value at coordinate (i+1, j) in the target foreground image of the adversarial sample; x i，j+1 is the pixel value at coordinate (i, j+1) in the target foreground image of the adversarial sample.

[0030] Furthermore, the model attention distraction loss, texture constraint loss and smoothness loss are integrated to calculate the total model loss, set the optimization object and optimization goal, and use the optimization algorithm for optimization update, including:

[0031] S31, assigning respective weight values ​​to the model's attention distraction loss, texture constraint loss and smoothness loss, and calculating the total model loss by weighted fusion;

[0032] S32. The adversarial sample texture is set as the optimization object, the minimization of the total model loss of the adversarial sample texture is set as the optimization goal, and the multi-strategy fusion Beluga optimization algorithm is used for optimization and update to obtain the optimal adversarial sample texture.

[0033] Furthermore, the multi-strategy fusion Beluga optimization algorithm is used for optimization and update, and the optimal adversarial sample textures are obtained, including:

[0034] S321. Using chaotic mapping to generate the initial population position, each beluga individual represents a different candidate perturbation of the adversarial sample texture, and using the total model loss to calculate the fitness value of each population individual;

[0035] S322. Use a modified decreasing control mechanism to calculate the balance factor between the exploration stage and the development stage, and use the cosine function to optimize the trend of the whale fall probability;

[0036] S323, in the exploration stage, the fusion Cauchy mutation operator is used to perform global position enhancement and calculate the position information of the population individuals in the exploration stage;

[0037] S324. In the development phase, an adaptive inertia weight factor is introduced according to fitness, the position information of the individuals in the population in the development phase is calculated, and the best individual in the current population is selected;

[0038] S325. When the maximum number of iterations is reached, stop the optimization and output the optimized adversarial sample texture.

[0039] Furthermore, the formula for calculating the position information of population individuals in the exploration phase is:

[0040]

[0041] In the formula, is the position information of individual i in the exploration phase at iteration t+1; is the optimal individual position; Cauchy(0,1) is the fused Cauchy mutation operator;

[0042] The formula for calculating the position information of population individuals in the exploration phase is:

[0043]

[0044] In the formula, is the position information of individual i in the development phase at iteration t+1; is the position information of individual i in the population at iteration t; is the position information of other whale individuals r at iteration t; C is the random jump intensity; r1 and r2 are both random numbers between (0, 1); L F is the levy flight function; ω zis the adaptive inertia weight factor.

[0045] Furthermore, the dataset is used for iterative training to obtain optimized three-dimensional adversarial samples including:

[0046] S41, sequentially extracting rendering parameters and scene image input value neural renderers of other groups in the data set, combining with the three-dimensional target model, to generate a foreground image and a scene image of the target;

[0047] S42, based on different foreground images and scene images, the multi-strategy fusion Beluga optimization algorithm is used to optimize and update the adversarial sample texture to obtain the optimal adversarial sample texture for each set of data;

[0048] S43. When the number of adversarial sample texture optimizations reaches a preset number of iterations, the finally optimized adversarial sample texture is applied to the three-dimensional target model to obtain a three-dimensional adversarial sample.

[0049] The beneficial effects of the present invention are:

[0050] 1. The three-dimensional adversarial samples generated by the present invention have good attack performance and attack migration capabilities, which can produce adversarial effects on image classifiers and target detectors constructed with different network models, taking into account the robustness, adversarial nature and appearance constraints of adversarial samples: by introducing three-dimensional models and neural rendering technology, the adversarial attack technology is extended to the three-dimensional field. Compared with traditional two-dimensional adversarial samples, three-dimensional adversarial samples have excellent robustness and relevance to the real world; based on the visualization interpretation technology of neural networks, the attention heat map of the network model is obtained, and by optimizing the texture of the adversarial samples, the attention distribution of the model is dispersed, thereby affecting the feature extraction ability of the model and reducing the recognition accuracy of the model, so that the three-dimensional adversarial samples have excellent adversarial performance; through texture constraints, the similarity between the adversarial samples and the clean samples is controlled, so that the three-dimensional adversarial samples have better visual features.

[0051] 2. The model-based attention distraction method of the present invention is a white-box adversarial attack method. Compared with the black-box adversarial attack method, the adversarial samples generated by this method have more advantages in performance. After several experimental analyses, ResNet-50 was selected as the white-box network model. The network model has moderate structural complexity and depth. The generated adversarial samples have excellent adversarial properties and adversarial transferability, and can achieve cross-model adversarial, including the adversarial between image classifiers and target detectors, effectively avoiding the adversarial overfitting problem under the white-box attack setting.

[0052] 3. Based on the original three-dimensional model, the present invention designs a method for generating three-dimensional adversarial samples along the idea of ​​optimizing texture and reducing loss. The method has no restrictions on the three-dimensional model and loss weight. The adversarial performance and visual features of different samples can be balanced by changing the model and loss weight. It supports users to customize the initial texture pattern of the three-dimensional target model, and the generated adversarial samples will retain the initial texture visual features to a large extent. It has rich application scenarios and can meet the adversarial needs in different environments, thus having high competitiveness and application value in the field of artificial intelligence adversarial attacks. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0054] Figure 1 is a flow chart of a method for generating three-dimensional adversarial samples based on model distraction according to an embodiment of the present invention;

[0055] Figure 2 is a technical block diagram of a method for generating three-dimensional adversarial samples based on model distraction according to an embodiment of the present invention;

[0056] Figure 3 is a schematic diagram of a target image generation method in a three-dimensional adversarial sample generation method based on model attention dispersion according to an embodiment of the present invention;

[0057] Figure 4 It is a schematic diagram of a model attention acquisition method in a three-dimensional adversarial sample generation method based on model attention dispersion according to an embodiment of the present invention. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0059] See also Figure 1 , a 3D adversarial sample generation method based on model distraction is provided, including:

[0060] S1. Collect data sets of three-dimensional target models in different virtual scenes, and perform neural rendering on the same group of data in the data sets to generate scene images containing the targets.

[0061] In the description of the present invention, collecting data sets of a three-dimensional target model in different virtual scenes, and performing neural rendering on the same group of data in the data sets to generate a scene image containing the target includes:

[0062] S11. Place a three-dimensional target model of the vehicle in the simulator environment, set different virtual scenes, use a camera sensor to collect scene images of the three-dimensional target model from multiple perspectives, and read the position information of the three-dimensional target model and the camera sensor as rendering parameters.

[0063] Specifically, the present invention places a three-dimensional target vehicle model in a simulator environment, and sets a camera sensor around the target vehicle so that the target vehicle model can be captured; the placement of the vehicle model and the camera sensor can be completed based on the API provided by the Carla simulator.

[0064] By changing the virtual scene and the observation distance and angle of the camera sensor, scene image data containing the target vehicle from multiple perspectives is collected; the combination of multiple observation distances and angles in the dataset is conducive to the robustness of generating three-dimensional adversarial samples, and is also convenient for comprehensively verifying the adversarial performance of three-dimensional adversarial samples.

[0065] While collecting images, the position data and posture data of the target vehicle model in the simulator and the position data and posture data of the camera sensor are read and saved as neural rendering parameters;

[0066] The position data is the absolute position of the object in the simulated environment, expressed by three-dimensional coordinates, including X-coordinate, Y-coordinate and Z-coordinate, which respectively indicate the position of the object in the horizontal, vertical and height directions; the attitude data is the direction of the object, expressed by three angles, including yaw angle, pitch angle and roll angle, which respectively indicate the rotation angle of the object relative to the north direction, the horizontal plane and its own longitudinal axis.

[0067] It should be noted that the Carla simulator is an open three-dimensional virtual environment for studying autonomous driving. It supports a variety of scenarios, vehicles, and sensor models, and can provide simulation effects close to the real world. Therefore, it is suitable for data set collection.

[0068] S12, merging each group of corresponding scene images and rendering parameters to form a data set.

[0069] Specifically, each set of corresponding scene image data and rendering parameters are merged into a data file, and the data file is saved in NPZ format, which can be read as a NumPy array using Python's NumPy library.

[0070] S13, read the vertex, face and texture data of the three-dimensional target model, replace the original texture data with the optimized texture data, read the scene image and rendering parameters of the same group in the data set, use the neural renderer to render, and generate the foreground image and mask image of the target.

[0071] Specifically, traditional rendering methods contain a discrete operation called rasterization, which makes it impossible to integrate the rendering process into neural networks. The neural renderer combines rendering with neural networks through rasterization that approximates gradients, enabling an end-to-end 3D adversarial sample training framework.

[0072] The method of generating three-dimensional adversarial samples in the present invention is to optimize the sample texture, so a texture data that can be optimized and trained is set based on the original model texture. There is no background in the foreground image and the mask image. The main difference between them is whether the target model texture is added in the neural rendering process. The textureless mask image fills the vehicle area with white.

[0073] S14, loading the scene image of the same group of data in the data set, combining the foreground image and the mask image of the target, and generating a scene image containing the target. This process can be expressed as:

[0074] I=I0×(1-M)+I obj *M;

[0075] Where I is the generated scene image containing the target; I0 ​​is the scene image containing the target read from the dataset; I obj is the target foreground image output by the neural renderer; M is the target mask image output by the neural renderer.

[0076] It should be noted that the process of generating a scene image containing a target is as follows: Figure 3 As shown, in essence, the original target foreground image collected from the simulator is deleted and replaced with the target foreground image generated by neural rendering, that is, the clean sample is replaced by the adversarial sample.

[0077] S2. Use the white-box model to analyze the scene image, calculate the model's attention distraction loss, and calculate the texture constraint loss and smoothness loss of the adversarial sample by generating target foreground images of different samples.

[0078] In the description of the present invention, the white box model is used to analyze the scene image, the model attention distraction loss is calculated, and the texture constraint loss and smoothness loss of the adversarial sample are calculated by generating target foreground images of different samples, including:

[0079] S21. Select the ResNet-50 network as the white box model, use the white box model to analyze the attention distribution of the scene image, and obtain the model attention heat map.

[0080] It should be noted that compared with the black-box method, the adversarial samples generated by the white-box method can show strong adversarial performance, but there will be problems with adversarial fitting and a certain type of neural network. Therefore, we choose to attack with similar feature extraction mechanisms between neural network models, and disrupt the model's attention so that the model extracts wrong features or misses features, thereby achieving the effect of adversarial attack and avoiding the adversarial overfitting problem in the white-box method.

[0081] The ResNet-50 network model selected by the present invention is moderate in structural complexity and depth. After comparative experiments with models such as ResNet-152, VGG, and DenseNet-201, it is found that the three-dimensional adversarial samples generated using the ResNet-50 network model have better adversarial resistance and adversarial transferability, and can achieve cross-model adversarial performance, including adversarial performance between image classifiers and target detectors, effectively avoiding the adversarial overfitting problem under the white-box attack setting.

[0082] Therefore, the present invention selects the ResNet-50 network as the white box model, and inputs the generated scene image into the white box model to make it reason. This process can be expressed as:

[0083] F(I)=y′.

[0084] In the formula, F is the white box network model; I is the input image; y′ is the result inferred by the model.

[0085] In the description of the present invention, the ResNet-50 network is selected as the white box model, and the attention distribution of the scene image is analyzed using the white box model to obtain the model attention heat map including:

[0086] S211. Extract the feature map output by the convolution block of the white box model during the forward propagation process.

[0087] Specifically, the present invention selects a general method for visual interpretation of deep neural networks, Grad-CAM, which utilizes the spatial correspondence between the feature map output by the convolution block and the original image, and processes the feature map of the model into a heat map, thereby obtaining the attention distribution of the model.

[0088] The present invention extracts the feature map output by the convolution block of the network model during the forward propagation process, which can be expressed as:

[0089] A=f CNN (I).

[0090] Where A is the feature map output by the convolutional block; CNN is the convolution block function of the network model; I is the input image of the network model.

[0091] It should be noted that the feature map is the intermediate result in the reasoning process of the neural network model and is the data that has not passed through the fully connected layer.

[0092] S212, using the back propagation of the white box model, obtain all channel weights of the target category in the feature map. This process can be expressed as:

[0093]

[0094] Where y is the true category label of the target in the input image; n is the channel number of the feature map; is the weight of category y in the feature map of the nth channel; W A is the width of the feature map; H A is the height of the feature map; s y The score of category y inferred by the network model; is the value at coordinate (i, j) in the feature map of the nth channel.

[0095] It should be noted that the above process essentially calculates the gradient of each feature map, and then globally averages these gradients to obtain a weight vector. The weight vector reflects the contribution of each feature map to the target category.

[0096] S213, combining the feature maps of each channel with the corresponding weights to obtain the attention distribution map of the white box model for the input scene image as the model attention heat map. This process can be expressed as:

[0097]

[0098] Where X is the attention heat map of the network model to the input; ReLU(·) is the linear rectification function; n is the number of channels of the feature map; is the weight of category y in the feature map of the nth channel; A n is the feature map of the nth channel.

[0099] It should be noted that if Figure 4 As shown in the figure, the weight vector is multiplied element-by-element by the feature map and then summed. After activation by the ReLU function, a heat map is obtained, which shows the contribution of different regions to the target category.

[0100] ReLU is a linear rectification function. As a commonly used activation function in deep learning, it is used to introduce nonlinear characteristics into neural networks so that the network can learn and fit complex data patterns. The formula is as follows:

[0101] f(x)=max(0,x)

[0102] S22. Calculate the model attention distraction loss based on the model attention heat map.

[0103] In the description of the present invention, the formula for calculating the model attention distraction loss is:

[0104]

[0105] Where, L a is the attention loss of the model; K is the number of attention regions in the model’s attention heat map; V k is the sum of the pixel values ​​of the kth attention area in the model attention heat map; W is the width of the model attention heat map; H is the height of the model attention heat map; S k is the area size of the kth attention region in the model's attention heat map.

[0106] It should be noted that the design of the model attention dispersion loss function aims to "cool down" or disperse the hot spots on the attention heat map into multiple small hot areas, and transfer the main hot areas from the target area to other areas as much as possible.

[0107] S23. Extract a set of rendering parameters from the data set, input the three-dimensional target model and texture through neural rendering, and generate the target foreground image of the clean sample and the adversarial sample under the same rendering parameters.

[0108] S24. Based on the target foreground images of clean samples and adversarial samples, measure the visual gap between adversarial samples and clean samples, and calculate the texture constraint loss and smoothness loss of adversarial samples.

[0109] Specifically, the Euclidean distance is often used to measure the distance between two individuals. The present invention applies it to the similarity between three-dimensional samples before and after quantitative training optimization, requiring that the corresponding pixel value difference between the front and back samples is within a certain range, so that the adversarial texture added to the three-dimensional adversarial sample will not appear out of control. Based on the sample texture constraint of the Euclidean distance, the target foreground image of the adversarial sample and the clean sample under the same rendering parameters is generated through the neural rendering method, the visual difference between the adversarial sample and the clean sample is measured, and the texture constraint loss and smoothness loss are calculated.

[0110] A set of rendering parameters is extracted from the data set, and the 3D target model and texture are input through the neural rendering method to generate the target foreground image of the clean sample and the adversarial sample under the same rendering parameters. This process can be expressed as:

[0111]

[0112] In the formula, I obj is the target foreground image of the adversarial sample; I obj0is the target foreground image of the clean sample; R is the neural rendering method function; O is the three-dimensional target model, including vertex information and face information; T0 is the original texture of the three-dimensional target model; T is the optimized texture, that is, the adversarial sample texture; ω is the rendering parameter.

[0113] It should be noted that the texture constraint is intended to control the degree of distortion of three-dimensional adversarial samples. By comparing the two-dimensional foreground image with the three-dimensional clean sample at multiple distances and angles, the optimization direction of the adversarial texture is constrained. Therefore, a neural renderer is used to generate the sample foreground image.

[0114] In the description of the present invention, the formulas for calculating the texture constraint loss and smoothness loss of the adversarial sample include:

[0115]

[0116] Where, L s is the texture constraint loss; I obj is the target foreground image of the adversarial sample; I obj0 is the target foreground image of the clean sample; L m is the smoothing loss; x i，j is the pixel value at coordinate (i, j) in the target foreground image of the adversarial sample; x i+1，j is the pixel value at coordinate (i+1, j) in the target foreground image of the adversarial sample; x i，j+1 is the pixel value at coordinate (i, j+1) in the target foreground image of the adversarial sample.

[0117] It should be noted that the texture constraint loss function essentially measures the Euclidean distance between the corresponding pixels of the adversarial sample and the clean sample on the image. The introduction of smoothness loss can effectively reduce the distortion on the adversarial sample.

[0118] S3. Comprehensive model attention distraction loss, texture constraint loss and smoothness loss, calculate the total model loss, set the optimization object and optimization goal, and use the optimization algorithm for optimization update.

[0119] In the description of the present invention, the model attention distraction loss, texture constraint loss and smoothness loss are integrated to calculate the total model loss, and the optimization object and optimization goal are set. The optimization algorithm is used for optimization update, including:

[0120] S31. Assign respective weight values ​​to the model's attention distraction loss, texture constraint loss and smoothness loss, and calculate the total model loss using a weighted fusion approach.

[0121] Specifically, the joint model attention distraction loss, texture constraint loss, and smoothness loss are used to calculate the total model loss of the adversarial sample generation algorithm. The formula is as follows:

[0122] L total =L a +λL s +μL m ;

[0123] Where, L total is the total loss; L a is the model attention loss; L s is the texture constraint loss; L m is the smoothness loss; λ is the weight of the texture constraint loss, which is used to control the distortion of the adversarial sample; μ is the weight of the smoothness loss.

[0124] It should be noted that the weight of the smoothing loss is generally a fixed value; the weight of the texture constraint loss is dynamically adjusted according to factors such as the three-dimensional model, initial texture, and application scenario. The smaller the weight, the greater the difference between the generated adversarial sample and the clean sample, and the stronger the adversarial performance; conversely, the closer the adversarial sample is to the clean sample, the weaker the adversarial performance.

[0125] S32. The adversarial sample texture is set as the optimization object, the minimization of the total model loss of the adversarial sample texture is set as the optimization goal, and the multi-strategy fusion Beluga optimization algorithm is used for optimization and update to obtain the optimal adversarial sample texture.

[0126] In the description of the present invention, the optimization update is performed by using the multi-strategy fusion Beluga optimization algorithm to obtain the optimal adversarial sample texture including:

[0127] S321. Use chaotic mapping to generate the initial population position. Each beluga individual represents a different candidate perturbation of the adversarial sample texture, and use the total model loss to calculate the fitness value of each population individual.

[0128] Specifically, the Logistic Chaos Mapping Factor is used to initialize the population and increase the diversity of the population. The position of the initial population is generated by chaos mapping, which can speed up the convergence speed. Introducing chaos into the White Whale algorithm can reduce the probability of the algorithm falling into the local optimum and speed up the convergence speed of the algorithm.

[0129] S322. A modified decreasing control mechanism is adopted to calculate the balance factor between the exploration stage and the development stage, and the cosine function is used to optimize the changing trend of the whale fall probability.

[0130] Specifically, the balance factor determines the balance between the exploration phase and the development phase of the algorithm. It can be adjusted dynamically according to the needs of the task. However, in the original Beluga algorithm, the balance factor changes linearly. For example, a value in the range of [0, 1] can be set, and the optimal value is usually determined by experimental optimization. However, it cannot accurately reflect the actual process of beluga whale groups preying on prey in nature. Therefore, the present invention draws on the change process of prey energy in the Harris Hawk optimization algorithm, designs a modified decreasing control mechanism, and the balance factor B f The nonlinearity decreases, and has a larger change rate in the early stage of algorithm iteration, which is conducive to rapid and comprehensive global search; it has a smaller change rate in the later stage of algorithm iteration, which reduces the search range and can greatly improve the tracking accuracy. The adjusted balance factor B f The formula is:

[0131]

[0132] Where B0 is a random number between (0, 1); R is a random operator that enhances exploration capability; t is the current iteration number; T max is the maximum number of iterations.

[0133] The whale fall probability is used in the algorithm to decide whether to introduce a whale fall (i.e., "detect" a new area). It can be dynamically adjusted according to the exploration and development stages in the optimization process. The present invention uses a cosine function to enhance the change trend of the whale fall probability of the original white whale algorithm, so that it has a higher whale fall probability at the early stage of iteration when it is far away from the prey, ensuring the retention of high-quality white whale individuals.

[0134] Using the cosine function to optimize the whale fall probability W f The calculation formula is:

[0135] W f =a·[cos(π·t / T max )+1]+b;

[0136] Where a is the control parameter of the cosine function amplitude, which is used to adjust the dynamic range of the whale fall probability; b is the offset, which controls the minimum value of the whale fall probability (i.e., the lower bound of the cosine function).

[0137] S323. In the exploration phase, the fused Cauchy mutation operator is used to perform global position enhancement and calculate the position information of the population individuals in the exploration phase.

[0138] In the description of the present invention, the formula for calculating the position information of the population individuals in the exploration phase is:

[0139]

[0140] In the formula, is the position information of individual i in the exploration phase at iteration t+1; is the optimal individual position; Cauchy(0,1) is the fused Cauchy mutation operator;

[0141] S324. In the development phase, an adaptive inertia weight factor is introduced according to fitness, the position information of the individuals in the population in the development phase is calculated, and the best individual in the current population is selected.

[0142] In the description of the present invention, the formula for calculating the position information of the population individuals in the exploration phase is:

[0143]

[0144] In the formula, is the position information of individual i in the development phase at iteration t+1; is the position information of individual i in the population at iteration t; is the position information of other whale individuals r at iteration t; C is the random jump intensity; r1 and r2 are both random numbers between (0, 1); L F is the levy flight function; ω z is the adaptive inertia weight factor.

[0145] It should be noted that the adaptive weight factor is used for adaptive adjustment when updating local details. This factor controls the intensity of local search and ensures that the algorithm will not fall into local extreme values ​​when searching for local optimal solutions. In the early stage of the optimization algorithm (global exploration stage), a larger inertia factor value (such as 0.9) is set, and a diverse initial population is generated in combination with the Logistic chaotic map. In the later stage (local development stage), the inertia factor is gradually reduced to a smaller value (such as 0.1), focusing on fine search in local areas.

[0146] The inertia factor is further adjusted through population fitness to adapt it to dynamic search needs.

[0147] S325. When the maximum number of iterations is reached, stop the optimization and output the optimized adversarial sample texture.

[0148] S4. Use the dataset for iterative training to obtain optimized three-dimensional adversarial samples.

[0149] Specifically, Figure 2 Shown is a block diagram of a 3D adversarial sample generation method based on model attention dispersion, which is mainly composed of three parts: neural rendering, model attention dispersion, and texture constraint. Through an iterative optimization framework, a 3D adversarial sample is finally generated.

[0150] In the description of the present invention, iterative training using a data set to obtain optimized generated three-dimensional adversarial samples includes:

[0151] S41, sequentially extract the rendering parameters and scene image input value neural renderer of other groups in the data set, combine with the three-dimensional target model, and generate the foreground image and scene image of the target.

[0152] Specifically, the data set contains rendering parameters at multiple distances and angles, which can fully reflect the surface features of three-dimensional samples through neural rendering. Therefore, the neural renderer is integrated into the adversarial sample generation method framework to expand the application of the generation algorithm to the three-dimensional field.

[0153] S42. Based on different foreground images and scene images, the multi-strategy fusion Beluga optimization algorithm is used to optimize and update the adversarial sample texture to obtain the optimal adversarial sample texture for each set of data.

[0154] Specifically, the attention distraction loss is related to the attack performance of the adversarial sample, and the texture constraint loss and smoothness loss are related to the visual characteristics of the adversarial sample. The two are actually contradictory because the adversarial ability of 3D adversarial samples comes from their own special surface features. This method aims to generate a more balanced 3D adversarial sample, so multiple loss functions are integrated into the total loss function.

[0155] S43. When the number of adversarial sample texture optimizations reaches a preset number of iterations, the finally optimized adversarial sample texture is applied to the three-dimensional target model to obtain a three-dimensional adversarial sample.

[0156] Specifically, the maximum number of iterations set in the experiment of the present invention is the number of files in the data set.

[0157] In summary, with the help of the above-mentioned technical scheme of the present invention, the three-dimensional adversarial samples generated by the present invention have good attack performance and attack migration ability, which can produce adversarial effects on image classifiers and target detectors constructed with different network models, and take into account the robustness, adversarialness and appearance constraints of adversarial samples: by introducing three-dimensional models and neural rendering technology, the adversarial attack technology is extended to the three-dimensional field. Compared with traditional two-dimensional adversarial samples, three-dimensional adversarial samples have excellent robustness and relevance to the real world; based on the visualization interpretation technology of neural networks, the attention heat map of the network model is obtained, and by optimizing the texture of the adversarial samples, the attention distribution of the model is dispersed, thereby affecting the feature extraction ability of the model, reducing the recognition accuracy of the model, and making the three-dimensional adversarial samples have excellent adversarial performance; through texture constraints, the similarity between the adversarial samples and the clean samples is controlled, so that the three-dimensional adversarial samples have better visual features. The present invention is a white-box adversarial attack method based on the model attention distraction method. Compared with the black-box adversarial attack method, the adversarial samples generated by this method have more advantages in performance. After several experimental analyses, ResNet-50 is selected as the white-box network model. The network model is moderate in structural complexity and depth. The generated adversarial samples have excellent adversarial and adversarial transferability, and can achieve cross-model adversarial, including the adversarial between image classifiers and target detectors, effectively avoiding the adversarial overfitting problem under the white-box attack setting. Based on the original three-dimensional model, the present invention designs a method for generating three-dimensional adversarial samples along the idea of ​​optimizing texture to reduce loss. The method does not limit the three-dimensional model and loss weight. It can balance the adversarial performance and visual features of different samples by changing the model and loss weight, and supports users to customize the initial texture pattern of the three-dimensional target model. The generated adversarial samples will retain the initial texture visual features to a large extent, have rich application scenarios, and can meet the adversarial needs in different environments, so as to have high competitiveness and application value in the field of artificial intelligence adversarial attacks.

[0158] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

Claims

1. A three-dimensional adversarial sample generation method based on model distraction, characterized in that: include: S1, collecting data sets of 3D target models in different virtual scenes, and performing neural rendering on the same group of data in the data sets to generate scene images containing the target; S2. Use the white box model to analyze the scene image, calculate the model's attention distraction loss, and calculate the texture constraint loss and smoothness loss of the adversarial sample by generating target foreground images of different samples; S3, integrate the model's attention distraction loss, texture constraint loss and smoothness loss, calculate the total model loss, set the optimization object and optimization goal, and use the optimization algorithm for optimization update; S4. Use the dataset for iterative training to obtain optimized three-dimensional adversarial samples.

2. According to the method for generating three-dimensional adversarial samples based on model distraction according to claim 1, it is characterized in that: The collecting of data sets of the three-dimensional target model in different virtual scenes, and performing neural rendering on the same group of data in the data sets to generate a scene image containing the target includes: S11, placing a three-dimensional target model of a vehicle in a simulator environment, setting different virtual scenes, using a camera sensor to collect scene images of the three-dimensional target model under multiple perspectives, and reading the position information of the three-dimensional target model and the camera sensor as rendering parameters; S1 2. Merge each set of corresponding scene images and rendering parameters to form a data set; S1 3. Read the vertex, face and texture data of the three-dimensional target model, replace the original texture data with the optimized texture data, read the scene image and rendering parameters of the same group in the data set, render with the neural renderer, and generate the foreground image and mask image of the target; S1 4. Load the scene image of the same group of data in the data set, combine the foreground image and mask image of the target, and generate a scene image containing the target.

3. According to the method of generating three-dimensional adversarial samples based on model distraction according to claim 1, it is characterized in that: The method of analyzing the scene image using the white box model, calculating the model attention distraction loss, and calculating the texture constraint loss and smoothness loss of the adversarial sample by generating target foreground images of different samples includes: S2 1. Select the ResNet-50 network as the white box model, use the white box model to analyze the attention distribution of the scene image, and obtain the model attention heat map; S22. Calculate the model attention distraction loss based on the model attention heat map; S23, extracting a set of rendering parameters from the data set, inputting the 3D target model and texture through neural rendering, and generating the target foreground image of the clean sample and the adversarial sample under the same rendering parameters; S24. Based on the target foreground images of clean samples and adversarial samples, measure the visual gap between adversarial samples and clean samples, and calculate the texture constraint loss and smoothness loss of adversarial samples.

4. The method for generating three-dimensional adversarial samples based on model distraction according to claim 3, characterized in that: The ResNet-50 network is selected as the white box model, and the attention distribution of the scene image is analyzed using the white box model to obtain the model attention heat map including: S2 11. Extract the feature map output by the convolutional block of the white-box model during the forward propagation process; S212, using the back propagation of the white box model to obtain all channel weights of the target category in the feature map; S2 1 3. Combine the feature maps of each channel with the corresponding weights to obtain the attention distribution map of the white-box model for the input scene image as the model attention heat map.

5. The method for generating three-dimensional adversarial samples based on model distraction according to claim 3, characterized in that: The formula for calculating the distraction loss of the model is: Where, L a is the attention loss of the model; K is the number of attention regions in the model’s attention heat map; V k is the sum of the pixel values ​​of the kth attention area in the model attention heat map; W is the width of the model attention heat map; H is the height of the model attention heat map; S k is the area size of the kth attention region in the model's attention heat map.

6. The method for generating three-dimensional adversarial samples based on model distraction according to claim 3, characterized in that: The formula for calculating the texture constraint loss and smoothness loss of the adversarial sample includes: Where, L s is the texture constraint loss; I obj is the target foreground image of the adversarial sample; I obj0 is the target foreground image of the clean sample; L m is the smoothing loss; x i，j is the pixel value at coordinate (i, j) in the target foreground image of the adversarial sample; x i+1，j is the pixel value at coordinate (i+1, j) in the target foreground image of the adversarial sample; x i，j+1 is the pixel value at coordinate (i, j+1) in the target foreground image of the adversarial sample.

7. The method for generating three-dimensional adversarial samples based on model distraction according to claim 1, characterized in that: The comprehensive model attention distraction loss, texture constraint loss and smoothness loss are calculated to calculate the total loss of the model, and the optimization object and optimization goal are set. The optimization algorithm is used for optimization update, including: S3 1. Assigning respective weight values ​​to the model attention distraction loss, texture constraint loss and the smoothness loss, and calculating the total loss of the model by weighted fusion; S32. The adversarial sample texture is set as the optimization object, the minimization of the total model loss of the adversarial sample texture is set as the optimization goal, and the multi-strategy fusion Beluga optimization algorithm is used for optimization and update to obtain the optimal adversarial sample texture.

8. The method for generating three-dimensional adversarial samples based on model distraction according to claim 7, characterized in that: The optimization update using multi-strategy fusion Beluga optimization algorithm to obtain the optimal adversarial sample texture includes: S321. Using chaotic mapping to generate the initial population position, each beluga individual represents a different candidate perturbation of the adversarial sample texture, and using the total model loss to calculate the fitness value of each population individual; S322. Use a modified decreasing control mechanism to calculate the balance factor between the exploration stage and the development stage, and use the cosine function to optimize the trend of the whale fall probability; S323, in the exploration stage, the fusion Cauchy mutation operator is used to perform global position enhancement and calculate the position information of the population individuals in the exploration stage; S324. In the development phase, an adaptive inertia weight factor is introduced according to fitness, the position information of the individuals in the population in the development phase is calculated, and the best individual in the current population is selected; S325. When the maximum number of iterations is reached, stop the optimization and output the optimized adversarial sample texture.

9. The method for generating three-dimensional adversarial samples based on model distraction according to claim 8, characterized in that: The formula for calculating the position information of population individuals in the exploration phase is: In the formula, is the position information of individual i in the exploration phase at iteration t+1; is the optimal individual position; Cauchy(0,1) is the fused Cauchy mutation operator; The formula for calculating the position information of population individuals in the exploration phase is: In the formula, is the position information of individual i in the development phase at iteration t+1; is the position information of individual i in the population at iteration t; is the position information of other whale individuals r at iteration t; C is the random jump intensity; r1 and r2 are both random numbers between (0, 1); L F is the levy flight function; ω z is the adaptive inertia weight factor.

10. The method for generating three-dimensional adversarial samples based on model distraction according to claim 1, characterized in that: The iterative training using the data set to obtain the optimized generated three-dimensional adversarial sample includes: S41, sequentially extracting rendering parameters and scene image input value neural renderers of other groups in the data set, combining with the three-dimensional target model, to generate a foreground image and a scene image of the target; S42, based on different foreground images and scene images, the multi-strategy fusion Beluga optimization algorithm is used to optimize and update the adversarial sample texture to obtain the optimal adversarial sample texture for each set of data; S43. When the number of adversarial sample texture optimizations reaches a preset number of iterations, the finally optimized adversarial sample texture is applied to the three-dimensional target model to obtain a three-dimensional adversarial sample.

Images