Method for generating adversarial samples, method, device and equipment for training neural network

Abstract

The embodiments of the present application disclose a method for generating adversarial samples, a method, an apparatus and a device for training a neural network, which include: obtaining a first gradient map of a target image, wherein the first gradient map contains the gradient of a loss function of the neural network to pixels in the target image, and the loss function is obtained by taking the target image as input of the neural network; transforming the first gradient map to obtain a second gradient map, and the second gradient map is used to make the target image affine invariant; adjusting the pixels of the target image based on the gradients of the pixels in the second gradient map to obtain a target image with affine invariance, and using the target image with affine invariance as an adversarial sample; the adversarial sample generated by this method still has a good attack effect even after affine transformation, thereby facilitating the discovery of deficiencies in the neural network attacked by the adversarial sample, so as to improve the robustness of the neural network.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of artificial intelligence technology, and in particular to a method for generating adversarial samples, a method, device and equipment for training a neural network. Background Art

[0002] Deep Learning (DL) is a new research direction in the field of Machine Learning (ML). It is introduced into machine learning to make it closer to its original goal - Artificial Intelligence (AI). With the development of deep learning technology, Deep Neural Networks (DNN) have been widely used in various fields.

[0003] However, recent studies have shown that deep neural networks are very vulnerable to adversarial examples, which significantly reduce the accuracy of image recognition. Adversarial examples refer to input samples formed by deliberately adding subtle interference to the data set, causing the neural network to give an incorrect output with high confidence. Figure 1 As shown, the neural network has 57.7% confidence in Figure 1 The original image shown is identified as a panda, but if Figure 1 Adding a disturbance to the image shown in the figure gives Figure 1 The perturbed image is shown in Figure 1. Although the perturbed image does not change visually compared to the original image, the neural network recognizes the perturbed image as a gibbon with 99.3% confidence. This shows that the existence of adversarial samples may cause security problems in many existing practical applications based on deep neural networks, and therefore has attracted great attention.

[0004] At present, adversarial attack algorithms are used to train neural networks. Among them, adversarial attack algorithms refer to the use of adversarial samples to attack neural networks during the training process to improve the recognition accuracy of neural networks for images. Attacks on neural networks using adversarial samples are mainly divided into two categories: white-box attacks, gray-box attacks, and black-box attacks. White-box attacks refer to attackers who know all the information about the neural network and can generate corresponding adversarial samples based on the information of the neural network to attack the neural network. Black-box attacks are the opposite of white-box attacks, which means that attackers know nothing about the internal structure, training parameters, defense methods, etc. of the attacked neural network, so they cannot use adversarial samples generated based on the information of the neural network to attack the neural network. Gray-box attacks are between white-box attacks and black-box attacks, which means that attackers only know part of the information of the neural network, so they can only use adversarial samples generated based on the limited information of the neural network to attack the neural network.

[0005] To this end, during the training process, the image is first input into the white-box model, then the perturbation is calculated, and finally an adversarial sample is generated based on the image and the perturbation. Based on the transferability of the adversarial sample, the adversarial sample can be used for black-box attack or gray-box attack; the white-box model refers to a neural network whose internal structure, training parameters, defense methods, etc. are known.

[0006] Therefore, a method is needed to generate adversarial samples with good transferability. Summary of the invention

[0007] The embodiments of the present application provide a method for generating adversarial samples, a method, apparatus and equipment for training neural networks. The method ensures that the generated adversarial samples still have a good attack effect even after affine transformation, thereby facilitating the discovery of deficiencies in the neural network attacked by the adversarial samples and improving the robustness of the neural network.

[0008] A first aspect of an embodiment of the present application provides a method for generating an adversarial sample, including:

[0009] A first gradient map of a target image is obtained. The target image can be represented by a pixel matrix. Each element in the pixel matrix represents a pixel of the target image. The pixel can be represented by a single channel or a three-channel. The first gradient map contains the gradient of the loss function of the neural network to the pixels in the target image. The loss function is obtained by taking the target image as the input of the neural network. The first gradient map is transformed to obtain a second gradient map. The second gradient map is used to make the target image affine invariant. Affine invariance can be understood as the characteristic that the target image can still maintain its original properties after affine transformation. The pixels of the target image are adjusted based on the gradients of the pixels in the second gradient map to obtain a target image with affine invariance, and the target image with affine invariance is used as an adversarial sample.

[0010] The first gradient map of the acquired target image is transformed to obtain a second gradient map. Since the second gradient map can make the target image affine invariant, the pixels of the target image are adjusted based on the gradients of the pixels in the second gradient map to obtain an adversarial sample with affine invariance. In this way, the generated adversarial sample still has a good attack effect even after affine transformation, which is conducive to discovering the deficiencies of the neural network attacked by the adversarial sample, so as to improve the robustness of the neural network.

[0011] As an implementation method, transforming the first gradient map to obtain the second gradient map includes: transforming the first gradient map to obtain a third gradient map, the third gradient map is used to make the target image have translation invariance, wherein the translation invariance can be understood as the characteristic that the target image can still maintain the original properties after the translation transformation; transforming the third gradient map to obtain a second gradient map, the second gradient map is used to make the target image have affine invariance.

[0012] The affine transformation is composed of a translation transformation and a linear transformation. Correspondingly, in this implementation, a third gradient map with translation invariance is first obtained by transformation, and then the third gradient map is transformed to obtain a second gradient map with affine invariance.

[0013] As an implementation method, transforming the third gradient map to obtain the second gradient map includes: performing polar coordinate transformation on the third gradient map, the polar coordinate transformation is used to convert the gradient of the pixels in the third gradient map from a rectangular coordinate system to a polar coordinate system; transforming the third gradient map after the polar coordinate transformation to obtain a fourth gradient map, the fourth gradient map is used to make the target image affine invariant, and the transformation of the third gradient map may include multiple transformations; performing polar coordinate inverse transformation on the fourth gradient map to obtain the second gradient map, the polar coordinate inverse transformation is used to convert the gradient of the pixels passing through the fourth gradient map from a polar coordinate system to a rectangular coordinate system.

[0014] In an embodiment of the present application, the gradient of the pixels in the third gradient map is converted from a rectangular coordinate system to a polar coordinate system through a polar coordinate transformation. The third gradient map in the polar coordinate system allows rotational invariance transformation and scaling invariance transformation to be directly implemented through convolution operations without the need for additional complex calculations. If the third gradient map is not subjected to polar coordinate transformation, rotational invariance transformation and scaling invariance transformation cannot be directly implemented through convolution operations. Therefore, the embodiment of the present application can reduce the amount of calculation, thereby reducing overhead and improving the efficiency of generating adversarial samples.

[0015] As an implementation method, adjusting the pixels of the target image based on the gradients of the pixels in the second gradient map to obtain a target image with affine invariance includes: calculating the perturbation amount of the pixels of the target image based on the gradients of the pixels in the second gradient map, wherein the perturbation amount of the pixels represents the adjustment amplitude of the pixels; and adjusting the pixels of the target image based on the perturbation amount of the pixels of the target image to obtain a target image with affine invariance.

[0016] Based on the gradient of the pixels in the second gradient map, the perturbation amount of the pixels of the target image is calculated, and then the pixels of the target image are adjusted based on the perturbation amount, thereby achieving the generation of adversarial samples.

[0017] As an implementation method, the gradient of the pixel in the second gradient map belongs to the current iterative calculation; based on the gradient of the pixel in the second gradient map, calculating the perturbation amount of the pixel of the target image includes: performing weighted sum processing on the gradient of the pixel in the second gradient map and the perturbation amount in the iterative calculation before the current iterative calculation, to obtain the perturbation amount of the pixel of the target image.

[0018] The perturbation amount in the previous iterative calculation and the gradient of the pixel in the second gradient map in the current iterative calculation are weighted so that the perturbation amount in the previous iterative calculation can play a corrective role in the current iterative calculation, that is, it can prevent the perturbation amount generated according to the gradient of the pixel in the second gradient map from being too targeted to the target image, causing the current perturbation amount to produce an overly specific change to the target image, thereby reducing the mobility of the generated adversarial sample.

[0019] As an implementation method, the gradient of the pixel in the second gradient map belongs to the current iterative calculation; based on the gradient of the pixel in the second gradient map, calculating the perturbation amount of the pixel of the target image includes: normalizing the gradient of the pixel in the second gradient map, and there are multiple methods for normalization; performing weighted summation processing on the gradient of the pixel in the second gradient map after normalization and the perturbation amount in the iterative calculation before the current iterative calculation, to obtain the perturbation amount of the pixel of the target image.

[0020] The gradient of the pixel in the second gradient map after normalization and the perturbation amount in the previous iterative calculation of the current iterative calculation are weighted and summed, so that the perturbation amount in the previous iterative calculation can play a corrective role in the current iterative calculation, that is, it can prevent the perturbation amount generated according to the gradient of the pixel in the second gradient map after normalization from being too targeted to the target image, causing the current perturbation amount to produce overly specific changes to the target image, so as to reduce the mobility of the generated adversarial samples; in addition, the gradient of the pixel in the gradient map after affine transformation is normalized, so as to control the gradient of the pixel in the gradient map after affine transformation within a certain range, so as to prevent the gradient of the pixel in the gradient map after affine transformation from being too different from the perturbation amount in the previous iterative calculation, so as to achieve the aforementioned correction effect through the weighted summation of the two.

[0021] As an implementation method, normalizing the gradients of pixels in the second gradient map includes: for any gradient in the second gradient map, obtaining the ratio of the gradient to the sum of all gradients in the second gradient map, and using the ratio as the gradient of the normalized pixel.

[0022] As an implementation method, normalizing the gradients of pixels in the second gradient map includes: for any gradient in the second gradient map, obtaining the ratio of the gradient to the maximum gradient in the second gradient map, and using the ratio as the gradient of the normalized pixel.

[0023] As an implementation method, adjusting the pixels of the target image based on the disturbance amount of the pixels of the target image to obtain the target image with affine invariance includes: obtaining the sign value of the disturbance amount of the pixels of the target image based on a sign function, specifically, when the disturbance amount is greater than 0, the corresponding sign value is 1; when the disturbance amount is equal to 0, the corresponding sign value is 0, and when the disturbance amount is less than 0, the corresponding sign value is -1; based on the product of the sign value and the disturbance amplitude, adjusting the pixels of the target image to obtain the target image with affine invariance.

[0024] The sign value of the disturbance amount of the pixel of the target image is obtained based on the sign function, so that the adjustment range of the pixel in the target image can be limited between -1 and 1. The pixel of the target image is adjusted based on the product of the sign value and the disturbance range, which can further limit the adjustment range of the pixel in the target image.

[0025] As an implementation method, obtaining the first gradient map of the target image includes: performing a scaling transformation on the target image, where the scaling transformation refers to reducing or increasing the number of pixels of the target image to reduce or increase the size of the image; using the scaled target image as the input of the neural network to obtain the loss function of the neural network; and back-propagating the neural network based on the loss function to obtain the first gradient map of the target image.

[0026] First, the target image is scaled and then input into the neural network to calculate the first gradient map of the target image. Since the scaling transformation can enhance the diversity of the target image, the adversarial sample is obtained by using the first gradient map obtained by this method. This can make the acquired adversarial sample have better transferability, thus ensuring that the adversarial sample has better robustness under scaling transformation.

[0027] A second aspect of an embodiment of the present application provides a method for training a neural network, comprising: training the neural network using adversarial samples generated by any generation method of the first aspect of the embodiment of the present application.

[0028] A third aspect of an embodiment of the present application provides a device for generating an adversarial sample, including:

[0029] A gradient map acquisition unit, used to acquire a first gradient map of a target image, wherein the first gradient map includes a gradient of a loss function of a neural network to pixels in the target image, and the loss function is acquired by taking the target image as an input of the neural network;

[0030] A transformation unit, used for transforming the first gradient map to obtain a second gradient map, wherein the second gradient map is used to make the target image have affine invariance;

[0031] An adjustment unit is used to adjust the pixels of the target image based on the gradients of the pixels in the second gradient map to obtain a target image with affine invariance, and use the target image with affine invariance as an adversarial sample.

[0032] As an implementation method, the gradient acquisition unit is used to transform the first gradient map to obtain a third gradient map, and the third gradient map is used to make the target image have translation invariance; transform the third gradient map to obtain a second gradient map, and the second gradient map is used to make the target image have affine invariance.

[0033] As an implementation method, the gradient acquisition unit is used to perform polar coordinate transformation on the third gradient map, and the polar coordinate transformation is used to convert the gradient of the pixels in the third gradient map from a rectangular coordinate system to a polar coordinate system; transform the third gradient map after the polar coordinate transformation to obtain a fourth gradient map, and the fourth gradient map is used to make the target image affine invariant; perform polar coordinate inverse transformation on the fourth gradient map to obtain a second gradient map, and the polar coordinate inverse transformation is used to convert the gradient of the pixels passing through the fourth gradient map from a polar coordinate system to a rectangular coordinate system.

[0034] As an implementation method, the adjustment unit is used to calculate the perturbation amount of the pixels of the target image based on the gradient of the pixels in the second gradient map; and adjust the pixels of the target image based on the perturbation amount of the pixels of the target image to obtain a target image with affine invariance.

[0035] As an implementation method, the gradient of the pixel in the second gradient map belongs to the current iterative calculation; the adjustment unit is used to perform weighted sum processing on the gradient of the pixel in the second gradient map and the disturbance amount in the previous iterative calculation of the current iterative calculation to obtain the disturbance amount of the pixel of the target image.

[0036] As an implementation method, the gradient of the pixel in the second gradient map belongs to the current iterative calculation; the adjustment unit is used to normalize the gradient of the pixel in the second gradient map; the gradient of the pixel in the second gradient map after normalization and the perturbation amount in the iterative calculation before the current iterative calculation are weighted summed to obtain the perturbation amount of the pixel of the target image.

[0037] As an implementation method, the adjustment unit is used to obtain the sign value of the disturbance amount of the pixel of the target image based on the sign function; based on the product of the sign value and the disturbance amplitude, the pixel of the target image is adjusted to obtain a target image with affine invariance.

[0038] As an implementation method, the gradient map acquisition unit is used to scale the target image; use the scaled target image as the input of the neural network to obtain the loss function of the neural network; and back-propagate the neural network based on the loss function to obtain a first gradient map of the target image.

[0039] Among them, for the specific implementation, relevant instructions and technical effects of the above units, please refer to the description of the first aspect of the embodiment of this application.

[0040] The fourth aspect of an embodiment of the present application provides a computer device, comprising: one or more processors and a memory; wherein the memory stores computer-readable instructions; the one or more processors read the computer-readable instructions to enable the training device to implement the method described in any implementation method of the first aspect or the second aspect.

[0041] A fifth aspect of an embodiment of the present application provides a computer-readable storage medium, including computer-readable instructions. When the computer-readable instructions are executed on a computer, the computer executes the method described in any implementation of the first aspect or the second aspect.

[0042] A sixth aspect of the present application provides a chip, comprising one or more processors, some or all of which are used to read and execute a computer program stored in a memory to execute the method in any possible implementation of the first or second aspect.

[0043] Optionally, the chip includes a memory, and the memory is connected to the processor through a circuit or a wire. Further optionally, the chip also includes a communication interface, and the processor is connected to the communication interface. The communication interface is used to receive data and / or information to be processed, and the processor obtains the data and / or information from the communication interface, processes the data and / or information, and outputs the processing result through the communication interface. The communication interface can be an input and output interface.

[0044] In some implementations, some of the one or more processors may implement some steps of the above method by means of dedicated hardware. For example, processing involving a neural network model may be implemented by a dedicated neural network processor or a graphics processor.

[0045] The method provided in the embodiment of the present application can be implemented by one chip or by multiple chips working together.

[0046] A seventh aspect of an embodiment of the present application provides a computer program product, which includes computer software instructions, and the computer software instructions can be loaded by a processor to implement the method described in any one of the implementation methods in the first aspect or the second aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 Schematic diagram of the original image and the image after adding disturbance;

[0048] Figure 2 Schematic diagram of the process of generating adversarial examples for existing technologies;

[0049] Figure 3 A structural diagram of the main framework of artificial intelligence;

[0050] Figure 4 A schematic diagram of a driving scenario provided in an embodiment of the present application;

[0051] Figure 5 A schematic diagram of a scenario for detecting the performance of a neural network provided in an embodiment of the present application;

[0052] Figure 6 A schematic diagram of training an object detection neural network using adversarial examples;

[0053] Figure 7 A system architecture diagram of a task processing system provided in an embodiment of the present application;

[0054] Figure 8 A schematic diagram of an embodiment of a method for generating an adversarial sample is provided for an embodiment of the present application;

[0055] Fig. 9 A schematic diagram of an embodiment of obtaining a first gradient map according to an embodiment of the present application;

[0056] Fig.10 A schematic diagram of an embodiment of obtaining a second gradient map according to an embodiment of the present application;

[0057] Fig.11 A schematic diagram of another embodiment of obtaining a second gradient map according to an embodiment of the present application;

[0058] Fig.12 A schematic diagram of an embodiment of generating adversarial samples for an embodiment of the present application;

[0059] Fig.13 A schematic diagram of an embodiment of calculating the disturbance amount in an embodiment of the present application;

[0060] Fig.14 A schematic diagram of a process for generating adversarial samples according to an embodiment of the present application;

[0061] Fig.15 is the original image in the embodiment of the present application;

[0062] Fig.16 For Fig.15 The image obtained by performing a rotation transformation on the original image shown;

[0063] Fig.17 For Fig.15 The image obtained by performing translation transformation on the original image shown;

[0064] Fig.18 For Fig.15 The image obtained by scaling the original image shown;

[0065] Fig.19 In order to adopt the method of the embodiment of the present application and based on Fig.15 The original image shown generates adversarial examples;

[0066] Fig. 20 For Fig.19 The image obtained by rotating the adversarial sample shown;

[0067] Fig.21 For Fig.19 The image obtained by translation transformation of the adversarial sample shown;

[0068] Fig. 22 For Fig.19 The image obtained by scaling the adversarial sample shown;

[0069] Fig.23 A schematic diagram of an embodiment of a device for generating adversarial samples provided in an embodiment of the present application;

[0070] Fig.24 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0071] The embodiments of the present application provide a method for generating adversarial samples, a method, apparatus and equipment for training neural networks. The method ensures that the generated adversarial samples still have a good attack effect even after affine transformation, thereby facilitating the discovery of deficiencies in the neural network attacked by the adversarial samples and improving the robustness of the neural network.

[0072] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0073] First of all, based on the attacker's understanding of the neural network, neural networks can be divided into white-box models, black-box models and gray-box models; the white-box model refers to a neural network whose internal structure, training parameters, defense methods, etc. are known, the black-box model refers to a neural network whose internal structure, training parameters, defense methods, etc. are unknown, and the gray-box model refers to a neural network whose internal structure, training parameters, defense methods, etc. are partially known.

[0074] For white-box models, since the internal structure, training parameters, defense methods, etc. are all known, effective adversarial samples can be generated based on this information; for black-box models and gray-box models, since they do not know the information of the neural network or only know part of the information of the neural network, they cannot generate effective adversarial samples based on the information of the neural network to effectively attack the neural network.

[0075] To this end, a white-box model is usually used to generate attack samples, and then the generated attack samples are used to attack the gray-box model or the black-box model based on the transferability of the attack samples. The attack on the gray-box model can also be called a gray-box attack, and the attack on the black-box model can also be called a black-box attack. The specific process is as follows.

[0076] like Figure 2 As shown, the initial sample is input into the white-box model, and then the perturbation is calculated based on the output of the white-box model. The perturbation is then added to the initial sample to obtain an adversarial sample. Based on the transferability of the adversarial sample, the adversarial sample can be used for gray-box attacks or black-box attacks.

[0077] It should be noted that migration can also be called migration capability, which specifically refers to the ability of attack samples obtained based on the white-box model to maintain the attack success rate on the gray-box model or the black-box model.

[0078] In order to ensure that the generated adversarial samples have good transferability, an embodiment of the present application provides a method for generating adversarial samples, which is based on the loss function of the neural network. The method calculates the gradient of the initial sample of the neural network, then performs a radial transformation on the gradient, and finally adjusts the initial sample according to the gradient after the affine transformation, thereby obtaining the adversarial sample; this method enables the generated adversarial sample to have affine invariance, and affine invariance can be understood as the adversarial sample still has a high attack success rate on the neural network after the affine transformation, which is equivalent to ensuring that the generated adversarial sample has good transferability.

[0079] The initial sample may be an image.

[0080] The technical solutions in the embodiments of the present invention will be described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0081] First, the overall workflow of the AI ​​system is described in Figure 3 , Figure 3 What is shown is a structural diagram of the main framework of artificial intelligence. The following explains the above artificial intelligence theme framework from two dimensions: "intelligent information chain" (horizontal axis) and "IT value chain" (vertical axis).

[0082] Among them, the "intelligent information chain" reflects a series of processes from data acquisition to processing. For example, it can be a general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, intelligent execution and output. In this process, data undergoes a condensed process of "data-information-knowledge-wisdom".

[0083] The "IT value chain" reflects the value that artificial intelligence brings to the information technology industry, from the underlying infrastructure of human intelligence, information (providing and processing technology implementation) to the system's industrial ecological process.

[0084] (1) Infrastructure:

[0085] The infrastructure provides computing power support for the AI ​​system, enables communication with the outside world, and supports it through the basic platform. It communicates with the outside world through sensors; computing power is provided by smart chips (CPU, NPU, GPU, ASIC, FPGA and other hardware acceleration chips); the basic platform includes distributed computing frameworks and networks and other related platform guarantees and support, which can include cloud storage and computing, interconnected networks, etc. For example, sensors communicate with the outside world to obtain data, and these data are provided to the smart chips in the distributed computing system provided by the basic platform for calculation.

[0086] (2) Data

[0087] The data on the upper layer of the infrastructure is used to represent the data sources in the field of artificial intelligence. The data involves graphics, images, voice, text, and IoT data of traditional devices, including business data of existing systems and perception data such as force, displacement, liquid level, temperature, and humidity.

[0088] (3) Data processing

[0089] Data processing usually includes data training, machine learning, deep learning, search, reasoning, decision-making and other methods.

[0090] Among them, machine learning and deep learning can symbolize and formalize data for intelligent information modeling, extraction, preprocessing, and training.

[0091] Reasoning refers to the process of simulating human intelligent reasoning in computers or intelligent systems, using formalized information to perform machine thinking and solve problems based on reasoning control strategies. Typical functions are search and matching.

[0092] Decision-making refers to the process of making decisions after intelligent information is reasoned, usually providing functions such as classification, sorting, and prediction.

[0093] (4) General ability

[0094] After the data has undergone the data processing mentioned above, some general capabilities can be further formed based on the results of the data processing, such as an algorithm or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0095] (5) Smart products and industry applications

[0096] Smart products and industry applications refer to the products and applications of artificial intelligence systems in various fields. They are the encapsulation of the overall artificial intelligence solution, which productizes intelligent information decision-making and realizes practical application. Its application areas mainly include: smart manufacturing, smart transportation, smart home, smart medical care, smart security, autonomous driving, smart city, smart terminal, etc.

[0097] The embodiments of the present application can be applied to the optimization design of the network structure of a neural network, and the neural network trained by the present application can be specifically applied in various sub-fields of the field of artificial intelligence, such as image recognition, medical image analysis, and autonomous driving.

[0098] by Figure 3 For example, the data in the data set acquired by the infrastructure in the embodiment of the present application can be multiple data of different types (also called training data, multiple training data constitute a training set) acquired through sensors such as cameras and radars, or it can be multiple image data or multiple video data.

[0099] Combine the following Figure 4 A specific application scenario of an embodiment of the present application is introduced.

[0100] In the fields of autonomous driving and assisted driving, it is necessary to obtain information about objects in the scene in real time, such as whether there are pedestrians crossing the road in front of the car or whether there are other vehicles blocking the current route, and then make corresponding decisions based on the information obtained. Specifically, in the autonomous driving system and assisted driving system, images or videos containing objects (such as pedestrians and cars) are input into the object detection neural network, and the object detection neural network outputs the category and location of the object. Finally, the decision system makes corresponding feedback based on the category and location of the object.

[0101] In this scenario, if Figure 5 As shown, the method provided in the embodiment of the present application can be used to generate corresponding adversarial samples, and the object detection neural network can be attacked by the adversarial samples. Then, statistical analysis is performed based on the attack results to obtain the performance of the object detection neural network for the adversarial samples, so as to evaluate the robustness of the object detection neural network based on the performance.

[0102] In this scenario, if Figure 6 As shown, the method provided in the embodiment of the present application can also be used to generate corresponding adversarial samples, and the adversarial samples and training sets (including multiple training samples) are used to train the object detection neural network, thereby obtaining an object detection neural network with better robustness.

[0103] Based on the relevant descriptions of the above application scenarios, it can be known that the adversarial samples generated by the method provided in the embodiments of the present application can be used to attack the neural network to detect the robustness of the neural network, and the adversarial samples generated by the method provided in the embodiments of the present application can be used to train the neural network to obtain a neural network with better robustness.

[0104] It should be noted that when the adversarial sample generated by the method provided in the embodiment of the present application is used to attack the neural network, the adversarial sample can be specifically applied to targeted attacks and untargeted attacks. A targeted attack means that the defense model (i.e., the attacked model) classifies the adversarial sample into the target category and the attack is considered successful. An untargeted attack means that the defense model classifies the adversarial sample into any category except the correct category and the attack is considered successful.

[0105] When training a neural network using adversarial samples generated by the method provided in the embodiment of the present application, the method provided in the embodiment of the present application can be combined with any other method for generating adversarial samples to generate more effective adversarial samples. For example, the method provided in the embodiment of the present application can be combined with the fast gradient sign method, the basic iterative method, the momentum iterative fast gradient sign method, the input diversity iterative fast gradient sign method, the translation invariant attack method, etc.

[0106] Among them, the fast gradient sign method: applies linear features to nonlinear models, derives the input image through the loss function of the neural network, back-propagates the derived gradient, and generates a perturbation direction about the input image; multiplies the maximum infinite norm of the perturbation in the perturbation direction to generate a certain perturbation; superimposes the perturbation with the input image to obtain an adversarial sample, which can be directly used to attack the neural network.

[0107] Basic Iterative Method: It is based on the fast gradient symbol method, that is, the gradient optimization method is changed from single-step derivation to iterative solution. Instead of using a single step length for calculation, it is divided into multiple small-step iterations, and the results of multiple iterations are clipped to prevent exceeding the upper limit of the perturbation. The attack performance of the basic iterative method is better than that of the fast gradient symbol method, but the efficiency of generating adversarial samples is reduced.

[0108] Momentum Iterative Fast Gradient Sign Method: Based on the basic iterative method, the concept of momentum is introduced into the generation process of adversarial samples. After each step of iterative perturbation, the optimization direction will be corrected to avoid falling into the local optimal value.

[0109] Input Diversity Iterative Fast Gradient Sign Method: It enhances the transferability of the generated adversarial samples by performing scaling and padding transformations on the input image with specific probabilities.

[0110] Translation invariant attack method: Compared with the several basic attack methods mentioned above, a convolution operation is performed after each gradient operation to achieve a translation transformation, so that the generated adversarial samples have strong mobility.

[0111] See also Figure 7 , Figure 7 A system architecture diagram of a task processing system provided in an embodiment of the present application, in Figure 7 In the example, the task processing system 100 includes an execution device 110, a training device 120, a database 130, a client device 140, a data storage system 150 and a data acquisition device 160. The execution device 110 includes a computing module 111. The data acquisition device 160 is used to obtain an open source large-scale data set (i.e., a training set) required by the user, and store the training set in the database 130. The training device 120 trains the target model / rule 101 based on the training set maintained in the database 130, and the trained neural network obtained by the training is then used on the execution device 110. The execution device 110 can call the data, code, etc. in the data storage system 150, and can also store data, instructions, etc. in the data storage system 150. The data storage system 150 can be placed in the execution device 110, or the data storage system 150 can be an external memory relative to the execution device 110.

[0112] Among them, the training set may include adversarial samples generated using the method provided in the embodiments of the present application.

[0113] The trained neural network obtained after training the target model / rule 101 by the training device 120 can be applied to different systems or devices (i.e., the execution device 110), which can be edge devices or end-side devices, such as mobile phones, tablets, laptops, monitoring systems (such as cameras), security systems, etc. Figure 7 In the embodiment, the execution device 110 is configured with an I / O interface 112 for data interaction with external devices, and a “user” can input data into the I / O interface 112 through a client device 140 . For example, the client device 140 may be a camera device of a monitoring system, and the target image captured by the camera device is input as input data to the computing module 111 of the execution device 110, and the computing module 111 detects the input target image and obtains the detection result, and then outputs the detection result to the camera device or directly displays it on the display interface of the execution device 110 (if any); in addition, in some embodiments of the present application, the client device 140 may also be integrated in the execution device 110, such as, when the execution device 110 is a mobile phone, the target task can be directly obtained through the mobile phone (such as the target image can be captured by the camera of the mobile phone, or the target video can be captured by the camera of the mobile phone, etc., and the target task is not limited here) or receives the target task sent by other devices (such as another mobile phone), and then the computing module 111 in the mobile phone detects the target task and obtains the detection result, and directly presents the detection result on the display interface of the mobile phone. The product form of the execution device 110 and the client device 140 is not limited here.

[0114] It is worth noting that Figure 7This is only a schematic diagram of a system architecture provided by an embodiment of the present application. The positional relationship between the devices, components, modules, etc. shown in the figure does not constitute any limitation. For example, in Figure 7 In the embodiment, the data storage system 150 is an external memory relative to the execution device 110. In other cases, the data storage system 150 may also be placed in the execution device 110. Figure 7 In the embodiment, the client device 140 is an external device relative to the execution device 110. In other cases, the client device 140 may also be integrated into the execution device 110.

[0115] Next, the method for generating adversarial samples provided in an embodiment of the present application is introduced.

[0116] like Figure 8 As shown, the embodiment of the present application provides an embodiment of a method for generating an adversarial sample, including:

[0117] Operation 201, obtaining a first gradient map of a target image, wherein the first gradient map includes a gradient of a loss function of a neural network with respect to pixels in the target image, and the loss function is obtained by taking the target image as an input of the neural network.

[0118] The target image can be represented by a pixel matrix, and each element in the pixel matrix represents a pixel of the target image.

[0119] Among them, there are two ways to represent pixels. One is a single-channel representation, that is, only a value between 0 and 255 is used to represent the color of the pixel; the other is a three-channel representation, that is, the value of the red channel, the value of the green channel and the value of the blue channel are used to represent the color of the pixel, that is, a set of RGB (red, green and blue) values ​​is used to represent the color of the pixel.

[0120] When a pixel is represented in a single-channel manner, the gradient of the pixel contains one value, and the gradient value of the pixel indicates the direction of change of the pixel on the channel; when the pixel is represented in a three-channel manner, the gradient of the pixel contains three values, and these three values ​​respectively indicate the direction of change of the pixel on the red channel, green channel, and blue channel.

[0121] Taking a three-channel pixel in a target image as an example, the pixel value of the pixel is (128, 0, 0). Correspondingly, the gradient value of the pixel can be (-0.5, 0.8, 2.3), where -0.5 in the gradient value represents the direction of change of the red channel value, 0.8 in the gradient value represents the direction of change of the green channel value, and 2.3 in the gradient value represents the direction of change of the blue channel value.

[0122] It should be noted that, usually, the first gradient map includes the gradients of all pixels in the target image. In other cases, the first gradient map may also include the gradients of only some pixels in the target image.

[0123] The process of obtaining the first gradient map of the target image is described below.

[0124] Specifically, the target image can be input into the neural network, the output of the neural network can be calculated through forward propagation, and then back propagation can be performed based on the output of the neural network to calculate the gradient of the loss function for the pixels in the target image; wherein, forward propagation refers to the order from the input layer to the output layer of the neural network, and the intermediate variables of the neural network are calculated and stored in sequence; back propagation refers to the order from the output layer to the input layer of the neural network, and the intermediate variables and gradients of the parameters of the neural network are calculated and stored in sequence. In the embodiment of the present application, the parameters mainly refer to the pixels in the target image.

[0125] It should be understood that an adversarial sample with good transferability means that after some transformation is performed on the adversarial sample, the adversarial sample can still maintain a high attack success rate on the neural network.

[0126] Therefore, in order to enhance the transferability of the generated adversarial samples, the target image can be scaled before being input into the neural network to increase the diversity of the target image, so that the generated adversarial samples can have better transferability.

[0127] Specifically, as an implementation method, Fig. 9 As shown, operation 201 includes:

[0128] Operation 301 : performing scaling transformation on a target image.

[0129] In operation 301, scaling transformation refers to reducing or increasing the number of pixels of a target image to reduce or increase the size of the image.

[0130] Exemplarily, the scaling transformation of the target image can be achieved through interpolation processing, wherein the interpolation processing is to use the grayscale values ​​of known neighboring pixels to generate the grayscale values ​​of unknown pixels, so as to regenerate an image with higher resolution from the original image.

[0131] In operation 302, the target image after scaling transformation is used as an input of the neural network to obtain a loss function of the neural network.

[0132] The target image after scaling transformation is input into the neural network, and then the loss function of the neural network can be obtained through forward propagation. It should be noted that the method of obtaining the loss function is a relatively mature technology and will not be described in detail in the embodiments of the present application.

[0133] In operation 303, back-propagation is performed on the neural network based on the loss function to obtain a first gradient map of the target image.

[0134] The gradient of the loss function for each pixel in the target image can be calculated through back propagation. It should be noted that back propagation is a relatively mature technology and will not be described in detail in the embodiments of the present application.

[0135] In this embodiment, the target image is first scaled and then input into a neural network to calculate a first gradient map of the target image. The adversarial sample is obtained by using the first gradient map obtained by this method. This enables the obtained adversarial sample to have better mobility, thereby ensuring that the adversarial sample has better robustness under scaling transformations.

[0136] In operation 202, the first gradient map is transformed to obtain a second gradient map, where the second gradient map is used to make the target image have affine invariance.

[0137] Since the second gradient map is used to make the target image affine invariant, the transformation performed on the first gradient map can also be called an affine invariant transformation.

[0138] In the embodiment of the present application, affine invariance can be understood as the characteristic that the target image can still maintain its original properties after affine transformation.

[0139] Among them, affine transformation refers to the process of performing a linear transformation (multiplying a matrix) and adding a translation transformation (adding a vector) in a vector space to transform into another vector space.

[0140] It should be noted that performing an affine transformation on an image means first performing a linear transformation on the image and then performing a translation transformation; in the embodiment of the present application, since the object to be processed is the gradient map of the target image rather than the target image itself, operation 202 may include: first performing a translation invariant transformation on the gradient map, and the gradient map after the translation invariant transformation can make the target image have translation invariance; then performing another transformation, and the gradient map after the translation invariant transformation and the transformation can make the target image have affine invariance.

[0141] Specifically, as a way to achieve this, Fig.10 As shown, operation 202 may specifically include:

[0142] In operation 401 , a first gradient map is transformed to obtain a third gradient map, where the third gradient map is used to make a target image translationally invariant.

[0143] Among them, translation invariance can be understood as the characteristic that the target image can still maintain its original properties after translation transformation; since the third gradient map is used to make the target image translation invariant, the transformation performed on the first gradient map can also be called translation invariant transformation.

[0144] Among them, the translation transformation refers to adding the specified horizontal offset and vertical offset to the pixel coordinates in the target image respectively; while the translation invariant transformation is different from the translation transformation, which refers to adjusting the gradient value of the pixel in the gradient map.

[0145] For a single-channel pixel, the gradient value corresponding to the pixel is adjusted; for a three-channel pixel, the three gradient values ​​in the three channel directions corresponding to the pixel are adjusted respectively. For example, if the gradient value of a pixel in the gradient map is (-0.5, 0.8, 2.3), then after the gradient map is translated and invariantly transformed, the gradient value of the pixel can be changed to (-0.2, 1.8, 2.4).

[0146] It should be noted that a variety of methods can be used to perform translation-invariant transformation on the gradient map; in the embodiment of the present application, a specific convolution kernel can be used to perform a convolution operation on the gradient map, thereby achieving translation-invariant transformation of the gradient map.

[0147] In operation 402, the third gradient map is transformed to obtain a second gradient map, where the second gradient map is used to make the target image have affine invariance.

[0148] It can be understood that the affine transformation is composed of a translation transformation and a linear transformation. Correspondingly, in order to obtain a second gradient map that can make the target image affine invariant, the first gradient map also needs to be transformed twice; these two transformations include the translation invariance transformation in operation 401 and the invariance transformation in operation 402. The invariance transformation corresponds to the linear transformation and can specifically include multiple transformations.

[0149] For example, the invariance transformation may include a rotation invariance transformation and / or a scaling invariance transformation; the rotation invariance transformation may be understood as a special transformation, and the gradient map after the transformation may make the target image rotation invariant, and the rotation invariance may be understood as the characteristic that the target image still has the original properties after the rotation transformation; similarly, the scaling invariance transformation may also be understood as a special transformation, and the gradient map after the transformation may make the target image scaling invariant, and the scaling invariance may be understood as the characteristic that the target image still has the original properties after the scaling transformation.

[0150] It should be noted that the transformation of the third gradient map may include multiple transformations, and when both length transformation and angle transformation are included, if the length transformation and angle transformation are directly performed on the third gradient map, a large amount of calculation may be caused. This is described in detail below.

[0151] First, it is assumed that the invariance transformation performed on the third gradient map includes a rotation invariance transformation and a scaling invariance transformation, and the rotation invariance transformation and the scaling invariance transformation make the target image have rotation invariance and scaling invariance.

[0152] A general method to make the target image specifically rotation invariant and scale invariant is to perform multiple rotation and scaling transformations on the target image within a limited range. After each rotation and scaling transformation within a limited range, the pixels of the target image are adjusted according to the gradient of the pixels of the target image after the rotation and scaling transformation according to the loss function; and multiple rotation and scaling transformations within a limited range are equivalent to aggregating the gradient values ​​of a sector-shaped area in the target image.

[0153] Therefore, the embodiment of the present application performs a rotation invariant transformation and a scaling invariant transformation on the third gradient map to replace the above-mentioned multiple limited-range rotation transformations and scaling transformations on the target image, that is, for each gradient value in the gradient map, it is updated to the aggregate value of the gradient values ​​of a fan-shaped area where the gradient value is located.

[0154] However, the rotation invariance transformation and scaling invariance transformation of the third gradient map are realized through convolution operation, while the gradient value of the fan-shaped area cannot be directly convolutionally operated using the convolution kernel, and the convolution operation can only be realized through additional complex calculations.

[0155] To this end, the embodiment of the present application transforms the third gradient map into a polar coordinate system through polar coordinate transformation, so as to map the gradient value of the sector area 2 to the gradient value of the matrix area.

[0156] Specifically, as a way to achieve this, Fig.11 As shown, operation 402 includes:

[0157] Operation 4021: performing polar coordinate transformation on the third gradient map. The polar coordinate transformation is used to convert the gradient of the pixels in the third gradient map from a rectangular coordinate system to a polar coordinate system.

[0158] It should be noted that the polar coordinate transformation of the third gradient image refers to establishing a plane rectangular coordinate system with the center pixel point of the target image as the origin and the width and height of the target image as the x-axis and y-axis respectively, and then mapping the gradient value of the coordinate (x, y) in the third gradient image in the plane rectangular coordinate system to the coordinate in the polar coordinate system. (∠(x, y) represents the angle between the line connecting the origin and this point and the positive direction of the x-axis).

[0159] It can be seen that before and after the polar coordinate transformation, the gradient in the third gradient map does not change, but the arrangement of the gradient in the gradient map changes.

[0160] Operation 4022: transform the third gradient map after polar coordinate transformation to obtain a fourth gradient map, where the fourth gradient map is used to make the target image have affine invariance.

[0161] Based on the above description, it can be known that the transformation of the third gradient map may include multiple transformations, and the embodiment of the present application does not specifically limit this.

[0162] The transformation of the third gradient map can be achieved by performing a convolution operation on the third gradient map using a specific convolution kernel.

[0163] Operation 4023: Perform an inverse polar coordinate transformation on the fourth gradient map to obtain a second gradient map. The inverse polar coordinate transformation is used to convert the gradient passing through the pixels in the fourth gradient map from a polar coordinate system to a rectangular coordinate system.

[0164] Among them, the polar coordinate inverse transformation is the inverse process of the polar coordinate transformation, and the specific understanding can refer to the relevant description of the polar coordinate transformation in operation 402. In the embodiment of the present application, the gradient of the pixel in the third gradient map is converted from the rectangular coordinate system to the polar coordinate system through the polar coordinate transformation. The third gradient map in the polar coordinate system allows the rotation invariance transformation and the scaling invariance transformation to be directly realized through the convolution operation without the need for additional complex calculations; if the third gradient map is not subjected to the polar coordinate transformation, the rotation invariance transformation and the scaling invariance transformation cannot be directly realized through the convolution operation. Therefore, the embodiment of the present application can reduce the amount of calculation, thereby reducing the overhead and improving the efficiency of generating adversarial samples.

[0165] Operation 203 : adjusting pixels of the target image based on the gradients of the pixels in the second gradient map to obtain a target image with affine invariance, and using the target image with affine invariance as an adversarial sample.

[0166] It can be understood that the gradient of the pixel in the second gradient map represents the transformation direction of the pixel. According to the gradient of the pixel, the pixel of the target image can be adjusted in a targeted manner to obtain an adversarial sample.

[0167] As a way to achieve this, Fig.12 As shown, operation 203 includes:

[0168] In operation 501, a disturbance amount of a pixel of a target image is calculated based on a gradient of a pixel in a second gradient map.

[0169] The pixel disturbance amount represents the adjustment amplitude of the pixel.

[0170] There are many methods for calculating the perturbation amount of a pixel, and the embodiments of the present application do not specifically limit this. Two of the calculation methods are introduced below, both of which are applied to the method of generating adversarial samples by iterative calculation, for example, they can be applied to the aforementioned basic iterative method, momentum iterative fast gradient sign method, and input diversity iterative fast gradient sign method.

[0171] As an implementation method, the gradient of the pixel in the second gradient map belongs to the current iterative calculation; based on this, operation 501 includes: performing weighted summation processing on the gradient of the pixel in the second gradient map and the perturbation amount in the previous iterative calculation of the current iterative calculation to obtain the perturbation amount of the pixel of the target image.

[0172] In this embodiment, the perturbation amount in the previous iterative calculation and the gradient of the pixel in the second gradient map in the current iterative calculation are weighted, that is, it is possible to prevent the perturbation amount generated according to the gradient of the pixel in the second gradient map from being too targeted to the current image, causing the current perturbation amount to produce overly specific changes to the target image, thereby reducing the mobility of the generated adversarial sample.

[0173] As another implementation, the gradient of the pixel in the second gradient map belongs to the current iteration calculation. Based on this, Fig.13 As shown, operation 501 includes:

[0174] In operation 601, normalize the gradients of pixels in the second gradient map.

[0175] It should be noted that there are many methods for normalization processing, and the embodiments of the present application do not specifically limit this.

[0176] For example, for any gradient in the second gradient map, the ratio of the gradient to the sum of all gradients in the second gradient map may be obtained, and the ratio may be used as the gradient of the normalized pixel.

[0177] For another example, for any gradient in the second gradient map, the ratio of the gradient to the maximum gradient in the second gradient map can be obtained, and the ratio is used as the gradient of the normalized pixel.

[0178] Operation 602 , weighted summing is performed on the gradient of the pixel in the normalized second gradient map and the disturbance amount in the previous iteration calculation of the current iteration calculation to obtain the disturbance amount of the pixel of the target image.

[0179] Since the aforementioned implementation method has described the weighted summation process, the weighted summation process may be understood by referring to the relevant description in the aforementioned implementation method.

[0180] In this embodiment, the gradient of the pixel in the second gradient map after normalization and the disturbance amount in the iterative calculation before the current iterative calculation are weighted and summed, so that the disturbance amount in the previous iterative calculation can play a corrective role in the current iterative calculation, that is, it can prevent the disturbance amount generated according to the gradient of the pixel in the second gradient map after normalization from being too targeted to the target image, causing the current disturbance amount to produce an overly specific change to the target image, so as to reduce the mobility of the generated adversarial sample; in addition, the gradient of the pixel in the gradient map after affine transformation is normalized, so as to control the gradient of the pixel in the gradient map after affine transformation within a certain range, so as to prevent the gradient of the pixel in the gradient map after affine transformation from being too different from the disturbance amount in the previous iterative calculation, so as to achieve the aforementioned correction effect by weighted summing the two.

[0181] In operation 502, pixels of the target image are adjusted based on the perturbation amount of the pixels of the target image to obtain a target image with affine invariance.

[0182] It should be noted that there are many methods for adjusting the pixels of the target image based on the disturbance amount of the pixels, and the embodiments of the present application do not specifically limit this.

[0183] As an implementation method, operation 502 includes:

[0184] Based on the product of the perturbation amount and the perturbation amplitude of the pixels of the target image, the pixels of the target image are adjusted to obtain a target image with affine invariance.

[0185] In this embodiment, the disturbance amount of the pixel of the target image is multiplied by the disturbance amplitude, and the pixel of the target image is adjusted according to the product, which can play a role in limiting the adjustment amplitude of the pixel in the target image.

[0186] As another implementation, operation 502 includes:

[0187] Obtaining a sign value of a perturbation amount of a pixel of a target image based on a sign function;

[0188] Based on the product of the sign value and the disturbance amplitude, the pixels of the target image are adjusted to obtain a target image with affine invariance.

[0189] In this embodiment, the sign operation is performed on the disturbance amount of the pixel of the target image through a sign function to obtain the sign value of the disturbance amount; specifically, when the disturbance amount is greater than 0, the corresponding sign value is 1; when the disturbance amount is equal to 0, the corresponding sign value is 0, and when the disturbance amount is less than 0, the corresponding sign value is -1.

[0190] Therefore, the sign value of the perturbations of different sizes corresponding to the pixels of the target image is only one of 0, 1 and -1.

[0191] In this embodiment, the sign value of the disturbance amount of the pixel of the target image is obtained based on the sign function, so that the adjustment amplitude of the pixel in the target image can be limited between -1 and 1. The pixel of the target image is adjusted based on the product of the sign value and the disturbance amplitude, which can further limit the adjustment amplitude of the pixel in the target image.

[0192] It should be noted that, in each of the above embodiments, the convolution kernel may be a Gaussian kernel, or may be other kernels except the Gaussian kernel, such as a uniform kernel and a linear kernel.

[0193] Based on the above description, Fig.14 Summarize the process of generating adversarial examples.

[0194] like Fig.14 As shown in Figure 2, the generation process of adversarial samples is as follows.

[0195] The first step is to scale the target image. The specific process can be understood by referring to the relevant description of operation 301.

[0196] In the second step, the scaled target image is input into the neural network.

[0197] The third step is to obtain the gradient map through back propagation The specific process can be understood by referring to the relevant description of operation 303, wherein J(T(x t ; p), y) is the loss function of the neural network, x t is the target image, p is the probability of scaling the target image, the value range of p is between 0 and 1, and y is the label of the target image.

[0198] by Figure 1 The original image shown is the target image, and the label of the target image is panda.

[0199] The fourth step is to use a specific convolution kernel W t Perform convolution operation on the gradient map to achieve translation invariance transformation of the gradient map and obtain a gradient map with translation invariance The gradient in this gradient map can be called a translation invariant perturbation, where is the real number field, and k is the convolution kernel size.

[0200] The fifth step is to use a specific convolution kernel W′ qA convolution operation is performed on the gradient map to achieve rotational invariance and scaling invariance transformation of the gradient map. The specific process can be understood by referring to the relevant descriptions of operations 401 to 404, thereby obtaining a gradient map with affine invariance. The gradient in this gradient map can be called an affine invariant perturbation, where is the real number field, k is the convolution kernel size, and They represent polar coordinate transformation and inverse polar coordinate transformation respectively.

[0201] The sixth step is to normalize the affine invariant perturbation, and add the perturbation in the previous iterative calculation after multiplying by the coefficient (i.e., the perturbation amplitude) to the affine invariant perturbation in this iterative calculation to obtain the perturbation of this iterative calculation.

[0202] The normalization process may be understood by referring to the relevant description of operation 601 .

[0203] The seventh step is to obtain the symbol value of the perturbation in this iterative calculation, and add the symbol value as the final perturbation to the target image, so as to obtain the adversarial sample of this iterative calculation.

[0204] It should be noted that the first to seventh steps can be executed to complete the iterative calculation in sequence. After the iterative calculation is completed, the adversarial sample of the current iterative calculation can be used as the target image of the next iterative calculation, and the first to seventh steps can be repeated to complete the next iterative calculation; after multiple iterative calculations, if the iteration conditions are met (for example, the number of iterative calculations reaches the preset number), the final adversarial sample can be output.

[0205] Among them, in multiple iterative calculations, the first step can be executed based on a certain probability, that is, scaling transformation is performed on the target image in a part of iterative calculations, and scaling transformation is not performed on the target image in another part of iterative calculations.

[0206] The effectiveness of the method for generating adversarial samples provided in the embodiments of the present application is illustrated below through specific examples.

[0207] First, the existing target neural network is used to Fig.15 , Fig.16 , Fig.17 and Fig.18 The image is recognized and Fig.15 , Fig.16 , Fig.17 and Fig.18 The probabilities of correctly identifying the images as docks are 95.7%, 86.3%, 93.1% and 93.5% respectively.

[0208] in, Fig.15 is the original image, the label of which is pier. Fig.16 , Fig.17 and Fig.18 The images are respectively Fig.15 The original image shown is the image after rotation, translation and scaling transformation.

[0209] It can be seen that even if the original image is rotated, translated, and scaled, the existing target neural network can still maintain a high recognition accuracy.

[0210] Then, based on Fig.15 The original image shown in FIG. 1 is generated by using the adversarial sample generation method provided in the embodiment of the present application to generate the corresponding adversarial sample as shown in FIG. Fig.19 As shown, Fig.19 compared to Fig.15 There was no significant change; Fig.19 The adversarial sample shown attacks the target neural network, and the target neural network will Fig.19 The image shown has a 49.3% chance of being misidentified as a straw hat.

[0211] It can be seen that the adversarial sample generation method provided in the embodiment of the present application is used to generate corresponding adversarial samples, which can form an effective attack on the target neural network.

[0212] Afterwards, Fig.19 The adversarial samples shown are rotated, translated, and scaled respectively, and the resulting images are as follows: Fig. 20 , Fig.21 and Fig. 22 As shown, using Fig. 20 , Fig.21 and Fig. 22 The image shown in the figure attacks the target neural network, and the target neural network will Fig. 20 , Fig.21 and Fig. 22 The images shown have probabilities of being misidentified as straw hats of 62%, 51.7%, and 87.2%, respectively.

[0213] It can be seen that the adversarial samples generated by the method provided in the embodiment of the present application are highly aggressive to the neural network, and even if various transformations are performed on the adversarial samples, the adversarial samples still have a high degree of aggressiveness to the neural network. Therefore, the adversarial sample generation method provided in the embodiment of the present application is used to generate the corresponding adversarial samples, which have good affine invariance.

[0214] See also Fig.23 The present application also provides an embodiment of a device for generating an adversarial sample, including:

[0215] A gradient map acquisition unit 701 is used to acquire a first gradient map of a target image, wherein the first gradient map includes a gradient of a loss function of a neural network to pixels in the target image, and the loss function is acquired by taking the target image as an input of the neural network;

[0216] A transformation unit 702, configured to transform the first gradient map to obtain a second gradient map, wherein the second gradient map is configured to make the target image have affine invariance;

[0217] The adjustment unit 703 is used to adjust the pixels of the target image based on the gradients of the pixels in the second gradient map to obtain a target image with affine invariance, and use the target image with affine invariance as an adversarial sample.

[0218] As an implementation method, the gradient acquisition unit 701 is used to transform the first gradient map to obtain a third gradient map, and the third gradient map is used to make the target image have translation invariance; transform the third gradient map to obtain a second gradient map, and the second gradient map is used to make the target image have affine invariance.

[0219] As an implementation method, the gradient acquisition unit 701 is used to perform polar coordinate transformation on the third gradient map, and the polar coordinate transformation is used to convert the gradient of the pixels in the third gradient map from a rectangular coordinate system to a polar coordinate system; transform the third gradient map after the polar coordinate transformation to obtain a fourth gradient map, and the fourth gradient map is used to make the target image have affine invariance; perform polar coordinate inverse transformation on the fourth gradient map to obtain a second gradient map, and the polar coordinate inverse transformation is used to convert the gradient of the pixels passing through the fourth gradient map from a polar coordinate system to a rectangular coordinate system.

[0220] As an implementation method, the adjustment unit 703 is used to calculate the perturbation amount of the pixels of the target image based on the gradient of the pixels in the second gradient map; and adjust the pixels of the target image based on the perturbation amount of the pixels of the target image to obtain a target image with affine invariance.

[0221] As an implementation method, the gradient of the pixel in the second gradient map belongs to the current iterative calculation; the adjustment unit 703 is used to perform weighted sum processing on the gradient of the pixel in the second gradient map and the disturbance amount in the previous iterative calculation of the current iterative calculation to obtain the disturbance amount of the pixel of the target image.

[0222] As an implementation method, the gradient of the pixel in the second gradient map belongs to the current iterative calculation; the adjustment unit 703 is used to normalize the gradient of the pixel in the second gradient map; the gradient of the pixel in the second gradient map after normalization and the perturbation amount in the iterative calculation before the current iterative calculation are weighted summed to obtain the perturbation amount of the pixel of the target image.

[0223] As an implementation method, the adjustment unit 703 is used to obtain the sign value of the disturbance amount of the pixel of the target image based on the sign function; based on the product of the sign value and the disturbance amplitude, adjust the pixel of the target image to obtain a target image with affine invariance.

[0224] As an implementation method, the gradient map acquisition unit 701 is used to scale the target image; use the scaled target image as the input of the neural network to obtain the loss function of the neural network; and back-propagate the neural network based on the loss function to obtain a first gradient map of the target image.

[0225] Among them, for the specific implementation, relevant instructions and technical effects of the above units, please refer to the description of the first aspect of the embodiment of this application.

[0226] The present application also provides an embodiment of a computer device, which may be a server. Fig.24 , Fig.24 is a schematic diagram of a structure of a computer device provided in an embodiment of the present application. The computer device 1800 may be deployed with Fig.11 or Fig.12 The training device of the neural network described in the corresponding embodiment is used to implement Fig.11 or Fig.12 The function of the training device of the corresponding neural network in the embodiment, specifically, the computer device 1800 is implemented by one or more servers, and the computer device 1800 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 1822 (for example, one or more processors) and memory 1832, one or more storage media 1830 (for example, one or more mass storage devices) storing application programs 1842 or data 1844. Among them, the memory 1832 and the storage medium 1830 can be short-term storage or permanent storage. The program stored in the storage medium 1830 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the computer device. Furthermore, the central processor 1822 can be configured to communicate with the storage medium 1830 and execute a series of instruction operations in the storage medium 1830 on the computer device 1800.

[0227] The computer device 1800 may also include one or more power supplies 1826, one or more wired or wireless network interfaces 1850, one or more input and output interfaces 1858, and / or one or more operating systems 1841, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0228] In the embodiment of the present application, the central processor 1822 is used to execute Fig.23 The method for generating adversarial samples executed by the adversarial sample generating device in the corresponding embodiment. Specifically, the central processor 1822 can be used to:

[0229] Acquire a first gradient map of a target image, wherein the first gradient map includes a gradient of a loss function of a neural network with respect to pixels in the target image, wherein the loss function is acquired by taking the target image as an input of the neural network;

[0230] Transforming the first gradient map to obtain a second gradient map, wherein the second gradient map is used to make the target image have affine invariance;

[0231] The pixels of the target image are adjusted based on the gradients of the pixels in the second gradient map to obtain a target image with affine invariance, and the target image with affine invariance is used as an adversarial sample.

[0232] The CPU 1822 can also be used to:

[0233] The adversarial samples generated by the above generation method are used to train the neural network.

[0234] The present application also provides a chip including one or more processors, some or all of which are used to read and execute a computer program stored in a memory to perform the methods of the above embodiments.

[0235] Optionally, the chip includes a memory, and the memory is connected to the processor through a circuit or a wire. Further optionally, the chip also includes a communication interface, and the processor is connected to the communication interface. The communication interface is used to receive data and / or information to be processed, and the processor obtains the data and / or information from the communication interface, processes the data and / or information, and outputs the processing result through the communication interface. The communication interface can be an input and output interface.

[0236] In some implementations, some of the one or more processors may implement some steps of the above method by means of dedicated hardware. For example, processing involving a neural network model may be implemented by a dedicated neural network processor or a graphics processor.

[0237] The method provided in the embodiment of the present application can be implemented by one chip or by multiple chips working together.

[0238] The embodiment of the present application also provides a computer storage medium, which is used to store computer software instructions used by the above-mentioned computer device, including a program designed for executing the computer device.

[0239] The computer device can be as described above Fig.23 The described adversarial sample generation device.

[0240] An embodiment of the present application further provides a computer program product, which includes computer software instructions. The computer software instructions can be loaded by a processor to implement the processes in the methods shown in the aforementioned embodiments.

[0241] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0242] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0243] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0244] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0245] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk.

Claims

1. A method for generating adversarial samples, characterized in that: include: Acquire a first gradient map of a target image, wherein the first gradient map includes a gradient of a loss function of a neural network with respect to pixels in the target image, wherein the loss function is acquired by taking the target image as an input of the neural network; Performing an affine invariance transformation on the first gradient map to obtain a second gradient map, wherein the second gradient map is used to make the target image affine invariant, and the affine invariance transformation includes a translation invariance transformation and an invariance transformation corresponding to a linear transformation; Adjusting pixels of the target image based on the gradients of the pixels in the second gradient map to obtain a target image with affine invariance, and using the target image with affine invariance as an adversarial sample; The performing an affine invariant transformation on the first gradient map to obtain a second gradient map comprises: Transforming the first gradient map to obtain a third gradient map, wherein the third gradient map is used to make the target image translationally invariant; Performing polar coordinate transformation on the third gradient map, wherein the polar coordinate transformation is used to convert the gradient of the pixels in the third gradient map from a rectangular coordinate system to a polar coordinate system; Transforming the third gradient map after the polar coordinate transformation to obtain a fourth gradient map, wherein the fourth gradient map is used to make the target image have affine invariance; The fourth gradient map is subjected to an inverse polar coordinate transformation to obtain the second gradient map, wherein the inverse polar coordinate transformation is used to convert the gradient of pixels in the fourth gradient map from a polar coordinate system to a rectangular coordinate system.

2. The generation method according to claim 1, characterized in that: The step of adjusting the pixels of the target image based on the gradients of the pixels in the second gradient map to obtain the target image with affine invariance includes: Calculating a disturbance amount of a pixel of the target image based on a gradient of a pixel in the second gradient map; The pixels of the target image are adjusted based on the disturbance amount of the pixels of the target image to obtain a target image with affine invariance.

3. The generation method according to claim 2, characterized in that: The gradient of the pixel in the second gradient map belongs to the current iteration calculation; The calculating the disturbance amount of the pixel of the target image based on the gradient of the pixel in the second gradient map comprises: The gradient of the pixel in the second gradient map and the disturbance amount in the previous iterative calculation of the current iterative calculation are weighted summed to obtain the disturbance amount of the pixel of the target image.

4. The generation method according to claim 2, characterized in that: The gradient of the pixel in the second gradient map belongs to the current iteration calculation; The calculating the disturbance amount of the pixel of the target image based on the gradient of the pixel in the second gradient map comprises: Normalizing the gradients of the pixels in the second gradient map; The gradient of the pixel in the second gradient map after the normalization processing and the disturbance amount in the previous iteration calculation of the current iteration calculation are weighted summed to obtain the disturbance amount of the pixel of the target image.

5. The generation method according to any one of claims 2 to 4, characterized in that: The step of adjusting the pixels of the target image based on the disturbance amount of the pixels of the target image to obtain the target image with affine invariance includes: Acquire a sign value of a disturbance amount of a pixel of the target image based on a sign function; Based on the product of the sign value and the disturbance amplitude, the pixels of the target image are adjusted to obtain a target image with affine invariance.

6. The generation method according to any one of claims 2 to 4, characterized in that: The obtaining of the first gradient map of the target image comprises: Performing a scaling transformation on the target image; Using the target image after the scaling transformation as an input of a neural network, and obtaining a loss function of the neural network; The neural network is back-propagated based on the loss function to obtain a first gradient map of the target image.

7. A neural network training method, characterized in that: include: The adversarial samples generated by the generation method described in any one of claims 1 to 6 are used to train the neural network.

8. A device for generating adversarial samples, characterized in that: include: A gradient map acquisition unit, used to acquire a first gradient map of a target image, wherein the first gradient map includes a gradient of a loss function of a neural network to pixels in the target image, and the loss function is acquired by taking the target image as an input of the neural network; A transformation unit, configured to perform an affine invariance transformation on the first gradient map to obtain a second gradient map, wherein the second gradient map is used to make the target image affine invariant, and the affine invariance transformation includes a translation invariance transformation and an invariance transformation corresponding to a linear transformation; an adjusting unit, configured to adjust pixels of the target image based on the gradients of the pixels in the second gradient map to obtain a target image with affine invariance, and use the target image with affine invariance as an adversarial sample; The transformation unit is specifically used to transform the first gradient map to obtain a third gradient map, and the third gradient map is used to make the target image have translation invariance; perform polar coordinate transformation on the third gradient map, and the polar coordinate transformation is used to convert the gradient of the pixels in the third gradient map from a rectangular coordinate system to a polar coordinate system; transform the third gradient map after the polar coordinate transformation to obtain a fourth gradient map, and the fourth gradient map is used to make the target image have affine invariance; The fourth gradient map is subjected to an inverse polar coordinate transformation to obtain the second gradient map, wherein the inverse polar coordinate transformation is used to convert the gradient of pixels in the fourth gradient map from a polar coordinate system to a rectangular coordinate system.

9. A neural network training device, characterized in that: The training device is deployed on a computer device so that a processor on the computer device trains a neural network using adversarial samples generated by the generation method described in any one of claims 1 to 6.

10. A computer device, characterized in that: include: One or more processors and memory; wherein the memory stores computer-readable instructions; The one or more processors read the computer-readable instructions to cause the computer device to implement the method according to any one of claims 1 to 7.

11. A computer-readable storage medium, characterized in that: The method comprises computer-readable instructions, which, when executed on a computer, cause the computer to execute the method according to any one of claims 1 to 7.

12. A computer program product, characterized in that The method comprises computer-readable instructions, which, when executed on a computer, cause the computer to execute the method according to any one of claims 1 to 7.

Images