Adversarial sample defense method and device based on data perturbation

Abstract

The present invention discloses a method and device for adversarial sample defense based on data perturbation. The method comprises the following steps: adding pixels that can interfere with the vehicle recognition of road signs as data perturbations to input samples to form defense samples; inputting the defense samples into the target neural network model for optimization, and outputting the trained data perturbations; adding the trained data perturbations to the samples to be recognized by the recognition neural network for recognition by the recognition neural network; the recognition neural network is a neural network model implanted in an autonomous driving vehicle. The present invention can improve the robustness of the neural network model and is particularly suitable for application in autonomous driving scenarios.

Description

Technical Field

[0001] The present invention relates to an adversarial sample defense method based on data perturbation for autonomous driving scenarios, and also relates to a corresponding data perturbation optimization device, belonging to the technical field of autonomous driving. Background Art

[0002] Adversarial samples refer to samples that can cause machine learning models to make wrong judgments by adding interference. In the past few years, deep neural networks have made remarkable achievements in a wide range of application fields such as computer vision, natural language processing, and speech recognition. However, relevant research has shown that deep neural networks are easily affected by artificially designed adversarial samples. These carefully constructed disturbances are imperceptible to humans, but can easily cause deep neural networks to make wrong judgments. This poses a safety challenge to the application of deep neural networks in scenarios with high reliability requirements, especially in autonomous driving scenarios.

[0003] In response to the vulnerability of deep neural networks to adversarial samples, relevant researchers have proposed a series of defensive methods to make deep neural networks more robust and more difficult to attack. Existing adversarial sample defense methods can be divided into three categories: adversarial training introduces adversarial samples dynamically generated by the model into the training process in the form of data augmentation, so that the model learns the features of adversarial samples, thereby improving the model's defense capabilities against adversarial samples; gradient masking methods hide the gradient information of the model through intentional or unintentional means to enhance the security of the model; adversarial detection methods use some means to pre-identify whether the input image is a normal sample or an adversarial sample to prevent adversarial samples from being input into the neural network model.

[0004] In the field of autonomous driving, studying the attack and defense of adversarial samples has a profound impact on the development of the autonomous driving industry. Although the existing adversarial sample defense methods for models have achieved good defense effects, they require targeted training and the training cycle is also long. During use, the defense methods are not flexible enough and are difficult to meet the actual needs of autonomous driving scenarios. Summary of the invention

[0005] The primary technical problem to be solved by the present invention is to provide an adversarial sample defense method based on data perturbation for autonomous driving scenarios.

[0006] Another technical problem to be solved by the present invention is to provide an adversarial sample defense device based on data perturbation.

[0007] To achieve the above object, the present invention adopts the following technical solutions:

[0008] According to a first aspect of an embodiment of the present invention, a method for defending against adversarial samples based on data perturbation is provided, comprising the following steps:

[0009] The pixels that can interfere with the vehicle's recognition of road signs are added to the input sample as data disturbances to form a defense sample;

[0010] Input the defense samples into the target neural network model for optimization and output the trained data perturbation;

[0011] The trained data disturbance is added to the samples to be identified by the recognition neural network for identification by the recognition neural network; the recognition neural network is a neural network model implanted in the autonomous driving vehicle.

[0012] Preferably, pixels that can interfere with the vehicle's recognition of road signs are added to the input sample as data disturbances, specifically including:

[0013] The current data perturbation replaces the pixels in the predetermined area of ​​the corresponding input sample.

[0014] Preferably, the step of inputting the defense sample into the target neural network model for optimization also includes:

[0015] Add natural noise to the defense samples;

[0016] Add adversarial perturbations to the defense samples after adding natural noise.

[0017] Preferably, the step of inputting the defense sample into the target neural network model for optimization specifically includes:

[0018] After inputting the defense sample into the target neural network model, the defense loss function is calculated;

[0019] The data perturbations in the defense samples are trained according to the defense loss function until the optimized data perturbations meet the threshold.

[0020] Preferably, the step of inputting the defense sample into the target neural network model for optimization further includes:

[0021] If the optimized data perturbation does not meet the threshold, the data perturbation that does not meet the threshold is added to the input sample.

[0022] Preferably, the expression of the defense sample is:

[0023] x′=A(p,x,l,t)

[0024] Among them, p is the data perturbation, x is the input sample, l is the location where the data perturbation is added, t is the transformation implemented on the data perturbation, and function A represents the perturbation attachment operation.

[0025] Preferably, the expression for counteracting disturbance is:

[0026]

[0027] in, is the defense sample after adding natural noise, y is the category label of the input sample, F(·) is the target neural network model, represents the gradient of the classification loss function, sign(·) represents the sign function, and the parameter ∈ represents the magnitude of the adversarial perturbation.

[0028] Preferably, the expression of the defense loss function is:

[0029]

[0030] in, is the defense sample after adding natural noise, δ is the adversarial perturbation, y is the category label of the input sample, F(·) is the target neural network model, Pr y (·) is the probability function of sample label y.

[0031] Preferably, the perturbation expression of the training completed data is:

[0032]

[0033] Among them, L d is the defense loss function, and p is the data perturbation.

[0034] According to a second aspect of an embodiment of the present invention, there is provided an adversarial sample defense device based on data perturbation, comprising a processor and a memory, wherein the processor reads a computer program in the memory to perform the following operations:

[0035] The pixels that can interfere with the vehicle's recognition of road signs are added to the input sample as data disturbances to form a defense sample;

[0036] Input the defense samples into the target neural network model for optimization and output the trained data perturbation;

[0037] The trained data disturbance is added to the samples to be identified by the recognition neural network for identification by the recognition neural network; the recognition neural network is a neural network model implanted in the autonomous driving vehicle.

[0038] Compared with the prior art, the adversarial sample defense method and device provided by the present invention have the following beneficial effects:

[0039] 1. During the data perturbation training process, natural noise processing and adversarial noise processing are performed, making the recognition neural network embedded in the autonomous driving vehicle more robust and able to cope with artificially constructed adversarial samples;

[0040] 2. During the optimization process, the defense loss function is used to optimize data perturbations. In addition to being robust to adversarial behavior, it can also improve the recognition accuracy of the recognition neural network. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A schematic diagram of the application of the adversarial sample defense method provided by the present invention in an autonomous driving scenario;

[0042] Figure 2 A schematic diagram of the process of the adversarial sample defense method provided by the present invention;

[0043] Figure 3 This is a schematic diagram of the training process of defense samples in an embodiment of the present invention;

[0044] Figure 4 A structural diagram of an adversarial sample defense device provided by an embodiment of the present invention;

[0045] Figure 5(a) is a graph showing the recognition efficiency of the target neural network model for adversarial samples;

[0046] FIG5( b ) is a graph showing the recognition efficiency of the target neural network model for adversarial samples carrying data perturbations obtained in an embodiment of the present invention;

[0047] Figure 6(a) is a graph showing the recognition efficiency of the target neural network model for abnormal background samples;

[0048] FIG6( b ) is a diagram showing the recognition efficiency of the target neural network model for abnormal background samples carrying data disturbances obtained in an embodiment of the present invention;

[0049] Figure 7(a) is a graph showing the recognition efficiency of the target neural network model for shadow samples;

[0050] FIG7( b ) is a graph showing the recognition efficiency of the target neural network model for shadow samples carrying data disturbances obtained in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The technical content of the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] Figure 1Schematic diagram of the application of the adversarial sample defense method provided by the present invention in an autonomous driving scenario. In the autonomous driving scenario, the neural network model H(·) is implanted into the autonomous driving vehicle as a recognition neural network, and the data disturbance is the interference factor that affects the autonomous driving vehicle's recognition of the image. The image here is the image information obtained by the autonomous driving vehicle during driving, such as road signs, road surfaces, roadblocks, and pedestrians.

[0053] like Figure 2 As shown, the adversarial sample defense method provided by the embodiment of the present invention at least includes the following steps:

[0054] 101. Add data disturbance to the input sample to form a defense sample;

[0055] The specific steps include:

[0056] 1011. Replace pixels in a predetermined area of ​​a corresponding input sample with the current data disturbance to form a defense sample.

[0057] The expression of the defense sample is:

[0058] x′=A(p,x,l,t) (1)

[0059] In formula (1), p is the data perturbation, x is the input sample, l is the location where the data perturbation is added, t is the transformation implemented on the data perturbation, and function A represents the perturbation attachment operation.

[0060] In the embodiment of the present invention, taking a road sign (stop) as an example, the pixel that can interfere with the vehicle's recognition of the road sign is used as the data disturbance p, and the initial value p0 is used as the data disturbance p to be added to any position on the road sign sample x to obtain a defense sample x'. The pixel that interferes with the vehicle's recognition of the road sign is the initial value p0, which can be attached to any position on the road sign (stop) to form a defense sample x'.

[0061] On this basis, the data perturbation p0 is initialized and attached to the input sample x through the perturbation attachment operation A(·) to obtain the defense sample. The data perturbation p can be divided into noise perturbation and patch perturbation. The perturbation size of each pixel of the data perturbation is restricted, but the number of perturbed pixels is not restricted, which constitutes noise perturbation; the number of perturbed pixels of the data perturbation is restricted, but the perturbation size of the perturbed pixels is not restricted, which constitutes patch perturbation. The size of the data perturbation p can be defined as needed.

[0062] 102. Input the defense sample into the target neural network model for optimization, and output the trained data perturbation.

[0063] Before step 102, the following steps are also included:

[0064] 201. Add natural noise to the defense samples;

[0065] In the embodiment of the present invention, natural noise processing is performed on the defense sample to obtain a defense sample after adding natural noise. The defense sample is processed with natural noise so that the data disturbance can be more robust to natural interference in the physical world, for example, to prevent the interference of rain, snow, fog and other weather in the real world on the data disturbance. Mathematical methods are used to add natural noise disturbance to simulate real scenes:

[0066]

[0067] In formula (2), x′ is the defense sample, and function C(·) represents adding natural noise disturbance to the defense sample x′. This is a defense sample after adding natural noise.

[0068] 202. Add adversarial perturbations to the defense samples after adding natural noise.

[0069] In an embodiment of the present invention, the defense sample after adding natural noise is subjected to adversarial noise processing to obtain the defended adversarial sample. In the training optimization process of data perturbation, adversarial noise interference is introduced to make the data perturbation more robust to adversarial samples and defensive against adversarial samples in the physical world. After adversarial perturbation processing, the expression of the defended adversarial sample is:

[0070]

[0071] In formula (3), y is the label of the sample, F(·) represents the target neural network, represents the gradient of the classification loss function, sign(·) represents the sign function, and the parameter ∈ represents the magnitude of the adversarial perturbation.

[0072] In an embodiment of the present invention, the defended adversarial sample δ is obtained by FGSM (fast gradient descent method). In other embodiments, the defended adversarial sample can also be obtained by PGD (gradient-based attack), CW (optimization-based attack), etc.

[0073] The defense samples are input into the target neural network model for optimization, which includes the following sub-steps:

[0074] 1021. After inputting the defense sample into the target neural network model, the defense loss function is calculated;

[0075] In the embodiment of the present invention, the defense sample is processed by natural noise and adversarial noise before being input into the target neural network model, that is, the defended adversarial sample is input into the target neural network model to calculate the defense loss function:

[0076]

[0077] In formula (4), is the defense sample after adding natural noise, δ is the adversarial perturbation, y is the category label of the input sample, F(·) is the target neural network model, Pr y (·) is the probability function of sample label y.

[0078] 1022. If the defense loss function is trained to defend against data disturbances in the sample, until the optimized data disturbance meets the threshold;

[0079] 1023. If the optimized data perturbation does not meet the threshold, the data perturbation that does not meet the threshold is added to the input sample.

[0080] In this embodiment of the present invention, the defense loss function L is used d Optimize the data perturbation, and determine whether the data perturbation meets the threshold value according to the prediction result of the neural network for the adversarial sample after defense. During the training process, it is necessary to judge the end of iteration. If it ends, stop the training and input the data perturbation. Otherwise, continue to iterate the new data perturbation. That is to say, if the data perturbation does not meet the threshold value, return to 1011. Of course, in other embodiments, the number of iterations can be used to judge the training. If the preset number of times is reached, stop the training. If it is not reached, return to continue the training.

[0081] The data perturbation expression after the training is:

[0082]

[0083] In formula (5), L d is the defense loss function, and p is the data perturbation.

[0084] like Figure 3 As shown, the above training process can be expressed as follows through Table 1:

[0085] Table 1

[0086]

[0087] In the autonomous driving scenario, the defense sample x′ is optimized through continuous iteration. Specifically, in each iteration, the defense sample x′ is subjected to natural noise processing C(·) and adversarial noise processing δ. Then it is input into the target neural network F(·) and the defense loss function L is calculated. d , thereby optimizing the data perturbation p. Figure 1 As shown in the figure, the road sign with p0 (i.e., the initial defense sample) is subjected to natural noise processing C(·) to form a defense sample with added natural noise (i.e., Figure 1Then, adversarial noise is added to the defense sample with natural noise added. The defense sample with natural noise and adversarial noise added is input into the target neural network F(·) for training. The loss function L d , constrained data perturbation p; the target neural network F(·) outputs the data perturbation p (well-treined patch) that satisfies the conditions.

[0088] 103. Add the trained data disturbance to the sample to be recognized by the recognition neural network for recognition by the recognition neural network.

[0089] In the embodiment of the present invention, the recognition neural network is a neural network that can recognize any sample. When the recognition neural network recognizes samples with the added perturbation of trained data, the accuracy is higher than that of samples without the added perturbation of trained data, thereby improving the accuracy of the target neural network model in recognizing samples.

[0090] In the autonomous driving scenario, posting the obtained data perturbation p on samples that are vulnerable to adversarial attacks—road signs—can improve the accuracy of the target neural network model H(·) in identifying road signs, and the road signs can be robust to adversarial samples, making it difficult for adversarial samples to have an effect on the road signs. In other application scenarios, each sample that needs to be identified (road signs, road surfaces, roadblocks, pedestrians, etc.) can also be posted with the output data perturbation p to improve the recognition accuracy of the neural network model H(·), but the actual operation efficiency is relatively low. Therefore, in a preferred embodiment of the present invention, only image information that is vulnerable to attack is selected from the samples that need to be identified for posting p, which can not only improve the recognition accuracy of the neural network model H(·) during the autonomous driving process, but also improve the recognition efficiency.

[0091] Of course, the adversarial sample defense method described in the embodiment of the present invention can also be applied to other scenarios such as product identification.

[0092] In order to implement the adversarial sample defense method provided by the present invention, the present invention further provides an adversarial sample defense device based on data perturbation. Figure 4As shown, the adversarial sample defense device includes a processor 42 and a memory 41, and may further include a communication component, a sensor component, a power component, a multimedia component, and an input / output interface according to actual needs. Among them, the memory, communication component, sensor component, power component, multimedia component, and input / output interface are all connected to the processor 42. As mentioned above, the memory 41 in the node device can be a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, etc., and the processor can be a central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a digital signal processing (DSP) chip, etc. Other communication components, sensor components, power components, multimedia components, etc. can all be implemented using common components in existing smart phones, which will not be described in detail here.

[0093] On the other hand, in the above-mentioned adversarial sample defense device based on data perturbation, the processor 42 reads the computer program in the memory 41 to perform the following operations:

[0094] The pixels that can interfere with the vehicle's recognition of road signs are added to the input sample as data disturbances to form a defense sample;

[0095] Input the defense samples into the target neural network model for optimization and output the trained data perturbation;

[0096] The trained data disturbance is added to the samples to be identified by the recognition neural network for identification by the recognition neural network; the recognition neural network is a neural network model implanted in the autonomous driving vehicle.

[0097] Below, three sets of experiments are used to compare the recognition efficiency of the target neural network model:

[0098] As shown in Figure 5(a), the sample in the figure is a road sign sample x with occlusion perturbation (white stripes on the sign, simulating the white stripes as adversarial attacks), i.e., adversarial samples. After testing, the recognition accuracy of the target neural network model for the road sign sample x with occlusion perturbation is 49.2%.

[0099] As shown in Figure 5(b), the sample in the figure is based on Figure 5(a) and the optimized data perturbation p (the colored bar on the left side of the sign) after optimization by the embodiment of the present invention is posted to obtain the optimized sample. After testing, the recognition accuracy of the neural network model H(·) for the road sign sample x with occlusion perturbation is 90.2%.

[0100] As shown in Figure 6(a), the sample in the figure is a road sign sample x with an abnormal background (the picture below the sign), that is, an abnormal background sample. After testing, the recognition accuracy of the neural network model H(·) for the road sign sample x with an abnormal background is 45.9%.

[0101] As shown in FIG6(b), the sample in the figure is based on FIG6(a) and the optimized data perturbation p (the colored bar on the left side of the sign) after optimization by the embodiment of the present invention is posted to obtain the optimized sample. After testing, the recognition accuracy of the neural network model H(·) for the road sign sample x with abnormal background is 91.3%.

[0102] As shown in Figure 7(a), the sample in the figure is a road sign sample x with shadow influence (shadow on the sign), that is, a shadow sample. After testing, the recognition accuracy of the neural network model H(·) for the road sign sample x with shadow influence is 47.5%.

[0103] As shown in FIG7(b), the sample in the figure is based on FIG7(a) and the optimized data perturbation p (the color bar on the left side of the sign) after optimization by the embodiment of the present invention is posted to obtain the optimized sample. After testing, the recognition accuracy of the neural network model H(·) for the road sign sample x affected by shadows is 91.3%.

[0104] The accuracy rates obtained through the above experimental comparisons indicate that the adversarial sample defense method and device provided by the present invention can effectively improve the robustness of the neural network model, and are particularly suitable for application in autonomous driving scenarios.

[0105] The above is a detailed description of the data perturbation-based adversarial sample defense method and device provided by the present invention. For those skilled in the art, any obvious changes made to it without departing from the essence of the present invention will constitute an infringement of the patent right of the present invention and will bear corresponding legal responsibilities.

Claims

1. A method for defending against adversarial samples based on data perturbation, characterized in that The steps include: Add pixels that can interfere with vehicle recognition of road signs as data disturbances to the input samples to form defense samples; add natural noise to the defense samples, and add adversarial disturbances to the defense samples after adding natural noise; In each iteration, the defense sample is subjected to natural noise processing and adversarial noise processing, and then input into the target neural network model, and the defense loss function is calculated, so as to train the data perturbation in the defense sample according to the defense loss function until the optimized data perturbation meets the threshold, and the trained data perturbation is output; wherein, The expression of the defense loss function is: The data perturbation expression after the training is: The expression of the anti-disturbance is: Among them, L d is the defense loss function, p is the data perturbation, This is the defense sample after adding natural noise. To combat disturbances, y is the category label of the input sample. is the target neural network model, For sample labels The probability function of represents the gradient of the classification loss function, Represents a symbolic function, with parameters represents the magnitude of the adversarial disturbance; The trained data disturbance is added to the samples to be recognized by the recognition neural network for recognition and training; the recognition neural network is a target neural network model implanted in the autonomous driving vehicle.

2. The adversarial sample defense method according to claim 1, characterized in that Pixels that can interfere with vehicle recognition of road signs are added to the input samples as data disturbances, including: The current data perturbation replaces the pixels in the predetermined area of ​​the corresponding input sample.

3. The adversarial sample defense method according to claim 1, characterized in that Also includes: If the optimized data perturbation does not meet the threshold, the data perturbation that does not meet the threshold is added to the input sample.

4. The adversarial sample defense method according to claim 1, characterized in that The expression of the defense sample is: in, For the defense sample, x is the input sample, l is the location where the data perturbation is added, t is the transformation implemented on the data perturbation, and function A represents the perturbation attachment operation.

5. A data perturbation-based adversarial sample defense device, characterized in that It comprises a processor and a memory, wherein the processor reads a computer program in the memory to execute the adversarial sample defense method based on data perturbation as described in any one of claims 1 to 4.

Images