Adversarial training method and device for autonomous driving recognition model

Abstract

The present disclosure provides an adversarial training method and device for an autonomous driving recognition model, which can be applied to the fields of autonomous driving technology and intelligent recognition technology. The method includes: using N perturbation block sub-images in a perturbation block image to process the original image to obtain N sample images to be denoised, the perturbation block sub-images are obtained by segmenting the perturbation block image, and N is an integer greater than 1; inputting the N sample images to be denoised into a denoiser to obtain N adversarial sample images; processing the N adversarial sample images and labels according to a recognition loss function to obtain a recognition loss value, the recognition loss function is used to train a recognition model, and the label represents the true recognition result of the adversarial sample image; adjusting the model parameters of the recognition model based on the recognition loss value.

Description

Technical Field

[0001] The present disclosure relates to the fields of autonomous driving technology and intelligent recognition technology, and in particular to an adversarial training method and device for an autonomous driving recognition model. Background Art

[0002] Recent studies have shown that deep learning models are vulnerable to subtle perturbations called adversarial examples. These barely perceptible changes pose serious safety risks to deep learning-based applications, especially in self-driving cars. For example, an adversary could change a traffic sign to falsely instruct an autonomous driving system to speed up instead of stop, potentially causing a serious accident.

[0003] In the process of implementing the concept of the present disclosure, the inventors discovered that there are at least the following problems in the related art: the adversarial robustness of the model is weak against various scene changes and random disturbances. Summary of the invention

[0004] In view of the above problems, the present disclosure provides an adversarial training method and device for an autonomous driving recognition model.

[0005] According to a first aspect of the present disclosure, an adversarial training method for an autonomous driving recognition model is provided, comprising: processing an original image using N perturbation block sub-images in a perturbation block image to obtain N sample images to be denoised, wherein the perturbation block sub-images are obtained by segmenting the perturbation block image, and N is an integer greater than 1; inputting the N sample images to be denoised into a denoiser to obtain N adversarial sample images; processing the N adversarial sample images and labels according to a recognition loss function to obtain a recognition loss value, wherein the recognition loss function is used to train a recognition model, and the labels represent the true recognition results of the adversarial sample images; and adjusting the model parameters of the recognition model based on the recognition loss value.

[0006] According to an embodiment of the present disclosure, the original image is processed using N disturbance block sub-images in the disturbance block image to obtain N sample images to be denoised, including: segmenting the original image to obtain N original sub-images and position information of the original sub-images, wherein the position information of the original sub-images represents the position of the original sub-images in the original image; determining a target disturbance block sub-image from the N disturbance block sub-images according to the position information of the original sub-images; and inserting the target disturbance block sub-image into the original image to obtain N sample images to be denoised.

[0007] According to an embodiment of the present disclosure, determining a target disturbed block sub-image from N disturbed block sub-images according to the position information of the original sub-image includes: determining a first original sub-image and the position information of the first original sub-image from the N original sub-images, where the first original sub-image is an original sub-image to be processed; and determining a target disturbed block sub-image having the same position information as the first original sub-image from the N disturbed block sub-images.

[0008] According to an embodiment of the present disclosure, the method further includes: determining a second original sub-image from the N original sub-images, the second original sub-image being an original sub-image that retains features of the original image.

[0009] According to an embodiment of the present disclosure, inserting a target perturbation block sub-image into an original image to obtain N sample images to be denoised includes: inserting a target perturbation block sub-image into a first original sub-image in the original image to obtain a first processed sub-image; and obtaining N sample images to be denoised based on the first processed sub-image and the second original sub-image.

[0010] According to an embodiment of the present disclosure, the loss function used to train the denoiser is the mean square error.

[0011] According to an embodiment of the present disclosure, the denoiser uses a nearest neighbour interpolation method for upsampling and does not use a maximum pooling layer in the denoiser for downsampling.

[0012] According to an embodiment of the present disclosure, adjusting the model parameters of the recognition model based on the recognition loss value includes: updating the perturbation block image based on the recognition loss value to obtain an updated perturbation block image; using the updated perturbation block image to process the original image to obtain N updated sample images to be denoised; inputting the N updated sample images to be denoised into a denoiser to obtain N updated adversarial sample images; and adjusting the model parameters of the recognition model using the N updated adversarial sample images.

[0013] According to an embodiment of the present disclosure, the method further includes: adjusting the model parameters of the denoiser using the N updated sample images to be denoised.

[0014] A second aspect of the present disclosure provides an adversarial training device for an autonomous driving recognition model, comprising: a first processing module, used to process an original image using N perturbed block sub-images in a perturbed block image to obtain N sample images to be denoised, where the perturbed block sub-images are obtained by segmenting the perturbed block image, and N is an integer greater than 1; an input module, used to input the N sample images to be denoised into a denoiser to obtain N adversarial sample images; a second processing module, used to process the N adversarial sample images and labels according to a recognition loss function to obtain a recognition loss value, where the recognition loss function is used to train a recognition model, and the labels represent the true recognition results of the adversarial sample images; and a first adjustment module, used to adjust the model parameters of the recognition model based on the recognition loss value.

[0015] A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above method.

[0016] The fourth aspect of the present disclosure further provides a computer-readable storage medium having an executable computer program stored thereon, which implements the steps of the above method when executed by a processor.

[0017] The fifth aspect of the present disclosure also provides a computer program product, including a computer program, which implements the steps of the above method when executed by a processor.

[0018] According to the adversarial training method and device for an autonomous driving recognition model provided by the present disclosure, the original image is processed using N perturbation block sub-images in the perturbation block image to obtain N sample images to be denoised, where the perturbation block sub-images are obtained by segmenting the perturbation block image, and N is an integer greater than 1; the N sample images to be denoised are input into a denoiser to obtain N adversarial sample images, thereby increasing the number of adversarial sample images for training the recognition model, and enhancing subsequent adversarial training.

[0019] N adversarial sample images and labels are processed according to the recognition loss function to obtain a recognition loss value. The recognition loss function is used to train the recognition model, and the label represents the true recognition result of the adversarial sample image. The model parameters of the recognition model are adjusted based on the recognition loss value, which combines the advantages of adversarial denoising and adversarial training. The training method of the present application improves the robustness of the recognition model. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above contents and other purposes, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0021] Figure 1 A diagram schematically illustrates an application scenario of an adversarial training method for an autonomous driving recognition model according to an embodiment of the present disclosure;

[0022] Figure 2 A flowchart of an adversarial training method for an autonomous driving recognition model according to an embodiment of the present disclosure is schematically shown;

[0023] Figure 3 A schematic diagram of the framework structure of adversarial denoising training according to an embodiment of the present disclosure is schematically shown;

[0024] Figure 4 A schematic diagram showing a comparison of data distribution of different training methods with the same amount of data on a data set according to an embodiment of the present disclosure is shown;

[0025] Figure 5 A block diagram schematically shows a structure of an adversarial training device for an autonomous driving recognition model according to an embodiment of the present disclosure; and

[0026] Figure 6A block diagram of an electronic device suitable for implementing an adversarial training method for an autonomous driving recognition model according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0027] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0028] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0029] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0030] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0031] In the technical solution of the present disclosure, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0032] In the scenario of using personal information for automated decision-making, the methods, devices, and systems provided by the embodiments of the present disclosure provide users with corresponding operation portals for users to choose to agree or reject the automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered. The expression "automated decision-making" here refers to the activity of automatically analyzing and evaluating a person's behavioral habits, interests and hobbies, or economic, health, credit status, etc. through computer programs, and making decisions. The expression "expert decision-making" here refers to the activity of making decisions by people who specialize in a certain field, have specialized experience, knowledge and skills, and have reached a certain level of professionalism.

[0033] An embodiment of the present disclosure provides an adversarial training method for an autonomous driving recognition model, comprising: processing an original image using N perturbed block sub-images in a perturbed block image to obtain N sample images to be denoised, wherein the perturbed block sub-images are obtained by segmenting the perturbed block image, and N is an integer greater than 1; inputting the N sample images to be denoised into a denoiser to obtain N adversarial sample images; processing the N adversarial sample images and labels according to a recognition loss function to obtain a recognition loss value, wherein the recognition loss function is used to train a recognition model, and the labels represent the true recognition results of the adversarial sample images; and adjusting the model parameters of the recognition model based on the recognition loss value.

[0034] Figure 1 The application scenario diagram of the adversarial training method for the autonomous driving recognition model according to an embodiment of the present disclosure is schematically shown.

[0035] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a car 101, a network 102, and a server 103. The network 102 is used to provide a medium for a communication link between the car 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0036] The user can use the car 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the car 101, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0037] The car 101 may be any electronic device having a display screen and supporting web browsing, including but not limited to a smart phone, a tablet computer, a laptop computer, a desktop computer, and the like.

[0038] The server 103 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the car 101. The background management server may analyze and process received data such as user requests, and feed back processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0039] It should be noted that the adversarial training method for the autonomous driving recognition model provided in the embodiment of the present disclosure can generally be executed by the server 103. Accordingly, the adversarial training device for the autonomous driving recognition model provided in the embodiment of the present disclosure can generally be set in the server 103. The adversarial training method for the autonomous driving recognition model provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 103 and can communicate with the car 101 and / or the server 103. Accordingly, the adversarial training device for the autonomous driving recognition model provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 103 and can communicate with the car 101 and / or the server 103.

[0040] It should be understood that Figure 1 The number of cars, networks and servers in the embodiment is only for illustration. Any number of cars, networks and servers may be provided as required.

[0041] Figure 2 A flowchart of an adversarial training method for an autonomous driving recognition model according to an embodiment of the present disclosure is schematically shown.

[0042] like Figure 2 As shown, the adversarial training method for the autonomous driving recognition model of this embodiment includes operations S210 to S240.

[0043] In operation S210, the original image is processed using N disturbed block sub-images in the disturbed block image to obtain N sample images to be denoised, where the disturbed block sub-images are obtained by segmenting the disturbed block image, and N is an integer greater than 1.

[0044] According to an embodiment of the present disclosure, the original image may be an image captured during the automatic driving process of the vehicle, for example, an image of the road ahead.

[0045] According to an embodiment of the present disclosure, the disturbed block image may be a noise image having different types of noise, such as Gaussian noise, salt and pepper noise, Poisson noise, etc. The disturbed image may also be a distorted image.

[0046] According to an embodiment of the present disclosure, the disturbed block sub-image is a part of the disturbed block image. Exemplarily, the disturbed sub-image may be a designated position image of the disturbed block image, or may be a random position image of the disturbed block image. For example, the disturbed sub-image may be the center position of the disturbed image.

[0047] According to an embodiment of the present disclosure, N disturbance block sub-images are superimposed on the original image to obtain N sample images to be denoised, and the superimposed position may also be the image recognition area of ​​the original image. The image recognition area may be the entire original image or an image recognition sensitive area in the original image.

[0048] In operation S220, N sample images to be denoised are input into a denoiser to obtain N adversarial sample images.

[0049] According to an embodiment of the present disclosure, a denoiser is used to reduce noise in a sample image to be denoised.

[0050] In operation S230, N adversarial sample images and labels are processed according to a recognition loss function to obtain a recognition loss value, where the recognition loss function is used to train a recognition model, and the label represents a true recognition result of the adversarial sample image.

[0051] According to an embodiment of the present disclosure, the recognition loss function may be cross entropy, or may be a loss function of Model Agnostic Theory (MAT) and a loss function of Training with Regularization for Adversarial Sample Detection and Evaluation (TRADES).

[0052] According to an embodiment of the present disclosure, the recognition model may be a neural network model, for example, a deep learning model, a support vector machine, a random forest model, etc., and the recognition model is not limited.

[0053] According to an embodiment of the present disclosure, the real recognition result may be the name of an object in the adversarial sample. For example, the adversarial sample may be a road condition photographed by an autonomous driving system. If there is a traffic light on the road, the real recognition result may be a red traffic light.

[0054] In operation S240 , a model parameter of the recognition model is adjusted based on the recognition loss value.

[0055] According to an embodiment of the present disclosure, the model parameters may be learnable variables of the recognition model, and the model parameters are adjusted by an optimization algorithm during the training process to minimize the loss value of the model.

[0056] According to an embodiment of the present disclosure, the original image is processed by using N perturbation block sub-images in the perturbation block image to obtain N sample images to be denoised, where the perturbation block sub-images are obtained by segmenting the perturbation block image, and N is an integer greater than 1; the N sample images to be denoised are input into a denoiser to obtain N adversarial sample images, thereby increasing the number of adversarial sample images for training a recognition model, and enhancing subsequent adversarial training.

[0057] N adversarial sample images and labels are processed according to the recognition loss function to obtain a recognition loss value. The recognition loss function is used to train the recognition model, and the label represents the true recognition result of the adversarial sample image. The model parameters of the recognition model are adjusted based on the recognition loss value, which combines the advantages of adversarial denoising and adversarial training. The training method of the present application improves the robustness of the recognition model.

[0058] According to an embodiment of the present disclosure, the original image is processed using N disturbance block sub-images in the disturbance block image to obtain N sample images to be denoised, including: segmenting the original image to obtain N original sub-images and position information of the original sub-images, wherein the position information of the original sub-images represents the position of the original sub-images in the original image; determining a target disturbance block sub-image from the N disturbance block sub-images according to the position information of the original sub-images; and inserting the target disturbance block sub-image into the original image to obtain N sample images to be denoised.

[0059] According to an embodiment of the present disclosure, the original image may be divided into N equal parts. For example, the original image may be divided into four equal parts, six equal parts, eight equal parts, etc. according to the size of the original image.

[0060] Alternatively, the original image may be segmented first to obtain a recognition region image, and then the recognition region image may be segmented.

[0061] According to an embodiment of the present disclosure, determining a target disturbed block sub-image from N disturbed block sub-images according to the position information of the original sub-image includes: determining a first original sub-image and the position information of the first original sub-image from the N original sub-images, where the first original sub-image is an original sub-image to be processed; and determining a target disturbed block sub-image having the same position information as the first original sub-image from the N disturbed block sub-images.

[0062] According to the embodiment of the present disclosure, the segmentation method of the perturbed image is the same as that of the original image, and the positions of the original sub-images and the perturbed sub-images obtained by segmentation can be sorted respectively. Then, each original sub-image has a perturbed sub-image with the same position sorting.

[0063] According to an embodiment of the present disclosure, a target disturbed block sub-image having the same position order is determined from N disturbed block sub-images according to the position information of the original sub-image.

[0064] According to an embodiment of the present disclosure, the method further includes: determining a second original sub-image from the N original sub-images, the second original sub-image being an original sub-image that retains features of the original image.

[0065] According to an embodiment of the present disclosure, inserting a target perturbation block sub-image into an original image to obtain N sample images to be denoised includes: inserting a target perturbation block sub-image into a first original sub-image in the original image to obtain a first processed sub-image; and obtaining N sample images to be denoised based on the first processed sub-image and the second original sub-image.

[0066] According to an embodiment of the present disclosure, when a target disturbance block sub-image is inserted into a first original sub-image in an original image to obtain a first processed sub-image, pixels of a second original sub-image are set to zero to obtain a sample image to be denoised, so as to maintain the size of the original image.

[0067] According to an embodiment of the present disclosure, the loss function used to train the denoiser is the mean square error.

[0068] The formula for using mean square error (MSE) is as follows:

[0069] (1)

[0070] L D represents the mean square error, M is the number of original images, N can be 4, and 4×M sample images to be denoised are trained separately. ij is the jth sample sub-image to be denoised in the sample image to be denoised corresponding to the i-th original image. For example, there are 10 original images in total, i can be 5, j is an integer less than N, and j can be 3. 53 It represents the sample image to be denoised that corresponds to the fifth original image and has a position ranking of 3, that is, there are four sample images to be denoised that correspond to the fifth original image, and the sample image to be denoised that has a position ranking of 3 is determined from the four.

[0071] Since adversarial training is relatively inefficient and the denoiser is used to increase the diversity of adversarial samples without requiring high-quality denoised images, the mean squared error (MSE) is a lightweight and efficient loss function that is sufficient to meet training requirements.

[0072] According to an embodiment of the present disclosure, the denoiser uses a nearest neighbour interpolation method for upsampling and does not use a maximum pooling layer in the denoiser for downsampling.

[0073] According to an embodiment of the present disclosure, the original sub-image may be the first original sub-image or the second original sub-image. Using the maximum pooling layer will result in downsampling, thereby reducing the resolution and possibly removing key information in these original sub-images, which is not ideal for denoising tasks and therefore needs to be discontinued. Using the nearest neighbor interpolation method in the upsampling stage to avoid introducing new values ​​so as not to distort the local details of the original sub-image helps to maintain the original structure and details of the original sub-image and improve the denoising effect.

[0074] Figure 3 A schematic diagram of the framework structure of adversarial denoising training according to an embodiment of the present disclosure is schematically shown.

[0075] For example, Figure 3 The adversarial denoising training shown in the figure is to divide an original image x into four original sub-images of equal size, and insert the target perturbation block sub-image into the original sub-image. The perturbation block image is δ in the figure. adv . De-noise the disturbance in one original sub-image at a time to obtain the reconstructed original sub-image x Padv， While keeping the other original sub-images unchanged, this process will generate adversarial sample images x Dadv , the original sub-image can be called a patch, and a patch-based adversarial denoiser is implemented to reduce the perturbations. By applying this method to each of the four original sub-images, the order of magnitude of the sample image is increased by 4 times.

[0076] The denoiser D is designed using the U-Net architecture. First, the denoiser operates only on the first original sub-image of the example, and the second original sub-image must be set to zero to maintain the original image size. Therefore, the max pooling layer is not used in the downsampling stage, and the nearest neighbor interpolation is used in the upsampling stage to enlarge the example, L D represents the loss function of the denoiser. Secondly, the learning goal of the structure is adversarial perturbation. In the training phase, the recognition loss function (L F )’s gradient to generate adversarial examples.

[0077] Then x Dadv The recognition model is also fitted with the full original images to ensure that the distribution does not show obvious deviations. No additional stage is required to train the adversarial denoiser, and the generated original and denoised images can be directly used to update the denoiser during training, thus achieving an end-to-end adversarial training approach.

[0078] According to an embodiment of the present disclosure, adjusting the model parameters of the recognition model based on the recognition loss value includes: updating the perturbation block image based on the recognition loss value to obtain an updated perturbation block image; using the updated perturbation block image to process the original image to obtain N updated sample images to be denoised; inputting the N updated sample images to be denoised into a denoiser to obtain N updated adversarial sample images; and adjusting the model parameters of the recognition model using the N updated adversarial sample images.

[0079] According to an embodiment of the present disclosure, the above method further includes: adjusting the model parameters of the denoiser using the N updated sample images to be denoised.

[0080] For example, the adversarial training method for autonomous driving recognition models can be flexibly combined with other defense methods that optimize loss functions. For example, using MAT as the optimization target:

[0081] (2)

[0082] represents the maximum likelihood estimate, express neighborhood, Y represents the label, and is a set of adversarial sample images. In MAT, we use the cross entropy loss as L and use the denoiser To train the recognition model .also, This indicates that the adversary fully understands the entire ADT (Adversarial Denoising-based Training) training framework, which includes the denoiser and Model Therefore, the defense capability of the adversarial denoising training framework under white-box attacks is demonstrated.

[0083] To demonstrate the effectiveness of the adversarial training approach for autonomous driving recognition models, other state-of-the-art optimization methods such as TRADES are also experimented.

[0084] (3)

[0085] represents the maximum likelihood estimate, express neighborhood, and is a set of adversarial sample images, Y represents the label, D is the denoiser, and F is the recognition model. λ is a regularization parameter used to balance the importance of natural error and robust error. is the cross entropy loss, is the Kullback-Leibler divergence loss.

[0086] In the inference phase, a similar strategy is adopted by reducing the perturbation of only one original sub-image and obtaining multiple adversarial sample images. Afterwards, the average of multiple adversarial sample images is taken as the final inference result, achieving excellent defense performance. The reason is that, regardless of the white-box or black-box setting, attacking the recognition model by adding adversarial perturbations to a single original sub-image or only the entire original image cannot guarantee a high success rate for the attacker. Therefore, it is more difficult to attack the model trained using the ADT framework than the recognition model trained using the traditional adversarial training method.

[0087] like Figure 3 As shown, a schematic diagram of the structure of adversarial denoising training, namely the ADT training framework. In order to utilize the sample quantity enhancement strategy to enhance adversarial training, and combine the advantages of adversarial denoising and adversarial training. The framework can be divided into two steps: a block-by-block adversarial denoising step and an adversarial training step. First, an adversarial perturbation is added to the original image to generate an adversarial sample image. The original image is divided into several original sub-images, and each first original sub-image is denoised to reduce the perturbation. Thereby, an adversarial sample image for training is created, while the second original sub-image remains unchanged. Among them, the first original sub-image can be any original sub-image in the original image, and the second original sub-image is the other original sub-image in the original image except the first original sub-image.

[0088] The denoiser is trained on sample images to be denoised and updated sample images to be denoised, and the recognition model can be trained on adversarial sample images and updated adversarial sample images to avoid bias in data distribution. After each training cycle, the denoiser and recognition model are updated to encourage the attacker to create new adversarial samples.

[0089] Figure 4 A schematic diagram showing a comparison of data distribution of different training methods with the same amount of data on a data set according to an embodiment of the present disclosure is shown.

[0090] Figure (a) shows the data distribution of adversarial sample images generated by ordinary methods, and Figure (b) shows the data distribution of adversarial sample images generated by adversarial training methods for autonomous driving recognition models.

[0091] According to the use of ADT, two main advantages are provided: increasing diversity and reducing the attack strength of adversarial examples. Figure 4As shown, compared with the traditional adversarial training method (a), the adversarial training method (b) for the autonomous driving recognition model significantly increases the number of training samples while ensuring that data of the same category is more concentrated in the feature space. In addition, in order to improve robustness, the traditional adversarial training method gradually increases the attack strength of adversarial samples. In the adversarial training method for the autonomous driving recognition model, the recognition model only needs to fit the denoised adversarial samples with lower attack strength. In summary, it is proved that adversarial training can be enhanced by using the sample number enhancement strategy. Under ADT, the advantages of adversarial denoising and adversarial training are combined. The diversity of adversarial sample images is increased and the attack strength of adversarial samples is reduced. Extensive analysis of various datasets shows that ADT outperforms other defense mechanisms in achieving excellent adversarial robustness.

[0092] ADT has high effectiveness on benchmark datasets under digital and physical attacks, can achieve higher robustness on traffic sign datasets, and also achieves better performance on commonly used adversarial datasets.

[0093] Figure 5 The structural block diagram of the adversarial training device for the autonomous driving recognition model according to an embodiment of the present disclosure is schematically shown.

[0094] like Figure 5 As shown, the adversarial training device 500 for the autonomous driving recognition model of this embodiment includes a first processing module 510, an input module 520, a second processing module 530 and an adjustment module 540.

[0095] The first processing module 510 is used to process the original image using N disturbed block sub-images in the disturbed block image to obtain N sample images to be denoised, where the disturbed block sub-images are obtained by segmenting the disturbed block image, and N is an integer greater than 1. In one embodiment, the first processing module 510 can be used to perform the operation S210 described above, which will not be described in detail herein.

[0096] The input module 520 is used to input N sample images to be denoised into the denoiser to obtain N adversarial sample images. In one embodiment, the input module 520 can be used to perform the operation S220 described above, which will not be described in detail here.

[0097] The second processing module 530 is used to process N adversarial sample images and labels according to the recognition loss function to obtain a recognition loss value, the recognition loss function is used to train the recognition model, and the label represents the real recognition result of the adversarial sample image. In one embodiment, the second processing module 530 can be used to perform the operation S230 described above, which will not be repeated here.

[0098] The first adjustment module 540 is used to adjust the model parameters of the recognition model based on the recognition loss value. In one embodiment, the adjustment module 540 can be used to perform the operation S240 described above, which will not be described in detail here.

[0099] According to an embodiment of the present disclosure, the first processing module 510 includes a slicing submodule, a first determining submodule and an inserting submodule. The slicing submodule is used to slice the original image to obtain N original subimages and position information of the original subimages, and the position information of the original subimages represents the position of the original subimages in the original image. The first determining submodule is used to determine the target disturbed block subimage from the N disturbed block subimages according to the position information of the original subimages. The inserting submodule is used to insert the target disturbed block subimage into the original image to obtain N sample images to be denoised.

[0100] According to an embodiment of the present disclosure, the first determination submodule includes a first determination unit and a second determination unit. The first determination unit is used to determine a first original subimage and position information of the first original subimage from N original subimages, and the first original subimage is an original subimage to be processed. The second determination unit is used to determine a target perturbation block subimage having the same position information as the first original subimage from the N perturbation block subimages.

[0101] According to an embodiment of the present disclosure, the above-mentioned device also includes a second determination module, which is used to determine a second original sub-image from the N original sub-images, and the second original sub-image is an original sub-image that retains the characteristics of the original image.

[0102] According to an embodiment of the present disclosure, the insertion submodule includes an insertion unit and an acquisition unit. The insertion unit is used to insert the target disturbance block subimage into the first original subimage in the original image to obtain the first processed subimage. The acquisition unit is used to obtain N sample images to be denoised based on the first processed subimage and the second original subimage.

[0103] According to an embodiment of the present disclosure, the loss function used to train the denoiser is the mean square error.

[0104] According to an embodiment of the present disclosure, the denoiser uses a nearest neighbour interpolation method for upsampling and does not use a maximum pooling layer in the denoiser for downsampling.

[0105] According to an embodiment of the present disclosure, the first adjustment module 540 includes an updating submodule, a processing submodule, an input submodule and an adjustment submodule. The updating submodule is used to update the perturbed block image based on the recognition loss value to obtain an updated perturbed block image. The processing submodule is used to process the original image using the updated perturbed block image to obtain N updated sample images to be denoised. The input submodule is used to input the N updated sample images to be denoised into the denoiser to obtain N updated adversarial sample images. The adjustment submodule is used to adjust the model parameters of the recognition model using the N updated adversarial sample images.

[0106] According to an embodiment of the present disclosure, the above-mentioned device further includes a second adjustment module, and the second adjustment module is used to adjust the model parameters of the denoiser using the N updated sample images to be denoised.

[0107] It should be noted that the adversarial training device part of the autonomous driving recognition model in the embodiments of the present disclosure corresponds to the adversarial training method part of the autonomous driving recognition model in the embodiments of the present disclosure. The description of the adversarial training device part of the autonomous driving recognition model specifically refers to the adversarial training method part of the autonomous driving recognition model, which will not be repeated here.

[0108] According to an embodiment of the present disclosure, any multiple modules of the first processing module 510, the input module 520, the second processing module 530 and the adjustment module 540 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the first processing module 510, the input module 520, the second processing module 530 and the adjustment module 540 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in a suitable combination of any of them. Alternatively, at least one of the first processing module 510 , the input module 520 , the second processing module 530 , and the adjustment module 540 may be at least partially implemented as a computer program module, and when the computer program module is executed, a corresponding function may be performed.

[0109] Figure 6 A block diagram of an electronic device suitable for implementing an adversarial training method for an autonomous driving recognition model according to an embodiment of the present disclosure is schematically shown.

[0110] like Figure 6 As shown, the electronic device 600 according to an embodiment of the present disclosure includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 to a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include an onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0111] In RAM 603, various programs and data required for the operation of electronic device 600 are stored. Processor 601, ROM 602 and RAM 603 are connected to each other via bus 604. Processor 601 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 602 and / or RAM 603. It should be noted that the program can also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0112] According to an embodiment of the present disclosure, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to the bus 604. The electronic device 600 may further include one or more of the following components connected to the input / output (I / O) interface 605: an input portion 606 including a keyboard, a mouse, etc.; an output portion 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 608 including a hard disk, etc.; and a communication portion 609 including a network interface card such as a LAN card, a modem, etc. The communication portion 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage portion 608 as needed.

[0113] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0114] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 602 and / or RAM 603 described above and / or one or more memories other than ROM 602 and RAM 603.

[0115] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the method provided by the embodiment of the present disclosure.

[0116] The above functions defined in the system / device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 601. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0117] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 609, and / or installed from a removable medium 611. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0118] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, apparatus, module, unit, etc. described above can be implemented by a computer program module.

[0119] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0120] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0121] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0122] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. An adversarial training method for an autonomous driving recognition model, characterized in that: The method comprises: Using N disturbance block sub-images in the disturbance block image to process the original image, N sample images to be denoised are obtained, wherein the disturbance block sub-images are obtained by segmenting the disturbance block image, and N is an integer greater than 1; Inputting the N sample images to be denoised into a denoiser to obtain N adversarial sample images; Processing the N adversarial sample images and labels according to a recognition loss function to obtain a recognition loss value, wherein the recognition loss function is used to train a recognition model, and the labels represent true recognition results of the adversarial sample images; adjusting a model parameter of the recognition model based on the recognition loss value; The method of processing the original image by using the N disturbance block sub-images in the disturbance block image to obtain N sample images to be denoised includes: Segmenting the original image to obtain N original sub-images and position information of the original sub-images, wherein the position information of the original sub-images represents the positions of the original sub-images in the original image; According to the position information of a first original sub-image in the original sub-images, determining a target disturbed block sub-image having the same position information as the first original sub-image from the N disturbed block sub-images, the first original sub-image being the original sub-image to be processed; Inserting the target disturbance block sub-image into the original image to obtain N sample images to be denoised.

2. The method according to claim 1, characterized in that Determining the target disturbed block sub-image from the N disturbed block sub-images according to the position information of the original sub-image comprises: Determine a first original sub-image and position information of the first original sub-image from the N original sub-images; The target disturbed block sub-image having the same position information as the first original sub-image is determined from the N disturbed block sub-images.

3. The method according to any one of claims 2, characterized in that The method further comprises: A second original sub-image is determined from the N original sub-images, where the second original sub-image is an original sub-image that retains the features of the original image.

4. The method according to claim 3, characterized in that Inserting the target disturbance block sub-image into the original image to obtain N sample images to be denoised comprises: Inserting the target disturbance block sub-image into the first original sub-image in the original image to obtain a first processed sub-image; According to the first processed sub-image and the second original sub-image, N sample images to be denoised are obtained.

5. The method according to claim 1, characterized in that The loss function used to train the denoiser is the mean square error.

6. The method according to claim 1, characterized in that The denoiser uses a nearest neighbour interpolation method for upsampling and does not use a maximum pooling layer in the denoiser for downsampling.

7. The method according to claim 1, characterized in that The adjusting the model parameters of the recognition model based on the recognition loss value comprises: updating the disturbance block image based on the recognition loss value to obtain an updated disturbance block image; The original image is processed using the updated disturbance block image to obtain N updated sample images to be denoised; Inputting the N updated sample images to be denoised into the denoiser to obtain N updated adversarial sample images; The model parameters of the recognition model are adjusted using the N updated adversarial sample images.

8. The method according to claim 7, characterized in that The method further comprises: The model parameters of the denoiser are adjusted using the N updated sample images to be denoised.

9. An adversarial training device for an autonomous driving recognition model, characterized in that: The device comprises: A first processing module is used to process the original image using N disturbance block sub-images in the disturbance block image to obtain N sample images to be denoised, wherein the disturbance block sub-images are obtained by segmenting the disturbance block image, and N is an integer greater than 1; An input module, used for inputting the N sample images to be denoised into a denoiser to obtain N adversarial sample images; A second processing module is used to process N adversarial sample images and labels according to a recognition loss function to obtain a recognition loss value, wherein the recognition loss function is used to train a recognition model, and the label represents a true recognition result of the adversarial sample image; A first adjustment module, configured to adjust a model parameter of the recognition model based on the recognition loss value; The first processing module comprises: A segmentation module, used for segmenting the original image to obtain N original sub-images and position information of the original sub-images, wherein the position information of the original sub-images represents the position of the original sub-images in the original image; a first determining submodule, configured to determine, according to position information of a first original subimage in the original subimages, a target disturbed block subimage having the same position information as the first original subimage from the N disturbed block subimages, wherein the first original subimage is an original subimage to be processed; The inserting submodule is used to insert the target disturbance block sub-image into the original image to obtain N sample images to be denoised.

Images