Target recognition model training method and device, equipment and medium

Abstract

The invention discloses a target recognition model training method and device, equipment and a medium, and relates to the field of deep learning, and the method comprises the following steps: generating an adversarial attack texture based on a DAS algorithm, and the adversarial attack texture is used for carrying out the interference when the target recognition model recognizes the target content in an image; fusing the anti-attack texture with the sample image data to generate first image data; target content in the first image data is recognized through a target recognition model, a first detection result is obtained, and the first detection result comprises the inclusion condition, predicted by the target recognition model, of the target content in the first image data; and training the target identification model based on the difference between the first detection result and the target content label. According to the invention, the ability of the target recognition model to confront the network attack can be determined, and targeted training is performed on the target recognition model in time, so that the target recognition model still has robustness when facing the physical confrontation attack.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of deep learning, and in particular to a method, apparatus, device, and medium for training a target recognition model. Background Art

[0002] Environmental perception technology is a key technology for the safe driving of autonomous vehicles. The vehicle's ability to accurately perceive the environment depends on the target recognition model. The target recognition model can identify and locate pedestrians, vehicles, traffic signs and other objects around the vehicle, providing the autonomous vehicle with the necessary information to make correct driving decisions.

[0003] In related technologies, target recognition models are usually trained based on a large amount of labeled data. Image data containing target objects is collected and manually labeled to determine the location and category of the target objects in the image. The image data is input as training data into the target recognition model for training, and the obtained target recognition model is applied to the field of autonomous driving.

[0004] However, when the target recognition model of an autonomous vehicle is subjected to adversarial attacks, the environmental images collected by the autonomous vehicle will be modified, and the model will not be able to accurately identify the target objects in the environmental images, causing safety issues. Therefore, using only ordinary image data to train the target recognition model cannot guarantee the robustness of the target recognition model. Summary of the invention

[0005] The embodiment of the present application provides a method, device, equipment, and medium for training a target recognition model, which can train a target recognition model. The technical solution is as follows:

[0006] In one aspect, a method for training a target recognition model is provided, the method comprising:

[0007] Generate an adversarial attack texture based on a dual attention distraction suppression attack DAS algorithm, wherein the adversarial attack texture is used to interfere when a target recognition model recognizes target content in an image, wherein the target content includes at least one of a vehicle and a pedestrian;

[0008] Fusion of the adversarial attack texture with sample image data to generate first image data, wherein the sample image data is used to simulate an environment image collected by an autonomous driving vehicle, and the sample image data is annotated with a target content label, and the target content label is used to indicate the distribution of target content in the sample image data;

[0009] Recognize the target content in the first image data by using the target recognition model to obtain a first detection result, where the first detection result includes the inclusion of the target content in the first image data predicted by the target recognition model;

[0010] The target recognition model is trained based on the difference between the first detection result and the target content label.

[0011] On the other hand, a training device for a target recognition model is provided, the device comprising:

[0012] A texture generation module, used to generate an adversarial attack texture based on a dual attention distraction suppression attack DAS algorithm, wherein the adversarial attack texture is used to interfere when a target recognition model recognizes target content in an image, wherein the target content includes at least one of a vehicle and a pedestrian;

[0013] An image generation module, configured to fuse the adversarial attack texture with sample image data to generate first image data, wherein the sample image data is used to simulate an environment image collected by an autonomous driving vehicle, and the sample image data is annotated with a target content label, and the target content label is used to indicate the distribution of target content in the sample image data;

[0014] an identification module, configured to identify target content in the first image data by using the target identification model to obtain a first detection result, wherein the first detection result includes the inclusion of the target content in the first image data predicted by the target identification model;

[0015] A training module is used to train the target recognition model based on the difference between the first detection result and the target content label.

[0016] In an optional embodiment, the image generation module is also used to obtain a target content configuration file, and the target content configuration file is used to generate a three-dimensional simulation model of the target content in the first image data; the first image data is generated based on the adversarial attack texture and the target content configuration file through an autonomous driving simulator, wherein the first image data contains the three-dimensional simulation model of the target content.

[0017] In an optional embodiment, the image generation module is also used to input the adversarial attack texture and the target content configuration file into the autonomous driving simulator, and output a three-dimensional simulation model of the target content integrated with the adversarial attack texture; and generate the first image data based on the three-dimensional simulation model of the target content.

[0018] In an optional embodiment, the image generation module is also used to obtain a test case code, wherein the test case code is used to instruct the generation of the first image data; based on the test case code, the autonomous driving simulator is controlled to generate a three-dimensional environment image, wherein the three-dimensional environment image is used to simulate the environment in which the autonomous driving vehicle is located; and the three-dimensional simulation model of the target content is added to the three-dimensional environment image to obtain the first image data.

[0019] In an optional embodiment, the texture generation module is also used to obtain a three-dimensional model configuration file of the target content, the three-dimensional model configuration file includes a model main body part and a model texture part, the model main body part is used to describe the three-dimensional shape of the target content, and the model texture part refers to the texture covering the surface of the model main body part; based on the DAS algorithm, the model texture part is transformed to generate the adversarial attack texture.

[0020] In an optional embodiment, the recognition module is also used to convert the format of the first image data to obtain test data that meets the preset format requirements; input the test data into the target recognition model; identify the presence of the target content in the test data through the target recognition model to obtain the first detection result; wherein the first detection result includes a bounding box first detection result, a global first detection result, a three-dimensional target first detection result and an average direction similarity first detection result; the bounding box first detection result is used to indicate the target recognition model's recognition of the position of the target content in the first image data, the global first detection result is used to indicate the target recognition model's recognition of the position of the target content in the first image data from a global perspective, the three-dimensional target first detection result is used to indicate the target recognition model's recognition of the target content in three-dimensional space, and the average direction similarity first detection result is used to indicate the target recognition model's recognition of the orientation of the target content in the first image data.

[0021] In an optional embodiment, the training module is also used to train the target recognition model based on adversarial attack training data when the difference between the first detection result and the target content label meets a preset adversarial training condition, wherein the adversarial attack training data is image data obtained based on the adversarial attack texture generated by the DAS algorithm.

[0022] On the other hand, a computer device is provided, which includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement a target recognition model training method as described in any of the above-mentioned embodiments of the present application.

[0023] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction, at least one program, a code set or an instruction set is stored in the storage medium, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement a target recognition model training method as described in any of the above-mentioned embodiments of the present application.

[0024] On the other hand, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the training method of the target recognition model described in any of the above embodiments.

[0025] The beneficial effects brought by the technical solution provided by the embodiment of the present application include at least:

[0026] Integrating adversarial attack textures into sample image data can simulate the impact of physical adversarial noise on autonomous vehicles when collecting images for recognition. Image data containing adversarial attack textures can be used to test the recognition effect of the target recognition model, determine the ability of the target recognition model to resist network attacks, and conduct targeted training on the target recognition model in a timely manner, so that the target recognition model is still robust in the face of physical adversarial attacks. The DAS algorithm can generate effective adversarial attack textures, generate training data in large quantities to train the target recognition model, and improve the training efficiency of the target recognition model. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0028] Figure 1 is a schematic diagram of a training system for a target recognition model provided by an exemplary embodiment of the present application;

[0029] Figure 2is a flowchart of testing the robustness of a target recognition model provided by an exemplary embodiment of the present application;

[0030] Figure 3 is a flow chart of a method for training a target recognition model provided by an exemplary embodiment of the present application;

[0031] Figure 4 It is a schematic diagram of the code relationship involved in generating an adversarial attack texture provided by an exemplary embodiment of the present application;

[0032] Figure 5 is a schematic diagram of an adversarial attack texture generated based on a first type of obstacle and a three-dimensional model of an obstacle superimposed with the adversarial attack texture provided by an exemplary embodiment of the present application;

[0033] Figure 6 is a schematic diagram of an adversarial attack texture generated based on a second type of obstacle and a three-dimensional model of an obstacle superimposed with the adversarial attack texture provided by an exemplary embodiment of the present application;

[0034] Figure 7 is a schematic diagram of image comparison before and after a physical attack on an image with a bus as an obstacle, provided by an exemplary embodiment of the present application;

[0035] Figure 8 is a structural block diagram of a training device for a target recognition model provided by an exemplary embodiment of the present application;

[0036] Fig. 9 It is a structural block diagram of a computer device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0037] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0038] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0039] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0040] It should be noted that the information and data involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0041] It should be understood that, although the terms first, second, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first parameter may also be referred to as the second parameter, and similarly, the second parameter may also be referred to as the first parameter. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0042] First, a brief introduction is given to the terms involved in the embodiments of this application:

[0043] Dual Attention Suppression (DAS): is an adversarial attack method for deep learning models that is able to generate adversarial camouflages that are highly effective in the physical world. The model specifically focuses on generating adversarial examples that are effective against a variety of models in a black-box setting, while also looking visually natural and effective. DAS works by using an attention mechanism to generate deceptive adversarial camouflages, tested and validated in both the digital and physical worlds. The model is able to generate these camouflages in a 3D simulation environment and shows good performance in a black-box setting.

[0044] In the present application, the model texture part of the three-dimensional model of the target content is stripped off, and the model texture part is transformed based on the DAS algorithm to generate an adversarial attack texture. The adversarial attack texture can be re-integrated with the model body of the three-dimensional model. An image of the three-dimensional model containing the target content is collected in a simulation environment. The image is integrated with the adversarial attack texture, which can interfere with the target recognition model when the target recognition model recognizes the target content in the image.

[0045] For example, a stop sign image refers to an image that contains a stop sign (as target content) that is integrated with an adversarial attack texture. That is, the stop sign image is integrated with the adversarial attack texture. The target recognition model inside the autonomous driving vehicle recognizes the stop sign image and identifies the stop sign in the stop signal image as a deceleration sign, resulting in a misjudgment. At this time, the autonomous driving vehicle is still moving.

[0046] Based on the principle that different models have similar attention patterns, DAS attempts to divert the model's attention from the target area to the non-target area, while generating reasonable patterns in the human attention pattern to achieve the effect of attack. In terms of implementation, the image is first input into the target model, the attention map of the target area is calculated, and the model's attention is diverted to the non-target area using a connectivity graph-based method. The corresponding loss function is calculated and gradient backpropagation is performed. Finally, the perturbed image is updated and the whole process is repeated.

[0047] KITTI / kitti format (Karlsruhe Institute of Technology and ToyotaTechnological Institute, a data format): KITTI format is a general term for a series of data formats related to the KITTI dataset, which is a large benchmark dataset widely used in computer vision and autonomous driving research. Exemplary, the specific file formats of the KITTI dataset include the following. Image files: usually stored in 8-bit PNG format, may contain cropped and distortion-corrected images; LiDAR point cloud data: stored in binary format, containing three-dimensional point cloud data obtained by LiDAR scanning, each point contains x, y, z coordinates and reflection intensity information.

[0048] SMOKE model (Single-Stage Monocular 3D Object Detection via KeypointEstimation): It directly predicts the 3D attribute information of the target through key point estimation and 3D (Dimension) box regression, instead of relying on traditional 2D detection methods, thus avoiding the noise that may be introduced by 2D detection and improving the performance of 3D detection.

[0049] Next, the training system of the target recognition model involved in the embodiment of the present application is described. For illustration, please refer to Figure 1 The implementation environment involves a terminal 110 and a server 120. The terminal 110 and the server 120 are connected via a communication network 130.

[0050] Taking the perception of the external environment by an autonomous vehicle as an example, the autonomous vehicle collects environmental images from its own perspective, recognizes the environmental images through the target recognition model, determines whether there is target content (which can be called obstacles) in the environmental images, and promptly feeds back the recognition results. The autonomous vehicle determines whether to continue driving or stop based on the recognition results. Therefore, testing the robustness of the target recognition model is the key to the autonomous vehicle's ability to correctly perceive the environment and travel safely.

[0051] Optionally, the terminal 110 includes a three-dimensional model of the obstacle, and the model texture part of the three-dimensional model is separated from the model main part. The terminal 110 transforms the model texture part through the DAS algorithm, generates an adversarial attack texture, and re-covers the adversarial attack texture on the model main part of the three-dimensional model, so as to obtain a three-dimensional model fused with the adversarial attack texture. The three-dimensional model fused with the adversarial attack texture can also be referred to as a DAS model.

[0052] The DAS model is actually used to simulate obstacles in the simulation environment (which can be vehicles, pedestrians, etc.) as target content to be identified.

[0053] Attacking the autonomous vehicle based on the DAS algorithm can cause obstacles with adversarial attack textures to appear in the environmental images collected by the autonomous vehicle. In other words, the environmental images collected at this time are actually images fused with adversarial attack textures. When the target recognition model recognizes images fused with adversarial attack textures, it is difficult to accurately identify the type of obstacles, which can lead to misjudgment of the autonomous vehicle and cause safety issues.

[0054] The terminal 110 also includes an autonomous driving simulator, which is used to fuse the sample image data and the adversarial attack texture to generate first image data, that is, the first image data actually refers to the image after the adversarial attack texture is superimposed.

[0055] The terminal 110 sends the first image data to the server 120. The server 120 is deployed with a target recognition model for receiving and recognizing the first image data sent from the terminal 110 to obtain a first detection result. The first image data is used to test whether the target recognition model can accurately recognize the physical noise attack. The first detection result is used to indicate whether the target recognition model contains target content in the first image data.

[0056] The target recognition model is used to identify the sample image data and the first image data respectively. By comparing the two recognition results, it is possible to determine whether the adversarial attack texture has effective interference on the target recognition model and the degree of interference. The training method for the target recognition model is determined based on the comparison results.

[0057] The difference between the sample image data and the first image data is whether noise is superimposed / the adversarial attack texture is fused. The sample image data refers to an image without the adversarial attack texture fused, that is, an environmental image collected when the obstacle three-dimensional model without the attack texture is running in a simulation environment.

[0058] That is, when the interference effect of the adversarial attack texture on the target recognition model reaches a preset condition, the DAS algorithm can be used to generate training data in batches, and the target recognition model can be trained based on the training data to improve the robustness of the target recognition model and enhance the target recognition model's ability to resist physical noise attacks. The training data has the same format as the first image data.

[0059] Figure 2 A flowchart for testing the robustness of a target recognition model provided by an exemplary embodiment of the present application includes the following steps.

[0060] S201, training to generate adversarial attack textures.

[0061] Using the 3D model of the obstacle, the model texture part of the 3D model is separated from the main part of the model to obtain the texture in the initial state. The DAS algorithm is used to attack the texture to obtain the adversarial attack texture.

[0062] S202, importing the texture into the obstacle model.

[0063] That is, the adversarial attack texture is re-covered on the main part of the obstacle three-dimensional model to obtain a three-dimensional model fused with the adversarial attack texture.

[0064] Among them, this process is executed by an autonomous driving simulator. The files corresponding to the adversarial attack textures and the files for generating the three-dimensional models of obstacles are imported into the relevant directories of the autonomous driving simulator, and the three-dimensional DAS model is generated by the autonomous driving simulator.

[0065] S203, running the test case to generate images and coordinate data.

[0066] The test case is a pre-prepared code file. Running the test case can control the autonomous driving simulator to generate a simulated 3D environment. The simulated 3D environment contains a 3D DAS model, which is used to simulate various situations that the autonomous driving vehicle may encounter while driving. For example, if the obstacle in front of the autonomous driving vehicle is a bus, the simulated 3D environment contains the 3D model of the autonomous driving vehicle and the 3D model of the bus, and the surface of the 3D model of the bus is fused with adversarial attack textures.

[0067] Obstacles and autonomous vehicles operate in a simulated three-dimensional environment, and images are collected from the perspective of the autonomous vehicle. The images contain obstacles, and the coordinate data is used to indicate the position of the obstacles in the images.

[0068] S204, the coordinate data is converted into KITTI format data.

[0069] The coordinate data is converted into a data format suitable for recognition by the target recognition model.

[0070] S205, testing the target recognition model.

[0071] The target recognition model in this application is a model trained with the SMOKE model as the prototype. The kitti format data is input into the target recognition model, and the target recognition model recognizes the image data fused with the adversarial attack texture, and outputs the recognition result. The recognition result includes the situation of the target recognition model recognizing the target content (obstacle), including the type of target content and the location information of the target content in the image data. The recognition result is used as a test result to test the accuracy of the target recognition model in recognizing images after being attacked by physical noise and the robustness of the model.

[0072] S206, test result evaluation.

[0073] The robustness of the target recognition model is evaluated based on the test results, and the target recognition model is subsequently trained in a targeted manner. The DAS algorithm is used to batch generate image data fused with adversarial attack textures as training data to train the target recognition model.

[0074] In summary, the recognition effect of the target recognition model is tested using image data containing adversarial attack textures, the ability of the target recognition model to resist network attacks is determined, and the target recognition model is trained in a timely manner so that the target recognition model is still robust in the face of physical adversarial attacks. The DAS algorithm can generate effective adversarial attack textures, generate training data in large quantities to train the target recognition model, and improve the training efficiency of the target recognition model.

[0075] The above-mentioned terminal can be a mobile phone, a tablet computer, a desktop computer, a portable laptop computer, a smart TV, a car terminal, a smart home device and other terminal devices in various forms, which is not limited in the embodiments of the present application.

[0076] It is worth noting that the above-mentioned servers can be independent physical servers, or they can be server clusters or distributed systems composed of multiple physical servers. They can also be cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), as well as big data and artificial intelligence platforms.

[0077] Among them, cloud technology refers to a hosting technology that unifies hardware, software, network and other resources in a wide area network or local area network to realize data computing, storage, processing and sharing. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model. It can form a resource pool, which is used on demand and flexible and convenient. Cloud computing technology will become an important support. The background services of the technical network system require a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the high development and application of the Internet industry, in the future, each item may have its own identification mark, and all need to be transmitted to the background system for logical processing. Data of different levels will be processed separately. All kinds of industry data require strong system backing support, which can only be achieved through cloud computing.

[0078] In some embodiments, the above server can also be implemented as a node in a blockchain system.

[0079] In combination with the above-mentioned noun introduction and application scenarios, the training method of the target recognition model provided by the present application is described. The method can be executed by a server or a terminal, or can be executed by a server and a terminal together. In the embodiment of the present application, the method is executed by a terminal as an example. Figure 3 As shown, Figure 3 FIG. 1 is a flow chart of a method for training a target recognition model provided by an exemplary embodiment of the present application. The method includes the following steps.

[0080] Step 310: Generate an adversarial attack texture based on a dual attention distraction suppression attack DAS algorithm.

[0081] The adversarial attack texture is used to interfere when the target recognition model recognizes the target content in the image, and the target content includes at least one of a vehicle and a pedestrian.

[0082] The generation of adversarial attack textures is based on the DAS principle, which attacks the target content to be identified in the image, changes the original texture of the target content, and makes the target recognition model unable to accurately identify the information of the target content, causing the target recognition model to misjudge. This application uses the application of the target recognition model to an autonomous driving vehicle as an example to illustrate, that is, in order to improve the accuracy of the results of the autonomous driving vehicle's perception of the external environment, it is necessary to train the target recognition model in a targeted manner, simulate the attack on the target recognition model, and interfere with the recognition process of the target recognition model.

[0083] During the driving process of the autonomous driving vehicle, the target recognition model mainly focuses on the obstacles in the driving environment. Therefore, the above target content can be referred to as obstacles. Obstacles include but are not limited to pedestrians, vehicles, animals, etc., which are obstacles that the autonomous driving vehicle needs to avoid during driving.

[0084] In some embodiments, the target content may also be road slogans, signboards, etc. For example, a signboard indicating that the road ahead is impassable to instruct the autonomous driving vehicle to stop. The target recognition model needs to accurately recognize the signboard in the collected image, determine the type of the signboard, and promptly feed back the recognition result to the autonomous driving vehicle, which determines whether to continue moving forward based on the recognition result.

[0085] The adversarial attack texture is generated through the DAS algorithm. Based on the principle that different models have similar attention patterns, it attempts to distract the model's attention from the target area to the non-target area, and at the same time generates reasonable patterns in the human attention pattern to achieve the effect of the attack. In terms of implementation, the image is first input into the target model, the attention map of the target area is calculated, and the model's attention is distracted to the non-target area using a connectivity graph-based method. The corresponding loss function is calculated and gradient backpropagation is performed. Finally, the perturbed image is updated and the whole process is repeated.

[0086] The field of computer vision image recognition involves a specific loss function to identify a certain type of object. The physical noise attack on the image in this application refers to changing the loss value of the original image (the original image refers to the image that has not been attacked) so that the original loss function's recognition of the target content decreases. Among them, the attention distraction loss Ld (attention distraction loss, Ld) tends to make the pixel gradient change less noticeable to the human eye, and the smoothing loss Ls (smooth loss, Ls) is to change the pixel gradient to interfere with the recognition of the intelligent driving perception end of the autonomous driving vehicle. Both are intended to change the texture of the target content.

[0087] Among them, the calculation principles of Ld loss and Ls loss are as follows.

[0088]

[0089] Among them, G k is the sum of the attention weights in the kth connected graph, N is the number of all pixels in the connected graph, and N k is the number of all pixels in the kth connected graph. From the design of the loss function, it can be seen that the number of all pixels in the connected graph N is a fixed value, so the optimization goal will be to continuously reduce N k With G k . N k The effect of reducing G is to increase the number of connected graphs, thereby achieving the effect of "distracting" attention; k The effect of the reduction is to continuously reduce the weight value of attention, thereby achieving the effect of "suppressing" attention. Compared with random texture attacks, DAS adversarial texture distracts the model's attention to other areas outside the detection object.

[0090] L s =∑(x i，j -x i+1，j ) 2 +(x i，j -x i，j+1 ) 2

[0091] where x i，j is image I adv The pixel value at coordinate (i, j). Ls further improves the naturalness of the disguise by reducing the squared difference between adjacent pixels and thus reducing the smoothness loss.

[0092] The specific process of generating adversarial attack texture is as follows: (1) The 3D model of the obstacle is divided into a pure white model (M) and a texture (T). The pure white model (M) refers to the main part of the 3D model, which is used to describe the 3D shape of the obstacle. The texture (T) refers to the model texture part of the 3D model, that is, the texture covering the surface of the 3D model. At this time, the texture (T) has not been attacked; (2) First, put the texture (T) into the optimizer; (3) Render the 3D model (M+T) onto a pre-prepared image through the neural renderer (R) to obtain the input image Input; (4) The slice Input is input into the target recognition model A to obtain the predicted value, and the heat map (Cam) is obtained by back propagation based on the logits; (5) The heat map (Cam) is used to calculate the loss value loss. The heat map can be decomposed into three dimensions, namely RGB (Red, Green, Blue), and the pixel gradient is calculated in the three dimensions to obtain the loss value; (6) The loss value loss is back propagated, and the texture (T) in the optimizer is updated according to the gradient; (7) Steps (3) to (6) are repeated to continuously optimize the texture (T), and after the training is completed, the adversarial attack texture (Tadv) is saved.

[0093] like Figure 4 As shown, Figure 4 It is a schematic diagram of the code relationships involved in generating adversarial attack textures.

[0094] The relationship diagram 400 involves a variety of variable classes, such as: a class 401 for configuring all parameters, an optimizer class 402, a data set class 403, a class for calculating loss 404, a training class 405, etc. These variable classes are respectively related to Figure 4 There is a corresponding relationship between adjacent code statements in the code. For example, the class 401 that configures all parameters refers to Figure 4 met / engine / Config.py, dataset class 403 refers to Figure 4 in dataset / MyDataset.py.

[0095] exist Figure 4 In the above example, the top of the general execution module is attack.py, and the core module is met / engine / Trainer.py in the middle. After executing attack.py, the general configuration is started.

[0096] met / engine / Config.py, optimizer met / engine / Textureoptimizer.py and met / engine / Trainer.py, met / engine / Config.py provides necessary parameter configuration for training and optimizer, and optimizer provides optimization code optimization for each round of training. During the training process, the dataset dataset / Mydatatest.py is called, and the loss value loss needs to be calculated through the heat map during training, so . / met / modules / det / DAS_Losscaculator.py is called. That is, the code attack.py is started, the first round of training begins, the dataset and configuration parameters are called, the heat map preprocessing begins, and then the loss value is calculated through the heat map. After the calculation, the loss value is returned, and optimization is performed before the next round of training begins.

[0097] Step 320: Fusing the adversarial attack texture with the sample image data to generate first image data.

[0098] The sample image data is used to simulate the environment images collected by the autonomous driving vehicle. The sample image data is annotated with target content labels, and the target content labels are used to indicate the distribution of target content in the sample image data.

[0099] Among them, the process of fusing the adversarial attack texture with the sample image data can also be called the process of importing the adversarial attack texture into the three-dimensional model of the obstacle, so that the surface of the three-dimensional model is covered with the adversarial attack texture to obtain the DAS model, and the DAS model is added to the three-dimensional simulation environment to perform image acquisition to obtain the first image data.

[0100] Optionally, a target content configuration file is obtained, and the target content configuration file is used to generate a three-dimensional simulation model of the target content in the first image data. The first image data is generated by an autonomous driving simulator based on the adversarial attack texture and the target content configuration file, wherein the first image data includes the three-dimensional simulation model of the target content.

[0101] Obtain a test case code, where the test case code is used to instruct generation of first image data; based on the test case code, control an autonomous driving simulator to generate a three-dimensional environment image, where the three-dimensional environment image is used to simulate the environment in which the autonomous driving vehicle is located; add a three-dimensional simulation model of the target content to the three-dimensional environment image to obtain the first image data.

[0102] The adversarial attack texture and the target content configuration file are input into an autonomous driving simulator, and a three-dimensional simulation model of the target content integrated with the adversarial attack texture is output, and first image data is generated based on the three-dimensional simulation model of the target content.

[0103] Exemplarily, the main process is as follows: (1) copy the trained adversarial attack texture to the import / Package01 / Props / [xxx] folder of CARLA (autonomous driving simulator); the trained adversarial attack texture is stored as the xxx_DAS.png file; (2) the final effect is that the .obj file and .mtl file of the obstacle and the original texture .png file and the adversarial texture xxx_DAS.png file are in the same directory;

[0104] (3) Modify the obstacle's .mtl file and change the texture name of the obstacle's .png file to the trained texture name; (4) After the modification, delete / root / Carla / Unreal / CarlaUE4 / Content / Package01; (5) Restart the terminal in the / root / carla folder and run make import; (6) Enter the Package01 configuration file directory and modify the Package01.Package.json file. At this time, you can see that the model has used the adversarial attack texture in the autonomous driving simulator.

[0105] Exemplarily, the first image data generated after running the test case code includes an image portion and coordinate data. Running the test case code generates test case data for the obstacle model after physical attack in the CARLA software platform, including image data (.png file) and other coordinate position data (.npz file).

[0106] For example, the test cases are divided according to the driving conditions of the autonomous vehicle, and include a total of 12 test cases. Each test case can generate 105 image data and corresponding coordinate data. Theoretically, a total of 1260 image data and coordinate data can be generated. In actual results, due to the freeze of the computer when running the test cases, a small amount of image data is less than 105, and the total result only includes 1256 image data and corresponding coordinate data. By comparison, it can be clearly seen that the texture of the obstacle after the physical attack is quite different from before.

[0107] The main steps of this process include: (1) running the required test case code on the CARLA software platform; (2) after all the required test case data are generated, all the data are integrated into the same folder, which only contains image data images, coordinate data locations and a corresponding labels file, where the coordinate data contains the location of the target content (obstacle) in the image, and the labels file is used to indicate the specific type of the target content, such as the target content is a motorcycle, a bus, etc.

[0108] Step 330: Identify the target content in the first image data by using the target recognition model to obtain a first detection result.

[0109] The first detection result includes the inclusion of target content in the first image data predicted by the target recognition model.

[0110] The first image data is format converted to obtain test data that meets the preset format requirements; the test data is input into the target recognition model; the target content in the test data is identified by the target recognition model to obtain a first detection result.

[0111] Among them, the first detection result includes a first detection result of a bounding box, a first global detection result, a first detection result of a three-dimensional target and a first detection result of an average directional similarity; the first detection result of a bounding box is used to indicate the position of the target content in the first image data recognized by the target recognition model, the first global detection result is used to indicate the position of the target content in the first image data recognized by the target recognition model from a global perspective, the first detection result of a three-dimensional target is used to indicate the recognition of the target content in three-dimensional space by the target recognition model, and the first detection result of an average directional similarity is used to indicate the orientation of the target content in the first image data recognized by the target recognition model.

[0112] Optionally, the format of the test data is KITTI format. After the first image data is converted into KITTI format data, it can be used as input data of the target recognition model. The test data is identified by the target recognition model to determine whether the test data contains target content, and then the ability of the target recognition model to identify images fused with adversarial attack textures is evaluated.

[0113] The detection accuracy of the target recognition model is tested using data containing adversarial attack textures. The first test result contains four dimensions: (1) bbox (2D bounding box detection), which is the dimension corresponding to the first detection result of the bounding box; (2) bev (bird's eye view detection), which is the dimension corresponding to the first global detection result; (3) 3D (3D target detection), which is the dimension corresponding to the first detection result of the three-dimensional target; (4) aos (average directional similarity detection), which is the dimension corresponding to the first detection result of the average directional similarity.

[0114] Among them, the detection indicator mainly uses mAP (average precision), and the test results will be saved in the log file after the test.

[0115] Exemplarily, the test results are as follows: for the same obstacle, the target recognition model detection uses 11-point interpolation method and 40-point interpolation method to calculate the average accuracy. In the same picture, small (pixel value lower than 80*80), medium (pixel value between 80*80 and 242*242), and large (pixel value higher than 242*242) ranges are used to detect the target recognition model, and their weighted average all is taken to calculate the overall detection effect.

[0116] As shown in Tables 1 and 2 below, Table 1 is the detection result of no attack on the obstacle (bus) in a sunny day, and Table 2 is the detection result after a physical attack (i.e., superimposing the adversarial attack texture) on the obstacle (bus) in a sunny day.

[0117] Table 1

[0118]

[0119]

[0120] For Table 1 and Table 2, the first column on the left shows the detection dimensions, namely bbox (2D bounding box detection), bev (bird's eye view detection), 3D (3D object detection), and aos (average directional similarity detection). Different intersection-over-union ratios are used for each type of detection dimension.

[0121] Explanation of the above Tables 1 and 2: Since the pixel range of the small part detection is within 80*80, and the pixels of the three-dimensional obstacle model actually generated are higher than 80*80, when the small pixel range is used to detect the target content (obstacle), the target recognition model fails to correctly identify the target content, so the detection result of the small part is 0.

[0122] In bbox detection, after physical attack, the overall (all) average precision of the target recognition model for bus detection drops from 84.1644 to 11.2991, the average precision in the medium range drops from 92.9647 to 6.3866, and the average precision in the large range drops from 73.8062 to 10.7034.

[0123] In BEV detection, after physical attack, the overall (all) average precision of the target recognition model for BUS detection dropped from 80.7078 to 17.0162, the average precision in the medium range dropped from 90.8316 to 11.4242, and the average precision in the large range dropped from 67.4487 to 18.6964.

[0124] In 3D detection, after physical attack, the overall (all) average precision of the object recognition model for bus detection drops from 80.7078 to 17.0162, the average precision in the medium range drops from 90.8316 to 11.4242, and the average precision in the large range drops from 67.4487 to 18.6964.

[0125] In AOS detection, after physical attack, the overall (all) average precision of the target recognition model for bus detection dropped from 41.49 to 4.98, the average precision in the medium range dropped from 45.53 to 2.79, and the average precision in the large range dropped from 36.63 to 4.76.

[0126] This result shows that the physical attack has a great interference on the target detection of the target recognition model, and such interference is often seen in the physical world, such as graffiti on vehicles, clothes of various colors on humans, etc., which is a major hidden danger in the field of intelligent driving technology. This application uses the images generated by training that incorporate adversarial attack textures to conduct targeted training on the target detection model, which can improve the detection accuracy of the target detection model for such obstacles and enhance the safety of intelligent driving.

[0127] Step 340 : training the target recognition model based on the difference between the first detection result and the target content label.

[0128] When the difference between the first detection result and the target content label meets the preset adversarial training conditions, the target recognition model is trained based on the adversarial attack training data, wherein the adversarial attack training data is image data obtained based on the adversarial attack texture generated by the pre-trained model.

[0129] like Figure 5 As shown, Figure 5 It is a schematic diagram of the adversarial attack texture generated based on the first type of obstacle and the three-dimensional model of the obstacle superimposed with the adversarial attack texture.

[0130] The adversarial attack texture 510 generated based on the first type of obstacle is superimposed on the original three-dimensional model of the first type of obstacle to obtain an adversarial texture three-dimensional model 520. Figure 5 shown.

[0131] like Figure 6 As shown, Figure 6 It is a schematic diagram of the adversarial attack texture generated based on the second type of obstacle and the three-dimensional model of the obstacle with the adversarial attack texture superimposed.

[0132] The adversarial attack texture 610 generated based on the second type of obstacle is superimposed on the original three-dimensional model of the second type of obstacle to obtain an adversarial texture three-dimensional model 620. The effect is as follows: Figure 6 shown.

[0133] Among them, the first type of obstacles are different from the second type of obstacles, for example, Figure 5 The first type of obstacle is pedestrians. Figure 6 The corresponding second type of obstacle is vehicles.

[0134] Indicatively, Figure 7 As shown, Figure 7 It is a schematic diagram of the image comparison before and after the physical attack on the image with the obstacle of a bus.

[0135] There is a difference between the image 701 before the attack and the image 702 after the attack, and the external texture of the obstacle 700 has changed.

[0136] In summary, the training method of the target recognition model provided by the present application integrates the adversarial attack texture into the sample image data, which can simulate the influence of physical adversarial noise on the autonomous driving vehicle when collecting images for recognition, and uses the image data containing the adversarial attack texture to test the recognition effect of the target recognition model, determine the ability of the target recognition model to resist network attacks, and timely conduct targeted training on the target recognition model, so that the target recognition model is still robust in the face of physical adversarial attacks. The DAS algorithm can generate effective adversarial attack textures, generate training data in large quantities to train the target recognition model, and improve the training efficiency of the target recognition model.

[0137] Figure 8 is a structural block diagram of a training device for a target recognition model provided by an exemplary embodiment of the present application, such as Figure 8 As shown, the device includes the following parts.

[0138] A texture generation module 810, configured to generate an adversarial attack texture based on a dual attention distraction suppression attack DAS algorithm, wherein the adversarial attack texture is used to interfere when a target recognition model recognizes target content in an image, wherein the target content includes at least one of a vehicle and a pedestrian;

[0139] An image generation module 820 is used to fuse the adversarial attack texture with sample image data to generate first image data, wherein the sample image data is used to simulate an environment image collected by an autonomous driving vehicle, and the sample image data is annotated with a target content label, and the target content label is used to indicate the distribution of target content in the sample image data;

[0140] The recognition module 830 is used to recognize the target content in the first image data by using the target recognition model to obtain a first detection result, wherein the first detection result includes the inclusion of the target content in the first image data predicted by the target recognition model;

[0141] The training module 840 is used to train the target recognition model based on the difference between the first detection result and the target content label.

[0142] In an optional embodiment, the image generation module 820 is also used to obtain a target content configuration file, which is used to generate a three-dimensional simulation model of the target content in the first image data; the first image data is generated based on the adversarial attack texture and the target content configuration file through an autonomous driving simulator, wherein the first image data contains the three-dimensional simulation model of the target content.

[0143] In an optional embodiment, the image generation module 820 is further used to input the adversarial attack texture and the target content configuration file into the autonomous driving simulator, and output a three-dimensional simulation model of the target content integrated with the adversarial attack texture; and generate the first image data based on the three-dimensional simulation model of the target content.

[0144] In an optional embodiment, the texture generation module 810 is also used to obtain a three-dimensional model configuration file of the target content, the three-dimensional model configuration file includes a model main body part and a model texture part, the model main body part is used to describe the three-dimensional shape of the target content, and the model texture part refers to the texture covering the surface of the model main body part; the model texture part is transformed based on the DAS algorithm to generate the adversarial attack texture.

[0145] In an optional embodiment, the image generation module 820 is also used to obtain a test case code, wherein the test case code is used to instruct the generation of the first image data; based on the test case code, the autonomous driving simulator is controlled to generate a three-dimensional environment image, wherein the three-dimensional environment image is used to simulate the environment in which the autonomous driving vehicle is located; and the three-dimensional simulation model of the target content is added to the three-dimensional environment image to obtain the first image data.

[0146] In an optional embodiment, the recognition module 830 is also used to convert the format of the first image data to obtain test data that meets the preset format requirements; input the test data into the target recognition model; identify the presence of the target content in the test data through the target recognition model to obtain the first detection result; wherein the first detection result includes a bounding box first detection result, a global first detection result, a three-dimensional target first detection result and an average direction similarity first detection result; the bounding box first detection result is used to indicate the target recognition model's recognition of the position of the target content in the first image data, the global first detection result is used to indicate the target recognition model's recognition of the position of the target content in the first image data from a global perspective, the three-dimensional target first detection result is used to indicate the target recognition model's recognition of the target content in three-dimensional space, and the average direction similarity first detection result is used to indicate the target recognition model's recognition of the orientation of the target content in the first image data.

[0147] In an optional embodiment, the training module 840 is also used to train the target recognition model based on adversarial attack training data when the difference between the first detection result and the target content label meets a preset adversarial training condition, wherein the adversarial attack training data is image data obtained based on the adversarial attack texture generated by the DAS algorithm.

[0148] It should be noted that the training device for the target recognition model provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the training device for the target recognition model provided in the above embodiment and the training method embodiment of the target recognition model belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0149] Fig. 9 The block diagram of the structure of a computer device 900 provided by an exemplary embodiment of the present application is shown. The computer device 900 may be: a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III, Moving Picture Experts Compression Standard Audio Layer 3), an MP4 (Moving Picture Experts Group Audio Layer IV, Moving Picture Experts Compression Standard Audio Layer 4) player, a laptop computer or a desktop computer. The computer device 900 may also be called a user device, a portable terminal, a laptop terminal, a desktop terminal or other names.

[0150] Typically, the computer device 900 includes a processor 901 and a memory 902 .

[0151] The processor 901 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 901 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 901 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 901 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 901 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0152] The memory 902 may include one or more computer-readable storage media, which may be non-transitory. The memory 902 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 902 is used to store at least one instruction, which is used to be executed by the processor 901 to implement the training method of the target recognition model provided in the method embodiment of the present application.

[0153] In some embodiments, the computer device 900 further includes some other components 903, and the type and quantity of the other components 903 can be selected based on the functional requirements of the computer device 900. Those skilled in the art will appreciate that Fig. 9 The structure shown in the figure does not constitute a limitation on the computer device 900, and the computer device 900 may include more or less components than those shown in the figure, or combine some components, or adopt a different arrangement of components.

[0154] Optionally, the computer readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a solid state drive (SSD), or an optical disk. Among them, the random access memory may include a resistance random access memory (ReRAM) and a dynamic random access memory (DRAM). The serial numbers of the above embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0155] An embodiment of the present application also provides a computer device, which includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the training method of the target recognition model as described in any of the above embodiments of the present application.

[0156] An embodiment of the present application also provides a computer-readable storage medium, in which at least one instruction, at least one program, a code set or an instruction set is stored. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement a target recognition model training method as described in any of the above embodiments of the present application.

[0157] The embodiment of the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the training method of the target recognition model described in any of the above embodiments.

[0158] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0159] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for training a target recognition model, characterized in that: The method comprises: Generate an adversarial attack texture based on a dual attention distraction suppression attack DAS algorithm, wherein the adversarial attack texture is used to interfere when a target recognition model recognizes target content in an image, wherein the target content includes at least one of a vehicle and a pedestrian; Fusion of the adversarial attack texture with sample image data to generate first image data, wherein the sample image data is used to simulate an environment image collected by an autonomous driving vehicle, and the sample image data is annotated with a target content label, and the target content label is used to indicate the distribution of target content in the sample image data; Recognize the target content in the first image data by using the target recognition model to obtain a first detection result, where the first detection result includes the inclusion of the target content in the first image data predicted by the target recognition model; The target recognition model is trained based on the difference between the first detection result and the target content label.

2. The method according to claim 1, characterized in that: The step of fusing the adversarial attack texture with the sample image data to generate the first image data includes: Acquire a target content configuration file, where the target content configuration file is used to generate a three-dimensional simulation model of the target content in the first image data; The first image data is generated based on the adversarial attack texture and the target content configuration file through an autonomous driving simulator, wherein the first image data includes the three-dimensional simulation model of the target content.

3. The method according to claim 2, characterized in that The generating the first image data based on the adversarial attack texture and the target content configuration file by the autonomous driving simulator includes: Inputting the adversarial attack texture and the target content configuration file into the autonomous driving simulator, and outputting a three-dimensional simulation model of the target content fused with the adversarial attack texture; The first image data is generated based on a three-dimensional simulation model of the target content.

4. The method according to claim 3, characterized in that The generating the first image data based on the three-dimensional simulation model of the target content includes: Acquire a test case code, where the test case code is used to instruct generation of the first image data; Controlling the autonomous driving simulator to generate a three-dimensional environment image based on the test case code, wherein the three-dimensional environment image is used to simulate the environment in which the autonomous driving vehicle is located; The three-dimensional simulation model of the target content is added to the three-dimensional environment image to obtain the first image data.

5. The method according to any one of claims 1 to 4, characterized in that: The dual attention distraction suppression attack DAS algorithm is used to generate an adversarial attack texture, including: Acquire a 3D model configuration file of the target content, wherein the 3D model configuration file includes a model main body part and a model texture part, wherein the model main body part is used to describe the 3D shape of the target content, and the model texture part refers to a texture covering a surface of the model main body part; The model texture portion is transformed based on the DAS algorithm to generate the adversarial attack texture.

6. The method according to any one of claims 1 to 4, characterized in that: The step of identifying the target content in the first image data by using the target recognition model to obtain a first detection result includes: Performing format conversion on the first image data to obtain test data that meets preset format requirements; inputting the test data into the target recognition model; The target recognition model is used to identify the presence of the target content in the test data to obtain the first detection result; wherein the first detection result includes a first bounding box detection result, a first global detection result, a first three-dimensional target detection result and a first average directional similarity detection result; the first bounding box detection result is used to indicate the position of the target content in the first image data identified by the target recognition model, the first global detection result is used to indicate the position of the target content in the first image data identified by the target recognition model from a global perspective, the first three-dimensional target detection result is used to indicate the recognition of the target content in three-dimensional space by the target recognition model, and the first average directional similarity detection result is used to indicate the orientation of the target content in the first image data identified by the target recognition model.

7. The method according to any one of claims 1 to 4, characterized in that: The training of the target recognition model based on the difference between the first detection result and the target content label includes: When the difference between the first detection result and the target content label meets the preset adversarial training condition, the target recognition model is trained based on adversarial attack training data, wherein the adversarial attack training data is image data obtained based on the adversarial attack texture generated by the DAS algorithm.

8. A training device for a target recognition model, characterized in that: The device comprises: A texture generation module, used to generate an adversarial attack texture based on a dual attention distraction suppression attack DAS algorithm, wherein the adversarial attack texture is used to interfere when a target recognition model recognizes target content in an image, wherein the target content includes at least one of a vehicle and a pedestrian; An image generation module, used to fuse the adversarial attack texture with sample image data to generate first image data, wherein the sample image data is used to simulate an environment image collected by an autonomous driving vehicle, and the sample image data is annotated with a target content label, and the target content label is used to indicate the distribution of target content in the sample image data; an identification module, configured to identify target content in the first image data by using the target identification model to obtain a first detection result, wherein the first detection result includes the inclusion of the target content in the first image data predicted by the target identification model; A training module is used to train the target recognition model based on the difference between the first detection result and the target content label.

9. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one program, and the at least one program is loaded and executed by the processor to implement the training method of the target recognition model as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores at least one program, and the at least one program is loaded and executed by the processor to implement the training method of the target recognition model as described in any one of claims 1 to 7.

Images