Training methods, devices, equipment, and media for target recognition models

Abstract

This application discloses a training method, apparatus, device, and medium for a target recognition model, relating to the field of deep learning. The method includes the following steps: generating adversarial attack textures based on the Dual Attention Dispersion Suppression Attack (DAS) algorithm, whereby the adversarial attack textures are used to interfere with the target recognition model's identification of target content in an image; fusing the adversarial attack textures with sample image data to generate first image data; identifying target content in the first image data using the target recognition model to obtain a first detection result, whereby the first detection result includes the target content's inclusion status predicted by the target recognition model in the first image data; and training the target recognition model based on the difference between the first detection result and the target content label. This method can determine the target recognition model's ability to resist network attacks, allowing for timely targeted training of the target recognition model, ensuring its robustness against physical adversarial attacks.

Description

Technical Field

[0001] This application relates to the field of deep learning, and in particular to a method, apparatus, device, and medium for training a target recognition model. Background Technology

[0002] Environmental perception technology is a key technology for the safe operation of autonomous vehicles. The vehicle's ability to accurately perceive these environments relies on object recognition models. These models can identify and locate objects such as pedestrians, vehicles, and traffic signs around the vehicle, providing essential information for autonomous vehicles to make correct driving decisions.

[0003] In related technologies, target recognition models are typically trained based on a large amount of labeled data. This involves collecting image data containing target objects, manually labeling the image data, determining the location and category of the target objects in the images, inputting the image data as training data into the target recognition model, and then applying the resulting target recognition model to the field of autonomous driving.

[0004] However, when the target recognition model of an autonomous vehicle is subjected to adversarial attacks, the environmental images captured by the autonomous vehicle can be modified, causing the model to be unable to accurately identify target objects in the environmental images and leading to safety issues. Therefore, training the target recognition model using only ordinary image data cannot guarantee its robustness. Summary of the Invention

[0005] This application provides a method, apparatus, device, and medium for training a target recognition model, capable of training a target recognition model. The technical solution is as follows:

[0006] On the one hand, a method for training a target recognition model is provided, the method comprising:

[0007] Adversarial attack textures are generated based on the Dual Attention Dispersion Suppression Attack (DAS) algorithm. These textures are used to interfere with the target recognition model when it identifies target content in an image. The target content includes at least one of vehicles and pedestrians.

[0008] The adversarial attack texture is fused with the sample image data to generate first image data. The sample image data is used to simulate environmental images collected by an autonomous vehicle. The sample image data is labeled with target content tags, which are used to indicate the distribution of target content in the sample image data.

[0009] The target content in the first image data is identified by the target recognition model to obtain a first detection result, which includes the inclusion of the target content in the first image data as predicted by the target recognition model.

[0010] The target recognition model is trained based on the difference between the first detection result and the target content tag.

[0011] On the other hand, a training device for a target recognition model is provided, the device comprising:

[0012] The texture generation module is used to generate adversarial attack textures based on the Dual Attention Dispersion Suppression Attack (DAS) algorithm. The adversarial attack textures are used to interfere with the target recognition model when it identifies target content in the image. The target content includes at least one of vehicles and pedestrians.

[0013] An image generation module is used to fuse the adversarial attack texture with sample image data to generate first image data. The sample image data is used to simulate environmental images collected by an autonomous vehicle. The sample image data is labeled with target content tags, which are used to indicate the distribution of target content in the sample image data.

[0014] The recognition module is used to recognize the target content in the first image data through the target recognition model to obtain a first detection result, wherein the first detection result includes the inclusion status of the target content in the first image data as predicted by the target recognition model.

[0015] The training module is used to train the target recognition model based on the difference between the first detection result and the target content label.

[0016] In an optional embodiment, the image generation module is further configured to obtain a target content configuration file, the target content configuration file being used to generate a three-dimensional simulation model of the target content in the first image data; and to generate the first image data using an autonomous driving simulator based on the adversarial attack texture and the target content configuration file, wherein the first image data contains the three-dimensional simulation model of the target content.

[0017] In an optional embodiment, the image generation module is further configured to input the adversarial attack texture and the target content configuration file into the autonomous driving simulator, and output a three-dimensional simulation model of the target content incorporating the adversarial attack texture; and generate the first image data based on the three-dimensional simulation model of the target content.

[0018] In an optional embodiment, the image generation module is further configured to acquire test case code, the test case code being used to instruct the generation of the first image data; control the autonomous driving simulator to generate a three-dimensional environment image based on the test case code, the three-dimensional environment image being used to simulate the environment in which the autonomous vehicle is located; and add the three-dimensional simulation model of the target content to the three-dimensional environment image to obtain the first image data.

[0019] In an optional embodiment, the texture generation module is further configured to obtain a three-dimensional model configuration file of the target content, the three-dimensional model configuration file including a model body part and a model texture part, the model body part being used to describe the three-dimensional shape of the target content, and the model texture part referring to the texture covering the surface of the model body part; the model texture part is modified based on the DAS algorithm to generate the adversarial attack texture.

[0020] In an optional embodiment, the recognition module is further configured to: convert the format of the first image data to obtain test data that meets preset format requirements; input the test data into the target recognition model; and use the target recognition model to identify cases where the target content exists in the test data to obtain the first detection result; wherein the first detection result includes a bounding box first detection result, a global first detection result, a three-dimensional target first detection result, and an average orientation similarity first detection result; the bounding box first detection result is used to indicate the target recognition model's identification of the position of the target content in the first image data, the global first detection result is used to indicate the target recognition model's identification of the position of the target content in the first image data from a global perspective, the three-dimensional target first detection result is used to indicate the target recognition model's identification of the target content in three-dimensional space, and the average orientation similarity first detection result is used to indicate the target recognition model's identification of the orientation of the target content in the first image data.

[0021] In an optional embodiment, the training module is further configured to train the target recognition model based on adversarial attack training data when the difference between the first detection result and the target content label meets preset adversarial training conditions, wherein the adversarial attack training data is image data obtained based on adversarial attack textures generated by the DAS algorithm.

[0022] On the other hand, a computer device is provided, the computer device including a processor and a memory, the memory storing at least one instruction, at least one program, code set or instruction set, the at least one instruction, the at least one program, the code set or instruction set being loaded and executed by the processor to implement the training method of the target recognition model as described in any of the above embodiments of this application.

[0023] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction, at least one program, code set, or instruction set is stored in the storage medium, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the training method of the target recognition model as described in any of the embodiments of this application above.

[0024] On the other hand, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the training method for the target recognition model described in any of the above embodiments.

[0025] The beneficial effects of the technical solutions provided in this application include at least the following:

[0026] Incorporating adversarial attack textures into sample image data can simulate the impact of physical adversarial noise encountered by autonomous vehicles when recognizing images. The recognition performance of the target recognition model is then tested using image data containing adversarial attack textures to determine its ability to resist network attacks. This allows for timely targeted training of the target recognition model, ensuring its robustness against physical adversarial attacks. The DAS algorithm can generate effective adversarial attack textures and generate large batches of training data to train the target recognition model, improving its training efficiency. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a schematic diagram of a training system for a target recognition model provided in an exemplary embodiment of this application;

[0029] Figure 2This is a flowchart illustrating a robustness test of a target recognition model provided in an exemplary embodiment of this application;

[0030] Figure 3 This is a flowchart of a training method for a target recognition model provided in an exemplary embodiment of this application;

[0031] Figure 4 This is a schematic diagram of the code relationships involved in generating adversarial attack textures according to an exemplary embodiment of this application;

[0032] Figure 5 This is a schematic diagram of an adversarial attack texture generated based on a first type of obstacle and a three-dimensional obstacle model with superimposed adversarial attack texture provided in an exemplary embodiment of this application;

[0033] Figure 6 This is a schematic diagram of an adversarial attack texture generated based on a second type of obstacle and a three-dimensional obstacle model with superimposed adversarial attack texture provided in an exemplary embodiment of this application;

[0034] Figure 7 This is a schematic diagram comparing images before and after a physical attack on an image using an obstacle, represented by a bus, provided by an exemplary embodiment of this application.

[0035] Figure 8 This is a structural block diagram of a training apparatus for a target recognition model provided in an exemplary embodiment of this application;

[0036] Figure 9 This is a structural block diagram of a computer device provided in an exemplary embodiment of this application. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0038] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0039] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0040] It should be noted that all information and data involved in this application are authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0041] It should be understood that although the terms first, second, etc., may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, a first parameter may also be referred to as a second parameter, and similarly, a second parameter may also be referred to as a first parameter. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0042] First, a brief introduction to the terms used in the embodiments of this application:

[0043] Dual Attention Suppression (DAS) is an adversarial attack method targeting deep learning models. It generates powerful adversarial camouflage in the physical world. This model focuses on generating adversarial examples that can attack various models in a black-box setting, while appearing visually natural and camouflaged. DAS works by using attention mechanisms to generate deceptive adversarial camouflage through testing and validation in both the digital and physical worlds. The model can generate these camouflages in a 3D simulation environment and demonstrates good attack performance in a black-box setting.

[0044] In this application, the texture portion of the 3D model of the target content is stripped off, and the texture portion is modified based on the DAS algorithm to generate an adversarial attack texture. The adversarial attack texture can be re-fused with the main body of the 3D model. An image of the 3D model containing the target content is acquired in a simulation environment. The adversarial attack texture is fused into the image, which can interfere with the target recognition model when the target recognition model recognizes the target content in the image.

[0045] For example, a stop signal image refers to an image containing a stop sign (as the target content), which incorporates adversarial attack textures. In other words, the stop signal image incorporates adversarial attack textures. The target recognition model inside the autonomous vehicle recognizes the stop signal image and identifies the stop sign in the stop signal image as a speed reduction sign, resulting in a misjudgment. At this time, the autonomous vehicle continues to drive.

[0046] DAS is based on the principle that different models have similar attention patterns. It attempts to divert the model's attention from the target region to non-target regions, while generating patterns that resemble human attention patterns to achieve the desired attack effect. In implementation, the image is first input into the target model, the attention map of the target region is calculated, and a connected graph-based method is used to divert the model's attention to non-target regions. The corresponding loss function is calculated and gradient backpropagation is performed. Finally, the perturbation image is updated, and the entire process is repeated.

[0047] KITTI / kitti format (Karlsruhe Institute of Technology and Toyota Technological Institute, a data format): The KITTI format is a collective term for a series of data formats associated with the KITTI dataset, a large benchmark dataset widely used in computer vision and autonomous driving research. For example, specific file formats of the KITTI dataset include the following: Image files: typically stored in 8-bit PNG format, possibly containing cropped and distortion-corrected images; LiDAR point cloud data: stored in binary format, containing 3D point cloud data obtained from LiDAR scans, with each point containing x, y, z coordinates and reflection intensity information.

[0048] The SMOKE model (Single-Stage Monocular 3D Object Detection via Keypoint Estimation) directly predicts the 3D attribute information of the target through keypoint estimation and 3D bounding box regression, instead of relying on traditional 2D detection methods. This avoids the noise that may be introduced by 2D detection and improves the performance of 3D detection.

[0049] Secondly, the training system for the target recognition model involved in the embodiments of this application will be described, for illustrative purposes only. Please refer to [reference needed]. Figure 1 This implementation environment involves a terminal 110 and a server 120. The terminal 110 and the server 120 are connected via a communication network 130.

[0050] Taking the perception of the external environment by autonomous vehicles as an example, the autonomous vehicle collects environmental images from its own perspective, identifies the environmental images using a target recognition model, determines whether there are target objects (which can be called obstacles) in the environmental images, and promptly reports the recognition results. Based on the recognition results, the autonomous vehicle decides whether to continue driving or stop. Therefore, testing the robustness of the target recognition model is crucial for autonomous vehicles to correctly perceive the environment and travel safely.

[0051] Optionally, the terminal 110 contains a three-dimensional model of obstacles. The model texture part of the three-dimensional model is separated from the main body of the model. After the terminal 110 modifies the model texture part through the DAS algorithm, it generates an adversarial attack texture. The adversarial attack texture is then overlaid on the main body of the three-dimensional model to obtain a three-dimensional model with adversarial attack texture. This three-dimensional model with adversarial attack texture can also be simply referred to as the DAS model.

[0052] The DAS model is actually used to simulate obstacles (such as vehicles and pedestrians) in a simulation environment as target content to be identified.

[0053] Attacking autonomous vehicles using the DAS algorithm can introduce obstacles with adversarial textures into the environmental images captured by the vehicle. In other words, the captured environmental image is actually an image incorporating adversarial textures. When object recognition models try to identify images with infused adversarial textures, they struggle to accurately identify the types of obstacles, leading to misjudgments by the autonomous vehicle and causing safety issues.

[0054] Terminal 110 also includes an autonomous driving simulator, which is used to fuse sample image data and adversarial attack textures to generate first image data. In other words, the first image data actually refers to the image after superimposing the adversarial attack texture.

[0055] Terminal 110 sends the first image data to server 120. Server 120 is equipped with a target recognition model, which receives the first image data sent by terminal 110 and performs recognition to obtain a first detection result. The first image data is used to test the target recognition model's ability to accurately identify targets when faced with physical noise attacks. The first detection result is used to indicate whether the target recognition model can identify targets in the first image data.

[0056] The target recognition model is used to identify the sample image data and the first image data respectively. By comparing the results of the two recognitions, it can be determined whether the adversarial attack texture effectively interferes with the target recognition model and the degree of interference. Based on the comparison results, the training method for the target recognition model can be determined.

[0057] The difference between the sample image data and the first image data lies in whether noise is superimposed / adversarial attack textures are fused. The sample image data refers to the image without adversarial attack textures, that is, the environmental image collected when the obstacle 3D model without attack textures is run in the simulation environment.

[0058] In other words, when the interference effect of the adversarial attack texture on the target recognition model reaches the preset conditions, the DAS algorithm can be used to generate training data in batches. Based on the training data, the target recognition model is trained to specifically improve its robustness and enhance its ability to resist physical noise attacks. The training data has the same format as the first image data.

[0059] Figure 2 This is a flowchart of a test for the robustness of a target recognition model provided in an exemplary embodiment of this application, including the following steps.

[0060] S201, training to generate adversarial attack textures.

[0061] Using a 3D model of the obstacle, the texture portion of the model is stripped from the main body to obtain the initial texture. The DAS algorithm is then used to attack the texture, resulting in an adversarial texture.

[0062] S202, import the texture into the obstacle model.

[0063] That is, the adversarial attack texture is overlaid on the main body of the obstacle 3D model to obtain a 3D model that incorporates the adversarial attack texture.

[0064] This process is executed by the autonomous driving simulator, which imports the files corresponding to the adversarial attack textures and the files that generate the 3D obstacle models into the relevant directories of the autonomous driving simulator, and then generates the 3D DAS model by the autonomous driving simulator.

[0065] S203, run the test cases to generate images and coordinate data.

[0066] The test cases are pre-prepared code files. Running these test cases controls the autonomous driving simulator to generate a simulated 3D environment. This simulated 3D environment contains a 3D DAS model to simulate various situations that an autonomous vehicle may encounter while driving. For example, if the obstacle in front of the autonomous vehicle is a bus, the simulated 3D environment contains 3D models of both the autonomous vehicle and the bus, with the bus's 3D model surface incorporating adversarial attack textures.

[0067] The obstacles and autonomous vehicles operate in a simulated 3D environment. Images are captured from the perspective of the autonomous vehicle, and these images contain the obstacles. The coordinate data is used to indicate the position of the obstacles in the images.

[0068] S204, coordinate data converted to kitti format.

[0069] The coordinate data is converted into a format suitable for the target recognition model.

[0070] S205, Test the target recognition model.

[0071] The target recognition model in this application is trained based on the SMOKE model. Kitti format data is input into the target recognition model, which then identifies image data infused with adversarial attack textures and outputs a recognition result. This result includes the target content (obstacles) identified by the model, including the type of target content and its location information within the image data. This recognition result serves as a test result to evaluate the accuracy and robustness of the target recognition model in recognizing images subjected to physical noise attacks.

[0072] S206, Evaluation of test results.

[0073] The robustness of the target recognition model is evaluated based on the test results. Subsequently, the target recognition model is trained in a targeted manner. The DAS algorithm is used to generate image data with adversarial attack textures in batches, which are then used as training data to train the target recognition model.

[0074] In summary, by testing the target recognition model's performance using image data containing adversarial attack textures, we can determine the model's ability to resist network attacks and enable timely targeted training to ensure its robustness against physical adversarial attacks. The DAS algorithm can generate effective adversarial attack textures and generate large batches of training data to train the target recognition model, thereby improving its training efficiency.

[0075] The aforementioned terminal can be various forms of terminal devices such as mobile phones, tablets, desktop computers, portable laptops, smart TVs, vehicle terminals, and smart home devices, and this application embodiment does not limit them.

[0076] It is worth noting that the aforementioned servers can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0077] Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to achieve data computation, storage, processing, and sharing. Based on the cloud computing business model, cloud technology encompasses network technology, information technology, integration technology, management platform technology, and application technology. It can form resource pools, providing flexible and convenient on-demand access. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, every item may have its own identification mark in the future, requiring transmission to backend systems for logical processing. Data at different levels will be processed separately, and various industry data will require robust system support, which can only be achieved through cloud computing.

[0078] In some embodiments, the server described above can also be implemented as a node in a blockchain system.

[0079] Based on the above-described terminology and application scenarios, the training method of the target recognition model provided in this application will be explained. This method can be executed by a server or a terminal, or by both a server and a terminal. In this embodiment, the method is illustrated by being executed by a terminal. Figure 3 As shown, Figure 3 This is a flowchart of a training method for a target recognition model provided in an exemplary embodiment of this application. The method includes the following steps.

[0080] Step 310: Generate adversarial attack textures based on the Dual Attention Dispersion Suppression Attack (DAS) algorithm.

[0081] The adversarial attack texture is used to interfere with the target recognition model when it identifies target content in an image, including at least one of vehicles and pedestrians.

[0082] The generation of adversarial attack textures is based on the DAS principle. It attacks the target content to be identified in an image, altering the original texture of the target content, making it impossible for the target recognition model to accurately identify the target content information, thus causing misjudgment. This application uses the application of a target recognition model to autonomous vehicles as an example. That is, in order to improve the accuracy of autonomous vehicles' perception of the external environment, it is necessary to specifically train the target recognition model by simulating attacks on the target recognition model to interfere with its recognition process.

[0083] During the operation of autonomous vehicles, the target recognition model mainly focuses on obstacles in the driving environment. Therefore, the above-mentioned target content can be referred to as obstacles, which include, but are not limited to, pedestrians, vehicles, animals, etc., obstacles that autonomous vehicles need to avoid during operation.

[0084] In some embodiments, the target content can also be road signs, directional signs, etc., such as signs indicating that the road ahead is impassable, to instruct the autonomous vehicle to stop. The target recognition model needs to accurately identify the signs in the acquired images, determine the type of signs, and promptly feed the recognition results back to the autonomous vehicle, which then determines whether to continue based on the recognition results.

[0085] Adversarial attack textures are generated using the DAS algorithm. Based on the principle that different models have similar attention patterns, the algorithm attempts to divert the model's attention from the target region to non-target regions, while simultaneously generating patterns that resemble human attention patterns to achieve the desired attack effect. In implementation, the image is first input into the target model, and the attention map of the target region is calculated. Then, a connected graph-based method is used to divert the model's attention to non-target regions. The corresponding loss function is calculated, and gradient backpropagation is performed. Finally, the perturbation image is updated, and the entire process is repeated.

[0086] The field of computer vision image recognition involves a specific loss function to identify a class of objects. The physical noise attack on the image in this application refers to altering the loss value of the original image (the original image being the unattacked image) to reduce the recognition accuracy of the original loss function for the target content. Specifically, the attention distraction loss (Ld) aims to make the pixel gradient changes less noticeable to the human eye, while the smoothness loss (Ls) is designed to alter the pixel gradient to interfere with the recognition by the intelligent driving perception end of autonomous vehicles. Both aim to change the texture of the target content.

[0087] The calculation principles for Ld loss and Ls loss are as follows.

[0088]

[0089] Among them, G k The sum of attention weights in the k-th connected graph, where N is the number of pixels in the connected graph. k Let N be the number of pixels in the k-th connected graph. As can be seen from the design of the loss function, the number of pixels N in the connected graph is a fixed value; therefore, the optimization objective is to continuously reduce N. k With G k N k The shrinking effect of G is to increase the number of connected graphs, thereby achieving the effect of "distracting" attention; k The effect of shrinking is to continuously reduce the weight value of attention, thereby achieving the effect of "suppressing" attention. Compared with random texture attacks, DAS adversarial textures divert the model's attention to other areas outside the detection object.

[0090] L s =∑(x i，j -x i+1，j ) 2 +(x i，j -x i，j+1 ) 2

[0091] Where x i，j Image I adv The pixel value at coordinates (i, j). Ls further improves the naturalness of the camouflage by reducing the smoothing loss by reducing the squared difference between adjacent pixels.

[0092] The specific process of generating adversarial attack textures is as follows: (1) Divide the three-dimensional model of the obstacle into a pure white model (M) and a texture (T). The pure white model (M) refers to the main body of the three-dimensional model, which is used to describe the three-dimensional shape of the obstacle. The texture (T) refers to the texture part of the three-dimensional model, that is, the texture covering the surface of the three-dimensional model. At this time, the texture (T) has not yet been attacked; (2) First, put the texture (T) into the optimizer; (3) Render the three-dimensional model (M+T) onto the pre-prepared image through the neural renderer (R) to obtain the input image Input; (4) Render the image (5) Input the image into the target recognition model A to obtain the predicted value, and backpropagate the heat map (Cam) according to logits; (6) Use the heat map (Cam) to calculate the loss value. The heat map can be decomposed into three dimensions, namely RGB (Red, Green, Blue). Calculate the pixel gradient in the three dimensions to obtain the loss value; (7) Backpropagate the loss value and update the texture (T) in the optimizer according to the gradient; (8) Repeat steps (3)-(6) to continuously optimize the texture (T). After training is completed, save the adversarial attack texture (Tadv).

[0093] like Figure 4 As shown, Figure 4 This is a schematic diagram of the code relationships involved in generating adversarial attack textures.

[0094] The relationship diagram 400 involves various variable classes, such as: class 401 for configuring all parameters, optimizer class 402, dataset class 403, class 404 for calculating loss, training class 405, etc. These variable classes are respectively related to... Figure 4 There is a correspondence between adjacent code statements; for example, class 401, which configures all parameters, refers to... Figure 4 The `met / engine / Config.py` file and the dataset class 403 refer to... Figure 4 The file is dataset / MyDataset.py.

[0095] exist Figure 4 In the main execution module, the top part is `attack.py`, and the core module is `met / engine / Trainer.py` in the middle. After executing `attack.py`, the main configuration is started.

[0096] The files `met / engine / Config.py`, `met / engine / Textureoptimizer.py`, and `met / engine / Trainer.py` are used for training and optimizing. `met / engine / Config.py` provides necessary parameter configurations for training and the optimizer, while the optimizer provides optimization code at the end of each training round. During training, the dataset `dataset / Mydatatest.py` is called. Since the loss value needs to be calculated using a heatmap, `. / met / modules / det / DAS_Losscaculator.py` is called. In other words, the startup code `attack.py` begins the first round of training, calls the dataset and configuration parameters, performs heatmap preprocessing, calculates the loss value using the heatmap, returns the loss value after calculation, performs optimization, and then starts the next round of training.

[0097] Step 320: The adversarial attack texture is fused with the sample image data to generate the first image data.

[0098] The sample image data is used to simulate environmental images collected by autonomous vehicles. The sample image data is labeled with target content tags, which are used to indicate the distribution of target content in the sample image data.

[0099] The process of fusing adversarial attack textures with sample image data can also be called the process of importing adversarial attack textures into the 3D model of the obstacle, so that the surface of the 3D model is covered with adversarial attack textures to obtain a DAS model. The DAS model is then added to the 3D simulation environment for image acquisition to obtain the first image data.

[0100] Optionally, a target content configuration file is obtained, which is used to generate a three-dimensional simulation model of the target content in the first image data. The first image data is generated by an autonomous driving simulator based on the adversarial attack texture and the target content configuration file, wherein the first image data contains a three-dimensional simulation model of the target content.

[0101] Obtain the test case code, which is used to instruct the generation of the first image data. Based on the test case code, control the autonomous driving simulator to generate a three-dimensional environment image, which is used to simulate the environment in which the autonomous vehicle is located. Add the three-dimensional simulation model of the target content to the three-dimensional environment image to obtain the first image data.

[0102] The adversarial attack texture and target content configuration file are input into the autonomous driving simulator, and a 3D simulation model of the target content with adversarial attack texture is output. The first image data is generated based on the 3D simulation model of the target content.

[0103] For example, the main process is as follows: (1) Copy the trained adversarial attack texture to the import / Package01 / Props / [xxx] folder of CARLA (autonomous driving simulator); the trained adversarial attack texture is stored as xxx_DAS.png file; (2) the final effect is that the obstacle's .obj file and .mtl file, the original texture .png file, and the adversarial texture xxx_DAS.png file are in the same directory;

[0104] (3) Modify the .mtl file of the obstacle and change the texture name of the .png file of the obstacle to the texture name after training; (4) After modification, delete / root / Carla / Unreal / CarlaUE4 / Content / Package01; (5) Restart the terminal in the / root / carla folder and run make import; (6) Enter the configuration file directory of Package01 and modify the Package01.Package.json file. At this time, you can see that the model has used adversarial attack textures in the autonomous driving simulator.

[0105] For example, the first image data generated after running the test case code includes the image portion and coordinate data. Running the test case code generates test case data in the CARLA software platform from the obstacle model after the physical attack, including image data (.png file) and other coordinate position data (.npz file).

[0106] For example, the test cases were divided according to the driving conditions of the autonomous vehicle, containing a total of 12 test cases. Each test case could generate 105 image data points and corresponding coordinate data, theoretically generating a total of 1260 image data points and coordinate data points. However, due to lag when the computer ran the test cases, a small number of image data points were less than 105, resulting in a total of only 1256 image data points and corresponding coordinate data points. A comparison clearly shows that the texture of the obstacle differs significantly after the physical attack.

[0107] The main steps of this process include: (1) running the required test case code on the CARLA software platform; (2) after all the required test case data is generated, integrating all the data into the same folder, which contains only image data, coordinate data, and a corresponding label file. The coordinate data contains the location of the target content (obstacle) in the image, and the label file is used to indicate the specific type of the target content, such as motorcycles, buses, etc.

[0108] Step 330: The target content in the first image data is identified by the target recognition model to obtain the first detection result.

[0109] The first detection result includes the inclusion of target content in the first image data predicted by the target recognition model.

[0110] The first image data is converted to a new format to obtain test data that meets the preset format requirements; the test data is input into the target recognition model; the target recognition model identifies the cases in the test data that contain target content to obtain the first detection result.

[0111] The first detection result includes the first detection result of bounding box, the first detection result of global data, the first detection result of 3D target data, and the first detection result of average orientation similarity. The first detection result of bounding box data is used to indicate the position of the target content in the first image data as identified by the target recognition model. The first detection result of global data is used to indicate the position of the target content in the first image data as identified by the target recognition model from a global perspective. The first detection result of 3D target data is used to indicate the recognition of the target content in 3D space by the target recognition model. The first detection result of average orientation similarity is used to indicate the orientation of the target content in the first image data as identified by the target recognition model.

[0112] Optionally, the test data is in Kitti format. After the first image data is converted into Kitti format, it can be used as input data for the target recognition model. The target recognition model identifies the test data to determine if the test data contains target content, and then evaluates the ability of the target recognition model to recognize images with adversarial attack textures.

[0113] The detection accuracy of the target recognition model was tested using data containing adversarial attack textures. The first test result included four dimensions: (1) bbox (2D bounding box detection), which is the dimension corresponding to the first detection result of the bounding box; (2) bev (bird's-eye view detection), which is the dimension corresponding to the first global detection result; (3) 3D (3D target detection), which is the dimension corresponding to the first detection result of the three-dimensional target; and (4) aos (average orientation similarity detection), which is the dimension corresponding to the first detection result of the average orientation similarity.

[0114] The detection metrics mainly use mAP (mean accuracy), and the test results will be saved in a log file after the test is completed.

[0115] For example, the test results are as follows: For the same obstacle, the target recognition model detection uses 11-point interpolation and 40-point interpolation to calculate the average accuracy. In the same image, three ranges of target recognition models are used respectively: small (pixel value below 80*80), medium (pixel value between 80*80 and 242*242), and large (pixel value above 242*242), and their weighted average all is taken to calculate the overall detection effect.

[0116] As shown in Tables 1 and 2 below, Table 1 shows the detection results for the obstacle (bus) without attack in sunny weather, and Table 2 shows the detection results for the obstacle (bus) after physical attack (i.e., after overlaying the anti-attack texture) in sunny weather.

[0117] Table 1

[0118]

[0119]

[0120] For Tables 1 and 2, the first column on the left shows the detection dimensions, namely bbox (2D bounding box detection), bev (bird's-eye view detection), 3D (3D object detection), and aos (average orientation similarity detection). Different crossover ratios are used for each type of detection dimension.

[0121] The following explanation is provided for Tables 1 and 2 above: Since the pixel range detected by the small part is within 80*80, while the actual generated obstacle 3D model has a pixel range higher than 80*80, when the target content (obstacle) is detected using the small pixel range, the target recognition model fails to correctly identify the target content, so the detection result of the small part is 0.

[0122] In bounding box detection, after a physical attack, the target recognition model's overall (all) average accuracy for bus detection decreased from 84.1644 to 11.2991, the average accuracy in the medium range decreased from 92.9647 to 6.3866, and the average accuracy in the large range decreased from 73.8062 to 10.7034.

[0123] In BEV detection, after a physical attack, the target recognition model's overall (all) average accuracy for BEV detection decreased from 80.7078 to 17.0162, the average accuracy in the medium range decreased from 90.8316 to 11.4242, and the average accuracy in the large range decreased from 67.4487 to 18.6964.

[0124] In 3D detection, after a physical attack, the overall average accuracy of the target recognition model for bus detection decreased from 80.7078 to 17.0162, the average accuracy in the medium range decreased from 90.8316 to 11.4242, and the average accuracy in the large range decreased from 67.4487 to 18.6964.

[0125] In AOS detection, after a physical attack, the target recognition model's overall (all) average accuracy for bus detection decreased from 41.49 to 4.98, the average accuracy in the medium range decreased from 45.53 to 2.79, and the average accuracy in the large range decreased from 36.63 to 4.76.

[0126] This result demonstrates that the physical attack significantly interferes with the target detection of the target recognition model. Such interference is frequently seen in the physical world, such as graffiti on vehicles and clothing of various colors worn by humans, posing a significant hidden danger in the field of intelligent driving technology. This application utilizes images fused with adversarial attack textures generated during training to specifically train the target detection model, thereby improving the model's detection accuracy for such obstacles and enhancing the safety of intelligent driving.

[0127] Step 340: Train the target recognition model based on the difference between the first detection result and the target content label.

[0128] If the difference between the first detection result and the target content label meets the preset adversarial training conditions, the target recognition model is trained based on adversarial attack training data, wherein the adversarial attack training data is image data obtained based on adversarial attack textures generated by the pre-trained model.

[0129] like Figure 5 As shown, Figure 5 This is a schematic diagram of the adversarial attack texture generated based on the first type of obstacle and the 3D model of the obstacle with the adversarial attack texture superimposed.

[0130] The adversarial attack texture 510 generated based on the first type of obstacle is superimposed on the original 3D model of the first type of obstacle to obtain the adversarial texture 3D model 520, with the effect as follows. Figure 5 As shown.

[0131] like Figure 6 As shown, Figure 6 This is a schematic diagram of the adversarial attack texture generated based on the second type of obstacle and the 3D model of the obstacle with the adversarial attack texture superimposed.

[0132] The adversarial attack texture 610 generated based on the second type of obstacle is superimposed on the original 3D model of the second type of obstacle to obtain the adversarial texture 3D model 620, with the effect as follows. Figure 6 As shown.

[0133] Among them, the first type of obstacle differs from the second type of obstacle, for example, with Figure 5 The corresponding first type of obstacle is pedestrians, and Figure 6 The corresponding second type of obstacle is a vehicle.

[0134] Indicative, such as Figure 7 As shown, Figure 7 This is a diagram comparing images before and after a physical attack on an image using an obstacle, represented by a bus.

[0135] There is a difference between the image 701 before the attack and the image 702 after the attack, and the external texture of the obstacle 700 has changed.

[0136] In summary, the target recognition model training method provided in this application incorporates adversarial attack textures into sample image data. This simulates the impact of physical adversarial noise encountered by autonomous vehicles when recognizing images. The recognition performance of the target recognition model is tested using image data containing adversarial attack textures, determining the model's ability to resist network attacks. Timely targeted training of the target recognition model ensures its robustness against physical adversarial attacks. The DAS algorithm can generate effective adversarial attack textures and generate large batches of training data to train the target recognition model, improving its training efficiency.

[0137] Figure 8 This is a structural block diagram of a training device for a target recognition model provided in an exemplary embodiment of this application, as shown below. Figure 8 As shown, the device includes the following parts.

[0138] The texture generation module 810 is used to generate adversarial attack textures based on the dual attention dispersion suppression attack (DAS) algorithm. The adversarial attack textures are used to interfere with the target recognition model when it identifies target content in the image. The target content includes at least one of vehicles and pedestrians.

[0139] The image generation module 820 is used to fuse the adversarial attack texture with the sample image data to generate first image data. The sample image data is used to simulate the environmental image collected by the autonomous vehicle. The sample image data is labeled with target content tags, which are used to indicate the distribution of target content in the sample image data.

[0140] The recognition module 830 is used to recognize the target content in the first image data through the target recognition model and obtain a first detection result. The first detection result includes the inclusion of the target content in the first image data as predicted by the target recognition model.

[0141] The training module 840 is used to train the target recognition model based on the difference between the first detection result and the target content label.

[0142] In an optional embodiment, the image generation module 820 is further configured to acquire a target content configuration file, the target content configuration file being used to generate a three-dimensional simulation model of the target content in the first image data; and to generate the first image data using an autonomous driving simulator based on the adversarial attack texture and the target content configuration file, wherein the first image data contains the three-dimensional simulation model of the target content.

[0143] In an optional embodiment, the image generation module 820 is further configured to input the adversarial attack texture and the target content configuration file into the autonomous driving simulator, and output a three-dimensional simulation model of the target content incorporating the adversarial attack texture; and generate the first image data based on the three-dimensional simulation model of the target content.

[0144] In an optional embodiment, the texture generation module 810 is further configured to obtain a three-dimensional model configuration file of the target content, the three-dimensional model configuration file including a model body part and a model texture part, the model body part being used to describe the three-dimensional shape of the target content, and the model texture part referring to the texture covering the surface of the model body part; the model texture part is modified based on the DAS algorithm to generate the adversarial attack texture.

[0145] In an optional embodiment, the image generation module 820 is further configured to acquire test case code, the test case code being used to instruct the generation of the first image data; control the autonomous driving simulator to generate a three-dimensional environment image based on the test case code, the three-dimensional environment image being used to simulate the environment in which the autonomous vehicle is located; and add the three-dimensional simulation model of the target content to the three-dimensional environment image to obtain the first image data.

[0146] In an optional embodiment, the recognition module 830 is further configured to convert the format of the first image data to obtain test data that meets preset format requirements; input the test data into the target recognition model; and use the target recognition model to identify the presence of the target content in the test data to obtain the first detection result; wherein the first detection result includes a bounding box first detection result, a global first detection result, a three-dimensional target first detection result, and an average orientation similarity first detection result; the bounding box first detection result is used to indicate the target recognition model's identification of the position of the target content in the first image data, the global first detection result is used to indicate the target recognition model's identification of the position of the target content in the first image data from a global perspective, the three-dimensional target first detection result is used to indicate the target recognition model's identification of the target content in three-dimensional space, and the average orientation similarity first detection result is used to indicate the target recognition model's identification of the orientation of the target content in the first image data.

[0147] In an optional embodiment, the training module 840 is further configured to train the target recognition model based on adversarial attack training data when the difference between the first detection result and the target content label meets preset adversarial training conditions, wherein the adversarial attack training data is image data obtained based on adversarial attack textures generated by the DAS algorithm.

[0148] It should be noted that the target recognition model training device provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the target recognition model training device and the target recognition model training method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0149] Figure 9 This illustration shows a structural block diagram of a computer device 900 provided in an exemplary embodiment of this application. The computer device 900 may be a smartphone, tablet computer, MP3 player (Moving Picture Experts Group Audio Layer III), MP4 player (Moving Picture Experts Group Audio Layer IV), laptop computer, or desktop computer. The computer device 900 may also be referred to as user equipment, portable terminal, laptop terminal, desktop terminal, or other names.

[0150] Typically, computer device 900 includes a processor 901 and a memory 902.

[0151] Processor 901 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 901 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 901 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 901 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 901 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0152] The memory 902 may include one or more computer-readable storage media, which may be non-transitory. The memory 902 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 902 are used to store at least one instruction, which is executed by the processor 901 to implement the training method of the target recognition model provided in the method embodiments of this application.

[0153] In some embodiments, the computer device 900 also includes other components 903, the type and number of which can be selected based on the functional needs of the computer device 900. Those skilled in the art will understand that... Figure 9 The structure shown does not constitute a limitation on the computer device 900, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0154] Optionally, the computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), solid-state drives (SSDs), or optical discs, etc. The random access memory may include resistive random access memory (ReRAM) and dynamic random access memory (DRAM). The sequence numbers of the embodiments in this application are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0155] This application also provides a computer device, which includes a processor and a memory. The memory stores at least one instruction, at least one program, a code set, or an instruction set. The at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the training method of the target recognition model as described in any of the above embodiments of this application.

[0156] This application also provides a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the training method of the target recognition model as described in any of the above embodiments of this application.

[0157] This application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the training method for the target recognition model described in any of the above embodiments.

[0158] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0159] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A training method for a target recognition model, characterized in that, The method includes: Obtain a 3D model configuration file of the target content. The 3D model configuration file includes a model body part and a model texture part. The model body part is used to describe the 3D shape of the target content. The model texture part refers to the texture covering the surface of the model body part. The target content includes at least one of vehicles and pedestrians. The texture part of the model is modified based on the dual attention dispersion suppression attack (DAS) algorithm to generate adversarial attack textures, which are used to interfere with the target recognition model when it identifies target content in the image. The adversarial attack texture is fused with the sample image data to generate first image data. The sample image data is used to simulate environmental images collected by an autonomous vehicle. The sample image data is labeled with target content tags, which are used to indicate the distribution of target content in the sample image data. The target content in the first image data is identified by the target recognition model to obtain a first detection result, which includes the inclusion of the target content in the first image data as predicted by the target recognition model. If the difference between the first detection result and the target content label meets the preset adversarial training conditions, the target recognition model is trained based on adversarial attack training data, wherein the adversarial attack training data is image data obtained based on adversarial attack textures generated by the DAS algorithm.

2. The method according to claim 1, characterized in that, The step of fusing the adversarial attack texture with sample image data to generate first image data includes: Obtain a target content configuration file, which is used to generate a three-dimensional simulation model of the target content in the first image data; The first image data is generated by an autonomous driving simulator based on the adversarial attack texture and the target content configuration file, wherein the first image data contains the three-dimensional simulation model of the target content.

3. The method according to claim 2, characterized in that, The step of generating the first image data using an autonomous driving simulator based on the adversarial attack texture and the target content configuration file includes: The adversarial attack texture and the target content configuration file are input into the autonomous driving simulator, and a three-dimensional simulation model of the target content incorporating the adversarial attack texture is output. The first image data is generated based on a three-dimensional simulation model of the target content.

4. The method according to claim 3, characterized in that The generation of the first image data based on the 3D simulation model of the target content includes: Obtain the test case code, which is used to instruct the generation of the first image data; Based on the test case code, the autonomous driving simulator is controlled to generate a three-dimensional environment image, which is used to simulate the environment in which the autonomous vehicle is located; The three-dimensional simulation model of the target content is added to the three-dimensional environment image to obtain the first image data.

5. The method according to any one of claims 1 to 4, characterized in that, The step of identifying the target content in the first image data using the target recognition model to obtain a first detection result includes: The first image data is converted to a different format to obtain test data that meets the preset format requirements; The test data is input into the target recognition model; The target recognition model identifies cases where the target content exists in the test data, resulting in the first detection result. The first detection result includes a bounding box first detection result, a global first detection result, a 3D target first detection result, and an average orientation similarity first detection result. The bounding box first detection result indicates the target recognition model's identification of the target content's position in the first image data. The global first detection result indicates the target recognition model's identification of the target content's position in the first image data from a global perspective. The 3D target first detection result indicates the target recognition model's identification of the target content in 3D space. The average orientation similarity first detection result indicates the target recognition model's identification of the target content's orientation in the first image data.

6. A training device for a target recognition model, characterized in that, The device includes: A texture generation module is used to obtain a 3D model configuration file of the target content. The 3D model configuration file includes a main model part and a model texture part. The main model part describes the 3D shape of the target content, and the model texture part refers to the texture covering the surface of the main model part. The target content includes at least one of vehicles and pedestrians. The model texture part is modified based on the Dual Attention Dispersion Suppression Attack (DAS) algorithm to generate adversarial attack textures. The adversarial attack textures are used to interfere with the target recognition model when it identifies the target content in the image. An image generation module is used to fuse the adversarial attack texture with sample image data to generate first image data. The sample image data is used to simulate environmental images collected by an autonomous vehicle. The sample image data is labeled with target content tags, which are used to indicate the distribution of target content in the sample image data. The recognition module is used to recognize the target content in the first image data through the target recognition model to obtain a first detection result, wherein the first detection result includes the inclusion status of the target content in the first image data as predicted by the target recognition model. The training module is used to train the target recognition model based on adversarial attack training data when the difference between the first detection result and the target content label meets the preset adversarial training conditions. The adversarial attack training data is image data obtained based on adversarial attack textures generated by the DAS algorithm.

7. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one program, which is loaded and executed by the processor to implement the training method of the target recognition model as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The storage medium stores at least one program segment, which is loaded and executed by a processor to implement the training method of the target recognition model as described in any one of claims 1 to 5.

Images