Target recognition model detection methods, devices, equipment, media and program products

Abstract

This application discloses a method, apparatus, device, medium, and program product for detecting a target recognition model, relating to the field of deep learning. The method includes the following steps: obtaining a weather scene configuration file and a working condition configuration file; generating a three-dimensional simulation driving environment based on the weather scene configuration file and the working condition configuration file, the three-dimensional simulation driving environment containing three-dimensional models of autonomous vehicles and obstacles; collecting sample image data based on the three-dimensional simulation driving environment; performing an iterative attack on the sample image data based on the projection gradient descent method to generate first image data; and recognizing the first image data using a target recognition model corresponding to the autonomous vehicle to obtain a first recognition result, the first recognition result being used to test the target recognition model. This method can detect the robustness of the target recognition model.

Description

Technical Field

[0001] This application relates to the field of deep learning, and in particular to a method, apparatus, device, medium, and program product for detecting a target recognition model. Background Technology

[0002] Environmental perception technology is a key technology for the safe operation of autonomous vehicles, and the vehicle's ability to accurately perceive targets depends on target recognition models.

[0003] In related technologies, target recognition models can collect environmental images, identify the target objects in the images (such as pedestrians, vehicles, traffic signs, etc. around autonomous vehicles), and provide necessary information for autonomous vehicles to make correct driving decisions.

[0004] However, the functions of autonomous vehicles rely on networks. If the environmental images are attacked during the acquisition of environmental images, the content of the environmental images may be interfered with and modified. The accuracy of the target recognition model in recognizing the interfered environmental images cannot be guaranteed. Therefore, accurately testing the robustness of the target recognition model before autonomous vehicles are put into use is an urgent problem to be solved. Summary of the Invention

[0005] This application provides a method, apparatus, device, medium, and program product for detecting a target recognition model, capable of detecting the robustness of the target recognition model. The technical solution is as follows:

[0006] On the one hand, a detection method for a target recognition model is provided, the method comprising:

[0007] The system obtains a weather scenario configuration file and a driving condition configuration file. The weather scenario configuration file is used to generate a simulated weather scenario when the autonomous vehicle is driving. The driving condition configuration file is used to generate a driving condition scenario when the autonomous vehicle is driving. The driving condition configuration file includes an obstacle configuration part and a driving condition configuration part. The obstacle configuration part includes the configuration of obstacles encountered by the autonomous vehicle when driving. The driving condition configuration part includes the working status of the autonomous vehicle when driving. The working status includes at least one of vehicle emergency braking, vehicle emergency avoidance, and vehicle traffic congestion.

[0008] A three-dimensional simulation driving environment is generated based on the weather scene configuration file and the working condition configuration file. The three-dimensional simulation driving environment includes three-dimensional models of the autonomous vehicle and the obstacles.

[0009] Sample image data is collected based on the three-dimensional simulated driving environment. The sample image data is an environmental image collected from the perspective of the autonomous vehicle, and the environmental image includes the obstacles.

[0010] The sample image data is iteratively attacked based on the projection gradient descent method to generate first image data, which is image data obtained by superimposing noise on the sample image data.

[0011] The first image data is identified by the target recognition model corresponding to the autonomous vehicle to obtain a first recognition result. The first recognition result includes the recognition status of the obstacle in the first image data by the target recognition model. The first recognition result is used to test the target recognition model.

[0012] On the other hand, a detection device for a target recognition model is provided, the device comprising:

[0013] The acquisition module is used to acquire a weather scene configuration file and a driving condition configuration file. The weather scene configuration file is used to generate a simulated weather scene when the autonomous vehicle is driving. The driving condition configuration file is used to generate a driving condition scene when the autonomous vehicle is driving. The driving condition configuration file includes an obstacle configuration part and a driving condition configuration part. The obstacle configuration part includes the configuration of obstacles encountered by the autonomous vehicle when driving. The driving condition configuration part includes the working status of the autonomous vehicle when driving. The working status includes at least one of vehicle emergency braking, vehicle emergency avoidance, and vehicle traffic congestion.

[0014] A generation module is used to generate a three-dimensional simulation driving environment based on the weather scene configuration file and the working condition configuration file. The three-dimensional simulation driving environment includes three-dimensional models of the autonomous vehicle and the obstacles.

[0015] The acquisition module is used to acquire sample image data based on the three-dimensional simulated driving environment. The sample image data is an environmental image acquired from the perspective of the autonomous vehicle, and the environmental image includes the obstacles.

[0016] The generation module is further configured to perform an iterative attack on the sample image data based on the projection gradient descent method to generate first image data, wherein the first image data is image data obtained by superimposing noise on the sample image data;

[0017] The recognition module is used to recognize the first image data through the target recognition model corresponding to the autonomous vehicle to obtain a first recognition result. The first recognition result includes the recognition status of the obstacle in the first image data by the target recognition model. The first recognition result is used to test the target recognition model.

[0018] In an optional embodiment, the generation module is further configured to input the weather scene configuration file into the autonomous driving simulator, and construct the background portion of the three-dimensional simulated driving environment through the autonomous driving simulator; input the obstacle configuration portion of the working condition configuration file into the autonomous driving simulator, and generate a three-dimensional model of the obstacle in the three-dimensional simulated driving environment through the autonomous driving simulator; the obstacle includes at least one of vehicles and pedestrians; input the driving working condition configuration portion of the working condition configuration file into the autonomous driving simulator, and generate a three-dimensional model of the autonomous vehicle in the three-dimensional simulated driving environment through the autonomous driving simulator.

[0019] In an optional embodiment, the acquisition module is further configured to control the movement of the three-dimensional models of the autonomous vehicle and the obstacles in the three-dimensional simulation driving environment through the autonomous driving simulator; acquire environmental images corresponding to the three-dimensional simulation driving environment based on the perspective of the autonomous vehicle to obtain the sample image data, wherein the sample image data includes the distribution and location information of the obstacles in the environmental images.

[0020] In an optional embodiment, the generation module is further configured to convert the sample image data into a format to obtain first test data that meets the preset format requirements; perform an iterative attack on the first test data based on the projection gradient descent method to generate the first image data, wherein the iterative attack refers to the process of adding noise to the first test data, and the number of iterations of the iterative attack is a preset threshold; wherein the data obtained after the i-th attack on the first test data is used as the data used in the (i+1)-th attack, and i is a positive integer.

[0021] In an optional embodiment, prior to the generation module, the apparatus further includes:

[0022] The category determination module is used to perform category analysis on the sample image data and determine the category label of the sample image data. The category label is used to indicate the category information of the obstacle in the sample image data.

[0023] In an optional embodiment, after the identification module, the device further includes:

[0024] The training module is used to determine that the target recognition model does not meet a preset accuracy requirement when recognizing image data if the difference between the first recognition result and the category label reaches a preset difference threshold; to acquire a sample image dataset containing multiple image data, the multiple image data having the same format as the sample image data; to perform an iterative attack on the sample image dataset based on the projection gradient descent method to generate a training image dataset; wherein, the training image dataset contains multiple training image data, the multiple training image data having the same format as the first image data; and to train the target recognition model based on the training image dataset.

[0025] On the other hand, a computer device is provided, the computer device including a processor and a memory, the memory storing at least one instruction, at least one program, code set or instruction set, the at least one instruction, the at least one program, the code set or instruction set being loaded and executed by the processor to implement the target recognition model detection method as described in any of the above embodiments of this application.

[0026] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction, at least one program, code set, or instruction set is stored in the storage medium, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the target recognition model detection method as described in any of the above embodiments of this application.

[0027] On the other hand, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the target recognition model detection method described in any of the above embodiments.

[0028] The beneficial effects of the technical solutions provided in this application include at least the following:

[0029] By customizing various working conditions and weather scenarios, an autonomous driving simulator is used to simulate corresponding simulation environments, comprehensively covering all situations that autonomous vehicles may encounter during operation, thus improving the simulation testing effect by closely resembling reality. Environmental image information is generated, simulating images that might be captured from the perspective of an autonomous vehicle. These images contain the target content to be identified. Images with adversarial perturbations are used as input data for the target recognition model of the autonomous vehicle. The accuracy and robustness of the target recognition model are determined based on whether it can accurately identify the perturbed target content. The projective gradient descent method is used to iteratively attack the images to generate adversarial perturbations, obtaining a large amount of perturbed image data. This image data can be used as training data for the target recognition model, specifically improving the training effect of the target recognition model and enabling it to possess the ability to resist attack perturbations. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is a schematic diagram of a target recognition model detection system provided in an exemplary embodiment of this application;

[0032] Figure 2 This is a flowchart of a target recognition model detection method provided in an exemplary embodiment of this application;

[0033] Figure 3 This is a schematic diagram comparing environmental images collected in a clear daytime state with a bus as an obstacle before and after being attacked, provided by an exemplary embodiment of this application.

[0034] Figure 4 This is a structural block diagram of a target recognition model detection device provided in an exemplary embodiment of this application;

[0035] Figure 5 This is a structural block diagram of a target recognition model detection device provided in another exemplary embodiment of this application;

[0036] Figure 6 This is a structural block diagram of a computer device provided in an exemplary embodiment of this application. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0038] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0039] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0040] It should be noted that all information and data involved in this application are authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0041] It should be understood that although the terms first, second, etc., may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, a first parameter may also be referred to as a second parameter, and similarly, a second parameter may also be referred to as a first parameter. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0042] First, a brief introduction to the terms used in the embodiments of this application:

[0043] Projected Gradient Descent (PGD) is an optimization algorithm typically used to solve constrained nonlinear optimization problems. In machine learning and deep learning, PGD can be used to train models, especially in adversarial training scenarios, to generate adversarial examples or for regularization in model optimization. The basic idea of ​​PGD is to iteratively search along the negative gradient direction of the objective function, but at each iteration, the solution is projected back into the feasible region to satisfy the problem's constraints. In adversarial training, the PGD algorithm can be used to generate adversarial examples by finding tiny perturbations in the input samples near the model's decision boundary. These perturbations are almost imperceptible to human vision but sufficient to cause the model to make incorrect predictions. PGD iteratively adjusts the input samples, each adjustment along the gradient direction of the model's output, and then uses a projection step to ensure that the generated adversarial examples satisfy specific constraints.

[0044] This application focuses on the perception of the external environment by autonomous vehicles. The autonomous vehicle acquires environmental images from its own perspective, uses a target recognition model to identify the presence of target content (obstacles) in the images, and promptly reports the recognition results. Based on these results, the autonomous vehicle determines whether to continue driving or stop. Therefore, testing the robustness of the target recognition model is crucial for autonomous vehicles to accurately perceive their environment and travel safely.

[0045] In this application, gradient projection descent is used to iteratively attack images, resulting in disturbed images. These disturbed images are then used as input data to the target recognition model of an autonomous vehicle to test the model's robustness. Specifically, the accuracy of the target recognition model's identification of target content (referred to as obstacles in this application, i.e., the subject the autonomous vehicle needs to avoid) in the image determines whether the model requires targeted training. If gradient projection descent effectively attacks the image, thus interfering with the target recognition model's identification, a large number of disturbed images can be generated using gradient projection descent as training data to specifically train the target recognition model, improving the model's accuracy and robustness.

[0046] KITTI / kitti format (a data format): The KITTI format is a collective term for a series of data formats associated with the KITTI dataset, a large benchmark dataset widely used in computer vision and autonomous driving research. For example, specific file formats of the KITTI dataset include the following: Image files: typically stored in 8-bit PNG (Portable Network Graphics) format, possibly containing cropped and distortion-corrected images; LiDAR point cloud data: stored in binary format, containing 3D point cloud data obtained from LiDAR scans, with each point containing x, y, z coordinates and reflection intensity information.

[0047] The SMOKE model (Single-Stage Monocular 3D Object Detection via Keypoint Estimation) directly predicts the 3D attribute information of the target through keypoint estimation and 3D bounding box regression, instead of relying on traditional 2D detection methods. This avoids the noise that may be introduced by 2D detection and improves the performance of 3D detection.

[0048] The target recognition model in this application is a model trained based on the SMOKE model. It is mainly used to perform target recognition on the acquired images, identify the location and type of target content (which can be obstacles, such as vehicles or pedestrians) in the image. This automatic recognition model is applied to the environmental perception of autonomous vehicles. The recognition results of the automatic recognition model serve as the basis for the autonomous vehicle's judgment and determine the next action of the autonomous vehicle.

[0049] CARLA (Autonomous Driving Simulator): Provides a modular and flexible application editing interface for solving a range of tasks in autonomous driving research and development.

[0050] In this application, an autonomous driving simulator is used to generate a three-dimensional simulation environment to simulate the environment in which an autonomous vehicle is driving. The three-dimensional simulation environment includes a specified weather scene, a three-dimensional model of obstacles, and a three-dimensional model of the autonomous vehicle. In the three-dimensional simulation environment, all vehicles and obstacles move according to a preset working condition scenario.

[0051] Secondly, the detection system of the target recognition model involved in the embodiments of this application will be described, for illustrative purposes only. Please refer to [the relevant documentation]. Figure 1 This implementation environment involves a terminal 110 and a server 120. The terminal 110 and the server 120 are connected via a communication network 130.

[0052] Terminal 110 includes an autonomous driving simulator, which can build a 3D simulation environment to simulate the driving environment of autonomous vehicles. To cover comprehensive weather scenarios and operating conditions, technicians first upload weather scenario configuration files and operating condition configuration files to Terminal 110. The weather scenario configuration file is used to generate simulated weather scenarios for autonomous vehicle operation, and the operating condition configuration file is used to generate operating condition scenarios for autonomous vehicle operation. The operating condition configuration file covers the autonomous vehicle's own driving conditions, as well as the types of obstacles and road conditions encountered by the autonomous vehicle.

[0053] The autonomous driving simulator in terminal 110 can build a corresponding three-dimensional simulation environment based on weather scene configuration files and operating condition configuration files. In the three-dimensional simulation environment, the autonomous vehicle drives as the main body and collects images.

[0054] For example, a 3D simulation environment simulates the driving of an autonomous vehicle on a road in a sunny condition. During the autonomous vehicle's driving process, there is an obstacle in its direction of travel, which is a bus parked on the side of the road.

[0055] After the terminal 110 constructs a 3D simulation environment, it acquires environmental images. These environmental images are taken from the perspective of the autonomous vehicle and are used to represent the surrounding environment during driving. Some environmental images include obstacles as target content, such as vehicles and pedestrians, which the autonomous vehicle needs to avoid or stop in a timely manner. In some embodiments, the target content also includes road conditions; for example, when the number of obstacles in the environmental image reaches a preset threshold, it represents current road congestion.

[0056] Terminal 110 performs an iterative attack on the acquired environmental images based on the Projected Gradient Descent (PGD) method. The purpose of this iterative attack is to interfere with the target recognition model of the autonomous vehicle when it recognizes the environmental images. In other words, the attack on the environmental images is actually to change some information in the environmental images, making the target recognition model of the autonomous vehicle unable to accurately identify the target content, making incorrect judgments, and affecting the driving safety of the autonomous vehicle.

[0057] Terminal 110 sends the first image data obtained after the iterative attack to server 120. Server 120 contains a target recognition model, which identifies the first image data and obtains a first recognition result. This first recognition result includes the target content identified by the target recognition model, including the type of target content and its location information in the image. The first recognition result is used to test the target recognition model, reflecting the accuracy and robustness of the target recognition model in recognizing the attacked image.

[0058] In some embodiments, the target recognition model cannot accurately identify the target content in the image after being attacked by the gradient projection descent method. Therefore, a batch of attacked images can be obtained based on the gradient projection descent method as training data to train the target recognition model.

[0059] The aforementioned terminal can be various forms of terminal devices such as mobile phones, tablets, desktop computers, portable laptops, smart TVs, vehicle terminals, and smart home devices, and this application embodiment does not limit them.

[0060] It is worth noting that the aforementioned servers can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0061] In some embodiments, the server described above can also be implemented as a node in a blockchain system.

[0062] Based on the above-described terminology and application scenarios, the target recognition model detection method provided in this application will be described. This method can be executed by a server or a terminal, or by both a server and a terminal. In this embodiment, the method is described using the terminal as an example. Figure 2 As shown, Figure 2 This is a flowchart of a target recognition model detection method provided in an exemplary embodiment of this application. The method includes the following steps.

[0063] Step 210: Obtain the weather scenario configuration file and the operating condition configuration file.

[0064] The weather scenario configuration file is used to generate simulated weather scenarios when autonomous vehicles are driving, and the operating condition configuration file is used to generate operating condition scenarios when autonomous vehicles are driving.

[0065] Optionally, the weather scene configuration file includes various weather images and descriptions corresponding to different weather conditions. For example, if the weather scene is snowy, the weather images include various images used to represent snowy weather, such as images of snow falling on city roads; the descriptions are as follows: the snowfall is 2.5 mm thick in 24 hours (i.e., the cumulative thickness on the ground is 2.5 mm), the color is white, and the shape is hexagonal.

[0066] The operating condition configuration file includes an obstacle configuration section and a driving operating condition configuration section. The obstacle configuration section contains the configuration of obstacles encountered by the autonomous vehicle when driving, and the driving operating condition configuration section contains the operating conditions of the autonomous vehicle when driving. The operating conditions include at least one of the following: vehicle emergency braking, vehicle emergency avoidance, and vehicle traffic congestion.

[0067] Optionally, the obstacle configuration section includes images and information about various types of obstacles. For example, if the obstacle is a bus, the obstacle configuration section includes images of buses with different shapes; the obstacle information includes: the obstacle's name and three-dimensional dimensions (length, width, and height).

[0068] Weather scenario configuration files and operating condition configuration files are files that technicians prepare in advance and upload to the terminal. In some embodiments, the terminal includes an application that can generate weather scenario configuration files and operating condition configuration files, and technicians can directly configure the above files within the terminal.

[0069] Optionally, the weather scene configuration file corresponds to 10 weather scenes, namely: (1) Sunny;

[0070] (2) Light rain; (3) Moderate rain; (4) Heavy rain; (5) Foggy day; (6) Snowy day; (7) Shadow: The view inside the vehicle is obstructed, resulting in shadows. For example, when an autonomous vehicle drives through the shade of a tree, the surrounding environment is covered by shadows from the perspective of the inside of the vehicle, and the light is weak; (8) Midnight: Refers to the scenario where the vehicle is driving during the midnight period. For example, the period from 00:00:00 to 4:00:00 on the same day can be regarded as midnight. The surrounding environment is dark or invisible from the perspective of the inside of the autonomous vehicle; (9) Backlight: The view inside the vehicle is affected by strong light, making it impossible to accurately observe the surrounding environment. For example, when an autonomous vehicle is driving, there is sunrise / strong light shining in front, and the vehicle is driving against the light; (10) Motion blur: When the autonomous vehicle is driving at a high speed, or when it makes a sharp turn or bumps, the view inside the vehicle becomes blurred due to motion.

[0071] Optionally, the obstacle configuration section corresponds to 7 types of obstacles: buses, trucks, motorcycles, special vehicles, cars, passenger cars, and pedestrians. These obstacles are all obstacle situations that autonomous vehicles may encounter during driving.

[0072] Optionally, the driving condition configuration section corresponds to 5 working conditions, namely: (1) Traffic congestion: The road where the autonomous vehicle is located is congested. For example, there are 10 vehicles visible in the vehicle's field of vision, and the distance between each vehicle is no more than 30 centimeters; (2) Emergency avoidance: Other obstacles suddenly appear during the autonomous vehicle's driving, and the autonomous vehicle needs to make emergency avoidance. For example, when the autonomous vehicle is passing through an intersection, a vehicle or a pedestrian suddenly appears at the intersection's turning point. At this time, the autonomous vehicle needs to make emergency avoidance; (3) Emergency braking: The autonomous vehicle suddenly brakes / stops during its driving. For example, when the autonomous vehicle is driving at a speed of 40 km / h, the vehicle in front suddenly stops due to a malfunction, and the autonomous vehicle makes emergency braking; (4) Vehicle in front leaves: The vehicle in front of the autonomous vehicle leaves the current lane. For example, after the vehicle in front changes lanes, there are no vehicles in front of the autonomous vehicle; (5) Adaptive cruise: When the autonomous vehicle is following a vehicle in front, it needs to maintain a following distance in real time according to the speed of the vehicle in front.

[0073] The obstacle configuration section contains files that describe the shape of obstacles and generate 3D models of obstacles. For example, the obstacle configuration section contains obstacle images that show the shape of obstacles in different views (including left view, right view, front view, etc.).

[0074] Step 220: Generate a three-dimensional simulation driving environment based on the weather scenario configuration file and the driving condition configuration file.

[0075] The 3D simulation driving environment includes 3D models of autonomous vehicles and obstacles. The terminal contains an autonomous driving simulator, which can generate corresponding 3D models based on the input configuration file to simulate the driving environment of autonomous vehicles.

[0076] Optionally, a weather scene configuration file can be input into the autonomous driving simulator to construct the background portion of the three-dimensional simulated driving environment.

[0077] The background section here refers to the physical environment in which the autonomous vehicle is driving under a specified weather scenario. For example, the weather scenario configuration file contains a section for configuring a light rain weather scenario. In the scenario built by the autonomous driving simulator, the autonomous vehicle is driving on a rural road. This background section is used to indicate the road conditions of the rural road under light rain weather. The autonomous vehicle will then simulate moving within the physical environment indicated by this background section.

[0078] Optionally, the obstacle configuration section of the working condition configuration file is input into the autonomous driving simulator, and the autonomous driving simulator generates a three-dimensional model of the obstacle in a three-dimensional simulated driving environment, wherein the obstacle includes at least one of vehicles and pedestrians.

[0079] For example, the work condition configuration file indicates that the obstacle is a bus or a motorcycle, and the obstacle configuration section includes pictures of the obstacle and information describing the size of the obstacle.

[0080] The autonomous driving simulator generates 3D models of obstacles based on the obstacle configuration and places them into the background of the 3D simulated driving environment. The ratio between the 3D obstacle model and the 3D model of the physical environment conforms to preset mapping requirements. For example, if the 3D model of the physical environment simulates a real-world space volume of 10,000 cubic meters, and the obstacle size information indicates an obstacle volume of 90 cubic meters, the actual volume of the 3D model displayed on the terminal screen is a proportionally scaled-down version, with a space volume: 3D model volume ratio of 10,000:1.

[0081] Optionally, the driving condition configuration section of the operating condition configuration file can be input into the autonomous driving simulator, and a three-dimensional model of the autonomous vehicle can be generated in a three-dimensional simulation driving environment through the autonomous driving simulator.

[0082] After the 3D models of obstacles, autonomous vehicles, and the physical environment are generated, a 3D simulation driving environment is obtained. The 3D models of obstacles and autonomous vehicles will perform corresponding activities based on the working conditions indicated by the driving condition configuration section.

[0083] For example, if the weather condition is light rain, the 3D simulation driving environment will simulate and display rainfall. If the physical environment is urban roads, the 3D simulation driving environment will simulate and display the intersecting relationships between different main roads and the traffic lights arranged on each road. The driving condition configuration section indicates that the autonomous vehicle is in a congested road condition, and the obstacle configuration section configures 10 cars and 2 pedestrians as obstacles. The 3D models of the above obstacles are distributed around the 3D model of the autonomous vehicle to form a congested road condition. In some embodiments, the obstacles (vehicles, pedestrians) and the autonomous vehicle travel slowly along the specified direction of the road.

[0084] For example, the specific implementation steps are as follows: S1: Model file import: Place the images of obstacle models such as vehicles and pedestrians into the import directory of the autonomous driving simulator (CARLA). Modify the relevant information in the configuration file to the actual information of the model file. In the CARLA directory, execute the make import command and wait for the compilation to complete. The actual information of the model file includes the shape, size, and name of the obstacle. S2: Weather scene construction: Based on the autonomous driving simulator, construct 10 weather scenes covering sunny, light rain, moderate rain, heavy rain, fog, snow, shadow, midnight, backlight, and motion blur. S3: Working condition scene construction: Based on the autonomous driving simulator, construct at least 12 test working conditions covering buses, trucks, motorcycles, special vehicles, cars, passenger cars, pedestrians, traffic congestion, emergency avoidance, emergency braking, vehicle leaving the front, and adaptive cruise control.

[0085] Step 230: Collect sample image data based on the three-dimensional simulated driving environment.

[0086] The sample image data consists of environmental images collected from the perspective of an autonomous vehicle, which contain obstacles.

[0087] Obstacles in the environmental image are the parts that the target recognition model is expected to be able to identify. If the target recognition model can accurately identify the type of obstacle and its location information in the environmental image, it means that the target recognition model has the ability to provide environmental perception for autonomous vehicles. In some embodiments, in addition to containing obstacles as target content, the environmental image also reflects the driving status of the autonomous vehicle itself, or the environmental image contains prompts to guide the autonomous vehicle's subsequent driving situation, such as signs, which are used to instruct the vehicle to stop, slow down, or return to the original route.

[0088] The system controls the movement of 3D models of autonomous vehicles and obstacles in a 3D simulated driving environment using an autonomous driving simulator.

[0089] Based on the perspective of autonomous vehicles, environmental images corresponding to the three-dimensional simulated driving environment are collected to obtain sample image data. The sample image data includes the distribution and location information of obstacles in the environmental images.

[0090] For example, based on the autonomous driving simulator, image information of autonomous vehicles under different weather and working conditions is collected. In each working condition, a total of 105 images are generated in a single scene. The image information consists of two parts: PNG (Portable Network Graphics, a common image file format) files and NPZ (NumPy Zip, a compressed file format) files. The PNG file contains the visual data of the image information, that is, the pixel information that the user can see. The NPZ file contains additional information related to the image, including the value of each pixel in the image.

[0091] In this embodiment, PNG files and NPZ files represent vehicle driving status and vehicle positioning information, respectively. Vehicle driving status refers to the driving condition of the autonomous vehicle in the 3D simulation driving environment and the driving condition of obstacles from its own perspective. Vehicle positioning information includes the location information of obstacles in the image, reflecting the distance and relative orientation between the autonomous vehicle and the obstacles.

[0092] Optionally, the obtained PNG and NPZ files are converted into testable data in Kitti format, and the files are integrated to generate category labels for the corresponding images. The category labels are used to indicate the name and type of obstacles in the image. For example, if an environmental image contains an obstacle called a bus, the environmental image is converted to Kitti format test data, and the category label based on this test data is "bus".

[0093] The purpose of converting the image data format is to obtain a data format suitable for the target recognition model to perform detection. The target recognition model is a model trained based on the SMOKE model. Image data in kitti format can be used as input data into the target recognition model, which then identifies the target content in the data and outputs the recognition result.

[0094] Step 240: Perform an iterative attack on the sample image data based on the projection gradient descent method to generate the first image data.

[0095] The first image data is the image data obtained by superimposing noise onto the sample image data.

[0096] Gradient projection descent attacks sample image data by adding noise to it. This noise is intended to interfere with the target recognition model's ability to identify the image. While the impact of this noise on the image is not immediately apparent to the human eye, even subtle changes can interfere with the target recognition model, leading to inaccurate results, incorrect judgments, and ultimately affecting the safety of autonomous vehicles.

[0097] The sample image data is converted to a new format to obtain the first test data that meets the preset format requirements.

[0098] For example, if the sample image data is converted into Kitti format, then the first test data is Kitti format test data, which is a data type that the target recognition model can recognize.

[0099] An iterative attack is performed on the first test data using the projective gradient descent method to generate the first image data. The iterative attack refers to the process of adding noise to the first test data, and the number of iterations of the iterative attack is a preset threshold. The data obtained after the i-th attack on the first test data is used as the data for the (i+1)-th attack, where i is a positive integer.

[0100] In other words, when attacking the first test data, the noise superposition process is cumulative. After the previous round of attack ends, an intermediate output result is obtained, and the next round of attack is carried out on the intermediate output result. That is, noise is continued to be superimposed on the data of the intermediate output result.

[0101] For example, a digital attack is performed on the first test data after it has been converted to the Kitti format. The attack method is gradient projection descent (PGD), which superimposes noise on the original image and updates the noise in the direction that maximizes the loss value of the target recognition model (SMOKE model in this application).

[0102] The principle is as follows:

[0103] In this method, noise is superimposed onto the image by changing the pixel values. x is the input to the target recognition model, specifically the pixel values ​​of the image; θ represents the parameter values ​​of the target model; y represents the label value of the target recognition model; and L(θ, x, y) is the loss function of the target recognition model. Let X be the gradient of the input x, sgn be the direction of the gradient, S be the added perturbation, t be the number of attack iterations, and α be the perturbation factor. t+1 This refers to the value of the pixel at the (t+1)th iteration.

[0104] The iterative process mainly involves the following key parameters: iteration count t = 20, threshold eps = 76, and perturbation factor α = 2. That is, the first test data is attacked 20 times. The threshold refers to the upper limit of the change to a single pixel in the image during each attack; the pixel value ranges from 0 to 255, and the change in a single pixel before and after each attack does not exceed 76. In digital attacks, especially when generating adversarial perturbations, the perturbation factor is a parameter used to quantify the strength of the adversarial perturbation. The perturbation factor is usually expressed as a small value, defining the limitation on the size or amplitude of the adversarial perturbation. The perturbation factor controls the difference between the adversarial sample (a sample superimposed with noise, such as the test data obtained after attacking the first test data in this application) and the original sample (such as the first test data in this application). For example, if the original input is x, the adversarial sample can be represented as x + α * noise, where α is the perturbation factor, and noise is the perturbation vector calculated according to the attack algorithm (such as the PGD algorithm in this application).

[0105] This is illustrative; please refer to it. Figure 3 , Figure 3 This is a comparative diagram of environmental images collected under clear weather conditions when the obstacle is a bus, before and after the attack. The difference between sample image 310 before the attack and sample image 320 after the attack is that the surface coating of the obstacle bus 330 has changed.

[0106] Step 250: The first image data is identified using the target recognition model corresponding to the autonomous vehicle to obtain the first recognition result.

[0107] The first recognition result includes the target recognition model's recognition of obstacles in the first image data. The first recognition result is used to test the target recognition model.

[0108] Optionally, before iteratively attacking the sample image data based on the projection gradient descent method to generate the first image data, a category analysis is performed on the sample image data to determine the category label of the sample image data. The category label is used to indicate the category information of obstacles in the sample image data.

[0109] If the difference between the first recognition result and the category label reaches a preset difference threshold, it is determined that the target recognition model does not meet the preset accuracy requirement when recognizing image data.

[0110] For example, if an obstacle in the sample image data is a sign indicating construction ahead and impassable, the autonomous vehicle should turn back. The category label would then identify the obstacle in the sample image data as a "construction sign." However, after iteratively attacking the sample image data using the projective gradient descent method, the target recognition model obtains the first image data. The first recognition result indicates that the automatic recognition model identifies the construction sign as a deceleration sign. This first recognition result differs from the meaning expressed by the category label, with a difference of 80%, reaching the preset difference threshold of 60%. Therefore, it is determined that the target recognition model does not meet the preset accuracy requirement when recognizing the image data.

[0111] In cases where the projected gradient descent method poses an effective attack on the target recognition model, a batch of attacked data can be generated based on this method to train the target recognition model, thereby enabling the target recognition model to resist digital attacks and improving the robustness and recognition accuracy of the target recognition model.

[0112] Optionally, a sample image dataset is obtained, which contains multiple image data sets, all of which have the same format as the sample image data. An iterative attack is performed on the sample image dataset using the projective gradient descent method to generate a training image dataset; wherein, the training image dataset contains multiple training image data sets, all of which have the same format as the first image data; the target recognition model is trained based on the training image dataset.

[0113] In summary, the method provided in this application, by customizing various working conditions and weather scenarios, uses an autonomous driving simulator to simulate the corresponding simulation environment, comprehensively covering various situations that autonomous vehicles may encounter during operation, thus closely resembling reality and improving the simulation testing effect. It generates environmental image information, simulating images that might be captured from the perspective of an autonomous vehicle. These images contain the target content to be identified. Images with adversarial perturbations are used as input data for the target recognition model of the autonomous vehicle. The accuracy and robustness of the target recognition model are determined based on whether it can accurately identify the perturbed target content. The method uses projective gradient descent to iteratively attack the images to generate adversarial perturbations, obtaining a large amount of perturbed image data. This image data can be used as training data for the target recognition model, specifically improving the training effect of the target recognition model and enabling it to possess the ability to resist attack perturbations.

[0114] Figure 4 This is a structural block diagram of a target recognition model detection device provided in an exemplary embodiment of this application, as shown below. Figure 4 As shown, the device includes the following parts.

[0115] The acquisition module 410 is used to acquire a weather scene configuration file and a driving condition configuration file. The weather scene configuration file is used to generate a simulated weather scene when the autonomous vehicle is driving. The driving condition configuration file is used to generate a driving condition scene when the autonomous vehicle is driving. The driving condition configuration file includes an obstacle configuration part and a driving condition configuration part. The obstacle configuration part includes the configuration of obstacles encountered by the autonomous vehicle when driving. The driving condition configuration part includes the working status of the autonomous vehicle when driving. The working status includes at least one of vehicle emergency braking, vehicle emergency avoidance, and vehicle traffic congestion.

[0116] The generation module 420 is used to generate a three-dimensional simulation driving environment based on the weather scene configuration file and the working condition configuration file. The three-dimensional simulation driving environment includes three-dimensional models of the autonomous vehicle and the obstacles.

[0117] The acquisition module 430 is used to acquire sample image data based on the three-dimensional simulated driving environment. The sample image data is an environmental image acquired from the perspective of the autonomous vehicle, and the environmental image contains the obstacles.

[0118] The generation module 420 is further configured to perform an iterative attack on the sample image data based on the projection gradient descent method to generate first image data, wherein the first image data is image data obtained by superimposing noise on the sample image data;

[0119] The recognition module 440 is used to recognize the first image data through the target recognition model corresponding to the autonomous vehicle to obtain a first recognition result. The first recognition result includes the recognition status of the obstacle in the first image data by the target recognition model. The first recognition result is used to test the target recognition model.

[0120] In an optional embodiment, the generation module 420 is further configured to input the weather scene configuration file into the autonomous driving simulator, and construct the background portion of the three-dimensional simulated driving environment through the autonomous driving simulator; input the obstacle configuration portion of the working condition configuration file into the autonomous driving simulator, and generate a three-dimensional model of the obstacle in the three-dimensional simulated driving environment through the autonomous driving simulator; the obstacle includes at least one of vehicles and pedestrians; input the driving working condition configuration portion of the working condition configuration file into the autonomous driving simulator, and generate a three-dimensional model of the autonomous vehicle in the three-dimensional simulated driving environment through the autonomous driving simulator.

[0121] In an optional embodiment, the acquisition module 430 is further configured to control the three-dimensional models of the autonomous vehicle and the obstacles to move in the three-dimensional simulation driving environment through the autonomous driving simulator; acquire environmental images corresponding to the three-dimensional simulation driving environment based on the perspective of the autonomous vehicle to obtain the sample image data, wherein the sample image data includes the distribution and location information of the obstacles in the environmental image.

[0122] In an optional embodiment, the generation module 420 is further configured to perform format conversion on the sample image data to obtain first test data that meets the preset format requirements; perform an iterative attack on the first test data based on the projection gradient descent method to generate the first image data, wherein the iterative attack refers to the process of adding noise to the first test data, and the number of iterations of the iterative attack is a preset threshold; wherein the data obtained after the i-th attack on the first test data is used as the data used in the (i+1)-th attack, and i is a positive integer.

[0123] In an optional embodiment, before the generation module 420, such as Figure 5 As shown, the device further includes:

[0124] The category determination module 450 is used to perform category analysis on the sample image data and determine the category label of the sample image data. The category label is used to indicate the category information of the obstacle in the sample image data.

[0125] In an optional embodiment, after the identification module, the device further includes:

[0126] The training module 460 is configured to determine that the target recognition model does not meet a preset accuracy requirement when recognizing image data if the difference between the first recognition result and the category label reaches a preset difference threshold; acquire a sample image dataset containing multiple image data, the multiple image data having the same format as the sample image data; perform an iterative attack on the sample image dataset based on the projection gradient descent method to generate a training image dataset; wherein the training image dataset contains multiple training image data, the multiple training image data having the same format as the first image data; and train the target recognition model based on the training image dataset.

[0127] In summary, the target recognition model detection device provided in this application can simulate corresponding simulation environments using an autonomous driving simulator by customizing various working conditions and weather scenarios. This comprehensively covers various situations that autonomous vehicles may encounter during operation, closely mirroring reality and improving simulation testing effectiveness. It generates environmental image information, simulating images that might be captured from the perspective of an autonomous vehicle. These images contain the target content to be identified. Images with adversarial perturbations are used as input data for the target recognition model of the autonomous vehicle. The accuracy and robustness of the target recognition model are determined based on whether it can accurately identify the perturbed target content. The projection gradient descent method is used to iteratively attack the images to generate adversarial perturbations, resulting in a large amount of perturbed image data. This image data can be used as training data for the target recognition model, specifically improving the training effect and enabling the target recognition model to possess the ability to resist attack perturbations.

[0128] It should be noted that the target recognition model detection device provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the target recognition model detection device and the target recognition model detection method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0129] Figure 6 This illustration shows a structural block diagram of a computer device 600 provided in an exemplary embodiment of this application. The computer device 600 may be a smartphone, tablet computer, MP3 player (Moving Picture Experts Group Audio Layer III), MP4 player (Moving Picture Experts Group Audio Layer IV), laptop computer, or desktop computer. The computer device 600 may also be referred to as a user device, portable terminal, laptop terminal, desktop terminal, or other names.

[0130] Typically, computer device 600 includes a processor 601 and a memory 602.

[0131] Processor 601 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 601 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 601 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 601 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 601 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0132] The memory 602 may include one or more computer-readable storage media, which may be non-transitory. The memory 602 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 602 are used to store at least one instruction, which is executed by the processor 601 to implement the target recognition model detection method provided in the method embodiments of this application.

[0133] In some embodiments, the computer device 600 also includes other components 603, the type and number of which can be selected based on the functional needs of the computer device 600. Those skilled in the art will understand that... Figure 6 The structure shown does not constitute a limitation on the computer device 600, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0134] Optionally, the computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), solid-state drives (SSDs), or optical discs, etc. The random access memory may include resistive random access memory (ReRAM) and dynamic random access memory (DRAM). The sequence numbers of the embodiments in this application are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0135] This application also provides a computer device, which includes a processor and a memory. The memory stores at least one instruction, at least one program, a code set, or an instruction set. The at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the target recognition model detection method as described in any of the above embodiments of this application.

[0136] This application also provides a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the target recognition model detection method as described in any of the above embodiments of this application.

[0137] This application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the target recognition model detection method described in any of the above embodiments.

[0138] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0139] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A detection method for a target recognition model, characterized in that, The method includes: The system obtains a weather scenario configuration file and a driving condition configuration file. The weather scenario configuration file is used to generate a simulated weather scenario when the autonomous vehicle is driving. The driving condition configuration file is used to generate a driving condition scenario when the autonomous vehicle is driving. The driving condition configuration file includes an obstacle configuration part and a driving condition configuration part. The obstacle configuration part includes the configuration of obstacles encountered by the autonomous vehicle when driving. The driving condition configuration part includes the working status of the autonomous vehicle when driving. The working status includes at least one of vehicle emergency braking, vehicle emergency avoidance, and vehicle traffic congestion. A three-dimensional simulation driving environment is generated based on the weather scene configuration file and the working condition configuration file. The three-dimensional simulation driving environment includes three-dimensional models of the autonomous vehicle and the obstacles. Sample image data is collected based on the three-dimensional simulated driving environment. The sample image data is an environmental image collected from the perspective of the autonomous vehicle, and the environmental image includes the obstacles. The sample image data is converted to a new format to obtain first test data that meets the preset format requirements; The first test data is iteratively attacked based on the projection gradient descent method to generate the first image data. The iterative attack refers to the process of adding noise to the first test data. The number of iterations of the iterative attack is a preset threshold. The data obtained after the i-th attack on the first test data is used as the data used in the (i+1)-th attack, where i is a positive integer. The first image data is identified by the target recognition model corresponding to the autonomous vehicle to obtain a first recognition result. The first recognition result includes the recognition status of the obstacle in the first image data by the target recognition model. The first recognition result is used to test the target recognition model.

2. The method according to claim 1, characterized in that, The generation of a three-dimensional simulated driving environment based on the weather scenario configuration file and the driving condition configuration file includes: The weather scene configuration file is input into the autonomous driving simulator, and the background part of the three-dimensional simulated driving environment is constructed through the autonomous driving simulator; The obstacle configuration section of the working condition configuration file is input into the autonomous driving simulator, and the autonomous driving simulator generates a three-dimensional model of the obstacle in the three-dimensional simulation driving environment; the obstacle includes at least one of vehicles and pedestrians. The driving condition configuration section of the working condition configuration file is input into the autonomous driving simulator, and the autonomous driving simulator generates a three-dimensional model of the autonomous vehicle in the three-dimensional simulation driving environment.

3. The method according to claim 2, characterized in that, The sample image data acquired based on the three-dimensional simulated driving environment includes: The autonomous driving simulator controls the movement of the autonomous vehicle and the three-dimensional model of the obstacle in the three-dimensional simulated driving environment. Based on the perspective of the autonomous vehicle, environmental images corresponding to the three-dimensional simulated driving environment are acquired to obtain sample image data, wherein the sample image data includes the distribution and location information of the obstacles in the environmental images.

4. The method according to any one of claims 1 to 3, characterized in that, Before generating the first image data by iteratively attacking the sample image data based on the projection gradient descent method, the method further includes: The sample image data is subjected to category analysis to determine the category label of the sample image data. The category label is used to indicate the category information of the obstacle in the sample image data.

5. The method according to claim 4, characterized in that, After obtaining the first recognition result by recognizing the first image data using the target recognition model corresponding to the autonomous vehicle, the method further includes: If the difference between the first recognition result and the category label reaches a preset difference threshold, it is determined that the target recognition model does not meet the preset accuracy requirement when recognizing image data. Obtain a sample image dataset, which contains multiple image data sets, and the multiple image data sets have the same format as the sample image data; The sample image dataset is iteratively attacked based on the projective gradient descent method to generate a training image dataset; wherein, the training image dataset contains multiple training image data, and the multiple training image data have the same format as the first image data; The target recognition model is trained based on the training image dataset.

6. A detection device for a target recognition model, characterized in that, The device includes: The acquisition module is used to acquire a weather scene configuration file and a driving condition configuration file. The weather scene configuration file is used to generate a simulated weather scene when the autonomous vehicle is driving. The driving condition configuration file is used to generate a driving condition scene when the autonomous vehicle is driving. The driving condition configuration file includes an obstacle configuration part and a driving condition configuration part. The obstacle configuration part includes the configuration of obstacles encountered by the autonomous vehicle when driving. The driving condition configuration part includes the working status of the autonomous vehicle when driving. The working status includes at least one of vehicle emergency braking, vehicle emergency avoidance, and vehicle traffic congestion. A generation module is used to generate a three-dimensional simulation driving environment based on the weather scene configuration file and the working condition configuration file. The three-dimensional simulation driving environment includes three-dimensional models of the autonomous vehicle and the obstacles. The acquisition module is used to acquire sample image data based on the three-dimensional simulated driving environment. The sample image data is an environmental image acquired from the perspective of the autonomous vehicle, and the environmental image includes the obstacles. The generation module is further configured to convert the sample image data into a format to obtain first test data that meets the preset format requirements; perform an iterative attack on the first test data based on the projection gradient descent method to generate first image data, wherein the iterative attack refers to the process of adding noise to the first test data, and the number of iterations of the iterative attack is a preset threshold; wherein the data obtained after the i-th attack on the first test data is used as the data used in the (i+1)-th attack, and i is a positive integer; The recognition module is used to recognize the first image data through the target recognition model corresponding to the autonomous vehicle to obtain a first recognition result. The first recognition result includes the recognition status of the obstacle in the first image data by the target recognition model. The first recognition result is used to test the target recognition model.

7. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one program, which is loaded and executed by the processor to implement the target recognition model detection method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The storage medium stores at least one program segment, which is loaded and executed by a processor to implement the target recognition model detection method as described in any one of claims 1 to 5.

9. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the detection method of the target recognition model as described in any one of claims 1 to 5.

Images