Object detection model influence autonomous driving safety evaluation method, device and equipment

Abstract

The present specification relates to the technical field of automatic driving, and provides an evaluation method, device and equipment for influence of an object detection model on automatic driving safety, which comprises: providing a driving scene in a driving scene set to an object detection model to obtain an obstacle identification result; performing dynamic and static obstacle classification on the obstacle identification result to obtain an obstacle classification result; determining a target vehicle destination area, a cost function and driving constraints in the driving scene; planning a motion trajectory of the target vehicle according to the obstacle classification result, the target vehicle destination area, the cost function and the driving constraints; controlling the target vehicle to perform simulated driving according to the motion trajectory to obtain a simulated driving result; and determining the influence of the object detection model on driving safety of the target vehicle according to the motion trajectory planning result and the simulated driving result. The embodiment of the present specification can evaluate the influence of the object detection model on automatic driving safety.

Description

Technical Field

[0001] This specification relates to the field of autonomous driving technology, and in particular to an assessment method, apparatus, and device for evaluating the impact of object detection models on autonomous driving safety. Background Technology

[0002] In recent years, autonomous driving systems have adopted numerous deep learning models. However, the application of these models has introduced new vulnerabilities that may compromise the safety of autonomous vehicles. For example, consider object detection models in autonomous driving systems (which identify obstacles from detection data output by visual sensors or LiDAR). Currently, many object detection models are derived from pre-trained deep learning models. Because the application of these models may introduce vulnerabilities, the object detection results obtained may be inaccurate, thus jeopardizing the driving safety risks of the trajectory planned by the autonomous driving system. Given that driving safety is a core concern in the field of autonomous driving, assessing the impact of object detection models on autonomous driving safety is of great significance. However, currently, there is a lack of technical solutions for evaluating the impact of object detection models on autonomous driving safety. Summary of the Invention

[0003] The purpose of the embodiments in this specification is to provide a method, apparatus, and device for evaluating the impact of object detection models on autonomous driving safety, so as to assess the impact of object detection models on autonomous driving safety.

[0004] To achieve the above objectives, on the one hand, embodiments of this specification provide a method for evaluating the impact of object detection models on autonomous driving safety, including: The driving scenarios in the set of driving scenarios are provided to the object detection model to obtain obstacle recognition results; The obstacle identification results are then classified into dynamic and static obstacles to obtain the obstacle classification results; Determine the target vehicle's destination area, cost function, and driving constraints under the stated driving scenario; Based on the obstacle classification results, the target area, the cost function, and the driving constraints, the motion trajectory of the target vehicle is planned. The target vehicle is controlled to perform simulated driving according to the motion trajectory, and the simulated driving results are obtained. The impact of the object detection model on the driving safety of the target vehicle is determined based on the motion trajectory planning results and the simulated driving results.

[0005] In the method for assessing the impact of object detection models on autonomous driving safety in the embodiments of this specification, the impact of the object detection model on the driving safety of the target vehicle is determined based on the motion trajectory planning results and the simulated driving results, including: The success rate is determined based on the motion trajectory planning results, and the collision rate is determined based on the simulated driving results. The safe driving rate of the object detection model is determined based on the successful planning rate and the collision rate.

[0006] In the evaluation method for the impact of object detection models on autonomous driving safety in the embodiments of this specification, the successful planning rate is based on the formula... Calculated; The collision rate is based on the formula Calculated; The safe driving rate is based on the formula Calculated; in, To increase the success rate of planning, The total number of driving scenarios in the set of driving scenarios. This represents the number of driving scenarios in the set of driving scenarios that can successfully generate motion trajectories. For collision rate, To successfully generate the number of driving scenarios corresponding to the motion trajectories in the motion trajectory where collisions occur. For safe driving rate.

[0007] The object detection model used in the embodiments of this specification also includes the following methods for assessing the impact of autonomous driving safety: The information value of the corresponding driving scenario is determined based on the smoothness of the motion trajectory, the distance between the target vehicle and the obstacle, and the cognitive uncertainty of the object detection model. The driving scenarios whose information value reaches a value threshold are collected from the set of driving scenarios and used as one of the training data for subsequent improvement of the object detection model.

[0008] In the evaluation method for the impact of object detection models on autonomous driving safety in the embodiments of this specification, the information value of the corresponding driving scenario is determined based on the smoothness of the motion trajectory, the distance between the target vehicle and the obstacle, and the cognitive uncertainty of the object detection model, including: According to the formula Calculate the information value of driving scenarios; in, For the first The information value of each driving scenario For the object detection model in the first Cognitive uncertainty in various driving scenarios for constant coefficients, For the first The smoothness of the motion trajectory corresponding to each driving scenario. for constant coefficients, For the first The distance between the target vehicle and obstacles in each driving scenario. for The constant coefficients.

[0009] In the evaluation method for the impact of object detection models on autonomous driving safety in the embodiments of this specification, the driving scenarios in the set of driving scenarios include: Visual images and / or point cloud data of multiple road types under different weather conditions; Visual images and / or point cloud data of multiple road types under different adversarial attacks; and, Visual images and / or point cloud data of multiple road types under different weather conditions and different adversarial attacks.

[0010] In the evaluation method for the impact of object detection models on autonomous driving safety in the embodiments of this specification, the driving scenarios in the set of driving scenarios are obtained in advance through the following methods: Select multiple typical driving scenarios containing different road types from the typical driving scenario dataset; Using weather conditions and / or counter-attack conditions as variables, each typical driving scenario is extended into different driving scenarios based on synthesis or simulation.

[0011] On the other hand, embodiments of this specification also provide an evaluation device for assessing the impact of object detection models on autonomous driving safety, including: A module is provided to provide driving scenarios from a set of driving scenarios to an object detection model in order to obtain obstacle recognition results; The classification module is used to classify the obstacle recognition results into dynamic and static obstacles to obtain obstacle classification results; The determination module is used to determine the target vehicle's destination area, cost function, and driving constraints in the driving scenario. The planning module is used to plan the motion trajectory of the target vehicle based on the obstacle classification results, the target area, the cost function, and the driving constraints. The control module is used to control the target vehicle to perform simulated driving according to the motion trajectory and obtain the simulated driving results; The evaluation module is used to determine the impact of the object detection model on the driving safety of the target vehicle based on the motion trajectory planning results and the simulated driving results.

[0012] On the other hand, embodiments of this specification also provide a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the computer program, when run by the processor, executes instructions for the above-described method.

[0013] On the other hand, embodiments of this specification also provide a computer storage medium storing a computer program thereon, which, when run by the processor of a computer device, executes instructions for the above-described method.

[0014] As can be seen from the technical solutions provided in the embodiments of this specification above, by providing the driving scenarios in the pre-configured set of driving scenarios to the object detection model, the obstacle recognition results under the corresponding driving scenarios can be obtained. After classifying the obstacle recognition results into dynamic and static obstacles, the motion trajectory of the target vehicle is planned according to the obstacle classification results, the target area of ​​the target vehicle, the cost function of the target vehicle, and the driving constraints of the target vehicle. The target vehicle is then controlled to perform simulated driving according to the motion trajectory to obtain the simulated driving results. Then, the impact of the object detection model on the driving safety of the target vehicle is determined based on the motion trajectory planning results and the simulated driving results, thereby realizing the evaluation of the impact of the object detection model on the safety of autonomous driving. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings: Figure 1 Flowcharts illustrating the evaluation methods for the impact of object detection models on autonomous driving safety in some embodiments of this specification are shown. Figure 2 Flowcharts of assessment methods for the impact of object detection models on autonomous driving safety are shown in other embodiments of this specification; Figure 3 The flowchart illustrates a process in some embodiments of this specification for determining the impact of an object detection model on the driving safety of a target vehicle based on motion trajectory planning results and simulated driving results. Figure 4 This specification shows a structural block diagram of an evaluation device for the impact of object detection models on autonomous driving safety in some embodiments; Figure 5 A structural block diagram of a computer device in some embodiments of this specification is shown.

[0016] [Explanation of Labels in the Attached Image]

[0017] 41. Provide modules; 42. Classification module; 43. Determine the module; 44. Planning Module; 45. Control module; 46. ​​Evaluation Module; 502. Computer equipment; 504, Processor; 506. Memory; 508. Drive mechanism; 510. Input / output interface; 512. Input devices; 514. Output devices; 516. Presentation equipment; 518. Graphical User Interface; 520. Network interface; 522. Communication link; 524. Communication bus. Detailed Implementation

[0018] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0019] Autonomous driving systems rely on the collaborative efforts of artificial intelligence, computer vision, radar, monitoring devices, and navigation and positioning systems to safely and automatically control motor vehicles without any active human intervention. Current autonomous driving systems employ numerous pre-trained deep learning models. However, the application of these models has introduced new vulnerabilities that may compromise the safety of autonomous vehicles. Therefore, it is necessary to assess the impact of deep learning models on the driving safety performance of autonomous driving systems.

[0020] The following embodiments in this specification use the evaluation of the impact of object detection models on autonomous driving safety as an example. However, those skilled in the art will understand that there are many deep learning models used in autonomous driving systems, and object detection models are only one of them. Besides object detection models involved in sensor data processing such as obstacle sensors, deep learning models may also be involved in data processing such as path planning, path execution, and vehicle status monitoring in autonomous driving systems. Therefore, the technical solutions for evaluating the impact of object detection models on autonomous driving safety in the embodiments of this specification should not be construed as the sole limitation of the embodiments in this specification. Adaptive modifications can also be made to other embodiments of this specification to evaluate the impact of other deep learning models on autonomous driving safety.

[0021] This specification provides an embodiment of a method for evaluating the impact of object detection models on autonomous driving safety, which can be applied to computer devices. (Reference) Figure 1 As shown, in some embodiments, the method for assessing the impact of the object detection model on autonomous driving safety may include the following steps: Step 101: Provide the driving scenes from the driving scene set to the object detection model to obtain obstacle recognition results.

[0022] Step 102: Perform dynamic and static obstacle classification on the obstacle identification results to obtain obstacle classification results.

[0023] Step 103: Determine the target vehicle's destination area, cost function, and driving constraints in the driving scenario.

[0024] Step 104: Based on the obstacle classification results, the target area, the cost function, and the driving constraints, plan the motion trajectory of the target vehicle.

[0025] Step 105: Control the target vehicle to perform simulated driving according to the motion trajectory and obtain the simulated driving results.

[0026] Step 106: Determine the impact of the object detection model on the driving safety of the target vehicle based on the motion trajectory planning results and the simulated driving results.

[0027] This embodiment of the specification provides driving scenarios from a pre-configured set of driving scenarios to the object detection model, thereby obtaining obstacle recognition results for the corresponding driving scenarios. After classifying the obstacle recognition results into dynamic and static obstacles, the target vehicle's trajectory is planned based on the obstacle classification results, the target vehicle's destination area, the target vehicle's cost function, and the target vehicle's driving constraints. The target vehicle is then controlled to perform simulated driving according to the trajectory to obtain simulated driving results. Finally, the impact of the object detection model on the target vehicle's driving safety is determined based on the trajectory planning results and the simulated driving results, thus achieving the assessment of the object detection model's impact on autonomous driving safety.

[0028] Considering that real-world driving scenarios may not fully cover all driving situations (especially some extreme driving scenarios), to facilitate rapid and comprehensive evaluation and targeted testing of the robustness of object detection models in complex driving scenarios, more driving scenarios can be obtained through techniques such as simulation or synthesis. For example, in some embodiments, multiple typical driving scenarios containing different road types can be selected from a typical driving scenario dataset, and each typical driving scenario can be expanded into different driving scenarios based on weather conditions and / or adversarial attack conditions as variables, using synthesis or simulation methods. Therefore, the pre-constructed set of driving scenarios can include both real-world driving scenarios (e.g., typical driving scenarios) and driving scenarios obtained through simulation or synthesis techniques.

[0029] In some embodiments, the driving scenarios in the driving scenario set may include: visual images and / or point cloud data of multiple road types under different weather conditions; visual images and / or point cloud data of multiple road types under different adversarial attacks; visual images and / or point cloud data of multiple road types under different weather conditions and different adversarial attacks, etc. It should be noted that this is only an illustrative description of driving scenarios. In other embodiments, the driving scenario set may include more types of driving scenarios, which are not limited in this specification.

[0030] Synthetic driving scene rendering primarily targets data generated by purely visual sensors. For example, extreme weather effects, such as raindrops, fog, or snowflakes, can be randomly overlaid in a two-dimensional format onto image data from the KITTI dataset for autonomous driving. The KITTI dataset is one of the most important test sets in the field of autonomous driving, mainly focusing on image processing techniques for autonomous driving, primarily applied to autonomous driving perception and prediction, and also involving localization and SLAM (Simultaneous Localization and Mapping) technologies.

[0031] The simulated driving scenario approach primarily targets data generated by the fusion of camera and LiDAR data. For example, in one embodiment, the CARLA simulator can be used to simulate point cloud and image data corresponding to LiDAR and cameras under different extreme weather conditions, based on four different driving scenarios (urban, rural, highway, and street). The CARLA simulator is an open-source simulator that can be used to simulate real-world traffic environments. Research indicates that simulation is currently one of the most effective methods for acquiring data under different weather conditions for LiDAR data.

[0032] In some embodiments, the driving scenes obtained based on the two methods described above can be converted into KITTI format for subsequent processing. In one exemplary embodiment, the driving scene set contains 5312 images. The dataset contains images of various sizes and 1600 point cloud data points. 3712 images were synthesized from object detection training images in the KITTI dataset; their data labels follow the original KITTI dataset annotations, and they do not have corresponding point cloud data. The remaining 1600 images and point cloud data were obtained through simulation, including four 20-second data segments collected at a 20Hz frequency from four different scenes. The data labels can be directly generated by the simulator. Furthermore, in another embodiment, a corresponding code library can be provided to allow users to customize and adjust weather effects. The synthesis code library supports parameterized adjustment of different weather parameters, such as fog concentration, to overlay different levels of weather data. The simulation code library also provides a simulator interface to support the generation of scenes under different weather conditions.

[0033] Adversarial attacks refer to malicious attacks on autonomous vehicles or autonomous driving systems, such as close-range sensor attacks, remote network attacks, perturbation attacks, and patch attacks. Because adversarial attacks can impact autonomous driving safety, this safety factor needs to be considered when generating driving scenario sets. Moreover, compared to extreme weather, adversarial attack scenario data is more difficult to collect, and readily available datasets are generally unavailable. Therefore, adversarial attack scenario data can be obtained by synthesizing adversarial examples from the KITTI dataset. Adversarial attack scenario data can include data from cameras and LiDAR under mainstream adversarial attacks. First, attack algorithms such as PGD (Project Gradient Descent) are used, which add perturbations or patches to the images to cause false or inaccurate detections by the detector. For LiDAR point cloud data, the LiDAR-Adv algorithm can be used to implement a deception attack, causing the 3D object detector (an object detection model) to produce false detections.

[0034] Object detection models can process input driving scenes and identify obstacles (such as other vehicles, pedestrians, etc.) within them. For example, when an object detection model identifies an obstacle in a driving scene image, it can use a two-dimensional or three-dimensional bounding box to outline the obstacle, thus obtaining the obstacle recognition result.

[0035] In actual testing, some or all of the driving scenarios can be selected from the set of driving scenarios as needed, and then input into the object detection model one by one for processing.

[0036] Obstacle recognition results only contain static information (such as object category, bounding box size, bounding box center position in 3D space, and confidence score). Based on static information from a single frame of data, it is difficult to distinguish between dynamic and static objects. However, subsequent trajectory planning requires dynamic information about the objects (i.e., obstacles) as part of its input. Therefore, it is necessary to classify obstacle recognition results into dynamic and static obstacles.

[0037] In some embodiments, a convolutional neural network (CNN) can be pre-trained to serve as a dynamic object classifier. This method uses consecutive frames of image data to distinguish between dynamic and static objects, and can label each object to indicate whether it is moving. By linking obstacle recognition results with dynamic information through this strategy, the process can be represented as... .in, Represents a dynamic object classifier The output (i.e., the obstacle classification result). For dynamic object classifiers The input parameters, specifically, These are the left and right images from binocular vision, respectively. This is the baseline distance (i.e., pupillary distance).

[0038] To plan a trajectory consistent with real-world driving scenarios, driving constraints need to be considered. These constraints include speed limits for different road types and dynamic constraints (such as acceleration, jolt, and energy) for different moving vehicles, to match realistic driving conditions. Since driving scenarios can change continuously in real-world driving, appropriate driving constraints need to be selected in a timely manner for driving safety assessment. In some embodiments, another CNN model can be pre-trained as a driving constraint selector. Driving constraint selector The driving scenario can be categorized by road type, and appropriate driving constraints can be selected for evaluation. The driving constraint selector can be represented as... .in, This represents the initial state of the vehicle. Within the permitted speed range of the road, This represents vehicle dynamics constraints. In the embodiments of this specification, the vehicle state can be represented as... ,in, These represent the vehicle's position, velocity, direction, and steering angle at a specific moment. Therefore, the vehicle's initial state... It can be characterized by a combination of the vehicle's position, speed, direction, and steering angle at the current moment.

[0039] The target vehicle in the embodiments of this specification is an autonomous vehicle, which may include, but is not limited to, fuel cell vehicles or electric vehicles with autonomous driving functions. The destination area is a defined region, the next destination the vehicle needs to reach, and the vehicle must enter this region within a defined time. The destination area of ​​the target vehicle can be determined based on user input. The cost function of the target vehicle is introduced to improve its energy efficiency and driving efficiency. Therefore, in the embodiments of this specification, the cost function may include energy consumption costs and driving time costs, etc., and the cost function can be pre-built and saved as configuration information. Therefore, the cost function of the target vehicle can be read from the configuration information. Once the driving scenario and destination area are determined, the possible driving segments of the target vehicle can be determined, and the speed constraints under the corresponding driving segments can also be determined. The dynamic constraints of the target vehicle can be read from the configuration information. Thus, the driving constraints of the target vehicle in the driving scenario are determined.

[0040] In some embodiments, the trajectory of the target vehicle can be planned based on the obstacle classification results, the target area, the cost function, and the driving constraints, using path planning algorithms such as search algorithms, random sampling, curve interpolation, or artificial potential field methods. The planned trajectory is the optimal trajectory for a continuous time series. The trajectory includes not only path planning but also speed planning. For example, in an exemplary embodiment, the A algorithm in the search algorithm can be used... The search algorithm plans the trajectory of the target vehicle.

[0041] Those skilled in the art will understand that the planning of the trajectory should be based on driving safety (i.e., avoiding collisions, etc.). Furthermore, in another embodiment, the planning of the trajectory may also take into account the passenger's tolerance to acceleration, avoiding discomfort to the passenger caused by sudden acceleration or braking.

[0042] To verify the effectiveness of the planned motion trajectory for subsequent driving safety assessments, the planned motion trajectory can be provided to the vehicle control module of the target vehicle. The vehicle control module can then control the target vehicle's accelerator, steering wheel, brakes, etc., to perform simulated driving according to the planned trajectory, thereby obtaining simulated driving results.

[0043] refer to Figure 2 As shown, in some other embodiments, the method for assessing the impact of the object detection model on autonomous driving safety may further include the following steps: Step 201: Provide the driving scenes from the driving scene set to the object detection model to obtain obstacle recognition results.

[0044] Step 202: Perform dynamic and static obstacle classification on the obstacle identification results to obtain obstacle classification results.

[0045] Step 203: Determine the target vehicle's destination area, cost function, and driving constraints in the driving scenario.

[0046] Step 204: Based on the obstacle classification results, the target area, the cost function, and the driving constraints, plan the motion trajectory of the target vehicle.

[0047] Step 205: Control the target vehicle to perform simulated driving according to the motion trajectory and obtain the simulated driving results.

[0048] Step 206: Determine the impact of the object detection model on the driving safety of the target vehicle based on the motion trajectory planning results and the simulated driving results.

[0049] Step 207: Determine the information value of the corresponding driving scenario based on the smoothness of the motion trajectory, the distance between the target vehicle and the obstacle, and the cognitive uncertainty of the object detection model. Step 207 can be executed after step 205 or after step 206.

[0050] Step 208: Collect driving scenarios from the driving scenario set whose information value reaches a value threshold, and use them as training data for subsequent improvements to the object detection model. This allows the retrained object detection model to achieve better training results with less data, thereby improving the accuracy and robustness of the object detection model.

[0051] In some embodiments, determining the information value of the corresponding driving scenario based on the distance between the target vehicle and the obstacle, and the cognitive uncertainty of the object detection model, may include: according to the formula Calculate the information value of driving scenarios. Among them, For the first The information value of each driving scenario For the object detection model in the first Cognitive uncertainty in various driving scenarios for constant coefficients, For the first The smoothness of the motion trajectory corresponding to each driving scenario. for constant coefficients, For the first The distance between the target vehicle and obstacles in each driving scenario. for The constant coefficient. Note that when the first... When there are multiple obstacles in a driving scenario, the formula It can be rewritten as ,in, For the first The number of obstacles in each driving scenario For the first The first driving scenario An obstacle.

[0052] refer to Figure 3 As shown, in some embodiments, determining the impact of the object detection model on the driving safety of the target vehicle based on the motion trajectory planning results and the simulated driving results may include the following steps: Step 301: Determine the success rate of the motion trajectory planning based on the results, and determine the collision rate based on the simulated driving results.

[0053] Whether a collision occurs during simulated driving according to the planned trajectory is a crucial factor in assessing driving safety; therefore, the collision rate needs to be calculated. Furthermore, if the planned trajectory fails, the target vehicle cannot perform the simulated driving; thus, the trajectory planning result can also serve as an important factor in assessing driving safety. Therefore, it is necessary to determine the successful planning rate based on the trajectory planning results and the collision rate based on the simulated driving results. That is, in the embodiments of this specification, the driving safety performance evaluation of the object detection model is jointly determined by the successful planning rate and the collision rate, which is a more intuitive measure of autonomous driving safety.

[0054] In some embodiments, the success rate can be calculated using the formula The calculation yielded the following result. To increase the success rate of planning, The total number of driving scenarios in the set of driving scenarios. This represents the number of driving scenarios in the set of driving scenarios that can successfully generate motion trajectories.

[0055] In some embodiments, the collision rate can be calculated using a formula. Calculated; where, For collision rate, To successfully generate the number of driving scenarios corresponding to the motion trajectories in the motion trajectory where collisions occur. This represents the number of driving scenarios in the set of driving scenarios that can successfully generate motion trajectories.

[0056] Step 302: Determine the safe driving rate of the object detection model based on the successful planning rate and the collision rate.

[0057] In some embodiments, the safe driving rate is calculated according to the formula The calculation yielded the following result. For safe driving rate, For collision rate, To ensure the success rate of planning.

[0058] Corresponding to the aforementioned method for assessing the impact of object detection models on autonomous driving safety, this specification also provides an apparatus for assessing the impact of object detection models on autonomous driving safety. (Reference) Figure 4 As shown, the assessment device for the impact of the object detection model on autonomous driving safety may include: Module 41 is provided, which can be used to provide driving scenes from the set of driving scenes to the object detection model in order to obtain obstacle recognition results; The classification module 42 can be used to classify the obstacle recognition results into dynamic and static obstacles to obtain obstacle classification results; The determination module 43 can be used to determine the target vehicle's destination area, cost function, and driving constraints in the driving scenario. Planning module 44 can be used to plan the motion trajectory of the target vehicle based on the obstacle classification results, the target area, the cost function, and the driving constraints. Control module 45 can be used to control the target vehicle to perform simulated driving according to the motion trajectory and obtain simulated driving results; The evaluation module 46 can be used to determine the impact of the object detection model on the driving safety of the target vehicle based on the motion trajectory planning results and the simulated driving results.

[0059] In some embodiments of the evaluation device, determining the impact of the object detection model on the driving safety of the target vehicle based on the motion trajectory planning results and the simulated driving results includes: The success rate is determined based on the motion trajectory planning results, and the collision rate is determined based on the simulated driving results; The safe driving rate of the object detection model is determined based on the successful planning rate and the collision rate.

[0060] In some embodiments of the evaluation device, the success rate is based on the formula Calculated; The collision rate is based on the formula Calculated; The safe driving rate is based on the formula Calculated; in, To increase the success rate of planning, The total number of driving scenarios in the set of driving scenarios. This represents the number of driving scenarios in the set of driving scenarios that can successfully generate motion trajectories. For collision rate, To successfully generate the number of driving scenarios corresponding to the motion trajectories in the motion trajectory where collisions occur. For safe driving rate.

[0061] In some embodiments, the evaluation apparatus may further include a value calculation module and a scene collection module. Wherein: The value calculation module can be used to determine the information value of the corresponding driving scenario based on the smoothness of the motion trajectory, the distance between the target vehicle and the obstacle, and the cognitive uncertainty of the object detection model. The scene collection module can be used to collect driving scenes in the driving scene set whose information value reaches a value threshold, so as to serve as one of the training data for subsequent improvement of the object detection model.

[0062] In some embodiments of the evaluation device, determining the information value of the corresponding driving scenario based on the smoothness of the motion trajectory, the distance between the target vehicle and the obstacle, and the cognitive uncertainty of the object detection model includes: According to the formula Calculate the information value of driving scenarios; in, For the first The information value of each driving scenario For the object detection model in the first Cognitive uncertainty in various driving scenarios for constant coefficients, For the first The smoothness of the motion trajectory corresponding to each driving scenario. for constant coefficients, For the first The distance between the target vehicle and obstacles in each driving scenario. for The constant coefficients.

[0063] In some embodiments of the evaluation device, the driving scenarios in the set of driving scenarios include: Visual images and / or point cloud data of multiple road types under different weather conditions; Visual images and / or point cloud data of multiple road types under different adversarial attacks; and, Visual images and / or point cloud data of multiple road types under different weather conditions and different adversarial attacks.

[0064] In some embodiments of the evaluation device, the driving scenarios in the set of driving scenarios are obtained in advance through the following methods: Select multiple typical driving scenarios containing different road types from the typical driving scenario dataset; Using weather conditions and / or counter-attack conditions as variables, each typical driving scenario is extended into different driving scenarios based on synthesis or simulation.

[0065] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0066] Although the process described above includes multiple operations that occur in a specific order, it should be clearly understood that these processes may include more or fewer operations that can be executed sequentially or in parallel (e.g., using parallel processors or a multithreaded environment).

[0067] Embodiments of this specification also provide a computer device. For example... Figure 5 As shown, in some embodiments of this specification, the computer device 502 may include one or more processors 504, such as one or more central processing units (CPUs) or graphics processing units (GPUs), each of which may implement one or more hardware threads. The computer device 502 may also include any memory 506 for storing any kind of information such as code, settings, data, etc. In one specific embodiment, a computer program is stored on the memory 506 and can run on the processor 504. When the processor 504 executes the program, it can perform instructions for the assessment method of the impact of the object detection model on autonomous driving safety described in any of the above embodiments. Non-limitingly, for example, the memory 506 may include any type of RAM, any type of ROM, flash memory, hard disk, optical disk, etc. More generally, any memory can use any technology to store information. Further, any memory can provide volatile or non-volatile retention of information. Further, any memory can represent a fixed or removable component of the computer device 502. In one case, when the processor 504 executes associated instructions stored in any memory or combination of memories, the computer device 502 can perform any operation of the associated instructions. Computer device 502 also includes one or more drive mechanisms 508 for interacting with any memory, such as hard disk drive mechanism, optical disk drive mechanism, etc.

[0068] Computer device 502 may also include an input / output interface 510 (I / O) for receiving various inputs (via input device 512) and providing various outputs (via output device 514). A specific output mechanism may include a presentation device 516 and an associated graphical user interface 518 (GUI). In other embodiments, the input / output interface 510 (I / O), input device 512, and output device 514 may be omitted, and the device may function solely as a computer device within a network. Computer device 502 may also include one or more network interfaces 520 for exchanging data with other devices via one or more communication links 522. One or more communication buses 524 couple the components described above together.

[0069] Communication link 522 can be implemented in any way, such as via a local area network, a wide area network (e.g., the Internet), a point-to-point connection, or any combination thereof. Communication link 522 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc., governed by any protocol or combination of protocols.

[0070] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to some embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processor to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processor, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0071] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processor to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0072] These computer program instructions may also be loaded onto a computer or other programmable data processor, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1The steps of the function specified in one or more boxes.

[0073] In a typical configuration, a computer device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0074] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0075] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by computer equipment. As defined in this specification, computer-readable media does not include transient media, such as modulated data signals and carrier waves.

[0076] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, the embodiments of this specification can take the form of computer program products implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0077] The embodiments described in this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. The embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processors connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0078] It should also be understood that, in the embodiments of this specification, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0079] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0080] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the embodiments of this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0081] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for evaluating the impact of object detection models on autonomous driving safety, characterized in that, include: The driving scenarios in the set of driving scenarios are provided to the object detection model to obtain obstacle recognition results; The obstacle identification results are then classified into dynamic and static obstacles to obtain the obstacle classification results; Determine the target vehicle's destination area, cost function, and driving constraints under the stated driving scenario; Based on the obstacle classification results, the target area, the cost function, and the driving constraints, the motion trajectory of the target vehicle is planned. The target vehicle is controlled to perform simulated driving according to the motion trajectory, and the simulated driving results are obtained. The success rate is determined based on the motion trajectory planning results, and the collision rate is determined based on the simulated driving results. The safe driving of the object detection model is determined based on the success rate and the collision rate.

2. The method for assessing the impact of object detection models on autonomous driving safety as described in claim 1, characterized in that, The success rate of the planning is based on the formula. Calculated; The collision rate is based on the formula Calculated; The safe driving rate is based on the formula Calculated; in, To increase the success rate of planning, The total number of driving scenarios in the set of driving scenarios. This represents the number of driving scenarios in the set of driving scenarios that can successfully generate motion trajectories. For collision rate, To successfully generate the number of driving scenarios corresponding to the motion trajectories in the motion trajectory where collisions occur. For safe driving rate.

3. The method for assessing the impact of object detection models on autonomous driving safety as described in claim 1, characterized in that, Also includes: The information value of the corresponding driving scenario is determined based on the smoothness of the motion trajectory, the distance between the target vehicle and the obstacle, and the cognitive uncertainty of the object detection model. The driving scenarios whose information value reaches a value threshold are collected from the set of driving scenarios and used as one of the training data for subsequent improvement of the object detection model.

4. The method for assessing the impact of object detection models on autonomous driving safety as described in claim 3, characterized in that, The information value of the corresponding driving scenario is determined based on the smoothness of the motion trajectory, the distance between the target vehicle and the obstacle, and the cognitive uncertainty of the object detection model, including: According to the formula Calculate the information value of driving scenarios; in, For the first The information value of each driving scenario For the object detection model in the first Cognitive uncertainty in various driving scenarios for constant coefficients, For the first The smoothness of the motion trajectory corresponding to each driving scenario. for constant coefficients, For the first The distance between the target vehicle and obstacles in each driving scenario. for The constant coefficients.

5. The method for assessing the impact of object detection models on autonomous driving safety as described in claim 1, characterized in that, The driving scenarios in the set of driving scenarios include: Visual images and / or point cloud data of multiple road types under different weather conditions; Visual images and / or point cloud data of multiple road types under different adversarial attacks; and, Visual images and / or point cloud data of multiple road types under different weather conditions and different adversarial attacks.

6. The method for assessing the impact of object detection models on autonomous driving safety as described in claim 5, characterized in that, The driving scenarios in the set of driving scenarios are obtained in advance through the following methods: Select multiple typical driving scenarios containing different road types from the typical driving scenario dataset; Using weather conditions and / or counter-attack conditions as variables, each typical driving scenario is extended into different driving scenarios based on synthesis or simulation.

7. An evaluation device for assessing the impact of object detection models on autonomous driving safety, characterized in that, include: A module is provided to provide driving scenarios from a set of driving scenarios to an object detection model in order to obtain obstacle recognition results; The classification module is used to classify the obstacle recognition results into dynamic and static obstacles to obtain obstacle classification results; The determination module is used to determine the target vehicle's destination area, cost function, and driving constraints in the driving scenario. The planning module is used to plan the motion trajectory of the target vehicle based on the obstacle classification results, the target area, the cost function, and the driving constraints. The control module is used to control the target vehicle to perform simulated driving according to the motion trajectory and obtain the simulated driving results; An evaluation module is used to determine the success rate based on the motion trajectory planning results and the collision rate based on the simulated driving results; and to determine the safe driving of the object detection model based on the success rate and the collision rate.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, When the computer program is run by the processor, it executes the instructions of the method according to any one of claims 1-6.

9. A computer storage medium having a computer program stored thereon, characterized in that, When the computer program is run by the processor of the computer device, it executes the instructions of the method according to any one of claims 1-6.

Images