Testing method, device, electronic device and storage medium for autonomous driving system

Abstract

The present application relates to the field of vehicle intelligent driving test technology, and in particular to a test method, device, electronic device and storage medium for an autonomous driving system, wherein the method comprises: obtaining the true value environment information of the simulation environment, the perception information of the target sensor and the input and output information of the autonomous driving system, obtaining the safety data, comfort data and violation data of the autonomous driving system under multiple driving conditions in the simulation environment, and calculating the test score of each data by weight according to the time ratio, and obtaining the test result of the autonomous driving system according to the test score of each data. The test method of the autonomous driving system of the embodiment of the present application automatically collects data through a data acquisition program, generates quantitative evaluation results through evaluation programs in multiple dimensions, and automatically analyzes the problems to reduce manual intervention, thereby saving costs and speeding up the test.

Description

Technical Field

[0001] The present application relates to the field of vehicle intelligent driving test technology, and in particular to a test method, device, electronic device and storage medium for an automatic driving system. Background Art

[0002] With the rapid development of the automotive industry and the increasing maturity of autonomous driving technology, the driving systems faced by autonomous driving vehicles will become more and more complex. Therefore, testing of vehicle autonomous driving systems is particularly important.

[0003] In the related technologies, the test and evaluation methods of the algorithms of the autonomous driving system rely on closed-field testing and road testing of real vehicles on the one hand, and on virtual simulation testing technology to accelerate the testing on the other hand, closing the tested algorithm into the simulation software for testing, which is mainly based on evaluation indicators of functional requirements and performance evaluation indicators based on scenarios. The evaluation indicators for functional requirements need to rely on case design. First, it is determined whether the algorithm can cope with these working conditions, and then a weighted score is given according to the difficulty of the working conditions. Finally, the performance evaluation based on the scenario will evaluate the robustness and safety of the algorithm based on the data of vehicle movement, and finally the performance score of the algorithm in the real environment is obtained.

[0004] However, the existing test and evaluation methods cannot fully meet the needs of intelligent driving tests. On the one hand, for the perception part, we hope to test whether the target list, lane lines, traffic signs and other information perceived by the algorithm based on the camera image and radar point cloud are consistent with the true value information of the simulation software, and score them based on consistency; on the other hand, the L2 algorithm cannot cope with all long-tail conditions. Therefore, after taking over at different times under dangerous conditions, the subsequent scoring should be based on different treatments and appropriate results, so as to prompt more efficient problem discovery and targeted optimization of the algorithm. Therefore, the intelligent driving car algorithm requires a more detailed and targeted method to conduct a more comprehensive and scientific simulation test. Summary of the invention

[0005] The present application provides a testing method, device, vehicle and storage medium for an autonomous driving system to solve the problems that the testing and evaluation methods in the related technologies cannot meet the autonomous driving testing requirements, the acquired test information does not match the simulation true value information, and lacks specificity.

[0006] The first aspect of the present application provides a method for testing an autonomous driving system, comprising the following steps:

[0007] Acquire true environmental information of a simulation environment, perception information of a target sensor, and input and output information of an autonomous driving system; construct the simulation environment according to the true environmental information, the perception information, and the input and output information, and acquire safety data, comfort data, and violation data of the autonomous driving system under a plurality of driving conditions under the simulation environment; and calculate a test score for each data by weighting the safety data, comfort data, and violation data of the autonomous driving system under the plurality of driving conditions according to a time ratio, and obtain a test result of the autonomous driving system according to the test score of each data.

[0008] According to the above technical means, based on the various test data information obtained in the simulation environment and the related data obtained, weighted calculations are performed and then a final analysis is performed to obtain the test results, which improves the matching degree between the test information and the true value information in the simulation environment and makes the test more comprehensive and scientific.

[0009] Furthermore, in one embodiment of the present application, the obtaining of safety data, comfort data, and violation data of the automatic driving system under multiple driving conditions includes: obtaining the deviation information of the vehicle from the center line of the lane, the lateral and longitudinal speeds of the vehicle, the vehicle heading angle, the vehicle steering wheel angle, the vehicle steering wheel angle rate, and the vehicle acceleration under the multiple driving conditions; obtaining the safety data, comfort data, and violation data of the automatic driving system according to the deviation information of the vehicle from the center line of the lane, the lateral and longitudinal speeds of the vehicle, the vehicle heading angle, the vehicle steering wheel angle, the vehicle steering wheel angle rate, and the vehicle acceleration.

[0010] According to the above technical means, the vehicle information obtained by testing under various working conditions is improved, which improves the accuracy of the vehicle's automatic driving system and makes the test more targeted.

[0011] Furthermore, in one embodiment of the present application, the driving condition includes at least two of a free cruising condition, a curve driving condition, an active lane changing condition, and a driver takeover condition.

[0012] Based on the above-mentioned technical means, by testing the vehicle's automatic driving system under different working conditions, the test results can be more comprehensive and detailed.

[0013] Furthermore, in one embodiment of the present application, the true value environmental information includes the current position, current speed, current geometric information, current classification information of dynamic and static traffic participants around the vehicle, as well as at least one of lane line information of the static road network and road topology information of the static road network.

[0014] According to the above technical means, by testing the true value environment information in multiple aspects, the test results are made more targeted, and problems can be discovered in time and optimized accordingly.

[0015] Furthermore, in one embodiment of the present application, the target sensor includes at least one of a camera, a millimeter-wave radar, an ultrasonic radar, a lidar, a high-precision map, and a combined navigation.

[0016] According to the above technical means, test information is collected through multiple sensors, which ensures test accuracy and improves test efficiency.

[0017] A second aspect of the present application provides a testing device for an autonomous driving system, comprising: an acquisition module for acquiring true environmental information of a simulation environment, perception information of a target sensor, and input and output information of an autonomous driving system; a construction module for constructing the simulation environment according to the true environmental information, the perception information, and the input and output information, and obtaining safety data, comfort data, and violation data of the autonomous driving system under multiple driving conditions under the simulation environment; and a calculation module for calculating a test score of each data according to a time ratio for the safety data, comfort data, and violation data of the autonomous driving system under the multiple driving conditions, and obtaining a test result of the autonomous driving system according to the test score of each data.

[0018] Furthermore, in one embodiment of the present application, the construction module includes: a first acquisition unit, used to obtain the deviation information of the vehicle from the lane centerline, the vehicle's lateral and longitudinal speeds, the vehicle's heading angle, the vehicle's steering wheel angle, the vehicle's steering wheel angle rate and the vehicle's acceleration under the multiple driving conditions; a second acquisition unit, used to obtain the safety data, comfort data and violation data of the automatic driving system based on the deviation information of the vehicle from the lane centerline, the vehicle's lateral and longitudinal speeds, the vehicle's heading angle, the vehicle's steering wheel angle, the vehicle's steering wheel angle rate and the vehicle's acceleration.

[0019] Furthermore, in one embodiment of the present application, the true value environmental information includes the current position, current speed, current geometric information, current classification information of dynamic and static traffic participants around the vehicle, as well as at least one of lane line information of the static road network and road topology information of the static road network.

[0020] Furthermore, in one embodiment of the present application, the true value environmental information includes the current position, current speed, current geometric information, current classification information of dynamic and static traffic participants around the vehicle, as well as at least one of lane line information of the static road network and road topology information of the static road network.

[0021] Furthermore, in one embodiment of the present application, the target sensor includes at least one of a camera, a millimeter-wave radar, an ultrasonic radar, a lidar, a high-precision map, and a combined navigation.

[0022] The third aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the testing method of the autonomous driving system as described in the above embodiment.

[0023] The fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the testing method of the autonomous driving system as described in the above embodiments.

[0024] The embodiment of the present application obtains the true value environment information of the simulation environment, the perception information of the target sensor and the input and output information of the automatic driving system, and obtains the safety data, comfort data and violation data of the automatic driving system under multiple driving conditions in the simulation environment, and calculates the test score of each data by weight according to the time ratio, and obtains the test result of the automatic driving system according to the test score of each data, collects data automatically through the data acquisition program, and generates quantitative evaluation results through the evaluation program of multiple dimensions, and analyzes the problems automatically to reduce manual intervention, thereby saving costs and speeding up the test. Therefore, it solves the problems that the test evaluation method in the related technology cannot meet the needs of automatic driving testing, the obtained test information does not match the simulation true value information, and lacks pertinence.

[0025] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0027] Figure 1 A flowchart of a method for testing an autonomous driving system according to an embodiment of the present application;

[0028] Figure 2 An example diagram of a simulation execution process flow according to an embodiment of the present application;

[0029] Figure 3 This is a flowchart of automated test execution according to one embodiment of the present application;

[0030] Figure 4 This is an example diagram of comfort and safety evaluation indicators when driving on a curve according to an embodiment of the present application;

[0031] Figure 5This is an example diagram of a testing device for an autonomous driving system according to an embodiment of the present application;

[0032] Figure 6 Schematic diagram of the structure of an electronic device according to an embodiment of the present application.

[0033] Figure numerals: 10 - testing device for autonomous driving system; 100 - acquisition module, 200 - construction module, 300 - calculation module. DETAILED DESCRIPTION

[0034] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0035] The following describes the test method, device, vehicle and storage medium of the automatic driving system of the embodiment of the present application with reference to the accompanying drawings. In view of the problem that the test evaluation method mentioned in the above background technology cannot meet the test requirements of automatic driving, the test information obtained does not match the simulation true value information, and lacks pertinence, the present application provides a test method for an automatic driving system, in which the safety data, comfort data and violation data of the automatic driving system under multiple driving conditions are obtained in the simulation environment by obtaining the true value environment information of the simulation environment, the perception information of the target sensor and the input and output information of the automatic driving system, and the test score of each data is calculated by weight according to the time ratio, and the test result of the automatic driving system is obtained according to the test score of each data, and the data is automatically collected through the data acquisition program, and the quantitative evaluation results are generated through the evaluation program of multiple dimensions, and the problem is automatically analyzed to reduce manual intervention, thereby saving costs and speeding up the test speed. Therefore, the test evaluation method in the related technology cannot meet the test requirements of automatic driving, the test information obtained does not match the simulation true value information, and lacks pertinence.

[0036] Specifically, Figure 1 A flowchart of a testing method for an autonomous driving system provided in an embodiment of the present application.

[0037] like Figure 1 As shown, the testing method of the automatic driving system includes the following steps:

[0038] In step S101, the true value environment information of the simulation environment, the perception information of the target sensor and the input and output information of the automatic driving system are obtained.

[0039] Optionally, the true value environment information of the embodiment of the present application may include the current position, current speed, current geometric information, current classification information of dynamic and static traffic participants around the vehicle, as well as lane line information of the static road network and road topology information of the static road network; the target sensor may include at least one of a camera, a millimeter wave radar, an ultrasonic radar, a lidar, a high-precision map and a combined navigation.

[0040] Specifically, in the simulation test process, the embodiment of the present application uses communication technology to obtain the true environmental information of the simulation environment, the perception information of the target sensor, and the input and output information of the autonomous driving system, so as to conduct a real-time evaluation of the functional requirements. That is, the execution script generated by the case design is used to determine whether the step event has been entered during the simulation operation and whether the verification criteria of the event are met.

[0041] Specifically, in the test of L3 (conditional autonomous driving) and lower level autonomous driving systems in the embodiments of the present application, it is necessary to test whether its functional logic is normal. If the logical function is normal, it is further determined whether the scenes within the executable range are correctly handled according to the required design. When the algorithm meets certain test conditions (including but not limited to external scene information and algorithm internal state), an event is triggered, such as driver operation, fault injection, control of traffic participant movement, etc., and the output of the observation algorithm meets the requirements. It should be noted that the above-mentioned observation algorithm output results include but are not limited to whether the perception is accurate, whether the decision is reasonable, and whether the control is executed normally.

[0042] Furthermore, if Figure 2 As shown in the figure, for the simulation execution process, it is an extension of the above steps in the time domain.

[0043] Specifically, when the simulation execution process of the embodiment of the present application is completed, the execution process of the case will have two states: passed or failed. For the passed case, when the algorithm meets certain test conditions and triggers an event, it is necessary to further determine whether the triggering event is the last step of the test. If so, the case is passed, otherwise it is necessary to re-determine whether the event meets the conditions for entering the test step; for the failed case, the external scene information and the internal state of the algorithm will be checked according to the entry conditions of the step to distinguish the reasons for the case blocking or failure. That is to say, the external scene information and the internal state of the algorithm can be checked by reading the entry conditions of the step to further determine whether the case is blocked because the scene construction does not meet the design or because the algorithm does not normally identify the scene information, thereby failing to make the expected decision correctly and failing to perform correct control, resulting in case failure.

[0044] On the one hand, for the above-mentioned case blocking caused by the scenario construction not conforming to the design, testers are needed to further analyze whether the test scenario construction is reasonable and whether the case design is wrong.

[0045] On the other hand, for the above-mentioned cases where the expected decision was not made correctly and thus the case failed, developers are also required to conduct failure error analysis in combination with the perceived consistency scores and data.

[0046] After the simulation test is completed, the time domain will be divided according to the working conditions identified during the simulation process, and scores will be given according to the pre-set evaluation modules for different working conditions.

[0047] In step S102, a simulation environment is constructed based on true environmental information, perception information, and input and output information, and in the simulation environment, safety data, comfort data, and violation data of the automatic driving system under multiple driving conditions are obtained.

[0048] Optionally, in one embodiment of the present application, safety data, comfort data and violation data of the automatic driving system under multiple driving conditions are obtained, including: obtaining the deviation information of the vehicle from the center line of the lane, the vehicle's lateral and longitudinal speeds, the vehicle's heading angle, the vehicle's steering wheel angle, the vehicle's steering wheel angle rate and the vehicle's acceleration under multiple driving conditions; obtaining the safety data, comfort data and violation data of the automatic driving system according to the deviation information of the vehicle from the center line of the lane, the vehicle's lateral and longitudinal speeds, the vehicle's heading angle, the vehicle's steering wheel angle, the vehicle's steering wheel angle rate and the vehicle's acceleration.

[0049] Optionally, the driving conditions used in the embodiment of the present application may include at least two of free cruising conditions, cornering conditions, active lane change conditions, and driver takeover conditions. It should be noted that the above conditions are only classified according to the logic of the algorithm itself, and the purpose is to call different evaluation criteria for different conditions to achieve reasonable scoring; for ordinary conditions, traffic regulations will be evaluated, while for driver takeover conditions, only safety will be evaluated.

[0050] Furthermore, the embodiments of the present application can obtain the deviation information of the vehicle from the lane centerline, the vehicle's lateral and longitudinal speeds, the vehicle's heading angle, the vehicle's steering wheel angle, the vehicle's steering wheel angle rate and the vehicle's acceleration under different working conditions in a simulation test environment, and obtain the safety data, comfort data and violation data of the autonomous driving system based on the deviation information of the vehicle from the lane centerline, the vehicle's lateral and longitudinal speeds, the vehicle's heading angle, the vehicle's steering wheel angle, the vehicle's steering wheel angle rate and the vehicle's acceleration.

[0051] In step S103, the safety data, comfort data and violation data of the automatic driving system under multiple driving conditions are weighted according to the time ratio to calculate the test score of each data, and the test result of the automatic driving system is obtained according to the test score of each data.

[0052] Specifically, after the embodiment of the present application obtains the safety data, comfort data, and violation data of the autonomous driving system, it is also necessary to calculate the test score of each data in a time-weighted manner for the algorithm's comfort data, safety data, and violation data.

[0053] For example, Figure 3 As shown, if the driving condition of the embodiment of the present application is in a free cruising condition, the requirements for trajectory centering and jerkiness (acceleration change rate) are high in the simulation test environment, and the threshold of this part of the data will be specially narrowed to obtain the experience score; then the deduction will be made for the pressure on the dotted line and the deviation from the center line of the lane. Preferably, the scoring formula is 1-∫vtOtdt / ∫vtdt, where v is the speed, t is the time, and O is the distance the vehicle deviates from the center point of the lane at the current time point.

[0054] Alternatively, if Figure 3 , Figure 4 As shown, if the driving condition of the embodiment of the present application is in a curved driving condition, higher requirements will be placed on speed and acceleration. For different curve radii, the speed and lateral acceleration of the safety index and comfort index will have different scoring criteria.

[0055] Alternatively, if Figure 3 As shown, if the driving condition of the embodiment of the present application is in an active lane change driving condition, it is not appropriate to use the trajectory centering score at this time. Therefore, it is necessary to score the safety performance such as the vehicle's lateral and longitudinal speed, vehicle heading angle, vehicle steering wheel angle, vehicle steering wheel angle rate, vehicle acceleration evaluation and vehicle spacing experienced by the driver. Among them, the specific threshold selection can be calibrated according to the performance of the actual vehicle model, and is not specifically limited here.

[0056] Alternatively, if Figure 3 As shown, if the driving condition of the embodiment of the present application is in the driver takeover condition, only the safety score is applicable at this time, and the comfort score and the traffic rule violation score are not applicable. It should be noted that all cases with takeover conditions in the embodiment of the present application need to clearly indicate the triggering of takeover in order to optimize the algorithm of the autonomous driving system test in a targeted manner.

[0057] Optionally, in addition to the different scoring criteria corresponding to the above working conditions, the information of the perception output can be compared with the environmental truth information to score the consistency, which can serve as an additional score as a reference for the optimization direction.

[0058] In summary, case studies are designed based on the evaluation indicators of functional requirements, and weighted scores are given according to the different working conditions that the algorithm copes with and the difficulty of the working conditions. Finally, an evaluation is conducted based on the performance evaluation of the scenario, vehicle motion data, and the robustness and safety of the algorithm to ultimately arrive at a performance score for the algorithm in a real environment.

[0059] According to the test method of the automatic driving system proposed in the embodiment of the present application, by obtaining the true value environment information of the simulation environment, the perception information of the target sensor and the input and output information of the automatic driving system, the safety data, comfort data and violation data of the automatic driving system under multiple driving conditions are obtained in the simulation environment, and the test score of each data is calculated by weight according to the time ratio, and the test result of the automatic driving system is obtained according to the test score of each data. The data is automatically collected through the data acquisition program, and the quantitative evaluation results are generated through the evaluation program of multiple dimensions. The problem is automatically analyzed to reduce manual intervention, thereby saving costs and speeding up the test. Therefore, the problems that the test evaluation method in the related technology cannot meet the test requirements of automatic driving, the obtained test information does not match the simulation true value information, and lacks pertinence are solved.

[0060] Next, a test device for an autonomous driving system according to an embodiment of the present application will be described with reference to the accompanying drawings.

[0061] Figure 5 It is a block diagram of a testing device for an automatic driving system according to an embodiment of the present application.

[0062] like Figure 5 As shown, the testing device 10 of the autonomous driving system includes: an acquisition module 100 , a construction module 200 and a calculation module 300 .

[0063] The acquisition module 100 is used to obtain the true value environment information of the simulation environment, the perception information of the target sensor and the input and output information of the automatic driving system;

[0064] A construction module 200 is used to construct a simulation environment according to true value environment information, perception information and input and output information, and obtain safety data, comfort data and violation data of the automatic driving system under multiple driving conditions in the simulation environment; and

[0065] The calculation module 300 is used to calculate the test score of each data by weighting the safety data, comfort data and violation data of the automatic driving system under multiple driving conditions according to the time ratio, and obtain the test result of the automatic driving system according to the test score of each data.

[0066] Optionally, in one embodiment of the present application, the construction module 200 includes: a first acquisition unit and a second acquisition unit.

[0067] The first acquisition unit is used to acquire the deviation information between the vehicle and the lane centerline, the vehicle's lateral and longitudinal speeds, the vehicle's heading angle, the vehicle's steering wheel angle, the vehicle's steering wheel angle rate, and the vehicle's acceleration under multiple driving conditions;

[0068] The second acquisition unit is used to obtain the safety data, comfort data and violation data of the automatic driving system according to the deviation information between the vehicle and the lane centerline, the vehicle's lateral and longitudinal speeds, the vehicle's heading angle, the vehicle's steering wheel angle, the vehicle's steering wheel angle rate and the vehicle's acceleration.

[0069] Optionally, in one embodiment of the present application, the true value environmental information includes the current position, current speed, current geometric information, current classification information of dynamic and static traffic participants around the vehicle, and at least one of lane line information of the static road network and road topology information of the static road network.

[0070] Optionally, in one embodiment of the present application, the true value environmental information includes the current position, current speed, current geometric information, current classification information of dynamic and static traffic participants around the vehicle, and at least one of lane line information of the static road network and road topology information of the static road network.

[0071] Optionally, in one embodiment of the present application, the target sensor includes at least one of a camera, a millimeter wave radar, an ultrasonic radar, a lidar, a high-precision map and a combined navigation.

[0072] According to the test device of the automatic driving system proposed in the embodiment of the present application, by obtaining the true value environment information of the simulation environment, the perception information of the target sensor and the input and output information of the automatic driving system, the safety data, comfort data and violation data of the automatic driving system under multiple driving conditions are obtained in the simulation environment, and the test score of each data is calculated weighted according to the time ratio, and the test result of the automatic driving system is obtained according to the test score of each data. The data is automatically collected through the data acquisition program, and the quantitative evaluation results are generated through the evaluation program of multiple dimensions, and the problems are automatically analyzed to reduce manual intervention, thereby saving costs and speeding up the test. Therefore, the problems that the test evaluation method in the related technology cannot meet the needs of automatic driving testing, the obtained test information does not match the simulation true value information, and lacks pertinence are solved.

[0073] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0074] A memory 601 , a processor 602 , and a computer program stored in the memory 601 and executable on the processor 602 .

[0075] When the processor 602 executes the program, the testing method of the automatic driving system provided in the above embodiment is implemented.

[0076] Optionally, the electronic device further includes:

[0077] The communication interface 603 is used for communication between the memory 601 and the processor 602 .

[0078] The memory 601 is used to store computer programs that can be executed on the processor 602 .

[0079] The memory 601 may include a high-speed RAM (Random Access Memory) memory, and may also include a non-volatile memory, such as at least one disk memory.

[0080] If the memory 601, the processor 602 and the communication interface 603 are implemented independently, the communication interface 603, the memory 601 and the processor 602 can be connected to each other through a bus and communicate with each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0081] Optionally, in a specific implementation, if the memory 601, the processor 602 and the communication interface 603 are integrated on a chip, the memory 601, the processor 602 and the communication interface 603 can communicate with each other through an internal interface.

[0082] The processor 602 may be a CPU (Central Processing Unit), or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application.

[0083] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned testing method for the autonomous driving system.

[0084] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0085] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0086] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0087] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array, a field programmable gate array, etc.

[0088] A person skilled in the art may understand that all or part of the steps in the above-mentioned embodiment method may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0089] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for testing an automatic driving system, characterized in that: The following steps are involved: Obtain the true value environment information of the simulation environment, the perception information of the target sensor, and the input and output information of the autonomous driving system; constructing the simulation environment according to the true value environment information, the perception information and the input and output information, and obtaining safety data, comfort data and violation data of the automatic driving system under multiple driving conditions in the simulation environment; as well as Calculating a test score of each data by weighting the safety data, comfort data, and violation data of the automatic driving system under the multiple driving conditions according to a time ratio, and obtaining a test result of the automatic driving system according to the test score of each data; The obtaining of safety data, comfort data, and violation data of the automatic driving system under multiple driving conditions includes: Obtaining information on the deviation of the vehicle from the center line of the lane, the lateral and longitudinal speeds of the vehicle, the heading angle of the vehicle, the steering wheel angle of the vehicle, the steering wheel angle rate of the vehicle, and the acceleration of the vehicle under the plurality of driving conditions; Obtaining safety data, comfort data and violation data of the automatic driving system according to the deviation information between the vehicle and the lane centerline, the lateral and longitudinal speeds of the vehicle, the heading angle of the vehicle, the steering wheel angle of the vehicle, the steering wheel angle rate of the vehicle and the acceleration of the vehicle; The driving conditions include at least two of a free cruising condition, a curve driving condition, an active lane changing condition, and a driver taking over condition; The information of the perception output is compared with the true value information of the environment to achieve consistency and score, which serves as an additional score. The test result of the autonomous driving system is obtained according to the test score of each data and the additional score. After the simulation test is completed, the working conditions identified during the simulation process are divided in the time domain, and scored according to the pre-set evaluation modules for different working conditions.

2. The method according to claim 1, characterized in that The true value environment information includes the current position, current speed, current geometric information, current classification information of dynamic and static traffic participants around the vehicle, and at least one of the lane line information of the static road network and the road topology information of the static road network.

3. The method according to claim 1, characterized in that The target sensor includes at least one of a camera, a millimeter wave radar, an ultrasonic radar, a laser radar, a high-precision map and a combined navigation.

4. A test device for an automatic driving system, characterized in that: include: An acquisition module is used to obtain the true value environment information of the simulation environment, the perception information of the target sensor, and the input and output information of the autonomous driving system; a construction module, configured to construct the simulation environment according to the true value environment information, the perception information, and the input and output information, and obtain safety data, comfort data, and violation data of the automatic driving system under multiple driving conditions in the simulation environment; as well as a calculation module, configured to weight the safety data, comfort data, and violation data of the automatic driving system under the plurality of driving conditions according to a time ratio and calculate a test score for each data, and obtain a test result of the automatic driving system according to the test score for each data; The building blocks include: A first acquisition unit is used to acquire the deviation information between the vehicle and the lane centerline, the vehicle lateral and longitudinal speeds, the vehicle heading angle, the vehicle steering wheel angle, the vehicle steering wheel angle rate and the vehicle acceleration under the plurality of driving conditions; a second acquisition unit, configured to obtain safety data, comfort data, and violation data of the automatic driving system according to the deviation information between the vehicle and the lane centerline, the lateral and longitudinal speeds of the vehicle, the heading angle of the vehicle, the steering wheel angle of the vehicle, the steering wheel angle rate of the vehicle, and the acceleration of the vehicle; The driving conditions include at least two of a free cruising condition, a curve driving condition, an active lane changing condition, and a driver taking over condition; The information of the perception output is compared with the true value information of the environment to achieve consistency and score, which serves as an additional score. The test result of the autonomous driving system is obtained according to the test score of each data and the additional score. After the simulation test is completed, the working conditions identified during the simulation process are divided in the time domain, and scored according to the pre-set evaluation modules for different working conditions.

5. The device according to claim 4, characterized in that The true value environment information includes the current position, current speed, current geometric information, current classification information of dynamic and static traffic participants around the vehicle, and at least one of the lane line information of the static road network and the road topology information of the static road network.

6. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a testing method for an autonomous driving system as described in any one of claims 1 to 3.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement a testing method for an autonomous driving system as described in any one of claims 1 to 3.

Images