Recharge test method and device for automatic driving algorithm, medium and equipment

Abstract

The invention provides a recharge test method and device for an automatic driving algorithm, a medium and equipment, and relates to the technical field of automatic driving. The method comprises the following steps: firstly, in response to a test instruction for a target automatic driving algorithm, obtaining a drive test data set corresponding to at least one scene test task; for any scene test task, a simulation environment is constructed based on the corresponding drive test data set, so that the target automatic driving algorithm executes the scene test task in the simulation environment, a simulation recharge result is obtained, and an evaluation report for the target automatic driving algorithm is generated; wherein the simulation environment is updated in real time based on the position information of the vehicle in the execution process of the scene test task. Therefore, the automatic driving algorithm is tested through the road test data, and the surrounding simulation environment is updated in real time based on the change condition of the position of the vehicle in the simulation process, so that the real driving environment is simulated, the recharge test effect of the automatic driving algorithm is improved, and the performance of the automatic driving algorithm is effectively evaluated.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a method, device, medium and equipment for recharging an autonomous driving algorithm. Background Art

[0002] With the continuous development of automobile intelligence, the automatic driving system has gradually become a standard feature of automobiles. For example, IACC (Integrated Adaptive Cruise Control) was launched in 2014. As a relatively mature L2-level assisted driving function, it has been widely used internationally. It can control the longitudinal speed of the vehicle by controlling the brakes and accelerator of the vehicle, and can control the steering of the vehicle by controlling the steering wheel of the vehicle, thereby freeing the driver's hands and feet, reducing the driver's driving burden, and improving driving safety.

[0003] However, with the increase in demand, autonomous driving is equipped with more and more sensors, the systems are becoming more complex, and the algorithms are becoming more sophisticated. Therefore, how to fully and effectively test and verify the autonomous driving system has become a problem that needs to be solved urgently. Summary of the invention

[0004] The present application provides a re-injection test method, device, medium and equipment for an autonomous driving algorithm, which can improve the re-injection test effect of the autonomous driving algorithm and realize effective evaluation of the performance of the autonomous driving algorithm.

[0005] A first aspect of an embodiment of the present application provides a recharge test method for an autonomous driving algorithm, the method comprising:

[0006] In response to a test instruction for a target autonomous driving algorithm, obtaining a road test data set corresponding to at least one scenario test task;

[0007] For any scenario test task, a simulation environment is constructed based on the road test data set corresponding to the scenario test task, so that the target autonomous driving algorithm performs the scenario test task in the simulation environment to obtain a simulation feedback result; wherein the simulation environment is updated in real time based on the vehicle position information during the execution of the scenario test task;

[0008] Based on the simulation feedback results corresponding to at least one scenario test task, an evaluation report for the target autonomous driving algorithm is generated.

[0009] Optionally, before obtaining a drive test data set corresponding to at least one scenario test task in response to a test instruction for a target autonomous driving algorithm, the method further includes:

[0010] Obtaining road test data for the target vehicle;

[0011] The drive test data are subjected to scene classification to obtain drive test data sets corresponding to different scene test tasks.

[0012] Optionally, the method further comprises:

[0013] Based on the current first posture information of the vehicle, determining the second posture information obtained by the vehicle executing the target automatic driving algorithm;

[0014] Based on the first posture information and the second posture information, determining a coordinate origin offset and a heading angle offset of a host vehicle coordinate system;

[0015] Based on the coordinate origin offset and the heading angle offset, coordinate transformation is performed on the environmental information in the drive test data set corresponding to the scenario test task to update the simulation environment.

[0016] Optionally, the method further comprises:

[0017] When the simulated driving behavior of the vehicle is different from the actual driving behavior in the road test data set, the driving state of the obstacle vehicle in the simulation environment is adjusted based on the simulated driving behavior.

[0018] Optionally, the simulation recharge result includes a decision result and an execution result;

[0019] Based on the simulation feedback results corresponding to at least one scenario test task, an evaluation report for the target autonomous driving algorithm is generated, including:

[0020] For any of the scenario test tasks, determine a first evaluation report for the decision result and a second evaluation report for the execution result;

[0021] Based on the first evaluation report and the second evaluation report corresponding to each of the scenario test tasks, an evaluation report for the target autonomous driving algorithm is generated.

[0022] Optionally, determining a first evaluation report for the decision result and a second evaluation report for the execution result includes:

[0023] For any of the scenario test tasks, determine a first evaluation indicator and a second evaluation indicator corresponding to the scenario test task;

[0024] Evaluate the decision result based on the first evaluation index to obtain the first evaluation report;

[0025] The execution result is evaluated based on the second evaluation index to obtain the second evaluation report.

[0026] Optionally, the method further comprises:

[0027] For any of the scenario test tasks, if any of the first evaluation indicators or the second evaluation indicators are not met, a problem defect report for the target autonomous driving algorithm is generated.

[0028] Based on the same inventive concept, a second aspect of an embodiment of the present application provides a recharge test device for an autonomous driving algorithm, the device comprising:

[0029] A drive test data acquisition module, configured to acquire a drive test data set corresponding to at least one scenario test task in response to a test instruction for a target autonomous driving algorithm;

[0030] A drive test data re-injection module is used to construct a simulation environment for any scenario test task based on the drive test data set corresponding to the scenario test task, so that the target autonomous driving algorithm performs the scenario test task in the simulation environment to obtain a simulation re-injection result; wherein the simulation environment is updated in real time based on the vehicle position information during the execution of the scenario test task;

[0031] The re-injection result evaluation module is used to generate an evaluation report for the target autonomous driving algorithm based on the simulation re-injection result corresponding to at least one scenario test task.

[0032] Based on the same inventive concept, the third aspect of an embodiment of the present application provides a storage medium, which stores machine executable instructions. When the machine executable instructions are executed by a processor, the re-injection test method of the autonomous driving algorithm proposed in the first aspect of the present application is implemented.

[0033] A fourth aspect of an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executed, implements a recharge test method for an autonomous driving algorithm as proposed in the first aspect of the present application.

[0034] Compared with the prior art, this application has the following advantages:

[0035] The embodiment of the present application provides a method for recharging an autonomous driving algorithm, which is configured to obtain a road test data set corresponding to at least one scenario test task in response to a test instruction for the target autonomous driving algorithm; for any scenario test task, a simulation environment is constructed based on the road test data set corresponding to the scenario test task, so that the target autonomous driving algorithm performs the scenario test task in the simulation environment and obtains a simulation recharging result; wherein the simulation environment is updated in real time based on the position information of the vehicle during the execution of the scenario test task; based on the simulation recharging result corresponding to at least one scenario test task, an evaluation report for the target autonomous driving algorithm is generated. Thus, the autonomous driving algorithm is tested through the road test data, and the road test data on the actual vehicle is reused for the second time, which is conducive to improving the recharging test effect of the autonomous driving algorithm. At the same time, based on the changes in the position of the vehicle during the simulation recharging process, the surrounding simulation environment is updated in real time to simulate the real driving environment, promote the effective testing of the autonomous driving algorithm, and make the performance evaluation results of the autonomous driving algorithm more real and reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0037] Figure 1 is a flow chart of a recharge test method of an autonomous driving algorithm in one embodiment of the present application;

[0038] Figure 2 is a flowchart of updating a simulation environment in an embodiment of the present application;

[0039] Figure 3 It is a schematic diagram of constructing the real-time coordinate system of the vehicle in one embodiment of the present application;

[0040] Figure 4 It is an application schematic diagram of a recharge test method of an autonomous driving algorithm in one embodiment of the present application;

[0041] Figure 5 It is a functional module schematic diagram of a recharge test device for an autonomous driving algorithm in one embodiment of the present application;

[0042] Figure 6 It is a structural schematic diagram of an electronic device in one embodiment of the present application. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0044] Please refer to Figure 1 , Figure 1 1 is a flow chart of a method for recharging an autonomous driving algorithm according to an embodiment of the present application. Figure 1 As shown, the method comprises the following steps:

[0045] S101: In response to a test instruction for a target autonomous driving algorithm, obtain a road test data set corresponding to at least one scenario test task.

[0046] In this implementation, the target autonomous driving algorithm refers to the new version of the autonomous driving algorithm to be submitted. Every time a new version of the autonomous driving algorithm is to be submitted, a road test data re-injection task will be triggered to test the target autonomous driving algorithm. Among them, the road test data re-injection task refers to the process of injecting the collected actual road test data (including sensor data, vehicle status information, driving behavior data, etc.) into the vehicle controller or domain controller for simulation and testing, so as to evaluate and verify the performance and algorithm of the autonomous driving system by reproducing the actual driving scenes and situations.

[0047] Specifically, the road test data feedback task includes a number of different scenario test tasks to detect whether the target autonomous driving system will respond accordingly in different scenarios. For example, when it is detected that the vehicle ahead is moving slowly and the system determines that overtaking is a safe and reasonable behavior, it is verified whether the target autonomous driving algorithm will issue a control instruction to change lanes and overtake to control the vehicle to change lanes and overtake. Or, when it is detected that the road ahead is congested or the exit ramp is about to be reached, it is verified whether the target autonomous driving algorithm will issue a control instruction to change lanes to optimize the driving route. Or, when approaching traffic lights, stop lines or intersections where deceleration or parking is required, it is verified whether the target autonomous driving algorithm will issue a control instruction to brake and control the vehicle to brake to comply with traffic regulations.

[0048] Furthermore, in this embodiment, the road test data refers to the relevant data automatically collected when the autonomous driving system is manually taken over by the driver. The road test data set corresponding to any scenario test task includes the driver's takeover time point, the environmental information around the vehicle, the driver's vehicle control information, etc. For example, data of 20 seconds before and after the manual takeover can be collected to form a road test data set to perform a re-injection test on the new version of the target autonomous driving algorithm.

[0049] In addition, road test data can also include relevant data manually triggered by the driver when he determines that the response of the autonomous driving system is unreasonable, or some classic and representative scene data marked by testers, or scene data of interest screened out by algorithms, etc.

[0050] S102: For any scenario test task, a simulation environment is constructed based on the road test data set corresponding to the scenario test task, so that the target autonomous driving algorithm performs the scenario test task in the simulation environment to obtain a simulation feedback result; wherein the simulation environment is updated in real time based on the vehicle position information during the execution of the scenario test task.

[0051] In this embodiment, when the target autonomous driving algorithm is subjected to scenario testing, the road test data is fed back into the vehicle simulation model to simulate the actual driving scenario and conduct simulation testing. The target autonomous driving algorithm is then run in the simulation environment to perform the scenario test task, and to detect whether the target autonomous driving system will respond accordingly in the scenario. Finally, the simulation feedback result of the target autonomous driving algorithm in the scenario is obtained based on the actual simulation data.

[0052] Furthermore, considering that when the control instructions of the vehicle change during simulation re-injection, the driving trajectory of the vehicle will also change, and the environmental information about obstacles and other environmental information in the collected road test data is based on the vehicle coordinate system, if the environmental information such as obstacles is not processed, the data received by the target autonomous driving algorithm during the actual re-injection process will be very different from the actual scene. Therefore, in this embodiment, in order to ensure the effectiveness of the simulation re-injection results, during the simulation process, a real-time vehicle coordinate system is constructed based on the vehicle position information, and the environmental information in the road test data set is converted to the corresponding coordinates to update the simulation environment in real time, thereby more realistically simulating the perception and decision-making process of the vehicle in actual road driving, which is conducive to improving the efficiency and authenticity of the re-injection test for the target autonomous driving algorithm.

[0053] S103: Generate an evaluation report for the target autonomous driving algorithm based on the simulation feedback results corresponding to at least one scenario test task.

[0054] In this embodiment, after obtaining the simulation re-injection results corresponding to each scenario test task through road test data simulation re-injection, the simulation re-injection results corresponding to each scenario test task are analyzed and evaluated using corresponding evaluation indicators to obtain the performance results of the target autonomous driving algorithm in multiple test scenarios. Then, the performance results of the target autonomous driving algorithm in multiple test scenarios are summarized, supplemented with explanations, etc., to generate an evaluation report for the target autonomous driving algorithm. That is, an evaluation report of a target autonomous driving algorithm is composed of the evaluation results of one or more scenario test tasks.

[0055] For example, in the final evaluation report, charts (such as curves, bar graphs, tables, etc.) can be used to assist in explaining the data and results of the simulation test, making the report more intuitive and easy to understand. At the same time, improvement suggestions and future research directions can be proposed for problems encountered during the simulation process, thereby generating a comprehensive, objective and accurate road test simulation feedback evaluation report, providing strong support for the development and improvement of autonomous driving algorithms.

[0056] This embodiment tests the autonomous driving algorithm through road test data and re-uses the road test data on the actual vehicle, which is conducive to ensuring the efficiency of the actual vehicle test verification. At the same time, a real-time vehicle coordinate system is constructed based on the vehicle position information, and the environmental information in the road test data set is converted accordingly to update the simulation environment in real time, which is conducive to improving the re-injection test effect and authenticity of the target autonomous driving algorithm, making the obtained simulation re-injection results more real and reliable, and by analyzing and evaluating the simulation re-injection results and forming an evaluation report, it can provide strong support for the optimization and improvement of the autonomous driving algorithm.

[0057] Optionally, before obtaining a drive test data set corresponding to at least one scenario test task in response to a test instruction for the target autonomous driving algorithm, the method further includes:

[0058] S201: Acquire road test data for a target vehicle.

[0059] In a feasible implementation, the road test data can be the data transmitted back by the target vehicle when the autonomous driving system is manually taken over. For example, during the autonomous driving process, the driver may take over the vehicle due to objective dangerous environment and other factors. When manual takeover occurs, the data feedback mechanism is automatically triggered, and the autonomous driving data is transmitted back to the manufacturer's server in the cloud through the network. In this way, road test data under various different situations can be collected and used for simulation re-injection testing of the next new version of the target autonomous driving algorithm.

[0060] Among them, the data from actual road tests that need to be collected and transmitted back include the time when the driver takes over, the vehicle's driving trajectory before and after the driver takes over, speed, acceleration, sensor data (such as cameras, radars, lidars, etc.) and environmental information (such as road conditions, traffic flow, weather conditions, etc.).

[0061] S202: Classify the drive test data by scenarios to obtain drive test data sets corresponding to different scenario test tasks.

[0062] In this implementation, the returned road test data is first pre-processed in the cloud, such as cleaning, denoising, calibration and synchronization, to ensure the accuracy and consistency of the data. Then, by writing and running data mining operators in the cloud, the returned road test data is classified into scenarios, including dangerous lane changes, untimely lane changes, abnormal suppression, abnormal rollback, ramp cruising, tunnel cruising, congested cruising, main road into ramp, ramp into main road, etc. At the same time, according to the classification results, scenario labels are configured for each road test data in the cloud data management system to establish road test data sets corresponding to different scenario test tasks. Among them, the road test data set corresponding to any scenario test task is composed of a set of road test data carrying the scenario label corresponding to the scenario.

[0063] It should be noted that a set of road test data can match multiple scene labels at the same time. For example, a set of road test data about braking may appear in a scenario where an obstacle (such as a vehicle, pedestrian, stationary object, etc.) is detected ahead and there is a risk of collision. In this case, the corresponding scene label is "avoid collision". At the same time, it may also appear in the process of following a vehicle, and dynamically adjust the distance to the vehicle in front according to factors such as the current vehicle speed, road conditions, and the speed of the vehicle in front to maintain a safe distance. In this case, the corresponding scene label is "safe following". In addition, it may also appear in scenarios where you need to slow down or stop, such as approaching traffic lights, stop lines, or intersections. In this case, the corresponding scene label is "obey traffic".

[0064] Therefore, during the feedback test of the new version of the target autonomous driving algorithm, different scenario tests are performed by pulling the road test data sets corresponding to the corresponding scenarios from the cloud according to different scenarios of concern, so as to detect and verify whether the target autonomous driving algorithm has made reasonable vehicle control instructions that meet the driver's requirements in different driver-takeover scenarios, and then conduct a comprehensive evaluation of the performance of the target autonomous driving algorithm under different scenario tests.

[0065] Optionally, the process of real-time updating of the simulation environment based on the vehicle position information in the road test data set in the above step S102 mainly includes two aspects: one is to convert the surrounding environment information by establishing the real-time coordinate system of the vehicle to maintain the consistency between the simulation environment and the actual road test environment; the other is to set up an intelligent response mechanism for surrounding obstacle vehicles to make basic responses to some common emergencies and enhance the flexibility of the simulation environment.

[0066] Among them, the first aspect of simulation environment update mainly includes:

[0067] S301: Based on the current first posture information of the vehicle, determine the second posture information obtained by the vehicle executing the target automatic driving algorithm.

[0068] In this embodiment, the first posture information is used to characterize the current posture state of the vehicle, and the second posture information is used to characterize the posture state of the vehicle after the vehicle executes the control instruction made by the target automatic driving algorithm according to the first posture information of the vehicle, that is, the posture state of the vehicle at the next moment. The first posture information and the second posture information of the vehicle include at least the position and heading angle of the vehicle.

[0069] Specifically, when determining the second posture information obtained by the vehicle executing the target automatic driving algorithm, if the information of the high-precision map is valid, the vehicle's position coordinates and heading can be directly determined based on the longitude and latitude coordinates and heading information of the vehicle provided in real time by the high-precision map. In the case where high-precision map information is not available, the response error of the vehicle control system can be ignored, and the vehicle's position and heading angle can be calculated based on the steering wheel angle, yaw angle, vehicle speed and vehicle acceleration received in real time by the vehicle.

[0070] S302: Based on the first pose information and the second pose information, determine the coordinate origin offset and the heading angle offset of the vehicle coordinate system.

[0071] In this embodiment, based on the coordinate origin position and heading angle of the vehicle at the current time t0, and the coordinate origin position and heading angle at the next time t1, the coordinate origin offset (including the horizontal and vertical coordinate offsets) and heading angle offset of the vehicle can be determined.

[0072] S303: Based on the coordinate origin offset and the heading angle offset, coordinate transformation is performed on the environmental information in the drive test data set corresponding to the scenario test task to update the simulation environment.

[0073] In this embodiment, based on the coordinate origin (including horizontal and vertical coordinates) and the change in heading angle from time t0 to time t1, the real-time coordinate system of the vehicle can be established, such as Figure 2 At the same time, in order to synchronize the simulation environment with the position change of the vehicle, it is necessary to transform the coordinates of the environmental information in the road test data set corresponding to the scenario test task, and integrate the environmental information into the real-time coordinate system of the vehicle, that is, map the environmental coordinate information under the original real-vehicle test conditions to the real-time coordinate system of the vehicle during the back injection, so as to update the simulation environment and ensure the consistency between the simulation environment and the real road test environment. Among them, the environmental information includes the coordinates and headings of the surrounding vehicles, the coordinates of obstacles, the lane lines and map points, etc.

[0074] For specific implementation, please refer to Figure 3 , Figure 3 FIG. 1 is a flowchart of updating the simulation environment in an embodiment of the present application. Figure 3As shown, firstly, the historical trajectory is calculated based on the yaw angle, steering wheel angle, vehicle speed and heading angle of the vehicle at time t0, and the origin coordinates (horizontal and vertical coordinates) and heading angle of the vehicle at time t1 are obtained. Then, by comparing with the origin coordinates at time t0, the coordinate origin offset (including the horizontal and vertical coordinate offsets) and heading angle offset of the vehicle from time t0 to time t1 can be obtained. After that, based on the coordinate origin offset and heading angle offset of the vehicle, the coordinates and headings of the surrounding vehicles, obstacle coordinates, lane lines, map points and other environmental information in the road test data are converted to obtain the converted road test data. Among them, the converted road test data characterizes the distribution of surrounding objects in the real-time coordinate system of the vehicle. Finally, the target autonomous driving algorithm is tested based on the converted road test data to realize the real-time update of the simulation environment and ensure the effectiveness and accuracy of the simulation re-injection test.

[0075] In this embodiment, when the simulation is fed back into the data, the autonomous driving algorithm calculates the actual position of the vehicle in real time based on the control instructions of the vehicle, constructs the real-time coordinate system of the vehicle, and converts the environmental information to the corresponding coordinate system of the vehicle, thereby unifying the data from different sensors (such as cameras, radars, lidars, etc.) into the same coordinate system, which helps to eliminate the data deviation caused by the difference in the coordinate systems of different sensors, ensure the accuracy and consistency of the data, and realize the effective fusion and utilization of multi-sensor information, further improving the performance of the autonomous driving system. At the same time, the fusion of environmental information into the real-time coordinate system of the vehicle can more conveniently process and analyze data, such as calculating the distance to obstacles, predicting trajectories, etc., which helps to simplify the calculation process, improve the efficiency of simulation testing, and can more realistically simulate the perception and decision-making process of the vehicle in actual road driving, which is conducive to improving the effectiveness of the target autonomous driving algorithm test.

[0076] The second aspect of updating the simulation environment mainly includes: when the simulated driving behavior of the vehicle is different from the actual driving behavior in the road test data set, adjusting the driving state of the obstacle vehicle in the simulation environment based on the simulated driving behavior.

[0077] In this embodiment, the simulated driving behavior refers to the driving behavior exhibited by the vehicle under the decision control made by the target autonomous driving algorithm, which may be the same as or different from the actual driving behavior in the collected road test data set. When the simulated driving behavior of the vehicle is the same as the actual driving behavior in the road test data set, that is, the vehicle in the simulated re-injection test shows the same results as after the driver took over during the road test, it means that the target autonomous driving algorithm has made the correct decision and has been correctly executed. When the simulated driving behavior of the vehicle is different from the actual driving behavior in the road test data set, it means that the target autonomous driving algorithm has made an error in decision or an execution error has occurred.

[0078] When the simulated driving behavior of the vehicle is different from the actual driving behavior in the road test data set, in order to more realistically simulate the actual driving situation, this embodiment introduces an intelligent response model for the surrounding obstacle vehicles. When the behavior of the vehicle changes, the driving state of the obstacle vehicles in the simulation environment is adjusted based on the simulated driving behavior, so that the surrounding obstacle vehicles can make basic responses according to the behavior of the vehicle, rather than mechanically driving according to the original vehicle driving trajectory in the collected road test data, so as to avoid collision between the surrounding obstacle vehicles and the vehicle.

[0079] For example, the actual driving behavior in the original road test data set is lane changing, but during the simulation, the vehicle performs a simulated driving behavior of braking under the control of the target autonomous driving algorithm. At this time, if the rear obstacle vehicle is still controlled according to the surrounding environment data in the original road test data set, it is very likely to cause the rear vehicle to collide with the vehicle. But in fact, if it is a real driving environment, when the front vehicle brakes, the rear vehicle will usually follow the brakes to avoid a collision. Therefore, this embodiment introduces an intelligent response model for the surrounding obstacle vehicles. When the vehicle brakes, the rear vehicle will make a basic response and follow the brakes to simulate the real driving environment and improve the accuracy of the target autonomous driving algorithm simulation re-injection test.

[0080] Optionally, the simulation feedback result includes a decision result and an execution result; based on the simulation feedback result corresponding to at least one scenario test task, an evaluation report for the target autonomous driving algorithm is generated, including:

[0081] S401: for any scenario test task, determining a first evaluation report for a decision result and a second evaluation report for an execution result;

[0082] In this embodiment, the simulation feedback results include decision results and execution results. The decision result refers to the driving decision made by the autonomous driving system based on the perception information after analysis by the target autonomous driving algorithm. The execution result refers to the specific action performed by the autonomous driving system to control the vehicle according to the decision result, that is, the final performance result of the vehicle.

[0083] When evaluating the simulation feedback results corresponding to any scenario test task, evaluation can be performed based on the decision results and execution results respectively to understand the performance of the autonomous driving system in terms of decision-making and control in detail. Specifically, the process mainly includes:

[0084] This step specifically includes: for any scenario test task, determining the first evaluation indicator and the second evaluation indicator corresponding to the scenario test task; evaluating the decision result based on the first evaluation indicator to obtain a first evaluation report; and evaluating the execution result based on the second evaluation indicator to obtain a second evaluation report.

[0085] In this embodiment, the first evaluation index is used to evaluate the decision result to obtain a first evaluation report. The first evaluation index includes multiple indicators such as decision accuracy, decision rationality and decision speed. Among them, decision accuracy requires analysis of whether the decision result is the same as the decision made by the driver taking over in the same scenario, and whether the decision result complies with traffic regulations, such as whether traffic rules, speed limits, traffic signals, etc. are followed. If the decision result is the same as the decision made by the driver taking over in the same scenario and the decision complies with traffic regulations, it is considered that the decision accuracy index of the target autonomous driving algorithm is up to standard. Decision rationality requires analysis of whether the decision result is logical and whether all relevant perceptual information is taken into account. If the decision result is logical and all relevant perceptual information is taken into account, it means that the decision result is reliable, and the decision rationality index is considered to be up to standard. Decision speed requires analysis of whether the time from the input of perceptual information to the output of decision results is in line with expectations, ensuring that the decision process is rapid and does not affect driving safety.

[0086] The second evaluation index is used to evaluate the execution results to obtain a second evaluation report. The second evaluation index includes multiple indicators such as execution accuracy, execution efficiency and safety. Among them, execution accuracy requires analyzing the error between the execution result and the decision-making instruction, such as direction control, speed control, etc., and evaluating the stability of the vehicle during the execution process, whether there are jitters, offsets and other phenomena. Execution efficiency requires analyzing whether the speed at which the vehicle executes the action meets expectations and whether the action can be completed within the specified time. Safety requires analyzing whether the vehicle is driving dangerously when executing the action, whether there is sudden acceleration, sudden deceleration, sudden steering, heavy braking, and whether it is driving on the solid line.

[0087] S402: Generate an evaluation report for the target autonomous driving algorithm based on the first evaluation report and the second evaluation report corresponding to each scenario test task.

[0088] In this implementation, for any scenario test task, after obtaining the first evaluation report on its decision result and the second evaluation report on its execution result, the evaluation report on the scenario test task can be obtained by merging or comprehensively analyzing the first evaluation report and the second evaluation report. Afterwards, by summarizing the evaluation reports of each scenario test task, an evaluation report on the target autonomous driving algorithm can be obtained to demonstrate the performance of the target autonomous driving algorithm in various scenarios or in multiple aspects, providing strong support for the subsequent development and improvement of the autonomous driving system.

[0089] Furthermore, while generating an evaluation report for the target autonomous driving algorithm, the above method also includes: for any scenario test task, if any first evaluation indicator or second evaluation indicator is not met, generating a problem defect report for the target autonomous driving algorithm.

[0090] In this implementation, for any scenario test task, when evaluating its decision results through multiple first evaluation indicators (or evaluating its execution results through multiple second evaluation indicators), if there is an indicator evaluation failure, a problem defect report for the evaluation is automatically generated. Then, the problem defect reports under all scenario test tasks are summarized to obtain a problem defect report for the target autonomous driving algorithm. The problem defect report can be generated separately or merged into the entire evaluation report. After that, the defect report is fed back to the corresponding developer through the defect handling system as a reference indicator for evaluating the pros and cons of subsequent different software versions.

[0091] Please refer to Figure 4 , Figure 4 Schematic diagram of the application of the recharge test method of the automatic driving algorithm in one embodiment of the present application. Figure 4 As shown in the figure, the simulation test process in actual application mainly includes:

[0092] S501: Collecting data records triggered by the driver taking over during the driving of the commodity vehicle and transmitting them back to the remote system;

[0093] S502: Filtering data of interest through cloud mining operators;

[0094] S503: According to the classification results screened by the cloud algorithm, labels are assigned to the data in the data management system;

[0095] S504: Establish a scene data set according to the tags configured by the cloud data;

[0096] S505: Establishing the task of recharging the simulation data of the vehicle, including establishing the real-time coordinate system of the vehicle and the intelligent response model of the surrounding obstacle vehicles;

[0097] S506: deploying the simulation re-injection task to the continuous integration pipeline, so that the cloud data re-injection task can be automatically triggered each time a new version of the software is submitted;

[0098] S507: running the actual vehicle test data in the cloud in the simulation recharging task, and outputting the recharging result data;

[0099] S508: using the evaluation operator to evaluate the re-injection result data outputted in S507, and determining whether the actual vehicle performance in the re-injection result meets the requirements;

[0100] S509: associating the evaluation data that failed in S508 with the defect management system to realize automatic generation of defect tickets from the problem data;

[0101] S510: Count the evaluation results of multiple scenarios and output an evaluation report.

[0102] Among them, the specific execution of the simulation recharging task and the result evaluation process please refer to the above embodiment, which will not be repeated here.

[0103] Please refer to Figure 5 Based on the same inventive concept, the second aspect of the embodiment of the present application provides a recharge test device for an autonomous driving algorithm. The recharge test device 500 for the autonomous driving algorithm includes:

[0104] A drive test data acquisition module 501 is used to obtain a drive test data set corresponding to at least one scenario test task in response to a test instruction for a target autonomous driving algorithm;

[0105] The drive test data re-injection module 502 is used to construct a simulation environment for any scenario test task based on the drive test data set corresponding to the scenario test task, so that the target autonomous driving algorithm performs the scenario test task in the simulation environment to obtain a simulation re-injection result; wherein the simulation environment is updated in real time based on the vehicle position information during the execution of the scenario test task;

[0106] The re-injection result evaluation module 503 is used to generate an evaluation report for the target autonomous driving algorithm based on the simulation re-injection result corresponding to at least one scenario test task.

[0107] Optionally, the device further comprises:

[0108] The drive test data feedback module is used to obtain the drive test data sent back by the target vehicle when the autonomous driving system is manually taken over;

[0109] The drive test data analysis module is used to classify the drive test data into different scenarios to obtain drive test data sets corresponding to different scenario test tasks.

[0110] Optionally, the device further comprises:

[0111] A position derivation module, used for determining a second position information obtained by executing the target automatic driving algorithm by the vehicle based on the current first position information of the vehicle;

[0112] An offset determination module, used to determine a coordinate origin offset and a heading angle offset of a vehicle coordinate system based on the first posture information and the second posture information;

[0113] The simulation update module is used to perform coordinate transformation on the environmental information in the drive test data set corresponding to the scenario test task based on the coordinate origin offset and the heading angle offset, so as to update the simulation environment.

[0114] Optionally, the device further comprises:

[0115] The obstacle vehicle response module is used to adjust the driving state of the obstacle vehicle in the simulation environment based on the simulated driving behavior when the simulated driving behavior of the vehicle is different from the actual driving behavior in the road test data set.

[0116] Optionally, the simulation recharge result includes a decision result and an execution result; the recharge result evaluation module 503 includes:

[0117] An evaluation submodule, for determining, for any of the scenario test tasks, a first evaluation report for the decision result and a second evaluation report for the execution result;

[0118] The report generation submodule is used to generate an evaluation report for the target autonomous driving algorithm based on the first evaluation report and the second evaluation report corresponding to each of the scenario test tasks.

[0119] Optionally, the evaluation submodule is specifically used for:

[0120] For any of the scenario test tasks, determine a first evaluation indicator and a second evaluation indicator corresponding to the scenario test task;

[0121] Evaluate the decision result based on the first evaluation index to obtain the first evaluation report;

[0122] The execution result is evaluated based on the second evaluation index to obtain the second evaluation report.

[0123] Optionally, the device further comprises:

[0124] A defect report generation module is used to generate a problem defect report for the target autonomous driving algorithm for any of the scenario test tasks when any of the first evaluation indicators or the second evaluation indicators are not met.

[0125] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0126] On the third aspect, based on the same inventive concept, an embodiment of the present application provides a storage medium, which stores machine executable instructions. When the machine executable instructions are executed by a processor, the re-injection test method of the autonomous driving algorithm proposed in the first aspect of the present application is implemented.

[0127] It should be noted that the specific implementation of the storage medium of the embodiment of the present application refers to the specific implementation of the re-injection test method of the autonomous driving algorithm proposed in the first aspect of the embodiment of the present application, and will not be repeated here.

[0128] In the fourth aspect, based on the same inventive concept, refer to Figure 6 An embodiment of the present application provides an electronic device 600, including a processor 601 and a memory 602; the memory 602 stores machine executable instructions that can be executed by the processor 601, and the processor 601 is used to execute the machine executable instructions to implement the re-injection test method of the autonomous driving algorithm proposed in the first aspect of the present application.

[0129] It should be noted that the specific implementation of the electronic device 600 in the embodiment of the present application refers to the specific implementation of the recharge test method of the autonomous driving algorithm proposed in the first aspect of the embodiment of the present application, and will not be repeated here.

[0130] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0131] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, devices, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0132] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0133] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0134] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0135] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0136] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0137] The above is a detailed introduction to the reinjection test method, device, medium and equipment of an autonomous driving algorithm provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A recharge test method for an autonomous driving algorithm, characterized in that: The method comprises: In response to a test instruction for a target autonomous driving algorithm, obtaining a road test data set corresponding to at least one scenario test task; For any scenario test task, a simulation environment is constructed based on the road test data set corresponding to the scenario test task, so that the target autonomous driving algorithm performs the scenario test task in the simulation environment to obtain a simulation feedback result; wherein the simulation environment is updated in real time based on the vehicle position information during the execution of the scenario test task; Based on the simulation feedback results corresponding to at least one scenario test task, an evaluation report for the target autonomous driving algorithm is generated.

2. The method according to claim 1, characterized in that Before obtaining a road test data set corresponding to at least one scenario test task in response to a test instruction for a target autonomous driving algorithm, the method further includes: Obtaining road test data for the target vehicle; The drive test data are subjected to scene classification to obtain drive test data sets corresponding to different scene test tasks.

3. The method according to claim 1, characterized in that The method further comprises: Based on the current first posture information of the vehicle, determining the second posture information obtained by the vehicle executing the target automatic driving algorithm; Based on the first posture information and the second posture information, determining a coordinate origin offset and a heading angle offset of a host vehicle coordinate system; Based on the coordinate origin offset and the heading angle offset, coordinate transformation is performed on the environmental information in the drive test data set corresponding to the scenario test task to update the simulation environment.

4. The method according to claim 1, characterized in that: The method further comprises: When the simulated driving behavior of the vehicle is different from the actual driving behavior in the road test data set, the driving state of the obstacle vehicle in the simulation environment is adjusted based on the simulated driving behavior.

5. The method according to claim 1, characterized in that The simulation recharge result includes a decision result and an execution result; Based on the simulation feedback results corresponding to at least one scenario test task, an evaluation report for the target autonomous driving algorithm is generated, including: For any of the scenario test tasks, determine a first evaluation report for the decision result and a second evaluation report for the execution result; Based on the first evaluation report and the second evaluation report corresponding to each of the scenario test tasks, an evaluation report for the target autonomous driving algorithm is generated.

6. The method according to claim 5, characterized in that Determining a first evaluation report for the decision result and a second evaluation report for the execution result includes: For any of the scenario test tasks, determine a first evaluation indicator and a second evaluation indicator corresponding to the scenario test task; Evaluate the decision result based on the first evaluation index to obtain the first evaluation report; The execution result is evaluated based on the second evaluation index to obtain the second evaluation report.

7. The method according to claim 6, characterized in that The method further comprises: For any of the scenario test tasks, if any of the first evaluation indicators or the second evaluation indicators are not met, a problem defect report for the target autonomous driving algorithm is generated.

8. A recharge test device for an autonomous driving algorithm, characterized in that: The device comprises: A drive test data acquisition module, configured to acquire a drive test data set corresponding to at least one scenario test task in response to a test instruction for a target autonomous driving algorithm; A drive test data re-injection module is used to construct a simulation environment for any scenario test task based on the drive test data set corresponding to the scenario test task, so that the target autonomous driving algorithm performs the scenario test task in the simulation environment to obtain a simulation re-injection result; wherein the simulation environment is updated in real time based on the vehicle position information during the execution of the scenario test task; The re-injection result evaluation module is used to generate an evaluation report for the target autonomous driving algorithm based on the simulation re-injection result corresponding to at least one scenario test task.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the recharge test method of the autonomous driving algorithm as described in any one of claims 1 to 7 is implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the re-injection test method of the autonomous driving algorithm as described in any one of claims 1 to 7.

Images