Automatic driving noa third party test and evaluation system

Abstract

The application relates to the technical field of automatic driving evaluation systems, and discloses an automatic driving NOA third-party test and evaluation system, which comprises the following steps: SS01, a scene construction module, which constructs an automatic driving scene, wherein the automatic driving scene is provided with a specified weather scene, a road scene and a driving scene; SS02, a feature recognition and labeling module, after the step S01, staff automatically label feature points and key points of the specified weather scene, the road scene and the driving scene; and after labeling is completed, a feature data set in the automatic driving scene is generated. When the automatic driving NOA third-party test and evaluation are carried out, the safety, the flow degree, the convenience and the accuracy of the automatic driving can be comprehensively evaluated according to four dimensions, i.e. driving result information of the automatic driving, driving collection and driving perception information in the automatic driving process, humanized evaluation information of a safety officer and driving simulation stability information of the automatic driving.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving evaluation systems, and more particularly to an automatic driving NOA third-party testing and evaluation system. BACKGROUND

[0002] Driving behavior is an important factor that both human drivers and automatic driving vehicles must consider, and most traffic accidents are caused by improper driving operation. Unhealthy driving habits are the main cause of traffic accidents. Automatic driving vehicles help improve the intelligence of the "human-vehicle-environment" system, thereby improving traffic efficiency and traffic safety. With the development of automatic driving technology, automatic driving vehicles are gradually popularized in closed scenarios, freeing human drivers from driving operations. However, there is still no reliable and effective evaluation method for objectively and accurately evaluating the driving behavior of automatic driving vehicles, so that automatic driving vehicles can absorb the correct driving methods of human drivers and abandon unhealthy driving habits of human drivers, and so that the driving behavior of automatic driving vehicles is as safe and effective as that of excellent drivers.

[0003] In the prior art, a patent document with publication number CN114692713A discloses a driving behavior evaluation method and device for automatic driving vehicles. The method includes the following steps: obtaining driving behavior information of an automatic driving vehicle on a to-be-evaluated route, the driving behavior information including: instantaneous vehicle speed, acceleration / deceleration, steering wheel angle, total vehicle operation time, total vehicle operation mileage, vehicle cumulative energy consumption, vehicle surrounding obstacle information, and vehicle surrounding road data information. The vehicle surrounding road data information includes slopes, curves, tunnels, accident-prone areas, road section speed limits, and traffic light information. The above driving behavior evaluation method can realize quantitative evaluation of the driving behavior of automatic driving vehicles and qualitative evaluation of the driving behavior of automatic driving vehicles. The above evaluation method mainly takes the automatic driving result as the evaluation value, and does not realize multi-dimensional testing and evaluation of automatic driving NOA during evaluation. Therefore, the present application provides an automatic driving NOA third-party testing and evaluation system to solve the technical problems raised in the background. SUMMARY

[0004] In order to overcome the shortcomings of the prior art, the present application provides an automatic driving NOA third-party testing and evaluation system. The present application can comprehensively evaluate the safety, flow, convenience, and accuracy of automatic driving based on four dimensions, including driving result information of automatic driving, driving collection and driving perception information during automatic driving, humanized evaluation information of safety officers, and driving simulation stability information of automatic driving, during the testing and evaluation of automatic driving NOA third-party. This effectively improves the evaluation accuracy of the automatic driving system and the credibility of the evaluation results.

[0005] To achieve the above object, the present application provides the following technical solutions: An automatic driving NOA third-party testing and evaluation system, comprising the following steps:

[0006] SS01, a scene construction module, which constructs an automatic driving scene, and the automatic driving scene is provided with a specified weather scene, a road scene and a driving scene;

[0007] SS02, a feature recognition and labeling module, after the step of SS01, a staff member performs automatic labeling of feature points and key points on the specified weather scene, the road scene and the driving scene, and after the labeling is completed, a feature data set in the automatic driving scene is generated;

[0008] SS03, a data acquisition module, which is constructed in an automatic driving vehicle, and the data acquisition module acquires real-time driving data when the automatic driving vehicle is running and generates a driving data set;

[0009] SS04, a sample reference construction module, a plurality of drivers are preset, the drivers should drive in the automatic driving scene for the first time, after the preset, a driving plan is established, the plurality of drivers drive in the automatic driving scene in turn according to the driving plan, and each driver generates a driving sample data set after driving simulation;

[0010] SS05, an average data construction module, which establishes an average data set according to the plurality of driving sample data sets in the step of SS04, and the average data set includes an average speed, an average steering wheel angle and a center trajectory parameter of the vehicle running at a specified driving time;

[0011] SS06, an evaluation sample construction module, the automatic driving vehicle drives in the automatic driving mode according to the driving plan in the step of SS03, after the automatic driving simulation is completed, the data acquisition module obtains a to-be-evaluated data set, and the automatic driving vehicle is provided with a safety officer during the automatic driving behavior;

[0012] SS07, a driving evaluation module, which performs driving weight ∑1 according to the to-be-evaluated data set obtained in the step of SS04, and the safety officer obtains a personalized evaluation weight value ∑2 according to the evaluation of the safety officer after the automatic driving behavior is completed;

[0013] SS08, a perception evaluation module, which judges driving feature information collected by the data acquisition module during driving through data analysis when the automatic driving vehicle drives in the automatic driving behavior, and calculates a perception information evaluation weight value ∑3 and a travel feature acquisition accuracy weight ∑4 according to the obtained driving feature information;

[0014] SS09, driving simulation weight evaluation module, after the automatic driving vehicle performs automatic driving behavior, the data collection module sets the collected driving data set into the computer driving simulation system for simulation reproduction, the computer driving simulation system obtains a simulation driving data set after each driving simulation, each simulation driving data set is obtained, the matching degree K of the simulation driving data set and the driving sample data set is automatically calculated, after the plurality of K values are calculated, the matching degree average KD, the matching range value KR and the automatic driving system stability value weight ∑5 are calculated;

[0015] SS10, comprehensive evaluation module, the comprehensive evaluation module calculates the evaluation value ∑ of the automatic driving NOA according to ∑1, ∑2, ∑3, ∑4 and ∑5;

[0016] ∑ = ∑1 * ∑2 * ∑3 * ∑4 * ∑5

[0017] After ∑ is calculated, it is automatically recorded and backed up.

[0018] As a preferred technical solution of the present application, in the SS01 step, the road scene includes the specified road type, the specified road marking, the specified signal lamp, the sign, the street lamp, the intersection type, the lane number, the lane line, the lane type, the speed limit type, the road type, the road angle, the weather scene includes the weather, the illumination, the road visibility and the environmental temperature and humidity, and the driving scene includes the traffic flow scene and the vehicle interference simulation scene.

[0019] As a preferred technical solution of the present application, in the SS02 step, the feature data set includes the total number of feature points and key points, and the type of feature points and key points.

[0020] As a preferred technical solution of the present application, in the SS03 step, the driving data set includes the driving time, the vehicle instantaneous speed, the steering wheel angle, the vehicle GPS position information, the vehicle edge driving track, the vehicle radar information, the automatic driving background running code and the driving video information.

[0021] As a preferred technical solution of the present application, in the SS07 step, when the driving weight ∑1 is calculated, the data of the to-be-evaluated data set in the SS06 step is taken as the calculation data, the average data set in the SS05 step is taken as the parameter data, the data difference X under each type of driving project is calculated, after the calculation of each type of data difference X is completed, the average difference value MD and the range value MR under the automatic driving project are calculated.

[0022]

[0023] When the range value MR is calculated, the maximum data difference value Xmax and the minimum data difference value Xmin after the size comparison of each project data difference is obtained through the numerical comparison algorithm.

[0024] MR=X max -X min

[0025]

[0026] After the calculation of ∑1 is completed, automatic recording and backup are performed.

[0027] As a preferred technical solution of the present application, in the SS08, the following is performed:

[0028]

[0029] In the calculation of ∑4, the matching degree of the driving feature collected by the data collection module and the feature data set is calculated first, and each feature corresponds to a feature matching degree F. After the calculation of the feature matching degrees FN of the various features is completed, the driving feature collection accuracy weight ∑4 is calculated.

[0030]

[0031] After the calculation of ∑4 is completed, automatic recording and backup are performed.

[0032] As a preferred technical solution of the present application, in the SS09, the following is performed:

[0033]

[0034] In the calculation of the matching difference KR, the maximum matching data difference value Kmax and the minimum matching data difference value Kmin after the size comparison of the various project data differences are obtained through a numerical comparison algorithm.

[0035] KR=K max -K min

[0036]

[0037] After the calculation of ∑5 is completed, automatic recording and backup are performed.

[0038] Compared with the prior art, the present application has the following beneficial effects:

[0039] In the present application, when the automatic driving NOA third-party test and evaluation are performed, the safety, flow, convenience and precision of the automatic driving can be comprehensively evaluated according to four dimensions, i.e., the driving result information of the automatic driving, the driving collection and driving perception information in the automatic driving process, the humanized evaluation information of the safety officer, and the driving simulation stability information of the automatic driving, thereby effectively improving the evaluation precision of the automatic driving system and the credibility of the evaluation results. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1A flow structure schematic diagram of an automatic driving NOA third-party test and evaluation system of the present application; DETAILED DESCRIPTION

[0041] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0042] As shown in Figure 1 The present application provides an automatic driving NOA third-party test and evaluation system, comprising the following steps:

[0043] SS01, a scene construction module, constructing an automatic driving scene, the automatic driving scene being provided with a specified weather scene, a road scene and a driving scene;

[0044] In the SS01 step, the road scene includes a specified road type, a specified road marking, a specified signal lamp, a sign, a street lamp, a crossroad type, a lane number, a lane line, a lane type, a speed limit type, a road type, a road angle, the weather scene includes weather, illumination, road visibility and environmental temperature and humidity, and the driving scene includes a traffic flow scene and a vehicle interference simulation scene;

[0045] SS02, a feature recognition and labeling module, after the SS01 step, a staff member automatically labels feature points and key points of the specified weather scene, the road scene and the driving scene, and after the labeling is completed, a feature dataset in the automatic driving scene is generated;

[0046] In the SS02 step, the feature dataset includes a total number of feature points and key points, and types of the feature points and the key points;

[0047] The feature dataset includes weather scene features, road scene features and driving scene features;

[0048] SS03, a data collection module, constructing the data collection module in an automatic driving vehicle, the data collection module collecting real-time driving data of the automatic driving vehicle when the automatic driving vehicle is running and generating a driving dataset;

[0049] In the SS03 step, the driving dataset includes driving time, vehicle instantaneous speed, steering wheel angle, vehicle GPS position information, vehicle edge driving track, vehicle radar information, automatic driving background running code and driving image information;

[0050] SS04, sample reference construction module, a plurality of drivers are preset, the driver should be the first time to drive in the automatic driving scene, after the preset, a driving plan is established, a plurality of drivers drive simulation in the automatic driving scene in turn according to the driving plan, each driver generates a driving sample data set after driving simulation;

[0051] The driving sample data set includes driving time, vehicle instantaneous speed, steering wheel angle, vehicle GPS position information, vehicle edge driving track, vehicle radar information, automatic driving background running code and driving video information;

[0052] SS05, average data construction module, the average data construction module establishes an average data set according to the plurality of driving sample data sets in SS04 step, the average data set includes average speed, average steering wheel angle and vehicle running center track parameters under specified driving time;

[0053] SS06, evaluation sample construction module, the automatic driving vehicle drives simulation according to the driving plan in SS03 step, after the automatic driving simulation is completed, a to-be-evaluated data set is obtained by the data acquisition module, and the automatic driving vehicle is configured with a safety officer during the automatic driving behavior;

[0054] SS07, driving evaluation module, the driving weight ∑1 is obtained according to the to-be-evaluated data set in SS04 step, and the safety officer obtains a personalized evaluation weight value ∑2 according to the evaluation of the safety officer after the automatic driving behavior is completed;

[0055] In SS07 step, when the driving weight ∑1 is calculated, each data in the to-be-evaluated data set in SS06 step is taken as calculation data, the average data set in SS05 step is taken as parameter data, data difference X in each driving project is calculated, after each type of data difference X is calculated, the average difference MD and the range value MR in the automatic driving project are calculated;

[0056]

[0057] If X1 is 2, X2 is 4, X3 is 6, X4 is 8 and X5 is 10 under certain specified conditions:

[0058]

[0059] When the range value MR is calculated, the maximum data difference Xmax and the minimum data difference Xmin after the size comparison of each project data difference are obtained through the numerical comparison algorithm;

[0060] MR=X max -X min

[0061] When X1 is 2, X2 is 4, X3 is 6, X4 is 8, and X5 is 10:

[0062] MR = 10-2 = 8

[0063] When X1 is 2, X2 is 4, X3 is 6, X4 is 8, and X5 is 10:

[0064]

[0065] ∑1 = 0.75

[0066] After ∑1 is calculated, it is automatically recorded and backed up;

[0067] SS08, a perception evaluation module, when the autonomous driving vehicle performs an autonomous driving behavior, the perception evaluation module judges the driving feature information collected by the data acquisition module during driving through data analysis, and the perception evaluation module calculates the perception information evaluation weight value ∑3 and the travel feature acquisition accuracy weight ∑4 according to the obtained driving feature information;

[0068] In SS08:

[0069]

[0070] When ∑4 is calculated, first, the matching degree of the driving feature collected by the data acquisition module and the feature data set is calculated, each feature corresponds to a feature matching degree F, after the feature matching degrees FN of various features are calculated, the travel feature acquisition accuracy weight ∑4 is calculated;

[0071]

[0072] After ∑4 is calculated, it is automatically recorded and backed up.

[0073] SS09, a driving simulation weight evaluation module, after the autonomous driving vehicle performs an autonomous driving behavior, the data acquisition module places the collected driving data set into a computer driving simulation system for simulation and reproduction, the computer driving simulation system obtains a simulation driving data set after each driving simulation, each simulation driving data set is obtained, and the matching degree K of the simulation driving data set and the driving sample data set is automatically calculated, after a plurality of K values are calculated, the matching degree average KD, the matching range value KR, and the autonomous driving system stability weight ∑5 are calculated;

[0074] In the step of SS09:

[0075]

[0076] When the matching range value KR is calculated, the maximum matching data difference value Kmax and the minimum matching data difference value Kmin after the size comparison of each project data difference are obtained through a numerical comparison algorithm;

[0077] KR = K max -K min

[0078]

[0079] After ∑5 is calculated, automatic recording and backup are performed;

[0080] SS10, a comprehensive evaluation module, automatically calculates the evaluation value ∑ of the autonomous driving NOA according to ∑1, ∑2, ∑3, ∑4 and ∑5;

[0081] ∑ = ∑1 x ∑2 x ∑3 x ∑4 x ∑5

[0082] After ∑ is calculated, automatic recording and backup are performed.

[0083] It should be noted that the relational terms herein such as first and second and the like are used solely to distinguish one from another entity or action without necessarily requiring or implying any actual relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0084] While the embodiments of the present application have been illustrated and described, it will be understood by those skilled in the art that various changes, modifications, substitutions, and alterations can be made hereto without departing from the principles and spirit of the application, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An autonomous driving NOA third-party testing and evaluation system, characterized in that, Includes the following steps: SS01, Scene Construction Module, constructs autonomous driving scenarios, which include preset weather scenarios, road scenarios, and driving scenarios; SS02, Feature Recognition and Labeling Module: After step SS01, staff automatically label feature points and key points for specified weather, road, and driving scenarios. After labeling, a feature dataset within the autonomous driving scenario is generated. SS03, Data Acquisition Module: A data acquisition module is built in the autonomous vehicle to collect real-time driving data during the operation of the autonomous vehicle and generate a driving dataset. SS04, Sample Reference Construction Module: Multiple drivers are preset, and the drivers are driving in this autonomous driving scenario for the first time. After the preset, a driving plan is established. According to the driving plan, the multiple drivers will conduct driving simulation in the autonomous driving scenario in turn. After each driver conducts a driving simulation, a line of vehicle sample dataset is generated. SS05, Average Data Construction Module, builds an average dataset based on multiple driving sample datasets in step SS04. The average dataset includes the average vehicle speed, average steering wheel angle, and center trajectory parameters of the vehicle under a specified driving time. SS06, Evaluation Sample Construction Module: The autonomous vehicle performs autonomous driving simulation according to the driving plan in step SS04. After the autonomous driving simulation is completed, the dataset to be evaluated is obtained. When the autonomous driving behavior is performed, the autonomous vehicle is equipped with a safety driver. SS07, the driving evaluation module, obtains the driving weight ∑1 based on the dataset to be evaluated obtained in step SS06. After the safety driver completes the autonomous driving behavior, the humanization evaluation weight value ∑2 is obtained based on the safety driver's evaluation. SS08, Perception Evaluation Module: When an autonomous vehicle performs autonomous driving behavior, the perception evaluation module uses data analysis to judge the driving feature information collected by the data acquisition module during the driving process. Based on the acquired driving feature information, the perception evaluation module calculates the perception information evaluation weight value ∑3 and the trip feature acquisition accuracy weight ∑4. SS09, Driving Simulation Weight Evaluation Module: After the autonomous vehicle performs autonomous driving behavior, the data acquisition module puts the collected driving dataset into the computer driving simulation system for simulation reproduction. After each driving simulation, the computer driving simulation system obtains a simulated driving dataset. After each simulated driving dataset is obtained, the matching degree K between the simulated driving dataset and the driving sample dataset is automatically calculated. After multiple K values ​​are calculated, the mean matching degree KD, the matching range KR, and the weight ∑5 of the stability value of the autonomous driving system are calculated. SS10, Comprehensive Evaluation Module: This module calculates the NOA (Noise of Account) evaluation value ∑ for autonomous driving based on ∑1, ∑2, ∑3, ∑4, and ∑5. ∑=∑1×∑2×∑3×∑4×∑5 After the calculation is completed, the data is automatically recorded and backed up.

2. The autonomous driving NOA third-party testing and evaluation system according to claim 1, characterized in that: In step SS01, the road scenario includes specified road type, specified road markings, specified traffic lights, signs, streetlights, intersection type, number of lanes, lane lines, lane type, speed limit type, road type, and road angle; the weather scenario includes weather, lighting, road visibility, and ambient temperature and humidity; and the driving scenario includes traffic flow scenario and vehicle interference simulation scenario.

3. The third-party testing and evaluation system for autonomous driving NOA as described in claim 1, characterized in that: In step SS02, the feature dataset includes the total number of feature points and key points, and the types of feature points and key points.

4. The third-party testing and evaluation system for autonomous driving NOA as described in claim 1, characterized in that: In step SS03, the driving dataset includes driving time, vehicle instantaneous speed, steering wheel angle, vehicle GPS location information, vehicle edge driving trajectory, vehicle radar information, autonomous driving background running code, and driving image information.

5. The third-party testing and evaluation system for autonomous driving NOA as described in claim 1, characterized in that: In step SS07, when calculating the driving weight ∑1, the data of each item in the dataset to be evaluated in step SS06 are used as the calculation data, and the average dataset in step SS05 is used as the parameter data. The data difference X under the driving item is calculated. After the data difference X of each type is calculated, the average difference MD and the range MR under the automatic driving item are calculated. When calculating the range (MR), the maximum data difference (Xmax) and the minimum data difference (Xmin) of each item are obtained by comparing the data differences of each item through a numerical comparison algorithm. MR=X max -X min ∑1= ÷100 After ∑1 is calculated, it is automatically recorded and backed up.

6. The third-party testing and evaluation system for autonomous driving NOA as described in claim 1, characterized in that: In SS08: When calculating ∑4, first calculate the matching degree between the driving features collected by the data acquisition module and the feature dataset. Each feature has a corresponding feature matching degree FN. After the feature matching degree FN of each feature is calculated, calculate the trip feature acquisition accuracy weight ∑4. ∑4 After ∑4 is calculated, it is automatically recorded and backed up.

7. The third-party testing and evaluation system for autonomous driving NOA as described in claim 1, characterized in that: In step SS09: KD Where KD is the mean matching degree; When calculating the matching range KR, the maximum matching data difference Kmax and the minimum matching data difference Kmin are obtained after comparing the data differences of each item through a numerical comparison algorithm. KR=K max -K min ∑5= ÷100 After ∑5 is calculated, it is automatically recorded and backed up.

Images