Automatic driving simulation evaluation method and simulation evaluation device

Abstract

The invention relates to an automatic driving simulation evaluation method and device, and belongs to the technical field of intelligent driving, and the method comprises the steps: constructing a digital twin 3D map; a vehicle simulation model is loaded in the digital twin 3D map, an automatic driving module used for loading an automatic driving algorithm is constructed, and an automatic driving evaluation unit based on traffic regulations is constructed; an automatic driving algorithm to be tested is loaded, the automatic driving module generates a control instruction based on environment data collected by the environment sensing assembly and issues the control instruction to the vehicle motion controller, and the vehicle motion controller controls the vehicle motion model to operate in the digital twin 3D map; the automatic driving evaluation unit collects controlled motion state parameters of the vehicle motion model and intervention degree parameters of the manual controller, and evaluates an automatic driving algorithm. According to the method, the defect that evaluation cannot be carried out according to the traffic regulations in the automatic driving algorithm evaluation process is effectively overcome, meanwhile, advantages and disadvantages of different algorithms can be compared, and the test cost is effectively reduced.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving technology, and in particular to an automatic driving simulation evaluation method and a simulation evaluation device. Background Art

[0002] With the development of science and technology, smart cars based on autonomous driving technology have shown great development potential in improving traffic safety and preventing and controlling traffic congestion, and are the direction of development of the automobile industry and traffic operations.

[0003] In order to improve the driving experience of driverless cars and prevent traffic accidents during the autonomous driving process, the autonomous driving algorithm needs to be tested multiple times, but the traditional real car road test method consumes a lot of manpower and material resources. Therefore, in the current autonomous driving research and development process, the effect of the driverless algorithm has begun to be tested through simulation. However, how to evaluate the pros and cons of the autonomous driving algorithm in simulation is the focus and difficulty of many current driverless simulation evaluation software.

[0004] The prior art has the following problems:

[0005] 1. Existing technologies for evaluating the quality of autonomous driving algorithms mostly rely on visual observation to manually determine whether a vehicle runs a red light, exceeds the speed limit, or drives in the wrong direction. They do not fully utilize the massive data in simulation scenarios to automatically evaluate the feasibility of autonomous driving algorithms based on these data.

[0006] 2. Existing technologies also use automatic evaluation methods, but they focus more on the stability and control accuracy of the algorithm, and involve less evaluation of traditional traffic regulations.

[0007] Therefore, in view of the shortcomings of the existing autonomous driving algorithm evaluation methods, there is an urgent need for an evaluation method that can build a simulation scenario and use the relevant information available in the simulation to implement an autonomous driving algorithm that is more in line with real-life scenarios. Summary of the invention

[0008] The purpose of the present invention is to provide an automatic driving simulation evaluation method and a simulation evaluation device.

[0009] To achieve the above-mentioned object of the invention, the present invention provides an automatic driving simulation evaluation method, comprising the following steps:

[0010] S1. Constructing a digital twin 3D map for simulating vehicle operation; wherein the digital twin 3D map includes: a static scene and a dynamic scene;

[0011] S2. Loading a vehicle simulation model in the digital twin 3D map, wherein the vehicle simulation model includes: a vehicle motion model, a vehicle motion controller, a manual controller, and an environmental perception component;

[0012] S3. Construct an autonomous driving module for loading an autonomous driving algorithm, and construct an autonomous driving evaluation unit based on traffic regulations;

[0013] S4. The autonomous driving module is loaded with the autonomous driving algorithm to be tested, and the autonomous driving module generates a control instruction based on the environmental data collected by the environmental perception component and sends it to the vehicle motion controller, and the vehicle motion controller controls the vehicle motion model to run in the digital twin 3D map based on the control instruction;

[0014] S5. The autonomous driving evaluation unit evaluates the autonomous driving algorithm to be tested based on the operating status of the vehicle motion model and outputs the evaluation result.

[0015] According to one aspect of the present invention, in step S1, the step of constructing a digital twin 3D map for simulating vehicle operation includes:

[0016] S11. Constructing the static scene based on the offline collected point cloud data and combining the 3D reconstruction method;

[0017] S12. Construct the dynamic scene based on the static scene based on a scene editor.

[0018] According to one aspect of the present invention, the step of constructing the static scene based on the offline collected point cloud data and in combination with a three-dimensional reconstruction method includes:

[0019] S111. Collecting point cloud data for the selected target area;

[0020] S112. Importing the collected point cloud data into a three-dimensional simulation environment, and constructing an initial static scene of the target area;

[0021] S113. Adding static traffic elements based on the initial static scene; wherein the static traffic elements include: vehicle driving roads, static obstacles, road lines, traffic lights, road materials and cracks;

[0022] S114. Draw a high-precision map line in the initial static scene using a high-precision map drawing tool, and add speed limit information to the high-precision map line according to a preset rule;

[0023] S115. Adding lighting elements, weather elements and a main camera module to the initial static scene based on preset rules to complete the generation of the static scene.

[0024] According to one aspect of the present invention, the step of constructing the dynamic scene based on the static scene based on a scene editor includes:

[0025] S121. Add pedestrian elements to the static scene based on the scene editor, and set the walking trajectory, pedestrian triggering timing scheme and walking speed scheme of the pedestrian elements; wherein the walking trajectory is generated based on a plurality of set trajectory points and connected by straight line segments or Besizer curves; the pedestrian triggering timing scheme includes: a time triggering mechanism triggered by a preset duration, a distance triggering mechanism triggered by the distance between the vehicle and the pedestrian element, and a position triggering mechanism triggered by the vehicle moving to a preset point;

[0026] S122. Based on the scene editor, an NPC vehicle element is added to the static scene, and a driving trajectory, a vehicle triggering timing scheme and a driving speed scheme of the NPC vehicle element are set; wherein the driving trajectory is generated based on a plurality of set trajectory points and connected by straight line segments or Besizer curves, and the vehicle triggering timing scheme includes: a time triggering mechanism triggered by a preset duration, a distance triggering mechanism triggered by the distance between the vehicle and the NPC vehicle element, and a position triggering mechanism triggered by the vehicle moving to a preset point;

[0027] S123. Add movable obstacle elements to the static scene based on the scene editor.

[0028] According to one aspect of the present invention, in step S2, the environmental perception component includes: a radar sensor for obstacle detection and passable area detection, a GPS sensor for real-time positioning of the simulated vehicle during driving, a camera sensor for collecting visual images in the forward direction of the simulated vehicle, and an IMU sensor for collecting the running speed, acceleration and rotation speed of the simulated vehicle.

[0029] According to one aspect of the present invention, in the step of constructing an automatic driving evaluation unit based on traffic regulations, the automatic driving evaluation unit includes: a controlled evaluation index for evaluating the controlled motion state parameter, and a manual intervention evaluation index for evaluating the intervention degree parameter; wherein the controlled evaluation index includes: an automatic driving duration index t0, a collision number index c0, a reverse driving number index c1, a speeding number index c2, and a red light running number index c3; the manual intervention evaluation index includes: a manual driving number index i0, and a manual intervention mileage index i1;

[0030] The automatic driving evaluation unit generates an evaluation formula for evaluating the automatic driving algorithm based on the controlled evaluation index and the manual intervention evaluation index, wherein the evaluation formula is expressed as:

[0031] R=t0+20*c0+15*c1+10*c2+18*c3+8*i0+20*i1 / (i1+L)

[0032] Among them, R is the comprehensive score value obtained by the evaluation formula.

[0033] According to one aspect of the present invention, the automatic driving duration index t0 is obtained in the following manner:

[0034] Under the control of the vehicle motion controller, the vehicle motion model starts running from a starting point and a timer is used to start timing until the vehicle motion model moves to an end point, and then the timing is stopped. During the running process of the vehicle motion model, it is continuously detected whether the manual controller intervenes. If the manual controller intervenes, the timing is suspended until the vehicle motion controller resumes control of the vehicle motion model, and then the timing is continued;

[0035] Obtaining the automatic driving duration index t0 based on the duration recorded by the timer between the starting point and the end point;

[0036] The manual driving times index i0 is obtained in the following way:

[0037] Under the control of the vehicle motion controller, the vehicle motion model starts running from a starting point and uses a counter to count the number of times the manual controller intervenes until the vehicle motion model moves to an end point, and then stops counting;

[0038] The number of interventions of the manual controller recorded by the counter between the starting point and the end point is used as the manual driving number index i0;

[0039] The collision number index c0 is obtained in the following way:

[0040] Under the control of the vehicle motion controller, the vehicle motion model starts running from a starting point and uses a counter to count the number of collisions of the vehicle motion model; wherein the collision between the vehicle motion model and any one of a static obstacle, a pedestrian element, and an NPC vehicle element is counted into the number of collisions;

[0041] Determine whether the collision of the vehicle motion model occurs under the control of the manual controller, if so, the counter stops accumulating the number of collisions, if not, the counter accumulates the number of collisions;

[0042] When the vehicle motion model moves to the end point, the counter stops counting;

[0043] The number of collisions recorded by the counter between the starting point and the end point is used as the collision number index c0;

[0044] The retrograde times index c1 is obtained in the following manner:

[0045] Under the control of the vehicle motion controller, continuously detecting whether the driving direction of the vehicle motion model is consistent with the direction of the road line in the vehicle driving road, and if not, using a counter to record the number of reverse driving times;

[0046] Determine whether the reverse movement of the vehicle motion model occurs under the control of the manual controller, if so, the counter gives up accumulating the number of reverse movements, if not, the counter accumulates the number of reverse movements;

[0047] When the vehicle motion model moves to the end point, the counter stops counting;

[0048] The number of retrograde movements recorded by the counter between the starting point and the end point is used as the retrograde movement number index c1;

[0049] The speeding times index c2 is obtained in the following manner:

[0050] Under the control of the vehicle motion controller, continuously detecting whether the driving speed of the vehicle motion model exceeds the speed limit information of the vehicle driving road, and if it exceeds, using a counter to record the number of speeding times;

[0051] Determine whether the speeding in the vehicle motion model occurs under the control of the manual controller, if so, the counter gives up accumulating the number of speeding times, if not, the counter accumulates the number of speeding times;

[0052] When the vehicle motion model moves to the end point, the counter stops counting;

[0053] The number of speeding times recorded by the counter between the starting point and the end point is used as the speeding times index c2;

[0054] The manual intervention mileage index i1 is obtained in the following way:

[0055] Under the control of the vehicle motion controller, continuously detecting whether the vehicle motion model is intervened by a manual controller, if the manual controller intervenes, continuously recording the intervention mileage, and suspending recording the intervention mileage if the vehicle motion controller resumes control of the vehicle motion model;

[0056] When the vehicle motion model moves to an end point, stop recording the intervention mileage;

[0057] The total length of the interference mileage recorded between the starting point and the end point is used as the manual intervention mileage index i1;

[0058] The red light running times index c3 is obtained in the following manner:

[0059] Under the control of the vehicle motion controller, when the red light state of the traffic light is detected, whether the vehicle motion model crosses the stop line of the intersection, and if so, a counter is used to record the number of red light violations;

[0060] When the vehicle motion model moves to the end point, the counter stops counting;

[0061] The number of red light running times recorded by the counter between the starting point and the end point is used as the red light running times index c3.

[0062] According to one aspect of the present invention, under the control of the vehicle motion controller, it is continuously detected whether the driving speed of the vehicle motion model exceeds the speed limit information of the vehicle driving road. If it exceeds, in the step of using a counter to record the number of speeding times, under the same speed limit information of the vehicle driving road, if the vehicle motion model exceeds the speed limit multiple times, the counter records the corresponding number of speeding times.

[0063] According to one aspect of the present invention, in the step of adding static traffic elements based on the initial static scene, if the traffic light is added, the signal light style, signal light color and signal light working status of the traffic light are set.

[0064] To achieve the above-mentioned object of the invention, the present invention provides an autonomous driving simulation evaluation device using the above-mentioned autonomous driving simulation evaluation method, comprising:

[0065] Offline data acquisition unit, used to collect point cloud data of real scenes;

[0066] A scene generation unit, which constructs a digital twin 3D map based on the point cloud data collected by the offline data collection unit;

[0067] A vehicle simulation unit, comprising a vehicle model generation module and an autonomous driving module, wherein the vehicle model generation module is used to generate a vehicle simulation model and load it into the digital twin 3D map, and the autonomous driving module is used to load the autonomous driving algorithm to be tested and output control instructions to the vehicle simulation model;

[0068] The vehicle simulation model includes: a vehicle motion model, a vehicle motion controller, an artificial controller and an environment perception component;

[0069] The automatic driving evaluation unit is used to collect the controlled motion state parameters of the vehicle motion model and the intervention degree parameters of the manual controller, and to evaluate the automatic driving algorithm to be tested based on the controlled motion state parameters and the intervention degree parameters and output the evaluation results.

[0070] According to a solution of the present invention, the present invention can not only intuitively display various violations of simulated vehicles during autonomous driving using evaluation indicators close to reality based on existing traffic rules, but also ensure that different algorithms are driving in the same environment to compare the advantages and disadvantages of the algorithm effects, greatly reducing the cost of algorithm testing for unmanned driving.

[0071] According to one solution of the present invention, the present invention makes full use of the relevant information available in the simulation scenario and automatically scores the autonomous driving algorithm based on traffic rules. This evaluation method can not only accurately identify the violations of the autonomous driving car according to the traffic regulations, but also score the effects of different algorithms in the same scenario in real time. The performance of the algorithm can be evaluated according to the score, which can greatly improve the work efficiency of algorithm engineers in development and testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 is a flowchart of a method for simulation evaluation of autonomous driving according to an embodiment of the present invention;

[0073] Figure 2 is a flow chart of an autonomous driving simulation evaluation method according to an embodiment of the present invention;

[0074] Figure 3 A structural block diagram of a simulation evaluation device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0075] The present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments. The embodiments cannot be described one by one here, but the embodiments of the present invention are not therefore limited to the following embodiments.

[0076] Combination Figure 1 and Figure 2 As shown, according to one embodiment of the present invention, an automatic driving simulation evaluation method of the present invention includes the following steps:

[0077] S1. Construct a digital twin 3D map for simulating vehicle operation; wherein the digital twin 3D map includes: a static scene and a dynamic scene;

[0078] S2. Loading a vehicle simulation model in the digital twin 3D map, wherein the vehicle simulation model includes: a vehicle motion model, a vehicle motion controller, a manual controller, and an environment perception component;

[0079] S3. Construct an autonomous driving module for loading an autonomous driving algorithm, and construct an autonomous driving evaluation unit based on traffic regulations;

[0080] S4. The autonomous driving module is loaded with the autonomous driving algorithm to be tested, and the autonomous driving module generates a control instruction based on the environmental data collected by the environmental perception component and sends it to the vehicle motion controller, and the vehicle motion controller controls the vehicle motion model to run in the digital twin 3D map based on the control instruction;

[0081] S5. The autonomous driving evaluation unit evaluates the autonomous driving algorithm to be tested based on the operating status of the vehicle motion model and outputs the evaluation results.

[0082] According to one embodiment of the present invention, in step S1, the step of constructing a digital twin 3D map for simulating vehicle operation includes:

[0083] S11. Constructing a static scene based on offline collected point cloud data and combining a 3D reconstruction method; specifically including:

[0084] S111. Collecting point cloud data for the selected target area; in this embodiment, the corresponding point cloud data can be collected in the target area by carrying a corresponding offline data collection unit on an actual motor vehicle;

[0085] S112. Importing the collected point cloud data into a three-dimensional simulation environment and constructing an initial static scene of the target area;

[0086] S113. Add static traffic elements based on the initial static scene; wherein the static traffic elements include: vehicle driving roads, static obstacles, road lines, traffic lights, road materials and cracks; in this embodiment, the traffic lights need to be set in the process of adding traffic lights, including the signal light style, signal light color and signal light working state. In the process of setting the working state of the traffic light, its current display color, pattern, display state and switching time need to be set separately. In this embodiment, by changing the density, depth, width and other parameters of the cracks on the driving road, it is simulated whether the driving road is a busy section or a busy intersection of the driving road.

[0087] S114. Use a high-precision map drawing tool to draw high-precision map lines in the initial static scene, and add speed limit information to the high-precision map lines selected according to preset rules; in this embodiment, through the received high-precision map lines, it is convenient to set speed limit information on each high-precision map line to verify whether the vehicle is speeding during the autonomous driving process. The use of high-precision maps and high-precision positioning can be used to determine the position of the vehicle motion model in the lane and help the vehicle motion model to drive along the middle of the lane; at the same time, the direction and speed of the vehicle motion model can be accurately controlled when the vehicle motion model is driving on a curve; high-precision maps can also provide reliable basis for the vehicle motion model to increase torque early when going uphill and reduce torque early when going downhill when driving on a slope, so as to achieve the purpose of stability;

[0088] S115. Add lighting elements, weather elements and a main camera module to the initial static scene based on preset rules to complete the generation of the static scene; in this embodiment, the lighting elements may include: lighting type (such as sunlight, street lights, lights emitted by buildings or vehicles, etc.), lighting intensity, lighting angle, etc. Weather elements include: rainy days and rainfall, foggy days and foggy days, snowy days and snowy days, etc.

[0089] S12. Building a dynamic scene based on the static scene based on the scene editor, which specifically includes:

[0090] S121. Add pedestrian elements to the static scene based on the scene editor, and set the walking trajectory, pedestrian triggering timing scheme and walking speed scheme of the pedestrian elements; in this embodiment, the walking trajectory can be set to walk along the vehicle driving road, cross the vehicle driving road, etc.; the pedestrian triggering timing scheme is used to set the time and position of the pedestrian walking along the walking trajectory; the walking speed scheme is used to set the walking speed of the pedestrian along the walking trajectory and the changing law of the walking speed. Among them, the walking trajectory is generated based on the set multiple trajectory points and connected by straight line segments or Besizer curves; in this embodiment, the intervals between the set trajectory points are the same or different. For example, in the case of crossing the vehicle driving road, the interval of the trajectory points on the driving road can be set to a small interval to increase the density of the trajectory points, so as to describe the pedestrian trajectory more carefully. When entering or leaving the driving road, the interval of the trajectory points can be set to a larger interval to increase the efficiency of pedestrian elements entering and leaving, and improve the simulation efficiency of the walking scene.

[0091] In this embodiment, the pedestrian trigger timing scheme includes: a time trigger mechanism triggered by a preset duration, a distance trigger mechanism triggered by the distance between the vehicle and the pedestrian element, and a position trigger mechanism triggered by the vehicle moving to a preset point;

[0092] In this embodiment, in the step of adding pedestrian elements to the static scene based on the scene editor, the action state of the pedestrian element can be further added, for example, the pedestrian element may fall, bend over, retreat, etc. while walking along the walking track, so as to further increase the flexibility of the use of the pedestrian element. In this embodiment, in order to realize the falling and bending actions, the pedestrian element can be deformed through pre-set instructions to realize the corresponding actions.

[0093] S122. Add NPC vehicle elements to the static scene based on the scene editor, and set the driving trajectory, vehicle trigger timing scheme and driving speed scheme of the NPC vehicle elements; in this embodiment, the driving trajectory of the NPC vehicle element can be set to drive along the vehicle driving road, merge into the vehicle driving road at the intersection, turn, make a U-turn, etc.; the vehicle trigger timing scheme is used to set the time and position of the NPC vehicle along the driving trajectory; the driving speed scheme is used to set the driving speed of the NPC vehicle along the driving trajectory and the law of driving speed change. Among them, the driving trajectory is generated based on the set multiple trajectory points and connected by straight line segments or Besizer curves. The vehicle trigger timing scheme includes: a time trigger mechanism triggered by a preset duration, a distance trigger mechanism triggered by the distance between the vehicle and the NPC vehicle element, and a position trigger mechanism triggered by the vehicle moving to a preset point;

[0094] In this embodiment, in the step of adding NPC vehicle elements to the static scene based on the scene editor, accident scenarios for traffic accidents involving NPC vehicle elements can be further added to simulate the emergency state when a traffic accident occurs to the NPC vehicle elements. For example, a traffic accident involving an NPC vehicle element causes the NPC vehicle element to invade the road on which the vehicle is traveling, and a traffic accident involving an NPC vehicle element causes debris, smoke, etc. to be generated on the road.

[0095] S123. Add movable obstacle elements to the static scene based on the scene editor. In this embodiment, the movable obstacle elements include: movable traffic cones, warning signs for marking accidents, maintenance signs for marking road maintenance, etc.

[0096] According to one embodiment of the present invention, in step S2, the environmental perception component includes: a radar sensor for obstacle detection and passable area detection, a GPS sensor for real-time positioning of the simulated vehicle during driving, a camera sensor for collecting visual images in the forward direction of the simulated vehicle, and an IMU sensor for collecting the running speed, acceleration and rotation speed of the simulated vehicle. In this embodiment, the radar sensor is used to measure the distance between the surface of the target object, analyze the reflected energy size, amplitude, frequency and phase of the reflected spectrum of the target object surface, so as to present the accurate three-dimensional structure information of the target object, and then realize obstacle detection and passable area detection. The GPS sensor is used to realize the real-time position positioning of the vehicle motion model during driving, so as to realize the automatic matching of the position of the vehicle motion model by the automatic driving module. The camera sensor is installed on the front side of the vehicle motion model, and is used to collect images in the forward direction in real time and transmit them to the automatic driving module. It is often used for lane line extraction, road information reading, traffic light recognition, traffic sign recognition, map construction and auxiliary positioning, etc.; the IMU sensor is installed on the vehicle motion model, and is used to measure the basic kinematic physical quantities such as vehicle motion speed, acceleration and rotation speed.

[0097] According to an embodiment of the present invention, in step S3, in the step of constructing an automatic driving evaluation unit based on traffic regulations, the automatic driving evaluation unit includes: a controlled evaluation index for evaluating controlled motion state parameters, and a manual intervention evaluation index for evaluating intervention degree parameters; wherein the controlled evaluation index includes: automatic driving duration index t0, collision number index c0, reverse number index c1, speeding number index c2, and red light running number index c3, and the manual intervention evaluation index includes: manual driving number index i0, manual intervention mileage index i1; in this embodiment, these indicators are assigned different weight values ​​according to the severity of traffic violations during actual road driving, and different evaluation results are calculated according to the violation of the indicators; specifically, the automatic driving evaluation unit generates an evaluation formula for evaluating the automatic driving algorithm based on the controlled evaluation index and the manual intervention evaluation index, wherein the evaluation formula is expressed as:

[0098] R=t0+20*c0+15*c1+10*c2+18*c3+8*i0+20*i1 / (i1+L)

[0099] Among them, R is the comprehensive score value obtained by the evaluation formula, the unit is second, t0 is second, c0, c1, c2, c3, i0 are times, L is the total length of the path, the unit is meter; after the simulation is completed, in addition to giving the comprehensive score of autonomous driving, the data of each evaluation indicator will also be displayed in detail for improving the algorithm.

[0100] According to an embodiment of the present invention, the automatic driving duration index is used to judge the driving speed, driving efficiency and driving reliability of the vehicle motion model. Then, the automatic driving duration index t0 is obtained in the following manner:

[0101] Under the control of the vehicle motion controller, the vehicle motion model starts running from the starting point and the timer starts timing until the vehicle motion model moves to the end point, and then the timing is stopped. During the running process of the vehicle motion model, it is continuously detected whether the manual controller intervenes. If the manual controller intervenes, the timing is suspended until the vehicle motion controller resumes control of the vehicle motion model, and then the timing is continued;

[0102] The autonomous driving duration index t0 is obtained by the duration recorded by the timer between the starting point and the end point.

[0103] According to an embodiment of the present invention, the manual driving times index i0 is used to judge the adaptability and obstacle avoidance capability of the automatic driving algorithm. Then, the manual driving times index is obtained in the following manner:

[0104] Under the control of the vehicle motion controller, the vehicle motion model starts running from the starting point and uses a counter to count the number of times the manual controller intervenes until the vehicle motion model moves to the end point, and then stops counting;

[0105] The number of interventions of the manual controller recorded by the counter between the starting point and the end point is used as the manual driving number index i0.

[0106] According to an embodiment of the present invention, collisions with traffic participants (pedestrians, NPC vehicle elements, etc.) or static obstacles during the autonomous driving process may easily cause serious traffic accidents. Therefore, the number of collisions index is an important index for judging the reliability and safety of the autonomous driving algorithm. The number of collisions index c0 is obtained in the following manner:

[0107] Under the control of the vehicle motion controller, the vehicle motion model starts running from the starting point and uses a counter to count the number of collisions of the vehicle motion model; wherein, the collision between the vehicle motion model and any of the static obstacles, pedestrian elements, and NPC vehicle elements is counted into the number of collisions;

[0108] Determine whether the collision of the vehicle motion model occurs under the control of the manual controller. If so, the counter stops accumulating the number of collisions. If not, the counter accumulates the number of collisions.

[0109] The counter stops counting until the vehicle motion model moves to the end point;

[0110] The number of collisions recorded by the counter between the starting point and the end point is used as the collision number index c0.

[0111] In this embodiment, in the process of counting the collisions of the vehicle motion model, it is also necessary to classify the static obstacles, pedestrian elements, and NPC vehicle elements that collide. Among them, the collision between the vehicle motion model and the pedestrian elements requires special annotation for special attention during the evaluation process.

[0112] According to an embodiment of the present invention, a vehicle driving in the wrong lane due to reverse driving is very likely to cause a traffic accident. Therefore, the reverse driving index c1 is an important index in the automatic driving evaluation process. Furthermore, the reverse driving index c1 is obtained in the following manner:

[0113] Under the control of the vehicle motion controller, the driving direction of the vehicle motion model is continuously detected to see whether it is consistent with the direction of the road line in the vehicle driving road. If not, a counter is used to record the number of reverse driving times.

[0114] Determine whether the reverse movement of the vehicle motion model occurs under the control of the manual controller, if so, the counter stops accumulating the number of reverse movements, if not, the counter accumulates the number of reverse movements;

[0115] The counter stops counting until the vehicle motion model moves to the end point;

[0116] The number of retrograde times recorded by the counter between the starting point and the end point is used as the retrograde times index c1.

[0117] According to an embodiment of the present invention, based on the speed limit information added in the above steps, it is possible to determine whether the current vehicle motion model is speeding by comparing the speed limit information with the speed of the current vehicle motion model. The accuracy of the automatic driving algorithm in recognizing the speed limit information can be evaluated by the obtained number of speeding times. Among them, the speeding times index c2 is obtained in the following way:

[0118] Under the control of the vehicle motion controller, the vehicle motion model is continuously detected to see whether the speed exceeds the speed limit information of the vehicle driving road. If it exceeds, a counter is used to record the number of speeding times.

[0119] Determine whether the speeding in the vehicle motion model occurs under the control of the manual controller. If so, the counter stops accumulating the number of speeding times. If not, the counter accumulates the number of speeding times.

[0120] The counter stops counting until the vehicle motion model moves to the end point;

[0121] The number of speeding times recorded by the counter between the starting point and the end point is used as the speeding times index c2.

[0122] In this embodiment, under the control of the vehicle motion controller, it is continuously detected whether the driving speed of the vehicle motion model exceeds the speed limit information of the vehicle driving road. If it exceeds, in the step of using a counter to record the number of speeding times, under the speed limit information of the same vehicle driving road, if the vehicle motion model exceeds the speed for multiple times, the counter records the corresponding number of speeding times. For example, when the vehicle motion model exceeds the speed for the first time on a road section with a speed limit of 40km / h, the speeding count is counted once, and when the speed of the vehicle motion model drops below the speed limit on the road section and then accelerates again to exceed the speed limit of 40km / h, the number of speeding times will be recorded again, that is, two speeding times.

[0123] According to an embodiment of the present invention, by counting the mileage of manual intervention during the automatic driving process, the automatic control effect of the automatic driving algorithm can be well evaluated. To this end, the manual intervention mileage index i1 is obtained in the following manner:

[0124] Under the control of the vehicle motion controller, the vehicle motion model is continuously detected to see if there is intervention of a manual controller. If there is intervention of a manual controller, the intervention mileage is continuously recorded. If the vehicle motion controller resumes control of the vehicle motion model, the intervention mileage is suspended.

[0125] When the vehicle motion model reaches the end point, the intervention mileage is stopped from being recorded;

[0126] The total length of the interference mileage recorded between the starting point and the end point is taken as the manual intervention mileage index i1.

[0127] According to one embodiment of the present invention, during the autonomous driving process, the state of the traffic light at the current intersection can be identified through the camera sensor provided, and uploaded to the autonomous driving module, which plans the movement of the vehicle motion model at the current traffic light intersection. In traffic accidents, accidents caused by running red lights at intersections account for a high proportion, so the number of red light running indicators is included as one of the evaluation indicators of the autonomous driving algorithm; wherein, the number of red light running indicators c3 is obtained in the following manner:

[0128] Under the control of the vehicle motion controller, when the red light state of the traffic light is detected, the vehicle motion model will check whether it crosses the stop line of the intersection. If so, a counter will be used to record the number of red light violations.

[0129] The counter stops counting until the vehicle motion model moves to the end point;

[0130] The number of red light running times recorded by the counter between the starting point and the end point is used as the red light running times index c3.

[0131] According to one embodiment of the present invention, in step S5, when the vehicle simulation model is controlled to perform autonomous driving in the digital twin 3D map, the autonomous driving evaluation unit can obtain the corresponding controlled evaluation index by collecting the controlled motion state parameters of the vehicle motion model (such as autonomous driving duration, number of collisions, number of reverse driving, number of speeding, number of red light running, etc.), and the autonomous driving evaluation unit can obtain the corresponding manual intervention evaluation index by collecting the intervention degree parameters of the manual controller (such as number of manual driving, manual intervention mileage, etc.). Finally, the autonomous driving algorithm to be tested is comprehensively evaluated according to the controlled evaluation index and the manual intervention evaluation index, and the evaluation result is output.

[0132] According to one embodiment of the present invention, the present invention provides an autonomous driving simulation evaluation device using the aforementioned autonomous driving simulation evaluation method, comprising:

[0133] Offline data acquisition unit, used to collect point cloud data of real scenes;

[0134] A scene generation unit constructs a digital twin 3D map based on the point cloud data collected by the offline data collection unit;

[0135] A vehicle simulation unit, including a vehicle model generation module and an autonomous driving module, wherein the vehicle model generation module is used to generate a vehicle simulation model and load it into the digital twin 3D map, and the autonomous driving module is used to load the autonomous driving algorithm to be tested and output control instructions to the vehicle simulation model;

[0136] The vehicle simulation model includes: vehicle motion model, vehicle motion controller, artificial controller and environment perception component;

[0137] The autonomous driving evaluation unit is used to collect the controlled motion state parameters of the vehicle motion model and the intervention degree parameters of the manual controller, and to evaluate the autonomous driving algorithm to be tested based on the controlled motion state parameters and the intervention degree parameters and output the evaluation results.

[0138] The above contents are merely examples of specific solutions of the present invention. For devices and structures not described in detail therein, it should be understood that they can be implemented by adopting general devices and general methods available in the art.

[0139] The above is only one solution of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An automatic driving simulation evaluation method, characterized in that: The following steps are involved: S1. Constructing a digital twin 3D map for simulating vehicle operation; wherein the digital twin 3D map includes: a static scene and a dynamic scene; S2. Loading a vehicle simulation model in the digital twin 3D map, wherein the vehicle simulation model includes: a vehicle motion model, a vehicle motion controller, a manual controller, and an environmental perception component; S3. Construct an autonomous driving module for loading an autonomous driving algorithm, and construct an autonomous driving evaluation unit based on traffic regulations; S4. The autonomous driving module is loaded with the autonomous driving algorithm to be tested, and the autonomous driving module generates a control instruction based on the environmental data collected by the environmental perception component and sends it to the vehicle motion controller, and the vehicle motion controller controls the vehicle motion model to run in the digital twin 3D map based on the control instruction; S5. The autonomous driving evaluation unit evaluates the autonomous driving algorithm to be tested based on the operating status of the vehicle motion model and outputs the evaluation result.

2. The automatic driving simulation evaluation method according to claim 1, characterized in that: In step S1, the step of constructing a digital twin 3D map for simulating vehicle operation includes: S11. Constructing the static scene based on the offline collected point cloud data and combining the 3D reconstruction method; S12. Construct the dynamic scene based on the static scene based on a scene editor.

3. The automatic driving simulation evaluation method according to claim 2, characterized in that: The step of constructing the static scene based on the offline collected point cloud data and combining the three-dimensional reconstruction method includes: S111. Collecting point cloud data for the selected target area; S112. Importing the collected point cloud data into a three-dimensional simulation environment, and constructing an initial static scene of the target area; S113. Adding static traffic elements based on the initial static scene; wherein the static traffic elements include: vehicle driving roads, static obstacles, road lines, traffic lights, road materials and cracks; S114. Draw a high-precision map line in the initial static scene using a high-precision map drawing tool, and add speed limit information to the high-precision map line according to a preset rule; S115. Adding lighting elements, weather elements and a main camera module to the initial static scene based on preset rules to complete the generation of the static scene.

4. The automatic driving simulation evaluation method according to claim 3, characterized in that: The step of constructing the dynamic scene based on the static scene based on the scene editor includes: S121. Add pedestrian elements to the static scene based on the scene editor, and set the walking trajectory, pedestrian triggering timing scheme and walking speed scheme of the pedestrian elements; wherein the walking trajectory is generated based on a plurality of set trajectory points and connected by straight line segments or Besizer curves; the pedestrian triggering timing scheme includes: a time triggering mechanism triggered by a preset duration, a distance triggering mechanism triggered by the distance between the vehicle and the pedestrian element, and a position triggering mechanism triggered by the vehicle moving to a preset point; S122. Based on the scene editor, an NPC vehicle element is added to the static scene, and a driving trajectory, a vehicle triggering timing scheme and a driving speed scheme of the NPC vehicle element are set; wherein the driving trajectory is generated based on a plurality of set trajectory points and connected by straight line segments or Besizer curves, and the vehicle triggering timing scheme includes: a time triggering mechanism triggered by a preset duration, a distance triggering mechanism triggered by the distance between the vehicle and the NPC vehicle element, and a position triggering mechanism triggered by the vehicle moving to a preset point; S123. Add movable obstacle elements to the static scene based on the scene editor.

5. The automatic driving simulation evaluation method according to claim 4, characterized in that: In step S2, the environmental perception component includes: a radar sensor for obstacle detection and passable area detection, a GPS sensor for real-time positioning of the simulated vehicle during driving, a camera sensor for collecting visual images in the forward direction of the simulated vehicle, and an IMU sensor for collecting the running speed, acceleration and rotation speed of the simulated vehicle.

6. The autonomous driving simulation evaluation method according to claim 5, characterized in that: In the step of constructing an automatic driving evaluation unit based on traffic regulations, the automatic driving evaluation unit includes: a controlled evaluation index for evaluating the controlled motion state parameter, and a manual intervention evaluation index for evaluating the intervention degree parameter; wherein the controlled evaluation index includes: an automatic driving duration index t0, a collision number index c0, a reverse driving number index c1, a speeding number index c2, and a red light running number index c3; the manual intervention evaluation index includes: a manual driving number index i0, and a manual intervention mileage index i1; The automatic driving evaluation unit generates an evaluation formula for evaluating the automatic driving algorithm based on the controlled evaluation index and the manual intervention evaluation index, wherein the evaluation formula is expressed as: R=t0+20*c0+15*c1+10*c2+18*c3+8*i0+20*i1 / (i1+L) Among them, R is the comprehensive score value obtained by the evaluation formula.

7. The automatic driving simulation evaluation method according to claim 6, characterized in that: The automatic driving duration index t0 is obtained in the following way: Under the control of the vehicle motion controller, the vehicle motion model starts running from a starting point and a timer is used to start timing until the vehicle motion model moves to an end point, and then the timing is stopped. During the running process of the vehicle motion model, it is continuously detected whether the manual controller intervenes. If the manual controller intervenes, the timing is suspended until the vehicle motion controller resumes control of the vehicle motion model, and then the timing is continued; Obtaining the automatic driving duration index t0 based on the duration recorded by the timer between the starting point and the end point; The manual driving times index i0 is obtained in the following way: Under the control of the vehicle motion controller, the vehicle motion model starts running from a starting point and uses a counter to count the number of times the manual controller intervenes until the vehicle motion model moves to an end point, and then stops counting; The number of interventions of the manual controller recorded by the counter between the starting point and the end point is used as the manual driving number index i0; The collision number index c0 is obtained in the following way: Under the control of the vehicle motion controller, the vehicle motion model starts running from a starting point and uses a counter to count the number of collisions of the vehicle motion model; wherein the collision between the vehicle motion model and any one of a static obstacle, a pedestrian element, and an NPC vehicle element is counted into the number of collisions; Determine whether the collision of the vehicle motion model occurs under the control of the manual controller, if so, the counter stops accumulating the number of collisions, if not, the counter accumulates the number of collisions; When the vehicle motion model moves to the end point, the counter stops counting; The number of collisions recorded by the counter between the starting point and the end point is used as the collision number index c0; The retrograde times index c1 is obtained in the following manner: Under the control of the vehicle motion controller, continuously detecting whether the driving direction of the vehicle motion model is consistent with the direction of the road line in the vehicle driving road, and if not, using a counter to record the number of reverse driving times; Determine whether the reverse movement of the vehicle motion model occurs under the control of the manual controller, if so, the counter gives up accumulating the number of reverse movements, if not, the counter accumulates the number of reverse movements; When the vehicle motion model moves to the end point, the counter stops counting; The number of retrograde movements recorded by the counter between the starting point and the end point is used as the retrograde movement number index c1; The speeding times index c2 is obtained in the following manner: Under the control of the vehicle motion controller, continuously detecting whether the driving speed of the vehicle motion model exceeds the speed limit information of the vehicle driving road, and if it exceeds, using a counter to record the number of speeding times; Determine whether the speeding in the vehicle motion model occurs under the control of the manual controller, if so, the counter gives up accumulating the number of speeding times, if not, the counter accumulates the number of speeding times; When the vehicle motion model moves to the end point, the counter stops counting; The number of speeding times recorded by the counter between the starting point and the end point is used as the speeding times index c2; The manual intervention mileage index i1 is obtained in the following way: Under the control of the vehicle motion controller, continuously detecting whether the vehicle motion model is intervened by a manual controller, if the manual controller intervenes, continuously recording the intervention mileage, and suspending recording the intervention mileage if the vehicle motion controller resumes control of the vehicle motion model; When the vehicle motion model moves to an end point, stop recording the intervention mileage; The total length of the interference mileage recorded between the starting point and the end point is used as the manual intervention mileage index i1; The red light running times index c3 is obtained in the following manner: Under the control of the vehicle motion controller, when the red light state of the traffic light is detected, whether the vehicle motion model crosses the stop line of the intersection, and if so, a counter is used to record the number of red light violations; When the vehicle motion model moves to the end point, the counter stops counting; The number of red light running times recorded by the counter between the starting point and the end point is used as the red light running times index c3.

8. The automatic driving simulation evaluation method according to claim 7, characterized in that: Under the control of the vehicle motion controller, whether the driving speed of the vehicle motion model exceeds the speed limit information of the vehicle driving road is continuously detected. If it exceeds, in the step of using a counter to record the number of speeding times, under the speed limit information of the same vehicle driving road, if the vehicle motion model exceeds the speed limit multiple times, the counter records the corresponding number of speeding times.

9. The automatic driving simulation evaluation method according to claim 8, characterized in that: In the step of adding static traffic elements based on the initial static scene, if the traffic light is added, the signal light style, signal light color and signal light working status of the traffic light are set.

10. An automatic driving simulation evaluation device using the automatic driving simulation evaluation method according to any one of claims 1 to 9, characterized in that: include: Offline data acquisition unit, used to collect point cloud data of real scenes; A scene generation unit, which constructs a digital twin 3D map based on the point cloud data collected by the offline data collection unit; A vehicle simulation unit, comprising a vehicle model generation module and an autonomous driving module, wherein the vehicle model generation module is used to generate a vehicle simulation model and load it into the digital twin 3D map, and the autonomous driving module is used to load the autonomous driving algorithm to be tested and output control instructions to the vehicle simulation model; The vehicle simulation model includes: a vehicle motion model, a vehicle motion controller, an artificial controller and an environment perception component; The automatic driving evaluation unit is used to collect the controlled motion state parameters of the vehicle motion model and the intervention degree parameters of the manual controller, and to evaluate the automatic driving algorithm to be tested based on the controlled motion state parameters and the intervention degree parameters and output the evaluation results.

Images