Automatic driving evaluation system based on simulation scene

Abstract

The embodiment of the invention relates to an automatic driving evaluation system based on a simulation scene. The system comprises an acquisition module, a scene server, an evaluation module, a scene cache module, a scene player and a self-vehicle simulator, the acquisition module is used for converting the real road condition data into a test scene; the evaluation module is used for selecting a matched scene from the scene server to initialize the scene cache module and the self-vehicle simulator, starting a test on the tested regulation and control module, and performing evaluation from four dimensions of safety, compliance, comfort and efficiency after the test is finished; and the scene player and the self-vehicle simulator are linked to complete the test of the to-be-tested regulation and control module. According to the method, the scene complexity can be improved, the richness and diversity of evaluation means are improved, the verification sufficiency of a regulation control module is improved, and the drive test cost is reduced.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to an autonomous driving evaluation system based on simulation scenarios. Background Art

[0002] The planning and control module (PCM) of an autonomous driving system plans the vehicle's driving trajectory based on the perception information output by the system's perception module (such as the movement trajectories of other traffic participants, obstacle locations, road conditions, and traffic light status). Based on the planned trajectory, it outputs a series of driving control commands (such as steering wheel angle, throttle / brake application, and gear position). The PCM testing phase is divided into two stages: the initial phase focuses on automated testing, while the subsequent phase focuses on road testing (referred to as "road testing"). Simply put, automated testing converts collected road condition data into perception information, which is then fed into the PCM for computation and the resulting driving control commands are compared with the expected commands. In principle, the more thorough the initial testing, the lower the cost of subsequent road testing. However, current automated testing methods for PCMs primarily rely on simple functional testing. These test scenarios lack sufficient complexity, and the test evaluation methodology is limited (only comparing output with expectations). This results in insufficient validation of the module's algorithmic performance in the initial phase and an inability to effectively reduce road testing costs in the later phase. Summary of the Invention

[0003] The purpose of the present invention is to provide an automatic driving evaluation system based on simulation scenarios in response to the defects of the prior art. The system includes: an acquisition module, a scenario server, an evaluation module, a scenario cache module, a scenario player, and a self-vehicle simulator; wherein the acquisition module is used to convert the real road condition data stored in the road network database into a test scenario and store it in the scenario server; the evaluation module is used to select a test scenario from the scenario server, set the test frame sequence, the loading frame index on the scene cache module, and the takeover frame index on the self-vehicle simulator, and perform simulation according to the test frame sequence by calling the scenario player; the scenario player sends corresponding test start / end instructions to the control module at the start and end of each round of simulation process, and also sends a playback completion status to the evaluation module at the end time. state; each round of simulation process of the scene player is a round of frame-by-frame loading and playback process; in each round of simulation process, as long as the current loaded frame is not the last frame, the scene player will generate the corresponding current perception information based on the current loaded frame and send it to the regulation and control module; the self-vehicle simulator is used to predict the self-vehicle motion state based on the loaded frame index and the current self-vehicle control instruction every time it receives a self-vehicle control instruction sent by the regulation and control module, and in each prediction process, if the loaded frame index is greater than or equal to the takeover frame index, the self-vehicle state of the next frame corresponding to the loaded frame index in the test frame sequence will be refreshed based on the current prediction result; the evaluation module is also used to evaluate this test from the four dimensions of safety, compliance, comfort and efficiency after receiving the playback completion status to obtain a corresponding evaluation report. The present invention can mine a large number of test scenarios of different complexities from a large amount of real road condition data, thereby achieving the purpose of increasing the complexity of the scenarios; it can improve the utilization rate of the test scenarios and the test efficiency of the regulatory control module based on the simulation test method of the scenario player + the vehicle simulator, and can improve the richness and diversity of the evaluation methods through the four-dimensional evaluation mechanism; through the present invention, the verification adequacy of the regulatory control module can be effectively improved in the early automatic testing stage, thereby further achieving the purpose of reducing road testing costs.

[0004] To achieve the above-mentioned purpose, an embodiment of the present invention provides an autonomous driving evaluation system based on a simulation scenario, the system comprising: an acquisition module, a scenario server, an evaluation module, a scenario cache module, a scenario player, and a self-driving vehicle simulator;

[0005] The acquisition module is connected to the road network database and the scenario server respectively; the evaluation module is connected to the scenario server, the scenario cache module, the scenario player and the vehicle simulator respectively; the scenario cache module is connected to the scenario player and the vehicle simulator respectively; the scenario player and the vehicle simulator are respectively connected to a regulation and control module under test;

[0006] The acquisition module is used to convert the real road condition data stored in the road network database into a test scene and store it in the scene server;

[0007] The scenario server is used to store a plurality of the test scenarios;

[0008] The evaluation module is used to receive batch tasks input by the user; and use each test scene identifier of the current batch task as the corresponding current identifier in turn; and select a matching scene from the scene server based on the current identifier to initialize the test frame sequence, loading frame index and takeover frame index on the scene cache module and the self-vehicle simulator; and send a play instruction to the scene player at the end of this initialization; and when the playback completion status is fed back by the scene player, perform a test evaluation based on the current identifier and the test frame sequence from the four dimensions of safety, compliance, comfort and efficiency to obtain a corresponding evaluation report; and feed back all the obtained evaluation reports to the current user; the batch task includes multiple test scene identifiers;

[0009] The scene cache module is used to store the test frame sequence and the loading frame index;

[0010] The scene player is used to identify the starting and target positions of the vehicle based on the test frame sequence when receiving the playback instruction and send a test start instruction carrying the recognition result to the regulation and control module, and perform a round of frame-by-frame loading and playback according to the test frame sequence when receiving the startup success sent back by the regulation and control module; and during this round of playback, if the current loaded frame is not the last frame, generate corresponding current perception information based on the current loaded frame and send it to the regulation and control module; and at the end of this round of playback, send a playback completion status to the evaluation module and send a test end instruction to the regulation and control module;

[0011] The ego vehicle simulator is configured to predict the ego vehicle's motion state based on the loading frame index and the current ego vehicle control instruction each time it receives an ego vehicle control instruction sent by the regulation and control module; and during each prediction process, if the loading frame index is greater than or equal to the takeover frame index, the ego vehicle state is refreshed for the next frame corresponding to the loading frame index based on the current prediction result.

[0012] Preferably, the road network database is used to store a plurality of the real road condition data in the real road network;

[0013] The real road condition data includes a participant dataset, a reference object dataset, a traffic light dataset, and a high-precision map of the area;

[0014] The participant data set includes multiple participant data; the participant data includes participant identification, participant type, and participant trajectory; the participant type includes vehicle, motorcycle, bicycle, pedestrian, and animal; the participant trajectory is composed of multiple first trajectory points; the first trajectory point includes a timestamp, a road section / traffic intersection identification, coordinates, heading angle, angular velocity, velocity, and acceleration;

[0015] The reference object data set includes a plurality of reference object data; the reference object data includes a reference object identifier, a reference object type, and a reference object coordinate; the reference object type includes at least a building, a pole, a plant, a traffic obstacle, and other obstacles;

[0016] When the signal light data set is not empty, it is composed of one or more signal light data; the signal light data includes a signal light identification, a signal light type, a signal light coordinate, and a signal light trajectory; the signal light type includes at least a motor vehicle signal light, a lane signal light, and a direction indicator signal light; the signal light trajectory is composed of a plurality of second trajectory points; the second trajectory point includes a timestamp and a signal light state; the signal light state includes at least red light, green light, yellow light, and flashing yellow light;

[0017] The regional high-precision map is a global high-precision map of the road section or intersection area corresponding to the current traffic data, including map element information of all road elements and non-road elements in the area;

[0018] The test scene includes scene identification, scene type, scene complexity, scene high-precision map, scene frame sequence, key frame index; the scene type includes straight road, curve, diverging road section, merging road section, merging road section, merging road section, cross intersection, T-shaped intersection; the scene frame sequence is composed of multiple first frame scenes; the first frame scene includes frame index, frame timestamp, self-vehicle trajectory point, other participant trajectory point set, reference object trajectory point set, signal light trajectory point set; the self-vehicle trajectory point includes self-vehicle coordinates, self-vehicle road section / traffic intersection identification, self-vehicle heading angle, self-vehicle speed, self-vehicle acceleration; the other participant trajectory point set includes multiple participant trajectory points. The participant trajectory point includes the participant identifier, the participant type, the participant coordinates, the participant road section / traffic intersection identifier, the participant orientation angle, the participant speed, and the participant acceleration; the reference object trajectory point set includes multiple reference object trajectory points, and the reference object trajectory point includes the reference object identifier, the reference object type, and the reference object coordinates; when the signal light trajectory point set is not empty, it is composed of one or more signal light trajectory points, and the signal light trajectory point includes the signal light identifier, the signal light type, the signal light coordinates, and the signal light state; the time interval between each two adjacent scenes in the scene frame sequence is a preset standard interval Δt std, the time span of the scene frame sequence is the preset scene duration L;

[0019] The test frame sequence consists of multiple first test frames; the first test frame includes a test frame index, a test frame timestamp, a self-vehicle label trajectory point, a self-vehicle simulation trajectory point, the trajectory point set of the other participants, the trajectory point set of the reference object, and the trajectory point set of the traffic light; the self-vehicle label trajectory point includes the self-vehicle label coordinates, the self-vehicle label road section / traffic intersection identification, the self-vehicle label heading angle, the self-vehicle label speed, and the self-vehicle label acceleration; the self-vehicle simulation trajectory point includes the self-vehicle simulation coordinates, the self-vehicle simulation road section / traffic intersection identification, the self-vehicle simulation heading angle, the self-vehicle simulation speed, and the self-vehicle simulation acceleration.

[0020] Preferably, the acquisition module is specifically configured to, when converting the real road condition data stored in the road network database into a test scenario and storing the converted data in the scenario server:

[0021] The real traffic condition data on the scene server with a time span exceeding the scene duration L is used as the corresponding current traffic condition data;

[0022] The earliest and latest timestamps in the current traffic condition data are used as the corresponding current start time and current end time; the current start time is converted to the corresponding start zero time; the current end time is converted relative to the start zero time to obtain the corresponding end time; the time period from the start zero time to the end time is used as the corresponding current traffic condition period; and the time period is calculated based on the standard interval Δt std The single-step interval is a time point sampling of the current road condition period to obtain corresponding multiple single-step time points; the time interval between each two adjacent single-step time points is the standard interval Δt std ;

[0023] And take the trajectories of each participant in the current road condition data as the corresponding current trajectory; and take the sub-period corresponding to the current trajectory in the current road condition period as the current trajectory period; and use the coordinates, heading angle, angular velocity, speed, and acceleration of all the first trajectory points in the current trajectory as observation quantities, and construct the corresponding Kalman filter equation according to the preset vehicle kinematic model; and based on the current Kalman filter equation, predict the state of the coordinates, heading angle, angular velocity, speed, and acceleration of the current participant at each single-step time point in the current trajectory period; and reconstruct the current trajectory based on the predicted data; in each reconstructed participant trajectory, the timestamp of each first trajectory point corresponds to a single-step time point, and the coordinates, heading angle, angular velocity, speed, and acceleration of each first trajectory point are the coordinate, heading angle, angular velocity, speed, and acceleration state prediction quantities corresponding to the timestamp of the current trajectory point, and the road section / traffic intersection identifier of each first trajectory point is the road section identifier or traffic intersection identifier corresponding to the current coordinate prediction quantity in the high-precision map of the area;

[0024] The driving distance of each reconstructed participant trajectory is calculated to obtain a corresponding first distance; the trajectory time of each reconstructed participant trajectory is calculated to obtain a corresponding first duration; the participant trajectory whose first distance exceeds a preset first distance threshold and whose first duration exceeds a preset first duration threshold is recorded as a corresponding candidate trajectory, and one of all the candidate trajectories is selected as the corresponding vehicle trajectory; the participant data corresponding to the vehicle trajectory in the participant data set of the current road condition data is collected as the corresponding vehicle data, and the other participant data is recorded as the corresponding other participant data;

[0025] and when the signal light data set of the current road condition data is not empty, each signal light track of the current signal light data set is used as the corresponding current track; and the sub-period corresponding to the current track in the current road condition period is used as the current track period; and each single-step time point in the current track period is recorded as the corresponding current time point, and the signal light state corresponding to the current time point is set according to the signal light state of the previous and / or next second track point of the current time point in the current track by interpolation, and the current time point and the corresponding signal light state form a corresponding reconstructed track point; and a new signal light track is formed by all the reconstructed track points corresponding to the current track period to replace the current track; in each reconstructed signal light track, the timestamp of each second track point corresponds to one single-step time point;

[0026] The scene duration L is used as the segment length to segment the current traffic period according to the preset single-step sliding step length to obtain corresponding multiple segment time periods; the single-step sliding step length is the standard interval Δt std an integer multiple of ;

[0027] And take each of the segmented time periods as the corresponding current segment; and assign a corresponding frame index to each of the single-step time points of the current segment in a point-by-point increment manner starting from 1; and take the single-step time point corresponding to each of the frame indexes as a corresponding frame timestamp; and use the road section / traffic intersection identification, coordinates, orientation angle, angular velocity, speed, and acceleration of the first trajectory point corresponding to each of the frame timestamps of the current segment in the vehicle trajectory as a group of corresponding vehicle road section / traffic intersection identification, vehicle coordinates, vehicle orientation angle, vehicle speed, and vehicle acceleration to form a corresponding vehicle trajectory point; and each of the The road section / traffic intersection identifier, coordinates, orientation angle, angular velocity, speed, and acceleration of the first trajectory point corresponding to each frame timestamp of the current segment in the participant trajectory of the other participant data are used as a group of corresponding participant road section / traffic intersection identifiers, participant coordinates, participant orientation angles, participant speeds, and participant accelerations; and the participant identifier, participant type, participant road section / traffic intersection identifier, participant coordinates, participant orientation angles, participant speeds, and participant accelerations corresponding to each of the other participant data form a corresponding participant trajectory point; and the current segment is used as a group of corresponding participant road section / traffic intersection identifiers, participant coordinates, participant orientation angles, participant speeds, and participant accelerations. All the participant trajectory points corresponding to each frame timestamp of the fragment form a corresponding set of other participant trajectory points; and each reference object data of the current road condition data is used as a corresponding reference object trajectory point, and all the reference object trajectory points form the reference object trajectory point set corresponding to each frame timestamp in the current fragment; and the signal light identification, the signal light type, the signal light coordinates and the signal light state corresponding to the second trajectory point corresponding to each frame timestamp of the current fragment in each signal light trajectory form a corresponding signal light trajectory point; and all the reference object data corresponding to each frame timestamp of the current fragment The signal light trajectory points constitute a corresponding signal light trajectory point set; and each frame index of the current segment and its corresponding frame timestamp, the vehicle trajectory point, the other participant trajectory point set, the reference object trajectory point set, and the signal light trajectory point set constitute a corresponding first frame scene; and all the first frame scenes corresponding to the current segment are sorted in chronological order to constitute a corresponding scene frame sequence; and the time period center point of the current segment is used as the corresponding current time period center point, and the frame index corresponding to the frame timestamp closest to the current time period center point in the scene frame sequence corresponding to the current segment is used as the corresponding key frame index;

[0028] and assigning a unique identifier to each of the scene frame sequences as the corresponding scene identifier;

[0029] and using the high-precision map of the area of ​​the current road condition data as the high-precision map of the scene corresponding to each of the scene frame sequences;

[0030] and forming a corresponding self-vehicle driving trajectory from a plurality of the self-vehicle trajectory points of each of the scene frame sequences; and querying the regional high-precision map of the current road condition data based on the self-vehicle road section / traffic intersection identifier of each of the self-vehicle driving trajectories, and confirming based on the query result whether the road section / traffic intersection covered by the current self-vehicle driving trajectory is a straight road, a curve, a diverging section, a merging section, an incoming section, an outgoing section, a cross intersection, or a T-intersection, and setting the corresponding scene type based on the confirmation result;

[0031] and respectively counting the total number of the participant identifiers and the total number of the participant types in each of the scene frame sequences to obtain a corresponding first total number and a second total number, respectively normalizing the first and second total numbers to obtain a corresponding first normalized total number and second normalized total number, performing a weighted sum on the first and second normalized total numbers, and using the sum result as the corresponding scene complexity;

[0032] Each scene frame sequence and its corresponding scene identifier, scene type, scene complexity, scene high-precision map and key frame index form a corresponding test scene and are stored in the scene server.

[0033] Preferably, the evaluation module is specifically configured to, when selecting a matching scene from the scene server based on the current identifier to initialize the test frame sequence, the loaded frame index on the scene cache module, and the takeover frame index on the self-driving vehicle simulator:

[0034] The test scene whose scene identifier on the scene server matches the current identifier is used as the corresponding matching scene; and the scene frame sequence and the key frame index of the matching scene are used as the corresponding current scene frame sequence and current key frame index;

[0035] And use the frame index and the frame timestamp of each first frame scene of the current scene frame sequence as the corresponding test frame index and the test frame timestamp; and use the self-vehicle trajectory point of each first frame scene of the current scene frame sequence as the corresponding self-vehicle label trajectory point and the self-vehicle simulation trajectory point; and each of the test frame indexes and its corresponding test frame timestamp, the self-vehicle label trajectory point, the self-vehicle simulation trajectory point, the other participant trajectory point set, the reference object trajectory point set, and the traffic light trajectory point set constitute a corresponding first test frame; and use all the obtained first test frames to form a new version of the test frame sequence to replace the old version of the test frame sequence on the scene cache module; and initialize the loaded frame index stored in the scene cache module to 0;

[0036] And the takeover frame index on the vehicle simulator is set as the current key frame index.

[0037] Preferably, the evaluation module is specifically configured to, when performing a test evaluation based on the current identifier and the test frame sequence from the four dimensions of safety, compliance, comfort, and efficiency to obtain a corresponding evaluation report:

[0038] Step 51: The high-precision map of the region and the key frame index corresponding to the test frame sequence on the scene cache module are used as the corresponding current high-precision map and current key frame index; and a subsequence in the test frame sequence whose test frame index is greater than or equal to the current key frame index is extracted as the corresponding current subsequence; and a corresponding trajectory A is constructed based on all the ego vehicle label trajectory points of the current subsequence; and a corresponding trajectory B is constructed based on all the ego vehicle simulation trajectory points of the current subsequence; and a corresponding trajectory C is constructed based on all the participant trajectory points of each participant identifier of the current subsequence. i , 1≤ participant index i≤N C , N C is the total number of participants; and constructs a corresponding trajectory D based on all the signal light trajectory points of each signal light identifier of the current subsequence k , 1≤semaphore index k≤N K , N K is the total number of signal lights;

[0039] Wherein, the trajectory A includes multiple trajectory points a j , 1≤indexj≤N F , N F is the total number of test frames of the current subsequence; the trajectory B includes N F Trajectory point b j The trajectory C i Including NF Trajectory point c i，j The trajectory D k Including N F Trajectory points d k,j Each of the trajectory points a j , the trajectory point b j , the trajectory point c i，j Each of the trajectory points d k,j corresponding to a set of the signal light coordinates and the signal light status;

[0040] Step 52: Based on the trajectory B and each of the trajectories C i Identify whether the vehicle and other participants have a collision event and obtain the corresponding recognition result r 1,i ; and based on the trajectory B and each of the trajectories C i Identify whether the collision time between the vehicle and other participants is lower than the preset collision time threshold and obtain the corresponding identification result r 2,i ; Based on the trajectories A, B and the current high-precision map, the vehicle is identified as to whether it has left the drivable area to obtain the corresponding recognition result r3; and based on all the recognition results r 1,i , the recognition result r 2,i Perform security scoring on the recognition result r3 to obtain a corresponding score S1;

[0041] Among them, the recognition result r 1,i Including yes, no; the recognition result r 2,i Including yes and no; the recognition result r3 includes yes and no;

[0042] Step 53: according to the track B and each track D k The current high-precision map is used to identify whether the vehicle has run a red light and obtain the corresponding recognition result r 4,k ; Based on the trajectory B and the current high-precision map, identify whether the vehicle has experienced a speeding event to obtain a corresponding recognition result r5; and based on all the recognition results r 4,k Compare the compliance score of the recognition result r5 to obtain the corresponding score S2;

[0043] Among them, the recognition result r 4,k Including yes and no; the recognition result r5 includes yes and no;

[0044] Step 54 : Identify whether a large lateral acceleration event has occurred on the vehicle based on the trajectory B to obtain a corresponding identification result r6 ; and identify whether a sudden braking event has occurred on the vehicle based on the trajectory B to obtain a corresponding identification result r7 ; and perform a comfort score based on the identification results r6 and r7 to obtain a corresponding score S3 ;

[0045] Wherein, the recognition result r6 includes yes and no; the recognition result r7 includes yes and no;

[0046] Step 55, according to the track B, all the tracks C i Using the current high-precision map, the vehicle is identified as having a low-speed driving event to obtain a corresponding recognition result r8; and based on the trajectories A and B, the vehicle is identified as having a navigation deviation event to obtain a corresponding recognition result r9; and efficiency scores are performed based on the recognition results r8 and r9 to obtain a corresponding score S4;

[0047] Wherein, the recognition result r8 includes yes and no; the recognition result r9 includes yes and no;

[0048] Step 56: Comprehensively score the scores S1, S2, S3, and S4 to obtain the corresponding score S. all ; and all the recognition results r 1,i , all the recognition results r 2,i , the recognition result r3, all the recognition results r 4,k , the recognition result r5, the recognition result r6, the recognition result r7, the recognition result r8, the recognition result r9, the score S1, the score S2, the score S3, the score S4 and the score S all Enter the preset evaluation report template to set the report text to obtain the corresponding evaluation report and save it.

[0049] Furthermore, the evaluation module is specifically used to evaluate the performance of the trajectory B and each trajectory C. i Identify whether the vehicle and other participants have a collision event and obtain the corresponding recognition result r 1,i When: each of the trajectories C i As the corresponding current trajectory; and for each of the trajectory points c of the current trajectory i，j The corresponding trajectory point b j The coordinate straight line spacing is calculated to obtain the corresponding trajectory point spacing; and all the obtained trajectory point spacings are identified to see whether they are greater than the preset collision spacing threshold. If so, the corresponding identification result r is set. 1,i If not, set the corresponding recognition result r 1,i For yes;

[0050] The evaluation module is specifically used to compare the trajectory B with the trajectory C. i Identify whether the collision time between the vehicle and other participants is lower than the preset collision time threshold and obtain the corresponding identification result r 2,i When: each of the trajectories C i As the corresponding current trajectory; and for each of the trajectory points c of the current trajectory i，j The corresponding trajectory point b j The collision time of the trajectory point is calculated to obtain the corresponding collision time; and whether the collision time of all the trajectory points obtained is higher than the collision time threshold is identified, and if so, the corresponding identification result r is set. 2,i If not, set the corresponding recognition result r 2,i For yes;

[0051] The evaluation module is specifically configured to, when the corresponding recognition result r3 is obtained by identifying whether the vehicle has left the drivable area based on the trajectories A, B and the current high-precision map: set the road area that the trajectory A passes through on the current high-precision map as a reference area, and set the connected area on the current high-precision map that is connected to the reference area and whose vehicle driving direction is not in a reverse relationship with the vehicle driving direction of the trajectory A as an expansion area, and when the number of the expansion areas is not zero, the reference area and all the expansion areas constitute a drivable area; and identify whether the trajectory B does not exceed the drivable area, and if so, set the corresponding recognition result r3 to no, and otherwise set the corresponding recognition result r3 to yes;

[0052] The evaluation module is specifically used to evaluate the trajectory B and each trajectory D. k The current high-precision map is used to identify whether the vehicle has run a red light and obtain the corresponding recognition result r 4,k When: each of the trajectories D k As the corresponding current trajectory; and the intersection range controlled by the traffic light corresponding to the current trajectory on the current high-precision map as the corresponding current intersection; and the trajectory point d where the traffic light state is red in the current trajectory k,j The corresponding test frame timestamp is recorded as the red light timestamp; and the track point b on the track B corresponding to the red light timestamp is recorded as the red light timestamp. j The coordinates of the track point are located in the current intersection for identification, if so, the corresponding track point running light state is set to yes, if not, the corresponding track point running light state is set to no; and all the track points corresponding to the current track are identified as no, if so, the corresponding identification result r is set. 4,kIf not, set the corresponding recognition result r 4,k For yes;

[0053] The evaluation module is specifically used to: when the corresponding identification result r5 is obtained by identifying whether the vehicle has a speeding event based on the trajectory B and the current high-precision map: j as the corresponding current trajectory point; and based on the coordinates of the current trajectory point and the road section / traffic intersection sign, query the current high-precision map to obtain the maximum driving speed corresponding to the current coordinates; and identify whether the speed of the current trajectory point exceeds the maximum driving speed, and if so, set the corresponding trajectory point speeding status to yes; otherwise, set the corresponding trajectory point speeding status to no; and identify whether the speeding status of all the trajectory points corresponding to the current trajectory is no, and if so, set the corresponding identification result r5 to no, otherwise, set the corresponding identification result r5 to yes;

[0054] The evaluation module is specifically used to: when the corresponding identification result r6 is obtained by identifying whether the vehicle has a large lateral acceleration event according to the trajectory B: j as the corresponding current trajectory point; and calculating the corresponding trajectory point lateral acceleration based on the coordinates, angle and acceleration of the current trajectory point; and querying a preset speed-lateral acceleration correspondence table based on the speed of the current trajectory point to obtain a corresponding lateral acceleration range; and identifying whether the lateral acceleration of the trajectory point exceeds the lateral acceleration range, if not, setting the corresponding trajectory point state to normal; if exceeded, setting the corresponding trajectory point state to abnormal; and identifying whether all the obtained trajectory point states are normal, if so, setting the corresponding identification result r6 to no, otherwise setting the corresponding identification result r6 to yes; the speed-lateral acceleration correspondence table includes multiple correspondence records; the correspondence record includes a speed range field and a lateral acceleration range field;

[0055] The evaluation module is specifically used to: when the corresponding identification result r7 is obtained by identifying whether the vehicle has a sudden braking event according to the trajectory B: j As the corresponding current track point; and the previous track point b of the current track point jas the corresponding previous trajectory point; and calculate the corresponding deceleration change rate with the acceleration of the previous trajectory point and the current trajectory point; and identify the acceleration and the deceleration change rate of the current trajectory point, if the current acceleration is less than a preset first acceleration threshold and the current deceleration change rate is less than a preset change rate threshold, then set the corresponding trajectory point emergency braking state to yes; if the current acceleration is greater than or equal to the first acceleration threshold, or the current deceleration change rate is greater than or equal to the change rate threshold, then set the corresponding trajectory point emergency braking state to no; and identify whether the emergency braking states of all the obtained trajectory points are no, if so, set the corresponding identification result r7 to no, otherwise set the corresponding identification result r7 to yes;

[0056] The evaluation module is specifically used to evaluate the trajectory B and all the trajectories C. i When the current high-precision map is used to identify whether the vehicle has a low-speed driving event and obtain the corresponding recognition result r8: each of the track points b of the track B is j as the corresponding current trajectory point; and the driving lane corresponding to the coordinates of the current trajectory point and the road section / traffic intersection sign on the current high-precision map is used as the corresponding current lane; and the same-direction lane adjacent to the current lane on the current high-precision map is used as the corresponding adjacent lane; and all the trajectory C i The trajectory point c whose timestamp corresponds to the current trajectory point and is in any of the adjacent lanes i，j Form a corresponding current trajectory point set; and all trajectory points c of the current trajectory point set i，j The average speed is calculated to obtain the corresponding current average speed; the product of the current average speed and a preset percentage parameter is used as the corresponding current speed threshold; and whether the speed of the current track point is lower than the current speed threshold is identified, and if so, the low speed state of the corresponding track point is set to yes; otherwise, the low speed state of the corresponding track point is set to no; and whether the low speed states of all the obtained track points are no is identified, and if so, the corresponding identification result r8 is set to no; otherwise, the corresponding identification result r8 is set to yes;

[0057] The evaluation module is specifically used for: when the corresponding identification result r9 is obtained by identifying whether the navigation deviation event occurs to the vehicle according to the trajectories A and B: j 、b jThe coordinate straight line spacing is calculated to obtain the corresponding ab track point spacing; and whether all the obtained ab track point spacings are less than the preset yaw spacing threshold is identified, if so, the corresponding identification result r9 is set to no, otherwise the corresponding identification result r9 is set to yes.

[0058] Preferably, the scene player is specifically configured to, when performing a round of frame-by-frame loading and playback according to the test frame sequence:

[0059] Perform a round of sequential traversal on all the first test frames of the test frame sequence of the scene cache module; and in this round of traversal, take the first test frame currently traversed as the corresponding current frame, and record the first test frame next to the current frame in the test frame sequence as the corresponding next frame, and when the next frame is not empty, confirm the time interval between the current frame and the next frame to obtain the corresponding next frame interval; and use the current time as the corresponding start time; and set the loading frame index of the scene cache module to the test frame index of the current frame; and form the corresponding current loading frame by the test frame timestamp of the current frame, the self-vehicle simulation trajectory point, the other participant trajectory point set, the reference object trajectory point set and the traffic light trajectory point set; and perform based on the current loading frame A two-dimensional or three-dimensional traffic scene is constructed, and the currently constructed scene is played; and when the next frame is not empty, the test frame timestamp, the self-vehicle simulation trajectory point, the other participant trajectory point set, the reference object trajectory point set and the traffic light trajectory point set corresponding to the currently loaded frame are brought into the perception information data template corresponding to the regulation and control module for data setting to obtain the corresponding current perception information and send it to the regulation and control module; and when the next frame is not empty, an independent timing is started from the current start time, and when the current timing duration matches the next frame interval, the current timing is stopped and the next first test frame is continued to traverse until the last first test frame traversal is completed; and at the end of this round of traversal, the playback completion status is sent to the evaluation module, and a test end instruction is sent to the regulation and control module.

[0060] Preferably, the ego vehicle simulator is specifically configured to, when performing a ego vehicle motion state prediction based on the loaded frame index and the current ego vehicle control instruction:

[0061] The test frame sequence and the loading frame index are obtained from the scene cache module; the high-precision map of the area corresponding to the test frame sequence is used as the corresponding current high-precision map; the first test frame corresponding to the current loading frame in the test frame sequence is used as the current frame, and the first test frame next to the current frame is used as the corresponding next frame; the time interval between the current frame and the next frame is used as the corresponding current prediction duration; the time point corresponding to the next frame is used as the corresponding current target time; the self-vehicle simulation coordinates, the self-vehicle simulation heading angle, the self-vehicle simulation speed, and the self-vehicle simulation acceleration of the self-vehicle simulation trajectory point in the current frame form a starting motion state; and the instruction parameters of the current self-vehicle control instruction are used as the corresponding current instruction parameters;

[0062] and identifying whether the current loading frame index is smaller than the takeover frame index stored locally in the simulator;

[0063] If the current loading frame index is less than the takeover frame index, a corresponding tag motion state is formed by the ego vehicle tag coordinates, the ego vehicle tag heading angle, the ego vehicle tag speed, and the ego vehicle tag acceleration of the ego vehicle tag trajectory point in the next frame; and the ego vehicle dynamics model built into the simulator predicts the coordinates, heading angle, speed, and acceleration of the ego vehicle at the current target moment based on the starting motion state, the current instruction parameters, and the current prediction duration, and the coordinates, heading angle, speed, and acceleration obtained in this prediction form a corresponding predicted motion state; and the model parameters of the ego vehicle dynamics model are optimized in a direction in which the differential state quantity of the predicted and tag motion states reaches a minimum value; the ego vehicle dynamics model is a mathematical model implemented based on a Kalman filter, or an intelligent model implemented based on a machine learning model or a deep learning model;

[0064] If the current loading frame index is greater than or equal to the takeover frame index, the vehicle dynamics model predicts the coordinates, heading angle, speed, and acceleration of the vehicle at the current target moment based on the starting motion state, the current instruction parameters, and the current prediction duration, and uses the coordinates, heading angle, speed, and acceleration obtained in this prediction as the corresponding current predicted coordinates, current predicted heading angle, current predicted speed, and current predicted acceleration; and uses the road section / traffic intersection identification corresponding to the current predicted coordinates in the current high-precision map as the corresponding current road section / traffic intersection identification; and uses the vehicle simulation road section / traffic intersection identification, the vehicle simulation coordinates, the vehicle simulation heading angle, the vehicle simulation speed, and the vehicle simulation acceleration of the vehicle simulation trajectory point in the next frame as the corresponding current road section / traffic intersection identification, the current predicted coordinates, the current predicted heading angle, the current predicted speed, and the current predicted acceleration.

[0065] An embodiment of the present invention provides an autonomous driving evaluation system based on a simulation scenario. As can be seen from the above content, the system includes: an acquisition module, a scenario server, an evaluation module, a scenario cache module, a scenario player, and a self-vehicle simulator; wherein the acquisition module is used to convert the real road condition data stored on the road network database into a test scenario and store it in the scenario server; the evaluation module is used to select a test scenario from the scenario server to set the test frame sequence, the loading frame index on the scene cache module, and the takeover frame index on the self-vehicle simulator, and perform simulation according to the test frame sequence by calling the scenario player; the scenario player sends the corresponding test start / end instructions to the control module at the start and end of each round of simulation process, and also sends the playback completion status to the evaluation module at the end time; each round of simulation process of the scenario player is A round of frame-by-frame loading and playback process; during each round of simulation, the scene player will generate corresponding current perception information based on the current loaded frame and send it to the regulation and control module as long as the current loaded frame is not the last frame; the self-vehicle simulator is used to predict the self-vehicle motion state based on the loaded frame index and the current self-vehicle control instruction every time it receives a self-vehicle control instruction sent by the regulation and control module, and in each prediction process, if the loaded frame index is greater than or equal to the takeover frame index, the self-vehicle state will be refreshed for the next frame corresponding to the loaded frame index in the test frame sequence based on the current prediction result; the evaluation module is also used to evaluate this test from the four dimensions of safety, compliance, comfort and efficiency after receiving the playback completion status to obtain a corresponding evaluation report. On the one hand, the embodiments of the present invention can mine massive test scenarios from a large amount of real road condition data, thereby improving the richness and complexity of the test scenarios; on the other hand, the simulation test method based on the scenario player + vehicle simulator improves the utilization rate of the test scenarios and the test efficiency of the regulation and control module; on the other hand, the richness and diversity of the evaluation methods are improved through the four-dimensional evaluation (safety, compliance, comfort and efficiency) mechanism; on the other hand, the verification adequacy of the regulation and control module is effectively improved in the early automatic testing stage and the later road testing costs are reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 A module structure diagram of an autonomous driving evaluation system based on simulation scenarios provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0067] To make the objectives, technical solutions, and advantages of the present invention more apparent, the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the embodiments described herein are merely some, rather than all, of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are intended to fall within the scope of protection of the present invention.

[0068] An embodiment of the present invention provides an autonomous driving evaluation system 1 based on a simulation scenario, such as Figure 1 As shown in the module structure diagram of an autonomous driving evaluation system based on a simulation scenario provided in an embodiment of the present invention, it mainly includes: an acquisition module 11, a scene server 12, an evaluation module 13, a scene cache module 14, a scene player 15, and a self-vehicle simulator 16.

[0069] The connection relationship between the modules of the system of the embodiment of the present invention is as follows: the acquisition module 11 is connected to the road network database 2 and the scenario server 12 respectively; the evaluation module 13 is connected to the scenario server 12, the scenario cache module 14, the scenario player 15 and the vehicle simulator 16 respectively; the scenario cache module 14 is connected to the scenario player 15 and the vehicle simulator 16 respectively; the scenario player 15 and the vehicle simulator 16 are respectively connected to a regulation and control module 3 under test.

[0070] (1) Acquisition module 11:

[0071] The acquisition module 11 in the embodiment of the present invention is used to convert the real road condition data stored in the road network database 2 into a test scenario and store it in the scenario server 12 .

[0072] Here, the road network database 2 of the embodiment of the present invention is used to store a plurality of real road condition data in a real road network.

[0073] The real traffic data of the embodiment of the present invention includes a participant dataset, a reference object dataset, a traffic light dataset, and a high-precision map of the area; wherein:

[0074] 1) The participant dataset includes multiple participant data; the participant data includes participant identification, participant type, and participant trajectory; participant types include vehicles, motorcycles, bicycles, pedestrians, and animals; the participant trajectory consists of multiple first trajectory points; the first trajectory point includes timestamp, road section / traffic intersection identification, coordinates, heading angle, angular velocity, velocity, and acceleration;

[0075] 2) The reference object data set includes multiple reference object data; the reference object data includes reference object identification, reference object type, and reference object coordinates; the reference object type includes at least buildings, poles, plants, traffic obstacles, and other obstacles;

[0076] 3) When the signal light dataset is not empty, it consists of one or more signal light data; the signal light data includes signal light identification, signal light type, signal light coordinates, and signal light trajectory; signal light types include at least motor vehicle signal lights, lane signal lights, and direction indicator signal lights; the signal light trajectory consists of multiple second trajectory points; the second trajectory points include timestamps and signal light states; the signal light states include at least red light, green light, yellow light, and flashing yellow light;

[0077] 4) Regional HD map is a global HD map of the road section or intersection area corresponding to the current traffic data, including map element information of all road elements and non-road elements in the area.

[0078] In a specific implementation of the embodiment of the present invention, the acquisition module 11 is specifically configured to convert the real road condition data stored in the road network database 2 into a test scenario and store it in the scenario server 12:

[0079] Step A1: using the real road condition data on the scene server 12 whose time span exceeds the preset scene duration L as the corresponding current road condition data;

[0080] Here, the scene duration L is a preset time length, for example, 10 minutes;

[0081] Step A2: The earliest and latest timestamps in the current traffic condition data are used as the corresponding current start time and current end time; the current start time is converted to the corresponding start time zero; the current end time is converted to the corresponding end time relative to the start time zero; the time period from the start time zero to the end time is used as the corresponding current traffic condition period; and the preset standard interval △t std The single-step interval is a time point sampling of the current traffic period to obtain multiple corresponding single-step time points; the time interval between each two adjacent single-step time points is the standard interval △t std ;

[0082] Here, the standard interval △t std is a preset time interval parameter, the standard interval △t std Much smaller than the scene duration L, and its time unit is usually in milliseconds;

[0083] Step A3, and use the trajectories of each participant in the current road condition data as the corresponding current trajectory; and use the sub-period corresponding to the current trajectory in the current road condition period as the current trajectory period; and use the coordinates, heading angle, angular velocity, speed, and acceleration of all first trajectory points in the current trajectory as observation quantities, and construct the corresponding Kalman filter equation according to the preset vehicle kinematic model; and based on the current Kalman filter equation, predict the coordinates, heading angle, angular velocity, speed, and acceleration of the current participant at each single-step time point in the current trajectory period; and reconstruct the current trajectory based on the predicted data; in each reconstructed participant trajectory, the timestamp of each first trajectory point corresponds to a single-step time point, and the coordinates, heading angle, angular velocity, speed, and acceleration of each first trajectory point are the coordinates, heading angle, angular velocity, speed, and acceleration state prediction quantities corresponding to the timestamp of the current trajectory point, and the road section / traffic intersection identifier of each first trajectory point is the road section identifier or traffic intersection identifier corresponding to the current coordinate prediction quantity in the regional high-precision map;

[0084] Step A4: Calculate the travel distance of each reconstructed participant trajectory to obtain a corresponding first distance; calculate the trajectory time of each reconstructed participant trajectory to obtain a corresponding first duration; record the participant trajectory whose first distance exceeds a preset first distance threshold and whose first duration exceeds a preset first duration threshold as a corresponding candidate trajectory, and select one of all the candidate trajectories as the corresponding self-vehicle trajectory; and use the participant data corresponding to the self-vehicle trajectory in the participant data set of the current road condition data as the corresponding self-vehicle data, and use the other participant data as the corresponding other participant data;

[0085] Here, the first distance threshold is a preset distance threshold parameter; the first duration threshold is a preset time duration threshold parameter;

[0086] In step A5, when the signal light dataset of the current road condition data is not empty, each signal light trajectory of the current signal light dataset is used as the corresponding current trajectory; and the sub-period corresponding to the current trajectory in the current road condition period is used as the current trajectory period; and each single-step time point in the current trajectory period is recorded as the corresponding current time point, and the signal light state corresponding to the current time point is set according to the signal light state of the previous and / or next second trajectory point in the current trajectory according to the interpolation method, and the current time point and the corresponding signal light state form a corresponding reconstructed trajectory point; and a new signal light trajectory is formed by all the reconstructed trajectory points corresponding to the current trajectory period to replace the current trajectory; in each reconstructed signal light trajectory, the timestamp of each second trajectory point corresponds to a single-step time point;

[0087] Step A6: Using the scene duration L as the segment length, the current traffic period is segmented according to the preset single-step sliding step length to obtain corresponding multiple segmented time periods;

[0088] Here, the single-step sliding step size is the standard interval △t std an integer multiple of ;

[0089] Step A7, and use each segment time period as the corresponding current segment; and assign a corresponding frame index to each single-step time point of the current segment in sequence starting from 1 and adding 1 point by point; and use the single-step time point corresponding to each frame index as a corresponding frame timestamp; and use the road section / traffic intersection identification, coordinates, heading angle, angular velocity, speed, and acceleration of the first trajectory point corresponding to each frame timestamp of the current segment in the vehicle trajectory as a group of corresponding vehicle road section / traffic intersection identification, vehicle coordinates, vehicle heading angle, vehicle speed, and vehicle acceleration to form a corresponding vehicle trajectory point; and use the road section / traffic intersection identification, coordinates, heading angle, angular velocity, speed, and acceleration of the first trajectory point corresponding to each frame timestamp of the current segment in the participant trajectory of each other participant data as a group of corresponding participant road section / traffic intersection identification, participant coordinates, participant heading angle, participant speed, and participant acceleration; and use the participant identification, participant type, participant road section / traffic intersection identification, participant coordinates, participant heading angle, participant speed, and participant acceleration corresponding to each other participant data to form a corresponding participant Trajectory points; and all participant trajectory points corresponding to each frame timestamp of the current segment form a corresponding other participant trajectory point set; and each reference object data of the current road condition data is used as a corresponding reference object trajectory point, and all reference object trajectory points form a reference object trajectory point set corresponding to each frame timestamp in the current segment; and the signal light identification, signal light type, signal light coordinates and signal light state corresponding to the second trajectory point corresponding to each frame timestamp of the current segment in each signal light trajectory form a corresponding signal light trajectory point; and all signal light trajectory points corresponding to each frame timestamp of the current segment form a corresponding signal light trajectory point set; and each frame index of the current segment and its corresponding frame timestamp, the vehicle trajectory point, the other participant trajectory point set, the reference object trajectory point set, and the signal light trajectory point set form a corresponding first frame scene; and all the first frame scenes corresponding to the current segment are sorted in chronological order to form a corresponding scene frame sequence; and the time period center point of the current segment is used as the corresponding current time period center point, and the frame index corresponding to the frame timestamp closest to the current time period center point in the scene frame sequence corresponding to the current segment is used as the corresponding key frame index;

[0090] Step A8, and assigning a unique identifier to each scene frame sequence as a corresponding scene identifier;

[0091] Step A9: Using the regional high-precision map of the current road condition data as the scene high-precision map corresponding to each scene frame sequence;

[0092] Step A10, forming a corresponding self-vehicle driving trajectory from the plurality of self-vehicle trajectory points in each scene frame sequence; querying a regional high-precision map of current road condition data based on the self-vehicle road segment / traffic intersection identifier of each self-vehicle driving trajectory; and confirming based on the query result whether the road segment / traffic intersection covered by the current self-vehicle driving trajectory is a straight road, a curve, a diverging section, a merging section, an incoming section, an outgoing section, a cross intersection, or a T-intersection, and setting a corresponding scene type based on the confirmation result;

[0093] Step A11: Count the total number of participant identifiers and the total number of participant types in each scene frame sequence to obtain a corresponding first total number and a second total number, normalize the first and second total numbers to obtain a corresponding first normalized total number and second normalized total number, perform a weighted sum on the first and second normalized total numbers, and use the sum result as the corresponding scene complexity;

[0094] In step A12, each scene frame sequence and its corresponding scene identifier, scene type, scene complexity, scene high-precision map and key frame index form a corresponding test scene and store it in the scene server 12.

[0095] (2) Scene Server 12:

[0096] The scenario server 12 in the embodiment of the present invention is used to store multiple test scenarios.

[0097] The test scenario of the embodiment of the present invention includes a scene identifier, a scene type, a scene complexity, a scene high-precision map, a scene frame sequence, and a key frame index; wherein:

[0098] 1) Each test scenario has a unique identifier, namely the scenario identifier;

[0099] 2) Scenario types include straight roads, curves, diverging sections, merging sections, merging sections, merging sections, cross intersections, and T-intersections;

[0100] 3) Scenario complexity is used to quantify the complexity of the current test scenario. As mentioned above, this parameter is calculated based on the number of participants and the number of participant types in the scenario.

[0101] 4) The scene frame sequence consists of multiple first-frame scenes. The time interval between every two adjacent frames in the scene frame sequence is the standard interval △t std , the time span of the scene frame sequence is the scene duration L;

[0102] The first frame scene includes a frame index, a frame timestamp, a self-vehicle trajectory point, a set of other participant trajectory points, a set of reference object trajectory points, and a set of signal light trajectory points; wherein, a) the self-vehicle trajectory point includes the self-vehicle coordinates, the self-vehicle road section / traffic intersection identification, the self-vehicle heading angle, the self-vehicle speed, and the self-vehicle acceleration; b) the set of other participant trajectory points includes multiple participant trajectory points, and the participant trajectory points include the participant identification, participant type, participant coordinates, the participant road section / traffic intersection identification, participant heading angle, participant speed, and participant acceleration; c) the set of reference object trajectory points includes multiple reference object trajectory points, and the reference object trajectory points include the reference object identification, reference object type, and reference object coordinates; d) when the signal light trajectory point set is not empty, it is composed of one or more signal light trajectory points, and the signal light trajectory points include the signal light identification, signal light type, signal light coordinates, and signal light status.

[0103] (III) Assessment Module 13:

[0104] The evaluation module 13 of the embodiment of the present invention is used to receive batch tasks input by the user; and use the various test scene identifiers of the current batch task as the corresponding current identifier in turn; and select a matching scene from the scene server 12 based on the current identifier to initialize the test frame sequence, loading frame index and takeover frame index on the scene cache module 14 and the self-vehicle simulator 16; and send a play instruction to the scene player 15 at the end of this initialization; and when the playback completion status is fed back by the scene player 15, perform a test evaluation based on the current identifier and the test frame sequence from the four dimensions of safety, compliance, comfort and efficiency to obtain a corresponding evaluation report; and feedback all the obtained evaluation reports to the current user. Here, the batch task includes multiple test scene identifiers.

[0105] In another specific implementation of the embodiment of the present invention, the evaluation module 13 is specifically configured to select a matching scene from the scene server 12 based on the current identifier to initialize the test frame sequence, the loaded frame index on the scene cache module 14, and the takeover frame index on the vehicle simulator 16:

[0106] Step B1: using the test scene whose scene identifier on the scene server 12 matches the current identifier as the corresponding matching scene; and using the scene frame sequence and key frame index of the matching scene as the corresponding current scene frame sequence and current key frame index;

[0107] Step B2, and use the frame index and frame timestamp of each first frame scene of the current scene frame sequence as the corresponding test frame index and test frame timestamp; and use the ego vehicle trajectory point of each first frame scene of the current scene frame sequence as the corresponding ego vehicle label trajectory point and ego vehicle simulation trajectory point; and each test frame index and its corresponding test frame timestamp, ego vehicle label trajectory point, ego vehicle simulation trajectory point, other participant trajectory point set, reference object trajectory point set, and traffic light trajectory point set form a corresponding first test frame; and use all the obtained first test frames to form a new version of the test frame sequence to replace the old version of the test frame sequence on the scene cache module 14; and initialize the loaded frame index stored in the scene cache module 14 to 0;

[0108] Step B3, and set the takeover frame index on the vehicle simulator 16 as the current key frame index.

[0109] In another specific implementation of the embodiment of the present invention, the evaluation module 13 is specifically configured to perform a test evaluation based on the current identifier and the test frame sequence from the four dimensions of safety, compliance, comfort, and efficiency to obtain a corresponding evaluation report:

[0110] Step C1: The high-precision map and key frame index of the region corresponding to the test frame sequence on the scene cache module 14 are used as the corresponding current high-precision map and current key frame index; and the subsequence with a test frame index greater than or equal to the current key frame index in the test frame sequence is extracted as the corresponding current subsequence; and the corresponding trajectory A is constructed based on all the ego vehicle label trajectory points of the current subsequence; and the corresponding trajectory B is constructed based on all the ego vehicle simulation trajectory points of the current subsequence; and the corresponding trajectory C is constructed based on all the participant trajectory points of each participant identifier of the current subsequence. i , 1≤ participant index i≤N C , N C is the total number of participants; and the corresponding trajectory D is constructed based on all the signal light trajectory points of each signal light identification in the current subsequence k , 1≤semaphore index k≤N K , N K is the total number of signal lights;

[0111] Among them, trajectory A includes multiple trajectory points a j , 1≤indexj≤N F , N F is the total number of test frames in the current subsequence; track B includes N F Trajectory point b j ; Track C i Including N F Trajectory point c i，j ; Track D k Including N F Trajectory points dk,j ; Each trajectory point a j , trajectory point b j , trajectory point c i，j Each trajectory point d k,j Corresponding to a set of signal light coordinates and signal light status;

[0112] Step C2, based on trajectory B and each trajectory C i Identify whether the vehicle and other participants have a collision event and obtain the corresponding recognition result r 1,i ; and based on trajectory B and each trajectory C i Identify whether the collision time between the vehicle and other participants is lower than the preset collision time threshold and obtain the corresponding identification result r 2,i ; Based on the trajectories A, B and the current high-precision map, the vehicle is identified to determine whether it has left the drivable area and obtain the corresponding recognition result r3; and based on all the recognition results r 1,i , recognition result r 2,i Perform security scoring on the recognition result r3 to obtain the corresponding score S1;

[0113] The collision time threshold is a pre-set time length parameter; the recognition result r 1,i Including yes, no; recognition result r 2,i Including yes and no; recognition result r3 includes yes and no;

[0114] Here, the present invention provides three types of evaluation factors for safety evaluation: collision evaluation factor, time to collision (TTC) evaluation factor, and drivable area evaluation factor; which correspond to three types of recognition results respectively: 1,i 、r 2,i , r3; Based on the identification results of these three types of evaluation factors, the corresponding security score S1 can be obtained by performing score conversion and summing up; when performing score conversion, the conversion rules can be customized based on application requirements, for example: r 1,i If it is “No”, the corresponding score is s 1,i =1, r 1,i If "yes", s 1,i =0, r 2,i If it is “No”, the corresponding score is s 2,i =1, r 2,i If "yes", s 2,i =0, r3 is "no" then the corresponding score s3 = 1, r3 is "yes" then s3 = 0; when totaling the scores, the totaling rules can be customized based on the application requirements, for example: if there is an s 1,i =0, then the corresponding score s1 = 0, if all s 1,i=1 then s1 = 1, if there is an s 2,i =0, then the corresponding score s2 = 0, if all s 2,i =1, then s2=1, S1=s1+s2+s3;

[0115] In another specific implementation of the embodiment of the present invention, the evaluation module 13 is specifically configured to evaluate the trajectory B and each trajectory C. i Identify whether the vehicle and other participants have a collision event and obtain the corresponding recognition result r 1,i When: Each track C i As the corresponding current trajectory; and for each trajectory point c of the current trajectory i，j The corresponding trajectory point b j The coordinate straight line spacing is calculated to obtain the corresponding trajectory point spacing; and all the obtained trajectory point spacings are identified to see whether they are greater than the preset collision spacing threshold. If so, the corresponding recognition result r is set. 1,i If not, set the corresponding recognition result r 1,i For yes;

[0116] Here, the collision distance threshold is a pre-set distance threshold parameter;

[0117] In another specific implementation of the embodiment of the present invention, the evaluation module 13 is specifically configured to evaluate the trajectory B and each trajectory C. i Identify whether the collision time between the vehicle and other participants is lower than the collision time threshold and obtain the corresponding recognition result r 2,i When: Each track C i As the corresponding current trajectory; and for each trajectory point c of the current trajectory i，j The corresponding trajectory point b j The collision time of the corresponding trajectory point is calculated; and the collision time of all the obtained trajectory points is identified to see whether it is higher than the collision time threshold. If so, the corresponding identification result r is set. 2,i If not, set the corresponding recognition result r 2,i For yes;

[0118] In another specific implementation of the embodiment of the present invention, the evaluation module 13 is specifically configured to, when obtaining a corresponding recognition result r3 for identifying whether the vehicle has exited the drivable area based on the trajectories A, B, and the current high-precision map: set the road area that trajectory A passes through on the current high-precision map as a reference area, and set the connected area on the current high-precision map that is connected to the reference area and whose vehicle driving direction is not in a reverse relationship with the vehicle driving direction of trajectory A as an expansion area, and when the number of expansion areas is not zero, the reference area and all the expansion areas constitute a drivable area; and identify whether trajectory B does not exceed the drivable area, and if so, set the corresponding recognition result r3 to no, and if not, set the corresponding recognition result r3 to yes;

[0119] Step C3, according to track B, each track D k The current high-precision map is used to identify whether the vehicle has run a red light and obtain the corresponding recognition result r 4,k ; Based on the trajectory B and the current high-precision map, identify whether the vehicle has experienced a speeding event and obtain the corresponding recognition result r5; and based on all the recognition results r 4,k The compliance score of the recognition result r5 is obtained by the corresponding score S2;

[0120] Among them, the recognition result r 4,k Including yes and no; recognition result r5 includes yes and no;

[0121] Here, the present invention provides two types of evaluation factors for compliance evaluation: red light running evaluation factor and speeding evaluation factor, which correspond to two types of recognition results respectively: r 4,k , r5; Based on the identification results of these two types of evaluation factors, the corresponding security score S2 can be obtained by performing score conversion and summing up; when performing score conversion, the conversion rules can be customized based on application requirements, for example: r 4,k If it is “No”, the corresponding score is s 4,k =1, r 4,k If "yes", s 4,k =0, r5 is "no" then the corresponding score s5 = 1, r5 is "yes" then s5 = 0; when totaling the scores, the totaling rules can be customized based on the application requirements, for example: if there is an s 4,k =0, then the corresponding score is s4 = 0. If all s 4,k =1 then s4=1, S2=s4+s5;

[0122] In another specific implementation of the embodiment of the present invention, the evaluation module 13 is specifically configured to evaluate the trajectory B and each trajectory D. k The current high-precision map is used to identify whether the vehicle has run a red light and obtain the corresponding recognition result r 4,k When: Each track D kAs the corresponding current trajectory; and the intersection range controlled by the traffic light corresponding to the current trajectory on the current high-precision map as the corresponding current intersection; and the trajectory point d where the traffic light state is red in the current trajectory k,j The corresponding test frame timestamp is recorded as the red light timestamp; and the trajectory point b on trajectory B corresponding to the red light timestamp is recorded as the red light timestamp. j The coordinates of the current track point are identified. If yes, the corresponding track point running light status is set to yes. If not, the corresponding track point running light status is set to no. And all track points corresponding to the current track are identified as no. If yes, the corresponding recognition result r is set. 4,k If not, set the corresponding recognition result r 4,k For yes;

[0123] In another specific implementation of the embodiment of the present invention, the evaluation module 13 is specifically configured to: when the corresponding identification result r5 is obtained by identifying whether the vehicle has experienced a speeding event based on the trajectory B and the current high-precision map: j as the corresponding current trajectory point; and based on the coordinates of the current trajectory point and the road section / traffic intersection sign, query the current high-precision map to obtain the maximum driving speed corresponding to the current coordinates; and identify whether the speed of the current trajectory point exceeds the maximum driving speed, if so, set the corresponding trajectory point speeding status to yes, otherwise set the corresponding trajectory point speeding status to no; and identify whether the speeding status of all trajectory points corresponding to the current trajectory is no, if so, set the corresponding identification result r5 to no, otherwise set the corresponding identification result r5 to yes;

[0124] Step C4: Identify whether a large lateral acceleration event has occurred on the vehicle based on trajectory B to obtain a corresponding identification result r6; and identify whether a sudden braking event has occurred on the vehicle based on trajectory B to obtain a corresponding identification result r7; and perform a comfort score based on the identification results r6 and r7 to obtain a corresponding score S3;

[0125] Among them, the recognition result r6 includes yes and no; the recognition result r7 includes yes and no;

[0126] Here, the present invention provides two types of evaluation factors for comfort evaluation: a large lateral acceleration evaluation factor and an emergency braking evaluation factor; each corresponds to two types of recognition results: r6 and r7; based on the recognition results of these two types of evaluation factors, score conversion and summation can be performed to obtain the corresponding safety score S3; when performing score conversion, the conversion rules can be customized based on application requirements, for example: if r6 is "no", the corresponding score is s6=1, if r6 is "yes", then s6=0; if r7 is "no", the corresponding score is s7=1, if r7 is "yes", then s7=0; when performing score summation, the summation rules can be customized based on application requirements, for example: S3=s6+s7;

[0127] In another specific implementation of the embodiment of the present invention, the evaluation module 13 is specifically configured to: when identifying whether a large lateral acceleration event occurs on the vehicle according to the trajectory B and obtaining a corresponding identification result r6: j as the corresponding current trajectory point; and calculate the corresponding trajectory point lateral acceleration based on the coordinates, angle and acceleration of the current trajectory point; and query the preset speed-lateral acceleration correspondence table based on the speed of the current trajectory point to obtain a corresponding lateral acceleration range; and identify whether the lateral acceleration of the trajectory point exceeds the lateral acceleration range, if not, set the corresponding trajectory point state to normal, if exceeded, set the corresponding trajectory point state to abnormal; and identify whether all the obtained trajectory point states are normal, if so, set the corresponding identification result r6 to no, otherwise set the corresponding identification result r6 to yes;

[0128] Here, the speed-lateral acceleration correspondence table of the embodiment of the present invention includes a plurality of correspondence records; each correspondence record includes a speed range field and a lateral acceleration range field;

[0129] In another specific implementation of the embodiment of the present invention, the evaluation module 13 is specifically configured to: when the corresponding identification result r7 is obtained by identifying whether the vehicle has a sudden braking event according to the trajectory B: j As the corresponding current track point in turn; and the previous track point b of the current track point j as the corresponding previous trajectory point; and calculate the corresponding deceleration change rate with the acceleration of the previous trajectory point and the current trajectory point; and identify the acceleration and deceleration change rate of the current trajectory point, if the current acceleration is less than the preset first acceleration threshold and the current deceleration change rate is less than the preset change rate threshold, then set the corresponding trajectory point emergency braking state to yes; if the current acceleration is greater than or equal to the first acceleration threshold, or the current deceleration change rate is greater than or equal to the change rate threshold, then set the corresponding trajectory point emergency braking state to no; and identify whether the emergency braking states of all the obtained trajectory points are no, if so, set the corresponding recognition result r7 to no, otherwise set the corresponding recognition result r7 to yes;

[0130] Here, the first acceleration threshold is a preset acceleration threshold parameter with a negative value, and the change rate threshold is a preset jerk with a negative value, that is, the acceleration change rate threshold parameter;

[0131] Step C5, based on track B and all tracks C iThe current high-precision map is used to identify whether the vehicle has experienced a low-speed driving event, thereby obtaining a corresponding recognition result r8. Furthermore, based on trajectories A and B, the vehicle is identified as having experienced a navigation deviation event, thereby obtaining a corresponding recognition result r9. Furthermore, efficiency scores are performed based on the recognition results r8 and r9, thereby obtaining a corresponding score S4.

[0132] Among them, the recognition result r8 includes yes and no; the recognition result r9 includes yes and no;

[0133] Here, the present invention provides two types of evaluation factors for efficiency evaluation: a low-speed evaluation factor and a navigation deviation evaluation factor; each corresponds to two types of recognition results: r8 and r9; the corresponding safety score S4 can be obtained by performing score conversion and summing based on the recognition results of these two types of evaluation factors; when performing score conversion, the conversion rules can be customized based on application requirements, for example: if r8 is "no", the corresponding score is s8=1, if r8 is "yes", then s8=0; if r9 is "no", the corresponding score is s9=1, if r9 is "yes", then s9=0; when performing score summing, the summing rules can be customized based on application requirements, for example: S4=s8+s9;

[0134] In another specific implementation of the embodiment of the present invention, the evaluation module 13 is specifically configured to evaluate the trajectory B and all trajectories C. i When the current high-precision map identifies whether the vehicle has a low-speed driving event and obtains the corresponding recognition result r8: each track point b of track B j as the corresponding current trajectory point; and the driving lane corresponding to the coordinates of the current trajectory point and the road section / traffic intersection sign on the current high-precision map is taken as the corresponding current lane; and the same-direction lane adjacent to the current lane on the current high-precision map is taken as the corresponding adjacent lane; and all trajectory C i The trajectory point c whose timestamp corresponds to the current trajectory point and is in any adjacent lane i，j Form the corresponding current trajectory point set; and for all trajectory points c in the current trajectory point set i，j The average speed is calculated to obtain the corresponding current average speed; the product of the current average speed and a preset percentage parameter is used as the corresponding current speed threshold; and whether the speed of the current track point is lower than the current speed threshold is identified, if so, the low speed state of the corresponding track point is set to yes, otherwise, the low speed state of the corresponding track point is set to no; and whether the low speed state of all the obtained track points is no is identified, if so, the corresponding identification result r8 is set to no, otherwise, the corresponding identification result r8 is set to yes;

[0135] Here, the percentage parameter is a preset percentage value between 0 and 1;

[0136] In another specific implementation of the embodiment of the present invention, the evaluation module 13 is specifically configured to: when the corresponding recognition result r9 is obtained by identifying whether the vehicle has a navigation deviation event according to the trajectories A and B, for each pair of trajectory points a in the trajectories A and B, j 、b j The coordinate straight line spacing is calculated to obtain the corresponding ab track point spacing; and whether all the obtained ab track point spacings are less than the preset yaw spacing threshold is identified, if so, the corresponding identification result r9 is set to no, otherwise the corresponding identification result r9 is set to yes;

[0137] Here, the yaw spacing threshold is a preset distance threshold parameter;

[0138] Step C6, perform comprehensive scoring based on scores S1, S2, S3, and S4 to obtain the corresponding score S all ; and all recognition results r 1,i , all recognition results r 2,i , recognition result r3, all recognition results r 4,k , recognition result r5, recognition result r6, recognition result r7, recognition result r8, recognition result r9, score S1, score S2, score S3, score S4 and score S all Import the preset evaluation report template, set the report text, obtain the corresponding evaluation report and save it.

[0139] Here, the comprehensive scoring mechanism of the embodiment of the present invention can customize the summary rules based on application requirements, for example, S all =S1+S2+S3+S4; the evaluation report template is a pre-set formatted report module.

[0140] (IV) Scene caching module 14:

[0141] The scene cache module 14 of the embodiment of the present invention is used to store the test frame sequence and the loading frame index.

[0142] Here, the test frame sequence of an embodiment of the present invention is composed of multiple first test frames; the first test frame includes a test frame index, a test frame timestamp, a self-vehicle label trajectory point, a self-vehicle simulation trajectory point, a trajectory point set of other participants, a reference object trajectory point set, and a signal light trajectory point set; the self-vehicle label trajectory point includes the self-vehicle label coordinates, the self-vehicle label road section / traffic intersection identification, the self-vehicle label heading angle, the self-vehicle label speed, and the self-vehicle label acceleration; the self-vehicle simulation trajectory point includes the self-vehicle simulation coordinates, the self-vehicle simulation road section / traffic intersection identification, the self-vehicle simulation heading angle, the self-vehicle simulation speed, and the self-vehicle simulation acceleration.

[0143] (V) Scene player 15:

[0144] The scene player 15 of the embodiment of the present invention is used to identify the starting and target positions of the vehicle based on the test frame sequence when receiving a playback instruction and send a test start instruction carrying the identification result to the regulation and control module 3, and perform a round of frame-by-frame loading and playback according to the test frame sequence when receiving the startup success sent back by the regulation and control module 3; and during this round of playback, if the current loaded frame is not the last frame, the corresponding current perception information is generated based on the current loaded frame and sent to the regulation and control module 3; and at the end of this round of playback, the playback completion status is sent to the evaluation module 13, and a test end instruction is sent to the regulation and control module 3.

[0145] Here, the scenario player 15 of the embodiment of the present invention is an autonomous driving simulation tool / simulator / simulation software, such as CARLA, Carmaker, CarSim, PreScan, etc.

[0146] In another specific implementation of an embodiment of the present invention, the scene player 15 is specifically used to: when identifying the starting and target positions of the ego vehicle based on the test frame sequence: use the ego vehicle label coordinates of the first and last ego vehicle label trajectory points of the test frame sequence of the scene cache module 14 as the ego vehicle starting position and the ego vehicle target position.

[0147] It should be noted that upon receiving the test start command, the control module 3 extracts the vehicle's starting and target positions and uses the starting position as the vehicle's initial position for the test, and the target position as the target position for the test, for subsequent step-by-step motion planning and control command output. Upon receiving the test end command, the control module 3 ceases internal operations and no longer receives perception information from the scenario player 15 or sends control commands to it.

[0148] In another specific implementation of the embodiment of the present invention, the scene player 15 is specifically configured to:

[0149] All the first test frames of the test frame sequence of the scene cache module 14 are traversed in sequence in one round; and in this round of traversal, the first test frame currently traversed is used as the corresponding current frame, and the first test frame next to the current frame in the test frame sequence is recorded as the corresponding next frame, and when the next frame is not empty, the time interval between the current frame and the next frame is confirmed to obtain the corresponding next frame interval; and the current time is used as the corresponding start time of this time; and the loading frame index of the scene cache module 14 is set to the test frame index of the current frame; and the corresponding current loading frame is composed of the test frame timestamp of the current frame, the self-vehicle simulation trajectory point, the trajectory point set of other participants, the reference object trajectory point set and the traffic light trajectory point set; and two-dimensional or three-dimensional intersection is performed based on the current loading frame. The scene is constructed and the currently constructed scene is played; and when the next frame is not empty, the test frame timestamp, self-vehicle simulation trajectory point, other participant trajectory point set, reference object trajectory point set and traffic light trajectory point set corresponding to the currently loaded frame are brought into the perception information data template corresponding to the regulation and control module 3 for data setting to obtain the corresponding current perception information and send it to the regulation and control module 3; and when the next frame is not empty, an independent timing is started from this start time, and when the timing duration matches the next frame interval, the timing is stopped and the next first test frame is continued to traverse until the last first test frame traversal is completed; and at the end of this round of traversal, the playback completion status is sent to the evaluation module 13, and the test end instruction is sent to the regulation and control module 3.

[0150] (6) Self-driving car simulator 16:

[0151] The ego vehicle simulator 16 of the embodiment of the present invention is configured to perform an ego vehicle motion state prediction based on the loading frame index and the current ego vehicle control command each time it receives an ego vehicle control command from the regulation and control module 3. Furthermore, during each prediction process, if the loading frame index is greater than or equal to the takeover frame index, the ego vehicle state is refreshed for the next frame corresponding to the loading frame index based on the current prediction result.

[0152] Here, the command parameters of the ego vehicle control command sent by the control module 3 include at least the steering wheel angle related to lateral control and the throttle / brake opening and closing degree related to longitudinal control. It should be noted that each time the control module 3 receives the current perception information from the scene player 15, it first extracts the ego vehicle simulation trajectory points (coordinates, orientation, speed, acceleration) as the current ego vehicle state, and extracts the trajectory points of other participants, the reference object trajectory points, and the traffic light trajectory points as the current environmental perception state. Then, based on the current ego vehicle state and the current environmental perception state, it plans the ego vehicle's driving trajectory toward the target location of this test. Then, based on the current planned trajectory, it predicts the ego vehicle's lateral motion control parameters and longitudinal motion control parameters. Finally, the predicted command parameters related to lateral / longitudinal control are combined into a driving control command, i.e., the ego vehicle control command, which is fed back to the ego vehicle simulator 16.

[0153] In another specific implementation of the embodiment of the present invention, the ego vehicle simulator 16 is specifically configured to:

[0154] Step D1, obtain the test frame sequence and the loading frame index from the scene cache module 14; use the high-precision map of the area corresponding to the test frame sequence as the corresponding current high-precision map; use the first test frame corresponding to the current loading frame in the test frame sequence as the current frame, and use the first test frame after the current frame as the corresponding next frame; use the time interval between the current frame and the next frame as the corresponding current prediction duration; use the time point corresponding to the next frame as the corresponding current target time; and use the self-vehicle simulation coordinates, self-vehicle simulation heading angle, self-vehicle simulation speed, and self-vehicle simulation acceleration of the self-vehicle simulation trajectory point in the current frame to form a starting motion state; and use the instruction parameters of the current self-vehicle control instruction as the corresponding current instruction parameters;

[0155] Step D2, and identifying whether the currently loaded frame index is less than the takeover frame index stored locally in the simulator;

[0156] In step D3, if the current loading frame index is less than the takeover frame index, the ego vehicle tag coordinates, ego vehicle tag heading angle, ego vehicle tag velocity, and ego vehicle tag acceleration of the next frame's ego vehicle tag trajectory point are used to form a corresponding tag motion state. The ego vehicle dynamics model built into the simulator predicts the ego vehicle's coordinates, heading angle, velocity, and acceleration at the current target time based on the starting motion state, current command parameters, and current prediction duration. The predicted coordinates, heading angle, velocity, and acceleration are used to form a corresponding predicted motion state. The model parameters of the ego vehicle dynamics model are optimized in a direction that minimizes the differential state quantity between the predicted and tag motion states.

[0157] Here, the vehicle dynamics model of the embodiment of the present invention is a mathematical model implemented based on a Kalman filter, or an intelligent model implemented based on a machine learning model or a deep learning model;

[0158] In step D4, if the current loading frame index is greater than or equal to the takeover frame index, the vehicle dynamics model predicts the coordinates, heading angle, velocity, and acceleration of the vehicle at the current target time based on the starting motion state, current command parameters, and current prediction duration, and uses the predicted coordinates, heading angle, velocity, and acceleration as the corresponding current predicted coordinates, current predicted heading angle, current predicted velocity, and current predicted acceleration; and uses the road section / traffic intersection identifier corresponding to the current predicted coordinates in the current high-precision map as the corresponding current road section / traffic intersection identifier; and uses the vehicle simulation road section / traffic intersection identifier, vehicle simulation coordinates, vehicle simulation heading angle, vehicle simulation velocity, and vehicle simulation acceleration of the vehicle simulation trajectory point in the next frame as the corresponding current road section / traffic intersection identifier, current predicted coordinates, current predicted heading angle, current predicted velocity, and current predicted acceleration.

[0159] An embodiment of the present invention provides an autonomous driving evaluation system based on a simulation scenario. As can be seen from the above content, the system includes: an acquisition module, a scenario server, an evaluation module, a scenario cache module, a scenario player, and a self-vehicle simulator; wherein the acquisition module is used to convert the real road condition data stored on the road network database into a test scenario and store it in the scenario server; the evaluation module is used to select a test scenario from the scenario server to set the test frame sequence, the loading frame index on the scene cache module, and the takeover frame index on the self-vehicle simulator, and perform simulation according to the test frame sequence by calling the scenario player; the scenario player sends the corresponding test start / end instructions to the control module at the start and end of each round of simulation process, and also sends the playback completion status to the evaluation module at the end time; each round of simulation process of the scenario player is A round of frame-by-frame loading and playback process; during each round of simulation, the scene player will generate corresponding current perception information based on the current loaded frame and send it to the regulation and control module as long as the current loaded frame is not the last frame; the self-vehicle simulator is used to predict the self-vehicle motion state based on the loaded frame index and the current self-vehicle control instruction every time it receives a self-vehicle control instruction sent by the regulation and control module, and in each prediction process, if the loaded frame index is greater than or equal to the takeover frame index, the self-vehicle state will be refreshed for the next frame corresponding to the loaded frame index in the test frame sequence based on the current prediction result; the evaluation module is also used to evaluate this test from the four dimensions of safety, compliance, comfort and efficiency after receiving the playback completion status to obtain a corresponding evaluation report. On the one hand, the embodiments of the present invention can mine massive test scenarios from a large amount of real road condition data, thereby improving the richness and complexity of the test scenarios; on the other hand, the simulation test method based on the scenario player + vehicle simulator improves the utilization rate of the test scenarios and the test efficiency of the regulation and control module; on the other hand, the richness and diversity of the evaluation methods are improved through the four-dimensional evaluation (safety, compliance, comfort and efficiency) mechanism; on the other hand, the verification adequacy of the regulation and control module is effectively improved in the early automatic testing stage and the later road testing costs are reduced.

[0160] Professionals should also be further aware that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0161] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0162] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An autonomous driving evaluation system based on simulation scenarios, characterized in that: The system includes: a collection module, a scene server, an evaluation module, a scene cache module, a scene player, and a self-vehicle simulator; The acquisition module is connected to the road network database and the scenario server respectively; the evaluation module is connected to the scenario server, the scenario cache module, the scenario player and the vehicle simulator respectively; the scenario cache module is connected to the scenario player and the vehicle simulator respectively; the scenario player and the vehicle simulator are respectively connected to a regulation and control module under test; The acquisition module is used to convert the real road condition data stored in the road network database into a test scene and store it in the scene server; The scenario server is used to store a plurality of the test scenarios; The evaluation module is used to receive batch tasks input by the user; and use each test scene identifier of the current batch task as the corresponding current identifier in turn; and select a matching scene from the scene server based on the current identifier to initialize the test frame sequence, loading frame index and takeover frame index on the scene cache module and the self-vehicle simulator; and send a play instruction to the scene player at the end of this initialization; and when the playback completion status is fed back by the scene player, perform a test evaluation based on the current identifier and the test frame sequence from the four dimensions of safety, compliance, comfort and efficiency to obtain a corresponding evaluation report; and feed back all the obtained evaluation reports to the current user; the batch task includes multiple test scene identifiers; The scene cache module is used to store the test frame sequence and the loading frame index; The scene player is used to identify the starting and target positions of the vehicle based on the test frame sequence when receiving the playback instruction and send a test start instruction carrying the recognition result to the regulation and control module, and perform a round of frame-by-frame loading and playback according to the test frame sequence when receiving the startup success sent back by the regulation and control module; and during this round of playback, if the current loaded frame is not the last frame, generate corresponding current perception information based on the current loaded frame and send it to the regulation and control module; and at the end of this round of playback, send a playback completion status to the evaluation module and send a test end instruction to the regulation and control module; The ego vehicle simulator is configured to predict the ego vehicle's motion state based on the loading frame index and the current ego vehicle control instruction each time it receives an ego vehicle control instruction sent by the regulation and control module; and during each prediction process, if the loading frame index is greater than or equal to the takeover frame index, the ego vehicle state is refreshed for the next frame corresponding to the loading frame index based on the current prediction result.

2. The autonomous driving evaluation system based on simulation scenarios according to claim 1, characterized in that: The road network database is used to store a plurality of real road condition data in the real road network; The real road condition data includes a participant dataset, a reference object dataset, a traffic light dataset, and a high-precision map of the area; The participant data set includes multiple participant data; the participant data includes participant identification, participant type, and participant trajectory; the participant type includes vehicle, motorcycle, bicycle, pedestrian, and animal; the participant trajectory is composed of multiple first trajectory points; the first trajectory point includes a timestamp, a road section / traffic intersection identification, coordinates, heading angle, angular velocity, velocity, and acceleration; The reference object data set includes a plurality of reference object data; the reference object data includes a reference object identifier, a reference object type, and a reference object coordinate; the reference object type includes at least a building, a pole, a plant, a traffic obstacle, and other obstacles; When the signal light data set is not empty, it is composed of one or more signal light data; the signal light data includes a signal light identification, a signal light type, a signal light coordinate, and a signal light trajectory; the signal light type includes at least a motor vehicle signal light, a lane signal light, and a direction indicator signal light; the signal light trajectory is composed of a plurality of second trajectory points; the second trajectory point includes a timestamp and a signal light state; the signal light state includes at least red light, green light, yellow light, and flashing yellow light; The regional high-precision map is a global high-precision map of the road section or intersection area corresponding to the current traffic data, including map element information of all road elements and non-road elements in the area; The test scene includes a scene identifier, a scene type, a scene complexity, a scene high-precision map, a scene frame sequence, and a key frame index; the scene types include straight roads, curves, diverging sections, merging sections, merging sections, merging sections, cross intersections, and T-intersections; the scene frame sequence consists of multiple first-frame scenes; The first frame scene includes a frame index, a frame timestamp, a self-vehicle trajectory point, a set of other participant trajectory points, a set of reference object trajectory points, and a set of signal light trajectory points; the self-vehicle trajectory point includes the self-vehicle coordinates, the self-vehicle road section / traffic intersection identification, the self-vehicle heading angle, the self-vehicle speed, and the self-vehicle acceleration; the other participant trajectory point set includes multiple participant trajectory points, and the participant trajectory points include the participant identification, the participant type, the participant coordinates, the participant road section / traffic intersection identification, the participant heading angle, the participant speed, and the participant acceleration; the reference object trajectory point set includes multiple reference object trajectory points, and the reference object trajectory points include the reference object identification, the reference object type, and the reference object coordinates; when the signal light trajectory point set is not empty, it is composed of one or more signal light trajectory points, and the signal light trajectory points include the signal light identification, the signal light type, the signal light coordinates, and the signal light status; the time interval between each two adjacent frames of the scene frame sequence is a preset standard interval Δt std , the time span of the scene frame sequence is the preset scene duration L; The test frame sequence consists of multiple first test frames; the first test frame includes a test frame index, a test frame timestamp, a self-vehicle label trajectory point, a self-vehicle simulation trajectory point, the trajectory point set of the other participants, the trajectory point set of the reference object, and the trajectory point set of the traffic light; the self-vehicle label trajectory point includes the self-vehicle label coordinates, the self-vehicle label road section / traffic intersection identification, the self-vehicle label heading angle, the self-vehicle label speed, and the self-vehicle label acceleration; the self-vehicle simulation trajectory point includes the self-vehicle simulation coordinates, the self-vehicle simulation road section / traffic intersection identification, the self-vehicle simulation heading angle, the self-vehicle simulation speed, and the self-vehicle simulation acceleration.

3. The autonomous driving evaluation system based on simulation scenarios according to claim 2, characterized in that: The acquisition module is specifically configured to convert the real road condition data stored in the road network database into a test scenario and store it in the scenario server: The real traffic condition data on the scene server with a time span exceeding the scene duration L is used as the corresponding current traffic condition data; The earliest and latest timestamps in the current traffic condition data are used as the corresponding current start time and current end time; the current start time is converted to the corresponding start zero time; the current end time is converted relative to the start zero time to obtain the corresponding end time; the time period from the start zero time to the end time is used as the corresponding current traffic condition period; and the time period is calculated based on the standard interval Δt std The single-step interval is a time point sampling of the current road condition period to obtain corresponding multiple single-step time points; the time interval between each two adjacent single-step time points is the standard interval Δt std ; And take the trajectories of each participant in the current road condition data as the corresponding current trajectory; and take the sub-period corresponding to the current trajectory in the current road condition period as the current trajectory period; and use the coordinates, heading angle, angular velocity, speed, and acceleration of all the first trajectory points in the current trajectory as observation quantities, and construct the corresponding Kalman filter equation according to the preset vehicle kinematic model; and based on the current Kalman filter equation, predict the state of the coordinates, heading angle, angular velocity, speed, and acceleration of the current participant at each single-step time point in the current trajectory period; and reconstruct the current trajectory based on the predicted data; in each reconstructed participant trajectory, the timestamp of each first trajectory point corresponds to a single-step time point, and the coordinates, heading angle, angular velocity, speed, and acceleration of each first trajectory point are the coordinate, heading angle, angular velocity, speed, and acceleration state prediction quantities corresponding to the timestamp of the current trajectory point, and the road section / traffic intersection identifier of each first trajectory point is the road section identifier or traffic intersection identifier corresponding to the current coordinate prediction quantity in the high-precision map of the area; and calculating the travel distance of each reconstructed participant trajectory to obtain a corresponding first distance; and calculating the trajectory time of each reconstructed participant trajectory to obtain a corresponding first duration; The participant trajectory whose first distance exceeds a preset first distance threshold and whose first duration exceeds a preset first duration threshold is recorded as a corresponding candidate trajectory, and one of all the candidate trajectories is selected as the corresponding vehicle trajectory; the participant data corresponding to the vehicle trajectory in the participant data set of the current road condition data is selected as the corresponding vehicle data, and the other participant data is selected as the corresponding other participant data; and when the signal light data set of the current road condition data is not empty, taking each signal light trajectory of the current signal light data set as the corresponding current trajectory; and taking the sub-period corresponding to the current road condition period of the current trajectory as the current trajectory period; Each of the single-step time points in the current trajectory period is recorded as a corresponding current time point, and the signal light state corresponding to the current time point is set according to the signal light state of the second trajectory point before and / or after the current time point in the current trajectory by interpolation, and the current time point and the corresponding signal light state form a corresponding reconstructed trajectory point; and a new signal light trajectory is formed by all the reconstructed trajectory points corresponding to the current trajectory period to replace the current trajectory; in each reconstructed signal light trajectory, the timestamp of each second trajectory point corresponds to a single-step time point; The scene duration L is used as the segment length to segment the current traffic period according to the preset single-step sliding step length to obtain corresponding multiple segment time periods; the single-step sliding step length is the standard interval Δt std an integer multiple of ; And take each of the segmented time periods as the corresponding current segment; and assign a corresponding frame index to each of the single-step time points of the current segment in a point-by-point increment manner starting from 1; and take the single-step time point corresponding to each of the frame indexes as a corresponding frame timestamp; and use the road section / traffic intersection identification, coordinates, orientation angle, angular velocity, speed, and acceleration of the first trajectory point corresponding to each of the frame timestamps of the current segment in the vehicle trajectory as a group of corresponding vehicle road section / traffic intersection identification, vehicle coordinates, vehicle orientation angle, vehicle speed, and vehicle acceleration to form a corresponding vehicle trajectory point; and each of the The road section / traffic intersection identifier, coordinates, orientation angle, angular velocity, speed, and acceleration of the first trajectory point corresponding to each frame timestamp of the current segment in the participant trajectory of the other participant data are used as a group of corresponding participant road section / traffic intersection identifiers, participant coordinates, participant orientation angles, participant speeds, and participant accelerations; and the participant identifier, participant type, participant road section / traffic intersection identifier, participant coordinates, participant orientation angles, participant speeds, and participant accelerations corresponding to each of the other participant data form a corresponding participant trajectory point; and the current segment is used as a group of corresponding participant road section / traffic intersection identifiers, participant coordinates, participant orientation angles, participant speeds, and participant accelerations. All the participant trajectory points corresponding to each frame timestamp of the fragment form a corresponding set of other participant trajectory points; and each reference object data of the current road condition data is used as a corresponding reference object trajectory point, and all the reference object trajectory points form the reference object trajectory point set corresponding to each frame timestamp in the current fragment; and the signal light identification, the signal light type, the signal light coordinates and the signal light state corresponding to the second trajectory point corresponding to each frame timestamp of the current fragment in each signal light trajectory form a corresponding signal light trajectory point; and all the reference object data corresponding to each frame timestamp of the current fragment The signal light trajectory points constitute a corresponding signal light trajectory point set; and each frame index of the current segment and its corresponding frame timestamp, the vehicle trajectory point, the other participant trajectory point set, the reference object trajectory point set, and the signal light trajectory point set constitute a corresponding first frame scene; and all the first frame scenes corresponding to the current segment are sorted in chronological order to constitute a corresponding scene frame sequence; and the time period center point of the current segment is used as the corresponding current time period center point, and the frame index corresponding to the frame timestamp closest to the current time period center point in the scene frame sequence corresponding to the current segment is used as the corresponding key frame index; and assigning a unique identifier to each of the scene frame sequences as the corresponding scene identifier; and using the high-precision map of the area of ​​the current road condition data as the high-precision map of the scene corresponding to each of the scene frame sequences; and forming a corresponding self-vehicle driving trajectory from a plurality of the self-vehicle trajectory points of each of the scene frame sequences; and querying the regional high-precision map of the current road condition data based on the self-vehicle road section / traffic intersection identifier of each of the self-vehicle driving trajectories, and confirming based on the query result whether the road section / traffic intersection covered by the current self-vehicle driving trajectory is a straight road, a curve, a diverging section, a merging section, an incoming section, an outgoing section, a cross intersection, or a T-intersection, and setting the corresponding scene type based on the confirmation result; and respectively counting the total number of the participant identifiers and the total number of the participant types in each of the scene frame sequences to obtain a corresponding first total number and a second total number, respectively normalizing the first and second total numbers to obtain a corresponding first normalized total number and second normalized total number, performing a weighted sum on the first and second normalized total numbers, and using the sum result as the corresponding scene complexity; Each scene frame sequence and its corresponding scene identifier, scene type, scene complexity, scene high-precision map and key frame index form a corresponding test scene and are stored in the scene server.

4. The autonomous driving evaluation system based on simulation scenarios according to claim 2, characterized in that: The evaluation module is specifically configured to, when selecting a matching scene from the scene server based on the current identifier, initialize the test frame sequence, the loaded frame index on the scene cache module, and the takeover frame index on the self-driving vehicle simulator: The test scene whose scene identifier on the scene server matches the current identifier is used as the corresponding matching scene; and the scene frame sequence and the key frame index of the matching scene are used as the corresponding current scene frame sequence and current key frame index; and using the frame index and the frame timestamp of each of the first frames of the current scene frame sequence as the corresponding test frame index and the test frame timestamp; and using the vehicle trajectory points of each of the first frame scenes in the current scene frame sequence as the corresponding vehicle label trajectory points and the vehicle simulation trajectory points; Each of the test frame indexes and its corresponding test frame timestamp, the vehicle label trajectory point, the vehicle simulation trajectory point, the other participant trajectory point set, the reference object trajectory point set, and the signal light trajectory point set constitute a corresponding first test frame; and forming a new version of the test frame sequence from all the first test frames obtained to replace the old version of the test frame sequence in the scene cache module; and initializing the loaded frame index stored in the scene cache module to 0; And the takeover frame index on the vehicle simulator is set as the current key frame index.

5. The autonomous driving evaluation system based on simulation scenarios according to claim 2, characterized in that: The evaluation module is specifically configured to, when performing a test evaluation based on the current identifier and the test frame sequence from the four dimensions of safety, compliance, comfort, and efficiency to obtain a corresponding evaluation report: Step 51: Using the high-precision map of the region and the key frame index corresponding to the test frame sequence on the scene cache module as the corresponding current high-precision map and current key frame index; Extracting a subsequence in the test frame sequence whose test frame index is greater than or equal to the current key frame index as the corresponding current subsequence; and constructing a corresponding trajectory A based on all the vehicle label trajectory points in the current subsequence; And construct the corresponding trajectory B based on all the vehicle simulation trajectory points of the current subsequence; and construct the corresponding trajectory C based on all the participant trajectory points of each participant identifier of the current subsequence i , 1≤ participant index i≤N C , N C is the total number of participants; and constructs a corresponding trajectory D based on all the signal light trajectory points of each signal light identifier of the current subsequence k , 1≤semaphore index k≤N K , N K is the total number of signal lights; Wherein, the trajectory A includes multiple trajectory points a j , 1≤indexj≤N F , N F is the total number of test frames of the current subsequence; the trajectory B includes N F Trajectory point b j The trajectory C i Including N F Trajectory point c i，j The trajectory D k Including N F Trajectory points d k,j Each of the trajectory points a j , the trajectory point b j , the trajectory point c i，j Each of the trajectory points d k,j corresponding to a set of the signal light coordinates and the signal light status; Step 52: Based on the trajectory B and each of the trajectories C i Identify whether the vehicle and other participants have a collision event and obtain the corresponding recognition result r 1,i ; and based on the trajectory B and each of the trajectories C i Identify whether the collision time between the vehicle and other participants is lower than the preset collision time threshold and obtain the corresponding identification result r 2,i ; Based on the trajectories A, B and the current high-precision map, the vehicle is identified as to whether it has left the drivable area to obtain the corresponding recognition result r3; and based on all the recognition results r 1,i , the recognition result r 2,i Perform security scoring on the recognition result r3 to obtain a corresponding score S1; Among them, the recognition result r 1,i Including yes, no; the recognition result r 2,i Including yes and no; the recognition result r3 includes yes and no; Step 53: according to the track B and each track D k The current high-precision map is used to identify whether the vehicle has run a red light and obtain the corresponding recognition result r 4,k ; Based on the trajectory B and the current high-precision map, identify whether the vehicle has experienced a speeding event to obtain a corresponding recognition result r5; and based on all the recognition results r 4,k Compare the compliance score of the recognition result r5 to obtain the corresponding score S2; Among them, the recognition result r 4,k Including yes and no; the recognition result r5 includes yes and no; Step 54 : Identify whether a large lateral acceleration event has occurred on the vehicle based on the trajectory B to obtain a corresponding identification result r6 ; and identify whether a sudden braking event has occurred on the vehicle based on the trajectory B to obtain a corresponding identification result r7 ; and perform a comfort score based on the identification results r6 and r7 to obtain a corresponding score S3 ; Wherein, the recognition result r6 includes yes and no; the recognition result r7 includes yes and no; Step 55, according to the track B, all the tracks C i Using the current high-precision map, the vehicle is identified as having a low-speed driving event to obtain a corresponding recognition result r8; and based on the trajectories A and B, the vehicle is identified as having a navigation deviation event to obtain a corresponding recognition result r9; and efficiency scores are performed based on the recognition results r8 and r9 to obtain a corresponding score S4; Wherein, the recognition result r8 includes yes and no; the recognition result r9 includes yes and no; Step 56: Comprehensively score the scores S1, S2, S3, and S4 to obtain the corresponding score S. all ; and all the recognition results r 1,i , all the recognition results r 2,i , the recognition result r3, all the recognition results r 4,k , the recognition result r5, the recognition result r6, the recognition result r7, the recognition result r8, the recognition result r9, the score S1, the score S2, the score S3, the score S4 and the score S all Enter the preset evaluation report template to set the report text to obtain the corresponding evaluation report and save it.

6. The autonomous driving evaluation system based on simulation scenarios according to claim 5, characterized in that: The evaluation module is specifically used to compare the trajectory B with the trajectory C. i Identify whether the vehicle and other participants have a collision event and obtain the corresponding recognition result r 1,i When: each of the trajectories C i As the corresponding current trajectory; and for each of the trajectory points c of the current trajectory i，j The corresponding trajectory point b j The coordinate straight line spacing is calculated to obtain the corresponding trajectory point spacing; and whether the obtained trajectory point spacing is greater than the preset collision spacing threshold is identified, and if so, the corresponding identification result r is set 1,i If not, set the corresponding recognition result r 1,i For yes; The evaluation module is specifically used to compare the trajectory B with the trajectory C. i Identify whether the collision time between the vehicle and other participants is lower than the preset collision time threshold and obtain the corresponding identification result r 2,i When: each of the trajectories C i As the corresponding current trajectory; and for each of the trajectory points c of the current trajectory i，j The corresponding trajectory point b j The collision time of the corresponding trajectory point is calculated; And identify whether the collision time of all the obtained trajectory points is higher than the collision time threshold, and if so, set the corresponding identification result r 2,i If not, set the corresponding recognition result r 2,i For yes; The evaluation module is specifically configured to, when the corresponding recognition result r3 is obtained by identifying whether the vehicle has left the drivable area based on the trajectories A, B and the current high-precision map: set the road area that the trajectory A passes through on the current high-precision map as a reference area, and set the connected area on the current high-precision map that is connected to the reference area and whose vehicle driving direction is not in a reverse relationship with the vehicle driving direction of the trajectory A as an expansion area, and when the number of the expansion areas is not zero, the reference area and all the expansion areas constitute a drivable area; and identify whether the trajectory B does not exceed the drivable area, and if so, set the corresponding recognition result r3 to no, and otherwise set the corresponding recognition result r3 to yes; The evaluation module is specifically used to evaluate the trajectory B and each trajectory D. k The current high-precision map is used to identify whether the vehicle has run a red light and obtain the corresponding recognition result r 4,k When: each of the trajectories D k as the corresponding current track; and the intersection range controlled by the traffic light corresponding to the current track on the current high-precision map is used as the corresponding current intersection; And the track point d in the current track where the traffic light state is red k,j The corresponding test frame timestamp is recorded as the red light timestamp; and the track point b on the track B corresponding to the red light timestamp is recorded as the red light timestamp. j The coordinates of the track point are located in the current intersection for identification, if so, the corresponding track point running light state is set to yes, if not, the corresponding track point running light state is set to no; and all the track points corresponding to the current track are identified as no, if so, the corresponding identification result r is set. 4,k If not, set the corresponding recognition result r 4,k For yes; The evaluation module is specifically used to: when the corresponding identification result r5 is obtained by identifying whether the vehicle has a speeding event based on the trajectory B and the current high-precision map: j as the corresponding current trajectory point; and based on the coordinates of the current trajectory point and the road section / traffic intersection sign, query the current high-precision map to obtain the maximum driving speed corresponding to the current coordinates; and identify whether the speed of the current trajectory point exceeds the maximum driving speed, and if so, set the corresponding trajectory point speeding status to yes; otherwise, set the corresponding trajectory point speeding status to no; and identify whether the speeding status of all the trajectory points corresponding to the current trajectory is no, and if so, set the corresponding identification result r5 to no, otherwise, set the corresponding identification result r5 to yes; The evaluation module is specifically used to: when the corresponding identification result r6 is obtained by identifying whether the vehicle has a large lateral acceleration event according to the trajectory B: j as the corresponding current trajectory point; and calculating the corresponding trajectory point lateral acceleration based on the coordinates, angle and acceleration of the current trajectory point; and querying a preset speed-lateral acceleration correspondence table based on the speed of the current trajectory point to obtain a corresponding lateral acceleration range; and identifying whether the lateral acceleration of the trajectory point exceeds the lateral acceleration range, if not, setting the corresponding trajectory point state to normal; if exceeded, setting the corresponding trajectory point state to abnormal; and identifying whether all the obtained trajectory point states are normal, if so, setting the corresponding identification result r6 to no, otherwise setting the corresponding identification result r6 to yes; the speed-lateral acceleration correspondence table includes multiple correspondence records; the correspondence record includes a speed range field and a lateral acceleration range field; The evaluation module is specifically used to: when the corresponding identification result r7 is obtained by identifying whether the vehicle has a sudden braking event according to the trajectory B: j As the corresponding current track point in sequence; and the previous track point b of the current track point j as the corresponding previous trajectory point; and calculate the corresponding deceleration change rate with the acceleration of the previous trajectory point and the current trajectory point; and identify the acceleration and the deceleration change rate of the current trajectory point, if the current acceleration is less than a preset first acceleration threshold and the current deceleration change rate is less than a preset change rate threshold, then set the corresponding trajectory point emergency braking state to yes; if the current acceleration is greater than or equal to the first acceleration threshold, or the current deceleration change rate is greater than or equal to the change rate threshold, then set the corresponding trajectory point emergency braking state to no; and identify whether the emergency braking states of all the obtained trajectory points are no, if so, set the corresponding identification result r7 to no, otherwise set the corresponding identification result r7 to yes; The evaluation module is specifically used to evaluate the trajectory B and all the trajectories C. i When the current high-precision map is used to identify whether the vehicle has a low-speed driving event and obtain the corresponding recognition result r8: each of the track points b of the track B is j as the corresponding current track point; and using the driving lane corresponding to the coordinates of the current track point and the road section / traffic intersection sign on the current high-precision map as the corresponding current lane; and taking the same-direction lane adjacent to the current lane on the current high-precision map as the corresponding adjacent lane; And all the trajectories C i The trajectory point c whose timestamp corresponds to the current trajectory point and is in any of the adjacent lanes i，j Form a corresponding current trajectory point set; and all trajectory points c of the current trajectory point set i，j The average speed is calculated to obtain the corresponding current average speed; The product of the current average speed and a preset percentage parameter is used as the corresponding current speed threshold; whether the speed of the current trajectory point is lower than the current speed threshold is identified, and if so, the low speed state of the corresponding trajectory point is set to yes; otherwise, the low speed state of the corresponding trajectory point is set to no; and whether the low speed states of all the obtained trajectory points are no is identified, and if so, the corresponding identification result r8 is set to no; otherwise, the corresponding identification result r8 is set to yes; The evaluation module is specifically used for: when the corresponding identification result r9 is obtained by identifying whether the navigation deviation event occurs to the vehicle according to the trajectories A and B: j 、b j The coordinate straight line spacing is calculated to obtain the corresponding ab track point spacing; and whether all the obtained ab track point spacings are less than the preset yaw spacing threshold is identified, if so, the corresponding identification result r9 is set to no, otherwise the corresponding identification result r9 is set to yes.

7. The autonomous driving evaluation system based on simulation scenarios according to claim 2, characterized in that: The scene player is specifically configured to: during a round of frame-by-frame loading and playback according to the test frame sequence: Performing a round of sequential traversal on all the first test frames of the test frame sequence of the scene cache module; and during this round of traversal, taking the first test frame currently traversed as the corresponding current frame, and recording the first test frame next to the current frame in the test frame sequence as the corresponding next frame, and confirming the time interval between the current frame and the next frame when the next frame is not empty to obtain the corresponding next frame interval; And take the current time as the corresponding start time; The loading frame index of the scene cache module is set to the test frame index of the current frame; and the corresponding current loading frame is composed of the test frame timestamp of the current frame, the self-vehicle simulation trajectory point, the other participant trajectory point set, the reference object trajectory point set and the traffic light trajectory point set; and constructing a two-dimensional or three-dimensional traffic scene based on the currently loaded frame, and playing the currently constructed scene; And when the next frame is not empty, the test frame timestamp, the self-vehicle simulation trajectory point, the other participant trajectory point set, the reference object trajectory point set and the traffic light trajectory point set corresponding to the currently loaded frame are brought into the perception information data template corresponding to the regulation and control module for data setting to obtain the corresponding current perception information and send it to the regulation and control module; and when the next frame is not empty, an independent timing is started from the current start time, and when the current timing duration matches the next frame interval, the current timing is stopped and the next first test frame is continued to traverse until the last first test frame traversal is completed; and at the end of this round of traversal, the playback completion status is sent to the evaluation module, and a test end instruction is sent to the regulation and control module.

8. The autonomous driving evaluation system based on simulation scenarios according to claim 2, characterized in that: The ego vehicle simulator is specifically configured to: when performing a ego vehicle motion state prediction based on the loaded frame index and the current ego vehicle control instruction: Obtain the test frame sequence and the loading frame index from the scene cache module; and use the high-precision map of the area corresponding to the test frame sequence as the corresponding current high-precision map; and taking the first test frame corresponding to the current loading frame in the test frame sequence as the current frame, and taking the first test frame next to the current frame as the corresponding next frame; and taking the time interval between the current frame and the next frame as the corresponding current prediction duration; and taking the time point corresponding to the next frame as the corresponding current target time; and forming a starting motion state by the self-vehicle simulated coordinates, the self-vehicle simulated heading angle, the self-vehicle simulated speed, and the self-vehicle simulated acceleration of the self-vehicle simulated trajectory point in the current frame; and taking the instruction parameters of the current vehicle control instruction as the corresponding current instruction parameters; and identifying whether the current loading frame index is smaller than the takeover frame index stored locally in the simulator; If the current loading frame index is less than the takeover frame index, a corresponding tag motion state is formed by the ego vehicle tag coordinates, the ego vehicle tag heading angle, the ego vehicle tag speed, and the ego vehicle tag acceleration of the ego vehicle tag trajectory point in the next frame; and the ego vehicle dynamics model built into the simulator predicts the coordinates, heading angle, speed, and acceleration of the ego vehicle at the current target moment based on the starting motion state, the current instruction parameters, and the current prediction duration, and the coordinates, heading angle, speed, and acceleration obtained in this prediction form a corresponding predicted motion state; and the model parameters of the ego vehicle dynamics model are optimized in a direction in which the differential state quantity of the predicted and tag motion states reaches a minimum value; the ego vehicle dynamics model is a mathematical model implemented based on a Kalman filter, or an intelligent model implemented based on a machine learning model or a deep learning model; If the current loading frame index is greater than or equal to the takeover frame index, the vehicle dynamics model predicts the coordinates, heading angle, speed, and acceleration of the vehicle at the current target time based on the starting motion state, the current instruction parameters, and the current prediction duration, and uses the predicted coordinates, heading angle, speed, and acceleration as the corresponding current predicted coordinates, current predicted heading angle, current predicted speed, and current predicted acceleration; and uses the road section / traffic intersection identifier corresponding to the current predicted coordinates in the current high-precision map as the corresponding current road section / traffic intersection identifier; The self-vehicle simulation road section / traffic intersection identifier, the self-vehicle simulation coordinates, the self-vehicle simulation heading angle, the self-vehicle simulation speed, and the self-vehicle simulation acceleration of the self-vehicle simulation trajectory point in the next frame are set as the corresponding current road section / traffic intersection identifier, the current predicted coordinates, the current predicted heading angle, the current predicted speed, and the current predicted acceleration.

Images