An evaluation method and system for an automatic driving trajectory planning module

Abstract

This invention discloses an evaluation method for an autonomous driving trajectory planning module, comprising the following steps: obtaining the trajectory planning results generated by the trajectory planning module under test and reference results for comparison; traversing all trajectory points of each trajectory in the trajectory planning results under test, comparing the velocity, lateral acceleration, longitudinal acceleration, lateral jerk, and longitudinal jerk of each trajectory point with the set limits for each item respectively; if at least one item exceeds the set limit, recording and storing the relative time on the trajectory of that trajectory point and the output time of the trajectory where that trajectory point is located; comparing each trajectory in the trajectory planning results under test with the reference trajectory corresponding to the same time in the reference results to obtain the difference between the two; calculating the index scores for each item based on the difference and the set scoring weights; and weighting the index scores of each item for each trajectory to obtain the evaluation score for that trajectory.

Description

Technical Field

[0001] This invention relates to an autonomous driving evaluation method and system, and more particularly to a driving path planning method and system. Background Technology

[0002] With the development of autonomous driving technology, autonomous driving algorithms are iterating rapidly, and their functions are constantly being enriched. Among these, the trajectory planning module, which receives all external perception information, integrates the situation on-site, makes decisions, and plans the vehicle's driving trajectory within a certain time for control execution, largely determines the performance of autonomous vehicles. Therefore, in the context of rapid iteration, it is necessary to conduct timely and accurate evaluations of newly developed trajectory planning modules.

[0003] Currently, the evaluation methods for trajectory planning modules mainly include the following two types:

[0004] The first approach is based on real-world driving trajectories collected from human drivers. The algorithm simulates driving in the same scenario, and the resulting trajectory is compared with the actual driving trajectory. This method only focuses on comparing the results and doesn't consider that the planning algorithm actually outputs a trajectory for a future time period—that is, a behavioral decision for a given timeframe. It fails to evaluate this capability and the impact of decision-making ability on actual driving performance.

[0005] The second method also uses real-vehicle data as a foundation. It employs machine learning, feeding real-vehicle trajectories and computationally generated trajectories from autonomous driving software in the same scenario as sample data into a network model. The evaluation network model is then trained using the real-vehicle trajectories as a reference. This evaluation network model is then used to evaluate the computationally generated trajectories from the autonomous driving software.

[0006] Both of these methods are based on real-vehicle data and focus on evaluating the calculated trajectory generated by the autonomous driving trajectory planning software after the entire scenario is completed. However, neither evaluates the decision-making ability of the planning algorithm. In particular, the second approach, which uses machine learning, requires a large amount of accurate data input to train a suitable evaluation network model. Insufficient data and the difficulty in quantifying driver behavior can all affect the model's performance. Furthermore, because the model's internal operation is uninterpretable, this approach also suffers from the problem of evaluation metrics and process being black boxes, making it difficult to adjust the evaluation metrics.

[0007] To address the above issues, we propose an evaluation method for the trajectory planning module of autonomous driving, which can analyze the planned trajectory output at each time step to evaluate the performance and decision-making ability of the planning module algorithm. Summary of the Invention

[0008] One objective of this invention is to provide an evaluation method for autonomous driving trajectory planning modules. This method considers both hard constraints and statistical indicators, focusing not only on the overall operational results of the scenario but also, more importantly, on analyzing the planned trajectory output at each time step to evaluate the performance and decision-making capabilities of the trajectory planning module. Furthermore, this method can be automated, analyze numerous scenarios, and provide a reference for the development of autonomous driving trajectory planning modules.

[0009] To achieve the above objectives, this invention proposes an evaluation method for an autonomous driving trajectory planning module, comprising the following steps:

[0010] Obtain the trajectory planning results generated by the trajectory planning module to be evaluated;

[0011] Obtain reference results for comparison with the trajectory planning results to be evaluated;

[0012] Iterate through all trajectory points of each trajectory included in the trajectory planning results to be evaluated, and compare the velocity, lateral acceleration, longitudinal acceleration, lateral jerk, and longitudinal jerk of each trajectory point with the set velocity limit V. limit Lateral acceleration limit a_lateral limit Longitudinal acceleration limit a_longitudinal limit Lateral jerk_latetal limit Longitudinal jerk_longitudinal limit Compare them separately. If at least one of them exceeds the limit, record and store the relative time of the trajectory point on the trajectory and the output time of the trajectory where the trajectory point is located.

[0013] Each trajectory included in the trajectory planning result to be evaluated is compared with the reference trajectory corresponding to the same moment in the reference result to obtain the differences in lateral position, longitudinal position, lateral acceleration, longitudinal acceleration, lateral jerk, longitudinal jerk, curvature, and planning efficiency between the two.

[0014] Based on the differences in lateral position, longitudinal position, lateral acceleration, longitudinal acceleration, lateral jerk, longitudinal jerk, curvature, and planning efficiency, and the set scoring weights, the index scores for each of the following are calculated: lateral position, longitudinal position, lateral acceleration, longitudinal acceleration, lateral jerk, longitudinal jerk, curvature, and planning efficiency.

[0015] The scores of each indicator for each trajectory are weighted and calculated to obtain the evaluation score for that trajectory.

[0016] In the evaluation method described in this invention, in order to evaluate whether the decisions made by the trajectory planning module of autonomous driving are safe and comfortable enough, each trajectory is calculated and checked to evaluate whether it exceeds the hard limit (i.e., the set speed limit V). limit Lateral acceleration limit a_lateral limit Longitudinal acceleration limit a_longitudinal limit Lateral jerk_latetal limit Longitudinal jerk_longitudinal limit The trajectory points of ) appear.

[0017] In this invention, both the horizontal and vertical directions are defined according to the Frenet coordinate system, where the vertical direction represents the direction of vehicle movement along the center line of the road, and the horizontal direction represents the direction perpendicular to the center line of the lane.

[0018] During the hard constraint evaluation phase, all trajectories to be evaluated are iterated and analyzed in a loop. Within each trajectory, all trajectory points are traversed, and the calculated velocity V, lateral acceleration a_lat, and longitudinal acceleration a_lon of each point are compared with V. limit ,a_lateral limit ,a_longitudinal limit The comparison is performed. If the calculation result exceeds the limit, the output time of the trajectory and the relative time of the point on the trajectory are recorded. The trajectory point that exceeds the limit exceeds the value of the limit index.

[0019] For jerk, which is not typically used as an output on the planned trajectory, it can be determined according to the definition. First, calculate the lateral and longitudinal jerk accelerometers at each moment, and then compare them with jerk_latetal. limit and jerk_longitudinal limit Compare them separately and record the results.

[0020] In this invention, the evaluation of hard constraints mainly focuses on whether the results generated by the trajectory planning module under evaluation violate traffic regulations and safety and comfort constraints.

[0021] In the evaluation method described in this invention, the comparison based on the reference result belongs to statistical index analysis, which is achieved by calculating the difference between the trajectory generated by the trajectory planning module to be evaluated and the trajectory at the same time in the reference result.

[0022] Unlike evaluations based on hard constraints, the purpose of statistical index analysis is to reflect the difference between the test result and the reference result, thereby guiding the improvement of the algorithm to make it closer to the desired result. Therefore, this invention uses a scoring model to evaluate the performance of the test trajectory.

[0023] Since the trajectories obtained by different trajectory planning modules, and between the trajectory planning module and the human driver, are obviously not completely consistent, in some embodiments of the present invention, a threshold can be set for each difference index based on practical experience. Only when the difference exceeds the corresponding threshold is it considered worthy of attention and included in the scoring model for calculation.

[0024] Furthermore, in the evaluation method for an autonomous driving trajectory planning module described in this invention, the trajectory planning module to be evaluated generates the trajectory planning result based on scene data, wherein the scene data includes: map positioning information, perception fusion information, and prediction information.

[0025] Furthermore, in some implementations, the reference result is generated by another trajectory planning module.

[0026] Furthermore, in some other embodiments, the reference result is the actual driving trajectory collected.

[0027] Furthermore, in the evaluation method for the autonomous driving trajectory planning module described in this invention, the following formulas are used to calculate the following index scores: cost_x for lateral position, cost_y for longitudinal position, cost_lat_a for lateral acceleration, cost_lon_a for longitudinal acceleration, cost_lat_jerk for lateral jerk, cost_lon_jerk for longitudinal jerk, cost_kappa for curvature, and cost_s for planning efficiency.

[0028]

[0029]

[0030]

[0031]

[0032]

[0033]

[0034]

[0035] cost_s = s_diff;

[0036] Where x_diff represents the lateral position difference, y_diff represents the longitudinal position difference, a_lat_diff represents the lateral acceleration difference, a_lon_diff represents the longitudinal acceleration difference, jerk_lat_diff represents the lateral jerk jerk_lon_diff represents the longitudinal jerk ...

[0037] The lateral position difference x_diff = x_ref – x, where x_ref represents the lateral position of the reference result and x represents the lateral position of the trajectory to be evaluated;

[0038] The vertical position difference y_diff = y_ref – y, where y_ref represents the vertical position of the reference result and y represents the horizontal position of the trajectory to be evaluated;

[0039] Lateral acceleration difference a_lat_diff = a_lat_ref - a_lat, where a_lat_ref represents the lateral acceleration of the reference result and a_lat represents the lateral acceleration of the trajectory to be evaluated;

[0040] The longitudinal acceleration difference a_lon_diff = a_lon_ref - a_lon, where a_lon_ref represents the longitudinal acceleration of the reference result and a_lon represents the longitudinal acceleration of the trajectory to be evaluated;

[0041] Lateral jerk_lat_diff = jerk_lat_ref - jerk_lat, where jerk_lat_ref represents the lateral jerk of the reference result and jerk_lat represents the lateral jerk of the trajectory to be evaluated.

[0042] The longitudinal jerk jerk_lon_diff = jerk_lon_ref - jerk_lon, where jerk_lon_ref represents the longitudinal jerk jerk of the reference result and jerk_lon represents the longitudinal jerk jerk of the trajectory to be evaluated.

[0043] The difference in curvature of trajectory points is kappa_diff = kappa_ref – kappa, where kappa_ref represents the curvature of the trajectory points in the reference result, and kappa represents the curvature of the trajectory points in the trajectory to be evaluated.

[0044] The planning efficiency difference s_diff = s_ref – s, where s_ref represents the planning efficiency of the reference result, s represents the planning efficiency of the trajectory to be evaluated, and s represents the farthest distance traveled by the planned route along the entire trajectory.

[0045] For the trajectory planning module, which generates a planned trajectory for several seconds per frame, this invention includes the planned trajectory generated at each moment across all time periods within the evaluation scope. Specifically, the first time period of the trajectory at the current moment has the greatest impact on the vehicle's actual driving and requires special attention; therefore, the weight for this period is set as w1, and in some embodiments, w1 can range from 0.6 to 0.5. The second time period has a secondary impact, and its weight is set as w2, and in some embodiments, w2 can range from 0.3 to 0.2. The trajectory in the third time period may have a relatively small impact on driving, and its weight is set as w3, and in some embodiments, w3 can range from 0.2 to 0.1.

[0046] Based on the above formula, the weighted calculation of the scores for each indicator of each trajectory yields the following formula for the evaluation score (cost) of that trajectory:

[0047]

[0048] Where cost_x represents the index score for the horizontal position, w x The weight of the indicator score representing the horizontal position is, in some implementations, ranges from 0.15 to 0.1; cost_y represents the indicator score representing the vertical position, w y The weight of the index score representing longitudinal position, in some implementations, ranges from 0.15 to 0.1; cost_lat_a represents the index score of lateral acceleration. The weight of the index score representing lateral acceleration, in some implementations, ranges from 0.2 to 0.15; cost_lon_a represents the index score representing longitudinal acceleration. The weight of the index score representing longitudinal acceleration, in some implementations, ranges from 0.2 to 0.15; cost_lat_jerk represents the index score representing lateral jerk. The weight of the index score representing lateral jerk, in some implementations, ranges from 0.2 to 0.1; cost_lon_jerk represents the index score representing longitudinal jerk. The weight of the index score representing longitudinal jerk is, in some implementations, ranged from 0.2 to 0.1; cost_kappa represents the index score of curvature, w kappaThe weight of the index score representing curvature, in some implementations, ranges from 0.15 to 0.1; cost_s represents the index score representing planning efficiency, w s The weight of the indicator score representing planning efficiency, in some implementations, ranges from 0.25 to 0.2.

[0049] Therefore, the present invention can also store the evaluation score (cost) and various index scores of each trajectory in the same array labeled with the trajectory output time.

[0050] Furthermore, the first time period can be less than 3 seconds, the second time period can be 3-5 seconds, and the third time period can be 5-7 seconds.

[0051] Furthermore, the evaluation method for the autonomous driving trajectory planning module described in this invention also includes the step of: obtaining at least one of the following based on the various index scores and evaluation scores of each trajectory included in the trajectory planning result to be evaluated: the average evaluation score of each trajectory in the trajectory planning result to be evaluated, the maximum and minimum values ​​of the evaluation scores, the distribution of the evaluation scores over time, and the normal distribution graph of the evaluation scores.

[0052] The performance of the trajectory planning test in a specific scenario can be obtained by analyzing the average evaluation score and the normal distribution of the evaluation scores. The maximum and minimum evaluation scores, as well as the time distribution of the evaluation scores, can indicate specific problems in the scenario, allowing for targeted analysis.

[0053] Another objective of this invention is to provide an evaluation system for an autonomous driving trajectory planning module.

[0054] To this end, the present invention also provides an evaluation system for an autonomous driving trajectory planning module, comprising:

[0055] The extraction module obtains the trajectory plan to be evaluated generated by the trajectory planning module and the reference results for comparison with the trajectory planning results.

[0056] The constraint evaluation module iterates through all trajectory points of each trajectory in the trajectory planning result to be evaluated, and compares the velocity, lateral acceleration, longitudinal acceleration, lateral jerk, and longitudinal jerk of each trajectory point with the set velocity limit, lateral acceleration limit, longitudinal acceleration limit, lateral jerk limit, and longitudinal jerk limit respectively. If at least one of them exceeds the limit, the relative time on the trajectory of that trajectory point and the output time of the trajectory where that trajectory point is located are recorded and stored.

[0057] The statistical analysis module performs the following steps: It compares each trajectory in the trajectory planning results to be evaluated with the corresponding reference trajectory in the reference results at the same time to obtain the differences in lateral position, longitudinal position, lateral acceleration, longitudinal acceleration, lateral jerk, longitudinal jerk, curvature, and planning efficiency between the two; based on the differences in lateral position, longitudinal position, lateral acceleration, longitudinal acceleration, lateral jerk, longitudinal jerk, curvature, and planning efficiency, and the set scoring weights, it calculates the index scores for each of the following: lateral position, longitudinal position, lateral acceleration, longitudinal acceleration, lateral jerk, longitudinal jerk, curvature, and planning efficiency; and it performs a weighted calculation of the index scores for each trajectory to obtain the evaluation score for that trajectory.

[0058] Furthermore, in the evaluation system for the autonomous driving trajectory planning module described in this invention, the reference result is generated by another trajectory planning module, or the reference result is the actual driving trajectory collected.

[0059] Furthermore, in the evaluation system for the autonomous driving trajectory planning module described in this invention, the statistical analysis module also obtains at least one of the following based on the various index scores and evaluation scores of each trajectory included in the trajectory planning result to be evaluated: the average evaluation score of each trajectory in the trajectory planning result to be evaluated, the maximum and minimum values ​​of the evaluation scores, the distribution of the evaluation scores over time, and the normal distribution graph of the evaluation scores.

[0060] The evaluation method for the trajectory planning module of autonomous driving described in this invention evaluates the operation results of the entire scenario from two dimensions: hard constraints and statistical indicators, and analyzes the planned trajectory output at each moment, thereby accurately evaluating the performance and decision-making ability of the trajectory planning module.

[0061] Furthermore, the evaluation method for the autonomous driving trajectory planning module described in this invention can be automated, thereby enabling the analysis of a large number of scenarios and providing a valid reference for the development of the autonomous driving trajectory planning module.

[0062] The evaluation system for the autonomous driving trajectory planning module described in this invention also has the aforementioned beneficial effects. Attached Figure Description

[0063] Figure 1 The diagram shows a flowchart of the evaluation method for the autonomous driving trajectory planning module of the present invention in one embodiment. Detailed Implementation

[0064] The evaluation method and system for the autonomous driving trajectory planning module of the present invention will be further explained and described below with reference to the accompanying drawings and specific embodiments. However, this explanation and description do not constitute an undue limitation on the technical solution of the present invention.

[0065] Figure 1 The diagram shows a flowchart of the evaluation method for the autonomous driving trajectory planning module of the present invention in one embodiment.

[0066] like Figure 1 As shown, in this embodiment, the evaluation method for the autonomous driving trajectory planning module includes the following steps:

[0067] 100: Obtain the trajectory planning result generated by the trajectory planning module to be evaluated: The trajectory planning result to be evaluated is generated by the trajectory planning module to be evaluated based on a large amount of collected test scenario data, which includes: map positioning information, perception fusion information and prediction information.

[0068] 200: Obtain a reference result for comparison with the trajectory planning result to be evaluated: In some implementations, the reference result may be generated by another trajectory planning module; in other implementations, the reference result may also be taken from the actual driving trajectory of the actual vehicle driver.

[0069] 301: Hard Constraint Analysis: Traverse all trajectory points of each trajectory included in the trajectory planning results to be evaluated, and compare the velocity, lateral acceleration, longitudinal acceleration, lateral jerk, and longitudinal jerk of each trajectory point with the set velocity limit V. limit Lateral acceleration limit a_lateral limit Longitudinal acceleration limit a_longitudinal limit Lateral jerk_latetal limit Longitudinal jerk_longitudinal limit Compare them separately. If at least one of them exceeds the limit, proceed to step 400 to output and record the evaluation results: record and store the relative time of the trajectory point on the trajectory and the output time of the trajectory where the trajectory point is located.

[0070] In step 301, all trajectories to be evaluated are traversed and parsed in a loop. For each trajectory, all trajectory points are traversed, and the calculated velocity V, lateral acceleration a_lat, and longitudinal acceleration a_lon of that point are compared with V. limit ,a_lateral limit ,a_longitudinal limitThe comparison is performed. If the calculation result exceeds the limit, the output time of the trajectory and the relative time of the point on the trajectory are recorded. The trajectory point that exceeds the limit exceeds the value of the limit index.

[0071] For jerk, which is not typically used as an output on the planned trajectory, it can be determined according to the definition. First, calculate the lateral and longitudinal jerk accelerometers at each moment, and then compare them with jerk_latetal. limit and jerk_longitudinal limit Compare them separately and record the results.

[0072] Step 302: Statistical indicator analysis:

[0073] 3021: Compare each trajectory included in the trajectory planning result to the reference trajectory in the reference result at the same time to obtain the differences in lateral position, longitudinal position, lateral acceleration, longitudinal acceleration, lateral jerk, longitudinal jerk, curvature, and planning efficiency between the two.

[0074] The lateral position difference x_diff = x_ref – x, where x_ref represents the lateral position of the reference result and x represents the lateral position of the trajectory to be evaluated;

[0075] The vertical position difference y_diff = y_ref – y, where y_ref represents the vertical position of the reference result and y represents the horizontal position of the trajectory to be evaluated;

[0076] Lateral acceleration difference a_lat_diff = a_lat_ref - a_lat, where a_lat_ref represents the lateral acceleration of the reference result and a_lat represents the lateral acceleration of the trajectory to be evaluated;

[0077] The longitudinal acceleration difference a_lon_diff = a_lon_ref - a_lon, where a_lon_ref represents the longitudinal acceleration of the reference result and a_lon represents the longitudinal acceleration of the trajectory to be evaluated;

[0078] Lateral jerk_lat_diff = jerk_lat_ref - jerk_lat, where jerk_lat_ref represents the lateral jerk of the reference result and jerk_lat represents the lateral jerk of the trajectory to be evaluated.

[0079] The longitudinal jerk jerk_lon_diff = jerk_lon_ref - jerk_lon, where jerk_lon_ref represents the longitudinal jerk jerk of the reference result and jerk_lon represents the longitudinal jerk jerk of the trajectory to be evaluated.

[0080] The difference in curvature of trajectory points is kappa_diff = kappa_ref – kappa, where kappa_ref represents the curvature of the trajectory points in the reference result, and kappa represents the curvature of the trajectory points in the trajectory to be evaluated.

[0081] The planning efficiency difference s_diff = s_ref – s, where s_ref represents the planning efficiency of the reference result, s represents the planning efficiency of the trajectory to be evaluated, and s represents the farthest distance traveled by the planned route along the entire trajectory.

[0082] 3022: Based on differences in lateral position, longitudinal position, lateral acceleration, longitudinal acceleration, lateral jerk, longitudinal jerk, curvature, and planning efficiency, and the set scoring weights, calculate the following indices: cost_x for lateral position, cost_y for longitudinal position, cost_lat_a for lateral acceleration, cost_lon_a for longitudinal acceleration, cost_lat_jerk for lateral jerk, cost_lon_jerk for longitudinal jerk, cost_kappa for curvature, and cost_s for planning efficiency.

[0083]

[0084]

[0085]

[0086]

[0087]

[0088]

[0089]

[0090] cost_s = s_diff;

[0091] Where x_diff represents the lateral position difference, y_diff represents the longitudinal position difference, a_lat_diff represents the lateral acceleration difference, a_lon_diff represents the longitudinal acceleration difference, jerk_lat_diff represents the lateral jerk jerk_lon_diff represents the longitudinal jerk ...

[0092] In some specific implementations, the value range of w1 can be 0.6 to 0.5, for example 0.6; the value range of w2 can be 0.3 to 0.2, for example 0.3; and the value range of w3 can be 0.2 to 0.1, for example 0.1.

[0093] In some specific implementations, the first time period can be less than 3 seconds, the second time period can be 3-5 seconds, and the third time period can be 5-7 seconds.

[0094] 3023: The weighted scores of each indicator for each trajectory are calculated to obtain the evaluation score (cost) for that trajectory.

[0095]

[0096] Where cost_x represents the index score for the horizontal position, w x The weight of the indicator score representing the horizontal position is, in some implementations, ranges from 0.15 to 0.1; cost_y represents the indicator score representing the vertical position, w y The weight of the index score representing longitudinal position, in some implementations, ranges from 0.15 to 0.1; cost_lat_a represents the index score of lateral acceleration. The weight of the index score representing lateral acceleration, in some implementations, ranges from 0.2 to 0.15; cost_lon_a represents the index score representing longitudinal acceleration. The weight of the index score representing longitudinal acceleration, in some implementations, ranges from 0.2 to 0.15; cost_lat_jerk represents the index score representing lateral jerk. The weight of the index score representing lateral jerk, in some implementations, ranges from 0.2 to 0.1; cost_lon_jerk represents the index score representing longitudinal jerk. The weight of the index score representing longitudinal jerk is, in some implementations, ranged from 0.2 to 0.1; cost_kappa represents the index score of curvature, w kappa The weight of the index score representing curvature, in some implementations, ranges from 0.15 to 0.1; cost_s represents the index score representing planning efficiency, w s The weight of the indicator score representing planning efficiency, in some implementations, ranges from 0.25 to 0.2.

[0097] Therefore, in some embodiments, the present invention can also store the evaluation score (cost) and various index scores of each trajectory in the same array labeled with the trajectory output time.

[0098] For further analysis, in some implementations, based on the various index scores and evaluation scores of each trajectory included in the trajectory planning results to be evaluated, at least one of the following is obtained: the average evaluation score of each trajectory in the trajectory planning results to be evaluated, the maximum and minimum values ​​of the evaluation scores, the distribution of the evaluation scores over time, and the normal distribution graph of the evaluation scores.

[0099] The performance of the trajectory planning under test in a certain scenario can be obtained from the average evaluation score and the normal distribution plot of the evaluation scores. The maximum and minimum values ​​of the evaluation scores, as well as the time distribution of the evaluation scores, can indicate specific problems in the scenario, enabling targeted analysis.

[0100] An embodiment of the present invention also provides an evaluation system for an autonomous driving trajectory planning module, comprising:

[0101] The extraction module obtains the trajectory plan to be evaluated generated by the trajectory planning module and the reference results for comparison with the trajectory planning results.

[0102] The constraint evaluation module iterates through all trajectory points of each trajectory in the trajectory planning result to be evaluated, and compares the velocity, lateral acceleration, longitudinal acceleration, lateral jerk, and longitudinal jerk of each trajectory point with the set velocity limit, lateral acceleration limit, longitudinal acceleration limit, lateral jerk limit, and longitudinal jerk limit respectively. If at least one of them exceeds the limit, the relative time on the trajectory of that trajectory point and the output time of the trajectory where that trajectory point is located are recorded and stored.

[0103] The statistical analysis module performs the following steps: It compares each trajectory in the trajectory planning results to be evaluated with the corresponding reference trajectory in the reference results at the same time to obtain the differences in lateral position, longitudinal position, lateral acceleration, longitudinal acceleration, lateral jerk, longitudinal jerk, curvature, and planning efficiency between the two; based on the differences in lateral position, longitudinal position, lateral acceleration, longitudinal acceleration, lateral jerk, longitudinal jerk, curvature, and planning efficiency, and the set scoring weights, it calculates the index scores for each of the following: lateral position, longitudinal position, lateral acceleration, longitudinal acceleration, lateral jerk, longitudinal jerk, curvature, and planning efficiency; and it performs a weighted calculation of the index scores for each trajectory to obtain the evaluation score for that trajectory.

[0104] In another embodiment of the present invention, the statistical analysis module in the evaluation system for the autonomous driving trajectory planning module further obtains at least one of the following based on the various index scores and evaluation scores of each trajectory included in the trajectory planning result to be evaluated: the average evaluation score of each trajectory in the trajectory planning result to be evaluated, the maximum and minimum values ​​of the evaluation scores, the distribution of the evaluation scores over time, and the normal distribution graph of the evaluation scores.

[0105] The evaluation method and system for autonomous driving trajectory planning modules described in this invention proposes for the first time a method to evaluate the performance of a planning algorithm's trajectory planning by comparing trajectories generated by different planning algorithms or comparing a planning algorithm with an actual driving trajectory. Since the generated trajectory for the entire time period (e.g., 7 seconds) is included in the evaluation scope, this invention can effectively achieve a comprehensive evaluation of the decision-making ability of the trajectory planning module under test in the tested scenario, and identify the worst-performing and more noteworthy points in multiple scenarios.

[0106] In evaluating the trajectory, this invention assigns different weights to each frame of the trajectory. The first time period of the trajectory has the greatest impact on vehicle movement and requires primary attention; the second time period has a secondary impact; and the trajectory in the third time period has little impact on movement, therefore the evaluation criteria should be more lenient than the previous two. Finally, the final evaluation is obtained by combining the weights and various indicators. Therefore, this method provides a more reasonable evaluation and can obtain comprehensive evaluation results. Based on the results, further targeted analysis of each individual result can be conducted.

[0107] The method of this invention can automatically parse large amounts of planning data and generate reports in batches, greatly improving the efficiency of testing and work.

[0108] The combination of the technical features in this case is not limited to the combination methods described in the claims of this case or the combination methods described in the specific embodiments. All technical features described in this case can be freely combined or combined in any way, unless they contradict each other.

[0109] It should also be noted that the embodiments listed above are merely specific embodiments of the present invention. Obviously, the present invention is not limited to the above embodiments, and similar changes or modifications made thereto are those that can be directly derived or easily conceived by those skilled in the art from the content disclosed in the present invention, and should all fall within the protection scope of the present invention.

Claims

1. An evaluation method for an autonomous driving trajectory planning module, characterized in that, Including the following steps: Obtain the trajectory planning results generated by the trajectory planning module to be evaluated; Obtain reference results for comparison with the trajectory planning results to be evaluated; Iterate through all trajectory points of each trajectory in the trajectory planning results to be evaluated, and compare the velocity, lateral acceleration, longitudinal acceleration, lateral jerk, and longitudinal jerk of each trajectory point with the set velocity limit, lateral acceleration limit, longitudinal acceleration limit, lateral jerk limit, and longitudinal jerk limit respectively. If at least one of them exceeds the limit, record and store the relative time of the trajectory point on the trajectory and the output time of the trajectory where the trajectory point is located. Each trajectory included in the trajectory planning result to be evaluated is compared with the reference trajectory corresponding to the same moment in the reference result to obtain the differences in lateral position, longitudinal position, lateral acceleration, longitudinal acceleration, lateral jerk, longitudinal jerk, curvature, and planning efficiency between the two. Based on the differences in lateral position, longitudinal position, lateral acceleration, longitudinal acceleration, lateral jerk, longitudinal jerk, curvature, and planning efficiency, and the set scoring weights, the index scores for each of the following are calculated: lateral position, longitudinal position, lateral acceleration, longitudinal acceleration, lateral jerk, longitudinal jerk, curvature, and planning efficiency. The scores of each indicator for each trajectory are weighted and calculated to obtain the evaluation score for that trajectory; The index score for the horizontal position is calculated using the following formula. Vertical position index score Horizontal acceleration index score Longitudinal acceleration index score Horizontal jerk index score longitudinal acceleration index score Curvature index score and the score of planning efficiency indicators. : ； in, Indicates differences in lateral position. Indicates longitudinal positional differences, Indicates the difference in lateral acceleration. Indicates longitudinal acceleration difference, Indicates the difference in lateral acceleration. Indicates longitudinal jerk difference, Indicates the difference in curvature. This indicates differences in planning efficiency. This indicates the scoring weight set for the first time period (0-t1) of the trajectory. This indicates the scoring weight set for the second time period t1-t2 of the trajectory. This indicates the scoring weight for the third time period t2-t3 of the trajectory.

2. The evaluation method for an autonomous driving trajectory planning module as described in claim 1, characterized in that, The trajectory planning module to be evaluated generates the trajectory planning result based on the scene data, which includes: map positioning information, perception fusion information, and prediction information.

3. The evaluation method for an autonomous driving trajectory planning module as described in claim 1, characterized in that, The reference results are generated by another trajectory planning module.

4. The evaluation method for an autonomous driving trajectory planning module as described in claim 1, characterized in that, The reference results are the actual driving trajectories collected.

5. The evaluation method for an autonomous driving trajectory planning module as described in claim 1, characterized in that, The first time interval is within 3 seconds, the second time interval is 3-5 seconds, and the third time interval is 5-7 seconds.

6. The evaluation method for an autonomous driving trajectory planning module as described in any one of claims 1-5, characterized in that, It also includes the following steps: based on the various index scores and evaluation scores of each trajectory included in the trajectory planning results to be evaluated, obtain at least one of the following: the average evaluation score of each trajectory in the trajectory planning results to be evaluated, the maximum and minimum values ​​of the evaluation scores, the distribution of the evaluation scores over time, and the normal distribution graph of the evaluation scores.

7. An evaluation system for an autonomous driving trajectory planning module, characterized in that, include: The extraction module obtains the trajectory plan to be evaluated generated by the trajectory planning module and the reference results for comparison with the trajectory planning results. The constraint evaluation module iterates through all trajectory points of each trajectory in the trajectory planning result to be evaluated, and compares the velocity, lateral acceleration, longitudinal acceleration, lateral jerk, and longitudinal jerk of each trajectory point with the set velocity limit, lateral acceleration limit, longitudinal acceleration limit, lateral jerk limit, and longitudinal jerk limit respectively. If at least one of them exceeds the limit, the relative time of the trajectory point on the trajectory and the output time of the trajectory where the trajectory point is located are recorded and stored. The statistical analysis module performs the following steps: Each trajectory included in the trajectory planning result to be evaluated is compared with the reference trajectory corresponding to the same moment in the reference result to obtain the differences in lateral position, longitudinal position, lateral acceleration, longitudinal acceleration, lateral jerk, longitudinal jerk, curvature, and planning efficiency between the two. Based on the differences in lateral position, longitudinal position, lateral acceleration, longitudinal acceleration, lateral jerk, longitudinal jerk, curvature, and planning efficiency, and the set scoring weights, the index scores for each of the following are calculated: lateral position, longitudinal position, lateral acceleration, longitudinal acceleration, lateral jerk, longitudinal jerk, curvature, and planning efficiency. The scores of each indicator for each trajectory are weighted and calculated to obtain the evaluation score for that trajectory; The index score for the horizontal position is calculated using the following formula. Vertical position index score Horizontal acceleration index score Longitudinal acceleration index score Horizontal jerk index score longitudinal acceleration index score Curvature index score and the score of planning efficiency indicators. : ； in, Indicates differences in lateral position. Indicates longitudinal positional differences, Indicates the difference in lateral acceleration. Indicates longitudinal acceleration difference, Indicates the difference in lateral acceleration. Indicates longitudinal jerk difference, Indicates the difference in curvature. This indicates differences in planning efficiency. This indicates the scoring weight set for the first time period (0-t1) of the trajectory. This indicates the scoring weight set for the second time period t1-t2 of the trajectory. This indicates the scoring weight for the third time period t2-t3 of the trajectory.

8. The evaluation system for an autonomous driving trajectory planning module as described in claim 7, characterized in that, The reference result is generated by another trajectory planning module, or the reference result is the actual driving trajectory collected.

9. The evaluation system for an autonomous driving trajectory planning module as described in claim 7 or 8, characterized in that, The statistical analysis module also obtains at least one of the following based on the various indicator scores and evaluation scores of each trajectory in the trajectory planning results to be evaluated: the average evaluation score of each trajectory in the trajectory planning results to be evaluated, the maximum and minimum values ​​of the evaluation scores, the distribution of the evaluation scores over time, and the normal distribution graph of the evaluation scores.

Images