A calibration method and system for end-to-end autonomous driving module

Abstract

The application discloses a kind of calibration methods and systems for end-to-end automatic driving module, establish the rule set that vehicle driving needs to comply with and corresponding atomic proposition;Get the planning path point output given by automatic driving module under current state;Two-dimensional grid is formed to driving environment, and the optimal route in the environment is explored by Monte Carlo random sampling method to meet the given vehicle driving rule set;According to the optimized route, the planning path point given by automatic driving module is adjusted to control vehicle driving.The application uses a small amount of artificial definition traffic rules that vehicle driving needs to comply with, thereby calibrating end-to-end automatic driving module, solve the problem that black box neural network model safety is difficult to guarantee.Because the rule set by artificial has explainability, the automatic driving module after calibration will comply with the rule set, so that the decision of automatic driving module can have explainability, so there is safety guarantee.

Description

TECHNICAL FIELD

[0001] The present application relates to a calibration method and system for end-to-end autonomous driving module, belonging to the technical fields of information technology, autonomous driving and the like. BACKGROUND

[0002] With the rapid development of deep neural networks, autonomous driving modules have also been successful. One trend is that existing autonomous driving modules use end-to-end neural network models to complete the entire driving task. However, the autonomous driving module sometimes has problems in actual application. End-to-end autonomous driving modules lack the constraints brought by explicit rules. Due to the black box nature of neural network models, we have no deterministic knowledge of the behavior of the autonomous driving module built on it. Neural network models are data-driven, so we cannot know what driving rules the autonomous driving module has learned, so as to check or correct these rules. The autonomous driving module may have learned some small probability of error driving methods from some error or contaminated data, which is difficult for humans to perceive, and it cannot be discovered until an accident occurs, which is very costly.

[0003] However, although sufficient testing can cover some boundary conditions, thus partially ensuring the safety of the autonomous driving module. Although testing is useful, it is also difficult to completely solve the problem. Autonomous driving testing is a very costly thing and has safety risks, and it is basically impossible to build a test scene that can cover all boundary conditions in a real scene. Some researchers also simulate testing in simulation scenes, but due to the problem of transferability of neural network models, simulation testing cannot completely replace real scene testing. SUMMARY

[0004] The present application provides a calibration method and system for end-to-end autonomous driving module, which solves the problems of poor explainability and difficulty in ensuring safety of end-to-end autonomous driving module in operation.

[0005] Technical scheme: A calibration method for end-to-end autonomous driving module, comprising the following steps:

[0006] Step 1: Establish a rule set that the vehicle must comply with and the corresponding atomic proposition;

[0007] Step 2: Obtain the planning path point output given by the autonomous driving module under the current state;

[0008] Step 3: Two-dimensional grid the driving environment, and record the ego vehicle data and related environment data, including ego vehicle position, ego vehicle speed, ego vehicle acceleration, position of other vehicles, speed of other vehicles, acceleration of other vehicles, and position of obstacles on the road;

[0009] Step 4: Add a random disturbance to the next planning path point given by the autonomous driving module to obtain an initial calibration path point, and then continue to generate new path points based on this point through random exploration to form a calibration path;

[0010] Step 5: Remove paths that do not meet the rule restrictions through rule set screening, and establish a scoring function to score and evaluate paths that meet the restrictions;

[0011] Step 6: Calibrate the path given by the autonomous driving module according to the score of the path to generate a calibrated path point.

[0012] To implement and optimize the above technical solutions, the following specific measures are taken:

[0013] Further, the rule set that the vehicle must follow during driving and the corresponding atomic proposition are established, including:

[0014] Step 101: Set the basic information set during vehicle driving, including the position of the ego vehicle, the speed of the ego vehicle, the acceleration of the ego vehicle, the position of the remaining vehicles, the speed of the remaining vehicles, the acceleration of the remaining vehicles, and the position of the remaining obstacles;

[0015] Step 102: Combine these basic information into first-order logic atomic propositions through artificial knowledge, i.e., the distance between the ego vehicle and the remaining vehicles should be greater than a preset threshold, the speed of the ego vehicle should be less than a certain preset threshold, and the acceleration of the ego vehicle should be less than a certain preset threshold.

[0016] Further, the planning path point output given by the autonomous driving module under the current state is obtained, including:

[0017] Step 201: Obtain the coordinates (Xw, Yw, Zw) of the planning path point output given by the autonomous driving module in the three-dimensional world coordinate system;

[0018] Step 202: Through coordinate transformation, obtain the coordinates (X1, Y1, Z1) of the planning path point output in the ego vehicle coordinate system.

[0019] Further, the driving environment is two-dimensionally gridded, and the ego vehicle data and related environment data are recorded, including the ego vehicle position, the ego vehicle speed, the ego vehicle acceleration, the position of the remaining vehicles, the speed of the remaining vehicles, the acceleration of the remaining vehicles, the position of the obstacles on the road, and the position of the center line on the map, including:

[0020] Step 301: Obtain the ego vehicle data and related environment data in the three-dimensional world coordinate system, including the ego vehicle position, the ego vehicle speed, the ego vehicle acceleration, the position of the remaining vehicles, the acceleration of the remaining vehicles, and the obstacles on the road.

[0021] Step 302: Establish a vehicle-centered planar coordinate system with the vehicle as the origin, the front of the vehicle as the positive x-axis direction, and the right side of the vehicle as the positive y-axis direction;

[0022] Step 303: Project the vehicle data and related environmental data in the three-dimensional world coordinate system into the vehicle planar coordinate system to obtain two-dimensional grid-based driving environment data.

[0023] Further, the next planning path point given by the automatic driving module is added with random disturbance to obtain an initial calibration path point, and then a new path point is generated based on the point as the base point through random exploration to form a calibration path, including:

[0024] Step 401: Randomly select a point (X2, Y2) in the neighborhood of the next planning point (X1, Y1) given by the automatic driving module, and satisfy X1 < X2 < X1 + 1m, Y1 - 1m < Y2 < Y1 + 1m;

[0025] Step 402: Randomly select a point (X3, Y3) in the neighborhood of (X2, Y2) obtained in step 401, and satisfy X2 < X3 < X2 + 1m, Y2 - 1m < Y3 < Y2 + 1m;

[0026] Step 403: Continue to generate (X4, Y4), (X5, Y5) and (X6, Y6) by imitating the above steps.

[0027] Step 404: Connect (X1, Y1), (X2, Y2), (X3, Y3), (X4, Y4), (X5, Y5) and (X6, Y6) in order to obtain a calibration path.

[0028] Further, the paths that do not meet the rule restrictions are removed through rule set screening, and a scoring function is established to score and evaluate the paths that meet the restrictions, including:

[0029] Step 501: Calculate the steering angle of the vehicle at each time according to the displacement change of the calibration path;

[0030] Step 502: For the calibration path whose steering angle exceeds the vehicle performance limit, it is determined that it does not meet the rule requirements and is deleted;

[0031] Step 503: For the remaining paths, calculate the score of the path, the calculation formula is: score = α * turning radius of the vehicle - β * minimum distance from the vehicle to other obstacles and vehicles - γ * distance from the vehicle to the center line. Wherein α, β, γ are constants greater than 0.

[0032] Further, the path given by the automatic driving module is calibrated according to the score of the path to generate a calibrated path point, including:

[0033] Step 601: select the calibration path with the highest score, and get its first output point;

[0034] Step 602: calibrate the output of the automatic driving module to the output point obtained in step 601, and control the vehicle to drive to the point.

[0035] A calibration system for an end-to-end automatic driving module, comprising the following modules:

[0036] Module 1: establish a rule set that the vehicle needs to follow when driving and the corresponding atomic proposition;

[0037] Module 2: obtain the planning path point output given by the automatic driving module under the current state;

[0038] Module 3: grid the driving environment in two dimensions, and record the ego vehicle data and related environment data, including the ego vehicle position, ego vehicle speed, ego vehicle acceleration, position of other vehicles, speed of other vehicles, acceleration of other vehicles, and position of obstacles on the road;

[0039] Module 4: add random disturbance to the next planning path point given by the automatic driving module to obtain an initial calibration path point, and then take the point as a base point to continue to generate new path points through random exploration to form a calibration path;

[0040] Module 5: remove paths that do not meet the rule restrictions through rule set screening, and establish a scoring function to score and evaluate paths that meet the restrictions;

[0041] Module 6: calibrate the path given by the automatic driving module according to the score of the path to generate a calibrated path point.

[0042] The implementation method of the calibration system for the end-to-end automatic driving module is the same as the above method, and will not be described again.

[0043] A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the above-mentioned computer program to realize the calibration method for the end-to-end automatic driving module as described above.

[0044] A computer readable storage medium storing a computer program for executing the calibration method for the end-to-end automatic driving module as described above.

[0045] Beneficial effects: compared with the prior art, the present application can calibrate the end-to-end automatic driving module using logical rules, realize safe and reliable calibration of the automatic driving module planning output according to the rule set, and solve the problem of uncontrollable end-to-end automatic driving module output and insufficient safety. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 Method flowchart for the embodiments of the present application;

[0047] Figure 2 Method schematic for the embodiments of the present application. DETAILED DESCRIPTION

[0048] The present application will be further illustrated below in conjunction with specific embodiments, which are intended to illustrate but not limit the scope of the present application. After reading the present application, those skilled in the art can make various modifications to the present application, which fall within the scope defined by the appended claims.

[0049] As shown in Figure 1 , the calibration method for end-to-end autonomous driving module provided by the embodiments of the present application comprises:

[0050] Step 1, establishing a rule set that the vehicle needs to comply with and corresponding atomic propositions.

[0051] Among them, the vehicle in the embodiments of the present application is the vehicle controlled by the automatic driving module to be calibrated, the rule set is the traffic rules that the vehicle to be calibrated needs to comply with, and the atomic proposition is the first-order proposition representation form of these traffic rules. In order to help calibrate the automatic driving module through the rule set, the rule set set needs to contain reasonable vehicle driving traffic rules, for example, when the target vehicle of the embodiments of the present application is driving on a straight road, the rule set can be that the vehicle and the preceding vehicle should maintain a certain distance.

[0052] Step 2, obtaining the planning path point output given by the automatic driving module under the current state.

[0053] Among them, the planning path point output can be the coordinate representation of the planning path given by the automatic driving module after accepting visual, radar, map and other information. The point cloud data can contain the whole or part of the path planning point output of the above-mentioned reference objects. For example, when the target automatic driving module of the embodiments of the present application is UniAD, the planning path point data can include the path point that the automatic driving module should plan to output every 0.5 seconds.

[0054] Step 3, two-dimensional grid of driving environment, and record the self-vehicle data and related environment data, including self-vehicle position, self-vehicle speed, self-vehicle acceleration, position of other vehicles, speed of other vehicles, acceleration of other vehicles, position of obstacles on the road.

[0055] The ego vehicle data can be obtained by vehicle sensors, GPS, IMU, etc. The relevant environment data can be obtained by vehicle sensors, map information, visual sensors, etc. The obtained three-dimensional data is projected onto the plane on which the vehicle travels, forming two-dimensional data. For example, when the autonomous driving module vehicle of the embodiment of the present application is driving in the Carla simulator, the ego vehicle information and the relevant environment information can be obtained by the sensors arranged on the ego vehicle and the surrounding vehicles in the Carla simulator.

[0056] Step 4: Add random disturbance to the next planning path point given by the autonomous driving module to obtain an initial calibration path point, and then continue to generate new path points based on this point by random exploration to form a calibration path.

[0057] In the next planning path point given by the autonomous driving module, the horizontal direction offset from the ego vehicle is represented. In one example, the coordinates of the next planning point P1 given by the autonomous driving module in the embodiment of the present application are (-1, 1), and after the random disturbance, the coordinates of the offset are C1(-0.5, 1.1). Then, taking C1(-0.5, 1.1) as the origin, continue to randomly generate path points P2(-0.5, -0.5), and so on, until all planning path points are generated.

[0058] Step 5: Remove paths that do not meet the rule restrictions by screening, and establish a scoring function to score and evaluate paths that meet the restrictions.

[0059] Among them, the paths that do not meet the rules can be screened according to the smoothness, length and other factors of the path result. For paths that meet the requirements, a scoring function can be established, which should include all the rules that should be included, and then the paths are sorted in descending order of score. In one example, in the embodiment of the present application, three paths are generated, among which path 1 does not meet the requirement that the turning angle of the path is less than 30 degrees and is screened out. Path 2 and path 3 meet the requirements and are scored, among which the score of path 2 is greater than that of path 3, and path 2 is selected.

[0060] Step 6: According to the score of the path, the path given by the autonomous driving module is calibrated to generate the calibrated path point. Among them, for the generated path, if it is composed of multiple points, the first one is taken as the planning output of the vehicle at the current time, and the subsequent path points can be used for the output at the subsequent time or recalculated at the subsequent time. For controlling the vehicle to reach the planning path point, traditional control algorithm can be used, or end-to-end model can be used. In one example, in the embodiment of the present application, the calibration system generates a planning path composed of 6 planning points, takes the first point as the planning point output, and moves the vehicle to the path point in the Carla simulator through transport api.

[0061] The system for calibrating end-to-end autonomous driving module includes the following modules:

[0062] Module one: Establish a rule set that the vehicle needs to follow and the corresponding atomic proposition. In the embodiment of the present application, the vehicle is the vehicle controlled by the autonomous driving module to be calibrated, the rule set is the traffic rules that the vehicle to be calibrated needs to follow, and the atomic proposition is the first-order proposition representation of these traffic rules. In order to help calibrate the autonomous driving module through the rule set, the rule set set needs to include reasonable vehicle driving traffic rules, for example, when the target vehicle of the embodiment of the present application is driving on a straight road, the rule set can be that the vehicle and the preceding vehicle should maintain a certain distance.

[0063] Module two: Obtain the planning path point output given by the autonomous driving module under the current state. The planning path point output can be the coordinate representation of the planning path given by the autonomous driving module after accepting visual, radar, map and other information, and the point cloud data can include the whole or part of the path planning point output of the above-mentioned reference object. For example, when the target autonomous driving module of the embodiment of the present application is UniAD, the planning path point data can include the path point that the autonomous driving module should plan to output every 0.5 seconds.

[0064] Module three: Two-dimensional grid the driving environment, and record the ego data and related environment data, including ego position, ego speed, ego acceleration, position of other vehicles, speed of other vehicles, acceleration of other vehicles, and position of obstacles on the road.

[0065] Add random disturbance to the next planning path point given by the autonomous driving module to obtain the initial calibration path point, and then take the point as the base point to continue to generate new path points in a random exploration manner to form a calibration path.

[0066] Module five: Remove the path that does not meet the rule limit through rule set screening, and establish a scoring function to score and evaluate the path that meets the limit.

[0067] Module six: Calibrate the path given by the autonomous driving module according to the score of the path to generate the calibrated path point.

[0068] Obviously, those skilled in the art should understand that each step of the calibration method for the end-to-end autonomous driving module of the above-mentioned embodiments of the present application or each module of the calibration system for the end-to-end autonomous driving module can be realized by a general computing device, which can be concentrated on a single computing device or distributed on a network composed of multiple computing devices, and optionally, they can be realized by program codes executable by the computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in different order, or they can be made into individual integrated circuit modules, or multiple modules or steps among them can be made into a single integrated circuit module. Thus, the embodiments of the present application are not limited to any specific combination of hardware and software.

Claims

1. A calibration method for an end-to-end autonomous driving module, characterized in that, The method comprises the following steps: Step 1: establishing a rule set that the vehicle driving needs to comply with and corresponding atomic propositions; Step 2: obtaining the planning path point output given by the automatic driving module under the current state; Step 3: two-dimensional griding the driving environment and recording the self-vehicle data and related environment data; Step 4: adding random disturbance to the next planning path point given by the automatic driving module to obtain an initial calibration path point, and then taking the point as a base point to continue to generate new path points in a random exploration manner to form a calibration path; Step 5: removing the paths that do not meet the rule restrictions through rule set screening, and establishing a scoring function to score and evaluate the paths that meet the restrictions; Step 6: calibrating the path given by the automatic driving module according to the score of the path to generate a calibrated path point.

2. The calibration method for end-to-end autonomous driving module according to claim 1, wherein, The establishment of the rule set that the vehicle driving needs to comply with and the corresponding atomic propositions comprises: Step 101: setting a basic information set in the vehicle driving process, including the position of the self-vehicle, the speed of the self-vehicle, the acceleration of the self-vehicle, the positions of the remaining vehicles, the speeds of the remaining vehicles, the accelerations of the remaining vehicles, and the positions of the remaining obstacles; Step 102: combining the basic information into first-order logic atomic propositions through artificial knowledge, that is, the distance between the self-vehicle and the remaining vehicles should be greater than a preset threshold, the speed of the self-vehicle should be less than a preset threshold, and the acceleration of the self-vehicle should be less than a preset threshold. 3.The end-to-end autonomous driving module-oriented calibration method according to claim 1, wherein, The obtaining of the planning path point output given by the automatic driving module under the current state comprises: Step 201: obtaining the coordinates (Xw, Yw, Zw) of the planning path point output given by the automatic driving module in the three-dimensional world coordinate system; Step 202: obtaining the coordinates (X1, Y1, Z1) of the planning path point output in the self-vehicle coordinate system through coordinate transformation. 4.The method of calibrating end-to-end autonomous driving modules according to claim 1, wherein, The two-dimensional griding of the driving environment and the recording of the self-vehicle data and related environment data comprise: Step 301: obtaining the self-vehicle data and related environment data in the three-dimensional world coordinate system, including the position of the self-vehicle, the speed of the self-vehicle, the acceleration of the self-vehicle, the positions of the remaining vehicles, the accelerations of the remaining vehicles, and the obstacles on the road; Step 302: establishing a self-vehicle plane coordinate system with the center of the vehicle as the origin, the front of the vehicle as the positive direction of the x-axis, and the right side of the vehicle as the positive direction of the y-axis; Step 303: projecting the self-vehicle data and related environment data in the three-dimensional world coordinate system to the self-vehicle plane coordinate system to obtain two-dimensional grided driving environment data. 5.The end-to-end autonomous driving module-oriented calibration method according to claim 1, wherein, The adding of random disturbance to the next planning path point given by the automatic driving module to obtain an initial calibration path point, and then taking the point as a base point to continue to generate new path points in a random exploration manner to form a calibration path comprise: Step 401: randomly taking a point (X2, Y2) in the neighborhood of the next planning point (X1, Y1) given by the automatic driving module, and satisfying X1 < X2 < X1 + 1m, Y1 - 1m < Y2 < Y1 + 1m; Step 402: randomly taking a point (X3, Y3) in the neighborhood of (X2, Y2) obtained in step 401, and satisfying X2 < X3 < X2 + 1m, Y2 - 1m < Y3 < Y2 + 1m; Step 403: Continue to generate (X4, Y4), (X5, Y5) and (X6, Y6) in the same way as the above steps; Step 404: Connect (X1, Y1), (X2, Y2), (X3, Y3), (X4, Y4), (X5, Y5) and (X6, Y6) in order to obtain a calibration path. 6.The end-to-end autonomous driving module-oriented calibration method according to claim 1, wherein, The rule set is used to filter out paths that do not meet the rule restrictions, and a scoring function is established to score and evaluate paths that meet the restrictions, including: Step 501: Calculate the steering angle of the ego vehicle at each time according to the displacement change of the calibration path; Step 502: For the calibration path whose steering angle exceeds the vehicle performance limit, it is determined that it does not meet the rule requirements and is deleted; Step 503: For the remaining paths, calculate the score of the path, the calculation formula is: score = a * self-turning radius - b * the minimum distance from the ego vehicle to other obstacles and vehicles - g * the distance from the ego vehicle to the center line; Where a, b, g are constants greater than 0.

7. The calibration method for end-to-end autonomous driving modules according to claim 1, wherein, The calibration path is generated according to the score of the path, including: Step 601: Select the calibration path with the highest score to obtain its first output point; Step 602: Calibrate the output of the automatic driving module to the output point obtained in step 601 to control the vehicle to travel to the point.

8. A calibration system for end-to-end autonomous driving modules, characterized in that, The method comprises the following modules: Module one: Establish a rule set that the vehicle must comply with and the corresponding atomic proposition; Module two: Obtain the planning path point output given by the automatic driving module under the current state; Module three: Two-dimensional grid the driving environment, and record the ego vehicle data and related environmental data; Module four: Add random disturbance to the next planning path point given by the automatic driving module to obtain an initial calibration path point, and then continue to generate new path points based on the point as the base point through random exploration to form a calibration path; Module five: Remove paths that do not meet the rule restrictions through the rule set, and establish a scoring function to score and evaluate paths that meet the restrictions; Module six: Calibrate the path given by the automatic driving module according to the score of the path to generate calibrated path points.

9. A computer device, comprising: The computer device comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor executes the above computer program to realize the calibration method for the end-to-end automatic driving module according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program for executing the calibration method for the end-to-end automatic driving module according to any one of claims 1-7.

Images