Calibration method and system for end-to-end automatic driving module

Abstract

The invention discloses a calibration method and system for an end-to-end automatic driving module. The method comprises the following steps: establishing a rule set which needs to be obeyed by vehicle driving and a corresponding atomic proposition; acquiring a planned path point output given by the automatic driving module in the current state; carrying out two-dimensional gridding on a driving environment, and exploring an optimal route in the environment through a Monte Carlo random sampling method to meet a given vehicle driving rule set; and according to the optimized route, a planned path point given by the automatic driving module is adjusted so as to control the vehicle to run. According to the method, traffic rules needing to be obeyed by vehicle driving are defined by a small amount of manpower, so that the end-to-end automatic driving module is calibrated, and the problem that the safety of a black-box neural network model is difficult to guarantee is solved. Because the manually set rule has interpretability, the calibrated automatic driving module complies with the set rule, so that the decision of the automatic driving module has interpretability, and the safety guarantee is provided.

Description

Technical Field

[0001] The present invention relates to a calibration method and system for an end-to-end autonomous driving module, belonging to the technical fields of information technology, autonomous driving, etc. Background Art

[0002] With the rapid development of deep neural networks, autonomous driving modules have also achieved many successes. One trend is that existing autonomous driving modules adopt end-to-end neural network models to complete the entire driving task. However, autonomous driving modules sometimes encounter problems in actual applications. The end-to-end autonomous driving module lacks the constraints brought by explicit rules. Due to the black-box nature of the neural network model, we have no deterministic understanding of the behavior of the autonomous driving module established thereon. The neural network model is data-driven, so we cannot know exactly what driving rules the autonomous driving module has learned, and thus cannot check or correct these rules. The autonomous driving module may very well learn some error-prone driving methods with low probability from some incorrect or contaminated data, which is difficult for humans to perceive until an accident occurs, and the cost is extremely high.

[0003] However, although sufficient testing can cover some boundary cases, thereby partially ensuring the safety of the autonomous driving module. Although testing is useful, it is also difficult to completely solve the problem. Autonomous driving testing is a very costly matter and also has safety risks. It is basically impossible to construct test scenarios that can cover all boundary conditions in real scenarios. Some researchers also conduct simulation tests in simulation scenarios, but due to the problem of transferability of the neural network model, the simulation test cannot completely replace the real-scenario test. Summary of the Invention

[0004] Object of the Invention: Aiming at the problems and deficiencies in the prior art, the present invention provides a calibration method and system for an end-to-end autonomous driving module, to solve the problems of poor interpretability and difficult-to-guarantee safety encountered by the end-to-end autonomous driving module during operation.

[0005] Technical Solution: A calibration method for an end-to-end autonomous driving module includes the following steps: Step 1: Establish a rule set that the vehicle needs to abide by and corresponding atomic propositions; Step 2: Obtain the planned path point output given by the autonomous driving module in the current state; Step 3: Grid the driving environment two-dimensionally, and record the vehicle's own data and relevant environmental data, including the vehicle's own position, the vehicle's own speed, the vehicle's own acceleration, the positions of other vehicles, the speeds of other vehicles, the accelerations of other vehicles, and the positions of obstacles on the road; Step 4: Add random perturbations to the next planned path point given by the autonomous driving module to obtain the initial calibration path point. Then, based on this point, continue to generate new path points through random exploration to form a calibration path; Step 5: Screen and remove paths that do not meet the rule restrictions through the rule set, and establish a scoring function to score and evaluate the paths that meet the restrictions; Step 6: Calibrate the path given by the autonomous driving module according to the score of the path to generate calibrated path points.

[0006] To implement and optimize the above technical solutions, the specific measures taken also include: Further, establishing the rule set that the vehicle needs to abide by during driving and the corresponding atomic propositions includes: Step 101: Set the set of basic information during vehicle driving, including the position of the host vehicle, the speed of the host vehicle, the acceleration of the host vehicle, the positions of other vehicles, the speeds of other vehicles, the accelerations of other vehicles, and the positions of other obstacles; Step 102: Through artificial knowledge, combine these basic information into atomic propositions of first-order logic, that is, the distance between the host vehicle and other vehicles should be greater than a preset threshold, the speed of the host vehicle should be less than a certain preset threshold, and the acceleration of the host vehicle should be less than a certain preset threshold.

[0007] Further, obtaining the output of the planned path point given by the autonomous driving module in the current state includes: Step 201: Obtain the coordinates (Xw, Yw, Zw) of the output of the planned path point given by the autonomous driving module in the three-dimensional world coordinate system; Step 202: Through coordinate transformation, obtain the coordinates (X1, Y1, Z1) of the output of the planned path point in the host vehicle coordinate system.

[0008] Further, two-dimensionally grid the driving environment and record the host vehicle data and relevant environment data, including the position of the host vehicle, the speed of the host vehicle, the acceleration of the host vehicle, the positions of other vehicles, the speeds of other vehicles, the accelerations of other vehicles, the positions of obstacles on the road, and the position of the center line on the map, including: Step 301: Obtain the host vehicle data and relevant environment data in the three-dimensional world coordinate system, including the position of the host vehicle, the speed of the host vehicle, the acceleration of the host vehicle, the positions of other vehicles, the accelerations of other vehicles, and the obstacles on the road.

[0009] Step 302: Establish a host vehicle plane coordinate system with the center of the vehicle as the origin, the front of the vehicle as the positive x-axis direction, and the right side of the vehicle as the positive y-axis direction; Step 303: Project the host vehicle data and relevant environment data in the three-dimensional world coordinate system onto the host vehicle plane coordinate system to obtain two-dimensionally gridded driving environment data.

[0010] Further, adding a random perturbation to the next planned path point given by the autonomous driving module to obtain an initial calibration path point, and then, based on this point, continuing to generate new path points through random exploration to form a calibration path, includes: Step 401: Randomly select a point (X2, Y2) within the relevant neighborhood of the next planned point (X1, Y1) given by the autonomous driving module, and satisfy X1 < X2 < X1 + 1m, Y1 - 1m < Y2 < Y1 + 1m; Step 402: Randomly select a point (X3, Y3) within the relevant neighborhood of (X2, Y2) obtained in Step 401, and satisfy X2 < X3 < X2 + 1m, Y2 - 1m < Y3 < Y2 + 1m; Step 403: Continuing to generate (X4, Y4), (X5, Y5) and (X6, Y6) by imitating the above steps.

[0011] Step 404: Connect (X1, Y1), (X2, Y2), (X3, Y3), (X4, Y4), (X5, Y5) and (X6, Y6) in sequence to obtain a calibration path.

[0012] Further, screening and removing paths that do not meet the rule restrictions through a rule set, and establishing a scoring function to score and evaluate the paths that meet the restrictions, includes: Step 501: Calculate the steering angle of the vehicle at each moment according to the displacement change of the calibration path; Step 502: For the calibration path whose steering angle exceeds the vehicle performance limit, it is determined that it does not meet the rule requirements and is deleted; Step 503: For the remaining paths, calculate the score of the path, and the calculation formula is: score = α * turning radius of the vehicle - β * minimum distance from the vehicle to other obstacles and vehicles - γ * distance from the vehicle to the center line. Where α, β, and γ are constants greater than 0.

[0013] Further, calibrating the path given by the autonomous driving module according to the score of the path to generate calibrated path points, includes: Step 601: Select the calibration path with the highest score and obtain its first output point; Step 602: Calibrate the output of the autonomous driving module to the output point obtained in Step 601, and control the vehicle to drive to this point.

[0014] A calibration system for an end-to-end autonomous driving module includes the following modules: Module 1: Establish a rule set that the vehicle needs to abide by and the corresponding atomic propositions; Module 2: Obtain the planned path point output given by the autonomous driving module in the current state; Module 3: Grid the driving environment two-dimensionally, and record the vehicle's own data and relevant environmental data, including the vehicle's own position, the vehicle's own speed, the vehicle's own acceleration, the positions of other vehicles, the speeds of other vehicles, the accelerations of other vehicles, and the positions of obstacles on the road; Module 4: Add random perturbations to the next planned path point given by the autonomous driving module to obtain an initial calibrated path point, and then, based on this point, continue to generate new path points through random exploration to form a calibrated path; Module 5: Screen and remove paths that do not meet the rule restrictions through a rule set, and establish a scoring function to score and evaluate the paths that meet the restrictions; Module 6: Calibrate the path given by the autonomous driving module according to the score of the path to generate calibrated path points.

[0015] The implementation method of the calibration system for the end-to-end autonomous driving module is the same as the above method and will not be elaborated here.

[0016] A computer device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the above computer program, it implements the calibration method for the end-to-end autonomous driving module as described above.

[0017] A computer-readable storage medium stores a computer program for executing the calibration method for the end-to-end autonomous driving module as described above.

[0018] Advantageous effects: Compared with the prior art, the present invention can calibrate the end-to-end autonomous driving module using logical rules, realize safe and reliable calibration of the planned output of the autonomous driving module according to the rule set, and solve the problems of uncontrollable output and insufficient safety of the end-to-end autonomous driving module. Description of the Drawings

[0019] Figure 1 It is the flowchart of the method of the embodiment of the present invention; Figure 2 It is the schematic diagram of the method of this embodiment. Detailed Embodiments

[0020] The following further clarifies the present invention in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and not to limit the scope of the present invention. After reading the present invention, various equivalent forms of modification of the present invention by those skilled in the art all fall within the scope defined by the appended claims of this application.

[0021] As Figure 1As shown in the figure, a calibration method for an end-to-end autonomous driving module provided by an embodiment of the present invention includes: Step 1, establish a rule set that the vehicle needs to abide by during driving and corresponding atomic propositions.

[0022] Among them, the vehicle in the embodiment of the present invention is the vehicle controlled by the autonomous driving module to be calibrated. The rule set is the traffic rules that the vehicle to be calibrated needs to abide by, and the atomic proposition is the first-order propositional representation form of these traffic rules. In order to help calibrate the autonomous driving module through the rule set, the set rule set needs to include reasonable vehicle driving traffic rules. For example, when the target vehicle in the embodiment of the present invention is driving on a straight road, the rule set can be that the vehicle and the vehicle in front should maintain a certain distance.

[0023] Step 2, obtain the planned path point output given by the autonomous driving module in the current state.

[0024] Among them, the planned path point output can be the coordinate representation of the planned path given after the autonomous driving module accepts information such as vision, radar, and map. The point cloud data can include the planned path point output of the whole or a part of the above reference object. For example, when the target autonomous driving module in the embodiment of the present invention is UniAD, the planned path point data can include the path points that the vehicle should plan to output every 0.5 seconds given by the autonomous driving module.

[0025] Step 3, grid the driving environment two-dimensionally, and record the vehicle's own data and relevant environmental data, including the vehicle's own position, the vehicle's own speed, the vehicle's own acceleration, the positions of other vehicles, the speeds of other vehicles, the accelerations of other vehicles, and the positions of obstacles on the road.

[0026] Among them, the vehicle's own data can be obtained through devices or methods such as vehicle sensors, GPS, and IMU. The relevant environmental data can be obtained through vehicle sensors, map information, vision sensors, etc. The obtained three-dimensional data will be projected onto the plane where the vehicle is driving to form two-dimensional data. For example, when the autonomous driving module vehicle in the embodiment of the present invention is driving in the Carla simulator, the vehicle's own information and relevant environmental information can be obtained by the sensors installed on the vehicle and surrounding vehicles in the Carla simulator.

[0027] Step 4, add random perturbations to the next planned path point given by the autonomous driving module to obtain the initial calibration path point, and then, based on this point, continue to generate new path points through random exploration to form a calibration path.

[0028] Among them, the next planned path point given by the autonomous driving module is represented as the horizontal direction offset from the host vehicle. In one example, the coordinates of the next planned point P1 given by the autonomous driving module in the embodiment of the present invention are (-1, 1). After random perturbation that complies with the rules, the offset coordinates are C1(-0.5, 1.1). Subsequently, with C1(-0.5, 1.1) as the origin, the path point P2(-0.5, -0.5) is randomly generated continuously, and so on until all planned path points are generated.

[0029] Step 5, filter out the paths that do not meet the rule restrictions through the rule set, and establish a scoring function to score and evaluate the paths that meet the restrictions.

[0030] Among them, to filter out the paths that do not meet the rules, it can be filtered according to factors such as the smoothness and length of the path results. For the paths that meet the requirements, a scoring function can be established. The function should include all the rules that should be included, and then the paths are sorted from high to low according to the scores. In one example, in the embodiment of the present invention, three paths are generated. Among them, path 1 does not meet the requirement that the steering angle of the path is less than 30 degrees and is filtered out. Path 2 and path 3 meet the requirements and are scored. Among them, the score of path 2 is greater than that of path 3, and path 2 is selected.

[0031] Step 6, calibrate the path given by the autonomous driving module according to the scores of the paths to generate calibrated path points. Among them, for the generated path, if it consists of multiple points, the first one is used as the planned output at the current moment of the vehicle. The output at subsequent moments can use the subsequent path points at the current moment, or can be recalculated at subsequent moments. To control the vehicle to reach the planned path point, traditional control algorithms can be used, or an end-to-end model can be used. In one example, in the embodiment of the present invention, the calibration system generates a planned path composed of 6 planned points, takes the first point as the planned point output, and moves the vehicle to this path point through the transport api in the Carla simulator.

[0032] A system for calibrating an end-to-end autonomous driving module includes the following modules: Module 1: Establish a rule set that the vehicle needs to abide by during driving and the corresponding atomic propositions. Among them, the vehicle in the embodiment of the present invention is the vehicle controlled by the autonomous driving module to be calibrated. The rule set is the traffic rules that the vehicle to be calibrated needs to abide by, and the atomic proposition is the first-order proposition representation form of these traffic rules. In order to help calibrate the autonomous driving module through the rule set, the set rule set needs to include reasonable vehicle driving traffic rules. For example, when the target vehicle in the embodiment of the present invention is driving on a straight road, the rule set can be that the vehicle and the vehicle in front should maintain a certain distance.

[0033] Module 2: Obtain the planned path point output given by the autonomous driving module in the current state. Among them, the planned path point output can be the coordinate representation of the planned path after planning by the autonomous driving module after receiving information such as vision, radar, and map. The point cloud data can include the planned path point output of the whole or a part of the above reference object. For example, when the target autonomous driving module of the embodiment of the present invention is UniAD, the planned path point data can include the path points that the vehicle should plan to output every 0.5 seconds given by the autonomous driving module.

[0034] Module 3: Grid the driving environment two-dimensionally and record the vehicle's own data and relevant environmental data, including the vehicle's own position, the vehicle's own speed, the vehicle's own acceleration, the positions of other vehicles, the speeds of other vehicles, the accelerations of other vehicles, and the positions of obstacles on the road.

[0035] Add random perturbations to the next planned path point given by the autonomous driving module to obtain the initial calibrated path point, and then, based on this point, continue to generate new path points through random exploration to form a calibrated path.

[0036] Module 5: Screen and remove the paths that do not meet the rule restrictions through a rule set, and establish a scoring function to score and evaluate the paths that meet the restrictions.

[0037] Module 6: Calibrate the path given by the autonomous driving module according to the score of the path to generate the calibrated path points.

[0038] Obviously, those skilled in the art should understand that each step of the above calibration method for the end-to-end autonomous driving module of the embodiment of the present invention or each module of the calibration system for the end-to-end autonomous driving module can be implemented by a general-purpose computing device. They can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. Optionally, they can be implemented by program codes executable by the computing device. Thus, they can be stored in a storage device and executed by the computing device. And in some cases, the steps shown or described can be executed in a different order than here, or they can be separately made into individual integrated circuit modules, or multiple modules or steps among them can be made into a single integrated circuit module to implement. In this way, the embodiments of the present invention are not limited to any specific combination of hardware and software.

Claims

1. A calibration method for an end-to-end autonomous driving module, characterized in that: The steps include: Step 1: Establish a set of rules that vehicles must follow and the corresponding atomic propositions; Step 2: Obtain the planned path point output given by the automatic driving module in the current state; Step 3: Grid the driving environment in two dimensions and record the vehicle data and related environment data; Step 4: Add random perturbations to the next planned path point given by the autonomous driving module to obtain the initial calibration path point, and then use this point as the base point to continue to generate new path points through random exploration to form a calibration path; Step 5: Remove the paths that do not meet the rule restrictions through rule set screening, and establish a scoring function to score and evaluate the paths that meet the restrictions; Step 6: Calibrate the path given by the autonomous driving module according to the path score and generate calibrated path points.

2. The calibration method for an end-to-end autonomous driving module according to claim 1, characterized in that: The establishment of a set of rules that the vehicle must comply with and the corresponding atomic propositions include: Step 101: setting a basic information set during vehicle driving, including the position of the vehicle, the speed of the vehicle, the acceleration of the vehicle, the positions of other vehicles, the speeds of other vehicles, the accelerations of other vehicles, and the positions of other obstacles; Step 102: Through artificial knowledge, the basic information is combined into an atomic proposition of first-order logic, that is, the distance between the vehicle and other vehicles should be greater than a preset threshold, the speed of the vehicle should be less than a preset threshold, and the acceleration of the vehicle should be less than a preset threshold.

3. The calibration method for an end-to-end autonomous driving module according to claim 1, characterized in that: Get the planned path point output given by the automatic driving module in the current state, including: Step 201: Obtain the coordinates (Xw, Yw, Zw) of the planned path point output given by the autonomous driving module in the three-dimensional world coordinate system; Step 202: Obtain the coordinates (X1, Y1, Z1) of the planned path point output in the vehicle coordinate system through coordinate transformation.

4. The calibration method for an end-to-end autonomous driving module according to claim 1, characterized in that: The two-dimensional gridding of the driving environment and recording of the vehicle data and related environmental data include: Step 301: Obtaining the vehicle data and related environment data in the three-dimensional world coordinate system, including the vehicle position, vehicle speed, vehicle acceleration, other vehicle positions, other vehicle accelerations, and obstacles on the road; Step 302: Establish a self-vehicle plane coordinate system with the center of the vehicle as the origin, the front of the vehicle as the positive direction of the x-axis, and the right side of the vehicle as the positive direction of the y-axis; Step 303: Project the vehicle data and related environment data in the three-dimensional world coordinate system to the vehicle plane coordinate system to obtain two-dimensional gridded driving environment data.

5. The calibration method for an end-to-end autonomous driving module according to claim 1, characterized in that: The method adds random disturbance to the next planned path point given by the autonomous driving module to obtain an initial calibration path point, and then uses this point as a base point to continue to generate new path points by random exploration to form a calibration path, including: Step 401: Randomly select a point (X2, Y2) in the relevant neighborhood of the next planning point (X1, Y1) given by the autonomous driving module, and satisfy X1 <X2<X1+1m,Y1-1m<Y2<Y1+1m; Step 402: Randomly select a point (X3, Y3) in the relevant neighborhood of (X2, Y2) obtained in step 401, and satisfy X2 <X3<X2+1m,Y2-1m<Y3<Y2+1m; Step 403: Following the above steps, continue to generate (X4, Y4), (X5, Y5) and (X6, Y6); Step 404: Connect (X1, Y1), (X2, Y2), (X3, Y3), (X4, Y4), (X5, Y5) and (X6, Y6) in sequence to obtain a calibration path.

6. The calibration method for an end-to-end autonomous driving module according to claim 1, characterized in that: The method of removing paths that do not meet the rule restrictions through rule set screening and establishing a scoring function to score and evaluate the paths that meet the restrictions includes: Step 501: Calculate the steering angle of the vehicle at each moment according to the displacement change of the calibration path; Step 502: For the calibration path whose steering angle exceeds the vehicle performance limit, it is determined that it does not meet the regulation requirements and is deleted; Step 503: For the remaining path, calculate the score of the path, and the calculation formula is: score = α* turning radius of the vehicle - β* minimum distance from the vehicle to other obstacles and vehicles - γ* distance from the vehicle to the center line; where α, β, γ are constants greater than 0.

7. The calibration method for an end-to-end autonomous driving module according to claim 1, characterized in that: The step of calibrating the path given by the autonomous driving module according to the score of the path and generating calibrated path points includes: Step 601: Select the calibration path with the largest score and obtain its first output point; Step 602: Calibrate the output of the autonomous driving module to the output point obtained in step 601, and control the vehicle to drive to this point.

8. A calibration system for an end-to-end autonomous driving module, characterized in that: Includes the following modules: Module 1: Establish a set of rules that vehicles must follow and the corresponding atomic propositions; Module 2: Obtain the planned path point output given by the automatic driving module in the current state; Module 3: Grid the driving environment in two dimensions and record the vehicle data and related environment data; Module 4: Add random perturbations to the next planned path point given by the autonomous driving module to obtain the initial calibration path point, and then use this point as the base point to continue to generate new path points through random exploration to form a calibration path; Module 5: Remove paths that do not meet the rule restrictions through rule set screening, and establish a scoring function to score and evaluate the paths that meet the restrictions; Module 6: Calibrate the path given by the autonomous driving module according to the path score and generate calibrated path points.

9. A computer device, characterized in that: The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, a calibration method for an end-to-end autonomous driving module as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program for executing a calibration method for an end-to-end autonomous driving module as described in any one of claims 1-7.

Images