An automatic driving planning and control collaborative training method and system
By employing a collaborative training method based on Transformer-based planning networks and model predictive controllers, the problems of decoupling between planning and control and training instability in autonomous driving are solved. This method generates dynamically executable trajectories that meet safety and traffic rules, thereby improving the safety and comfort of autonomous driving.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- TONGJI UNIV
- Filing Date
- 2026-04-24
- Publication Date
- 2026-07-03
Smart Images
Figure CN122087462B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of autonomous driving technology, and in particular to an autonomous driving planning and control collaborative training method and system. Background Technology
[0002] Autonomous driving systems typically consist of modules for perception, prediction, planning, and control. The planning module generally outputs a future reference trajectory or path, while the control module tracks the planning results based on the vehicle's dynamics model and outputs control commands (such as steering, longitudinal driving force / braking force, etc.). This modular structure has advantages such as interpretability and debuggability, but it still suffers from the problem of decoupling between planning and control: planning often generates trajectories based on simplified kinematic or geometric constraints, making it difficult to explicitly reflect dynamic limitations such as tire sideslip, center of gravity sideslip angle, and yaw rate. This results in the planned trajectory being difficult to track stably when executed by a real vehicle, requiring the controller to have large compensation inputs and even causing stability risks.
[0003] In recent years, end-to-end learning methods have emerged, directly inputting predictive control commands from the environment and reducing manual module design. However, due to the lack of explicit intermediate constraint structures, they often struggle to guarantee traffic rules and safety constraints, and may produce unexplained anomalous behaviors in distributed scenarios. To balance engineering controllability and learning generalization, integrated planning and control methods have emerged, attempting to incorporate planning and control into training through differentiable structures, so that the planning output inherently satisfies executability and safety. However, existing integrated planning and control methods still have shortcomings:
[0004] (1) The planning variables mostly take trajectory points as the optimization objects, which is inconsistent with the control input space, resulting in "the trajectory is feasible but the control is difficult to track";
[0005] (2) Cost terms are often processed by uniform weighted summation, and safety and rules may be compromised by comfort, making it difficult to reflect the constraint hierarchy of "safety first";
[0006] (3) The lack of cost scheduling strategy during the training phase leads to unstable gradients in the early stage, getting stuck in local optima or training divergence;
[0007] (4) Due to the limitations of the algorithm type, the solution efficiency of the planning and control algorithm is low, which makes it difficult to meet the strict real-time requirements of the embedded vehicle platform.
[0008] Therefore, it is necessary to propose a new integrated planning and control training scheme to unify planning and control variables, adopt hierarchical constraint expression of safety priority relationships, and improve training stability and closed-loop reliability through multi-stage cost scheduling. Summary of the Invention
[0009] The purpose of this invention is to provide a method and system for collaborative training of autonomous driving planning and control, which solves the problems of planning and control decoupling, dynamic infeasibility and training instability. It enables the trajectory output by the planner to balance comfort and efficiency while meeting the hard constraints of safety and traffic rules, and can be collaboratively optimized through the dynamic feedback gradient of the controller.
[0010] To achieve the above objectives, the present invention provides a method for collaborative training of autonomous driving planning and control, comprising the following steps:
[0011] S1. Obtain the current state information of the autonomous vehicle, the state information of surrounding traffic participants, and reference road information;
[0012] S2. Input the acquired information into the Transformer-based planning network, output the control sequence in the future time domain, and input the control sequence into the vehicle kinematics model to generate the planned trajectory;
[0013] S3. Input the planned trajectory into a neural network controller based on model predictive control for dynamic tracking optimization to obtain the executed control quantity and vehicle dynamic state prediction;
[0014] S4. Construct the control loss based on the vehicle dynamics state prediction, and backpropagate the gradient of the control loss with respect to the control sequence along the differentiable link to the Transformer-based planning network to update the parameters of the Transformer-based planning network.
[0015] S5. In the process of jointly training the Transformer-based planning network and the model predictive control-based neural network controller, the planning cost term is calculated according to the planning trajectory, and the planning cost term is subjected to constraint hierarchical processing. The safety and traffic rule related terms are treated as hard constraints, the comfort related terms are treated as soft constraints, and a multi-stage cost scheduling strategy is adopted to adjust the constraint strength or weight of different stages.
[0016] Preferably, the control sequence in S2 includes longitudinal acceleration and steering angle, wherein the steering angle is the front wheel steering angle or equivalent steering angle.
[0017] Preferably, the vehicle dynamics state prediction in S3 includes the center of gravity sideslip angle, yaw rate, and lateral acceleration.
[0018] Preferably, the control loss in S4 includes a stability penalty term for the center of mass sideslip angle, yaw rate, and lateral acceleration.
[0019] Preferably, the hard constraints in S5 include collision safety distance constraints, red light stop line constraints, speed limit constraints, and lane boundary constraints, and the hard constraints are approximately hardened through differentiable obstacle functions or augmented Lagrangian forms; the soft constraints include acceleration amplitude, acceleration, steering angle amplitude, and steering angle change rate.
[0020] Preferably, the formula for calculating the joint loss function used in S5 for joint training is as follows:
[0021] ;
[0022] in, To imitate the loss, This is a soft constraint term. For the index of soft constraint terms, Hard constraint function Differentiable penalty function, To control the loss, For the index of hard constraint terms, , , , These are the corresponding weighting coefficients.
[0023] Preferably, the multi-stage cost scheduling strategy in S5 includes: the first stage trains a Transformer-based planning network with imitation loss as the main method; the second stage gradually strengthens the hard constraints related to road geometry and traffic rules in the safety and traffic rule related items; the third stage gradually strengthens the safety-related hard constraints in the safety and traffic rule related items, and introduces the soft constraints related to comfort and the control loss in S4 for joint optimization; the constraint strength or weight is smoothly adjusted between each stage using linear annealing or cosine annealing.
[0024] An autonomous driving planning and control collaborative training system includes:
[0025] The environmental information acquisition module outputs the current status information of the autonomous vehicle, the status information of surrounding traffic participants, and reference road information.
[0026] The planning module has a built-in Transformer-based planning network and vehicle kinematics model. The input of the planning module is connected to the output of the environmental information acquisition module. The planning module outputs the future time domain control sequence from the Transformer-based planning network, and the vehicle kinematics model converts the control sequence into a planned trajectory.
[0027] The control module has a built-in neural network controller based on model predictive control. The input of the control module is connected to the output of the planning module. The control module performs dynamic tracking optimization on the planned trajectory and outputs the control quantity and vehicle dynamic state prediction.
[0028] The constraint layering module is used to layer the constraints of the planning cost items corresponding to the planning trajectory, treating safety and traffic rule-related items as hard constraints and comfort-related items as soft constraints.
[0029] The training scheduling module is used to execute multi-stage cost scheduling strategies and adjust the constraint strength or weight of different stages.
[0030] The parameter update module is used to backpropagate the gradient of the control loss relative to the control sequence along the differentiable link to the Transformer-based planning network in the planning module, so as to update the parameters of the Transformer-based planning network.
[0031] Preferably, the neural network controller based on model predictive control includes a differentiable optimization layer, a learnable iterative solver, and a neural network approximation solver.
[0032] Therefore, the present invention employs the above-mentioned method and system for collaborative training of autonomous driving planning and control, which has the following beneficial effects:
[0033] (1) This invention uses safety constraints and traffic rule constraints as hard constraints and comfort constraints as soft constraints for hierarchical optimization, which explicitly reflects the constraint hierarchy of safety priority, avoids the compromise of safety performance by comfort indicators, reduces the sensitivity to weight parameter settings, and effectively reduces traffic violations.
[0034] (2) The present invention adopts a multi-stage cost scheduling strategy, which gradually introduces imitation learning, rule constraints, safety constraints and comfort constraints in different training stages, reduces the difficulty of non-convex training, improves the model convergence speed and training stability, and avoids the problem of early gradient instability or getting trapped in local optima.
[0035] (3) This invention enables the planning network to have dynamic perception capabilities by backpropagating the gradient of the control loss relative to the control sequence along differentiable links. The generated planning trajectory inherently satisfies dynamic executability, thereby reducing the compensation intensity of the controller and improving driving comfort and system robustness.
[0036] (4) The present invention directly outputs the control sequence through the planning network, avoiding the iterative solution process between the planning layer and the control layer in the traditional method, and significantly reducing the online calculation delay; at the same time, the controller can use a neural network approximation solver to replace the traditional numerical optimization, further improving the solution efficiency and meeting the strict real-time requirements of the embedded vehicle platform.
[0037] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description
[0038] Figure 1 This is a flowchart of a collaborative training method for autonomous driving planning and control according to the present invention;
[0039] Figure 2 This is a schematic diagram of trajectory generation based on a Transformer-based planning network, which is part of the autonomous driving planning and control collaborative training method of the present invention.
[0040] Figure 3 This is a flowchart of a multi-stage cost scheduling strategy for an autonomous driving planning and control collaborative training method according to the present invention.
[0041] Figure 4 This is a diagram of the architecture of an autonomous driving planning and control collaborative training system according to the present invention. Detailed Implementation
[0042] The following detailed description of embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely illustrates selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.
[0043] Example
[0044] like Figure 1 As shown, this invention provides a method for collaborative training of autonomous driving planning and control, comprising the following steps:
[0045] S1. Acquire the current state information of the autonomous vehicle, the state information of surrounding traffic participants, and reference road information; provide necessary environmental and autonomous vehicle information for subsequent network planning. The current state information includes the vehicle's position, speed, and heading angle; the state information of surrounding traffic participants includes the position, speed, and predicted trajectory of neighboring vehicles; and the reference road information includes road centerlines, lane boundaries, speed limits, traffic lights, etc. This information is obtained through onboard sensors and high-precision maps.
[0046] S2. Input the acquired information into the Transformer-based planning network, output the control sequence in the future time domain, and input the control sequence into the vehicle kinematics model to generate the planned trajectory; the control sequence includes longitudinal acceleration and steering angle, and the steering angle is the front wheel steering angle or equivalent steering angle.
[0047] like Figure 2As shown, the Transformer-based planning network employs an encoder-decoder structure. The encoder performs multi-head attention fusion on the autonomous vehicle's state token, neighboring vehicle's state token, prediction token, and reference line token; the decoder uses the temporal embedding as the query and outputs the control sequence for the next T steps. The control sequence is input into the vehicle's kinematics model, and the planned trajectory is obtained through integration.
[0048] Taking a bicycle model as an example, the formula for the vehicle's kinematics model is as follows:
[0049] ;
[0050] ;
[0051] ;
[0052] ;
[0053] in, Wheelbase For time step, For longitudinal acceleration, For the front wheel steering angle, For vehicles at any time speed, For heading angle, , This represents the vehicle's position in the global coordinate system. The resulting trajectory... Used for calculating planning costs and serving as a reference input for the control module, this step unifies the planning variable and control input spaces, avoiding the problem of "feasible trajectory but difficult control tracking" in traditional methods.
[0054] S3. Input the planned trajectory into a neural network controller based on model predictive control for dynamic tracking optimization to obtain the control quantity to be executed and the predicted vehicle dynamic state; the predicted vehicle dynamic state includes the center of gravity sideslip angle, yaw rate and lateral acceleration.
[0055] The neural network controller based on model predictive control performs rolling optimization of the planned trajectory under the constraints of the vehicle dynamics model, outputs the actual control commands (such as steering torque and longitudinal force), and predicts dynamic states such as sideslip angle, yaw rate, and lateral acceleration. These dynamic states reflect the vehicle's stability margin and provide a basis for the subsequent construction of control losses.
[0056] S4. Construct the control loss based on the vehicle dynamics state prediction, and backpropagate the gradient of the control loss with respect to the control sequence along the differentiable link to the Transformer-based planning network to update the parameters of the Transformer-based planning network; the control loss includes stability penalty terms for the centroid sideslip angle, yaw rate and lateral acceleration.
[0057] By constructing a control loss (e.g., penalizing large centroid sideslip angles, excessively high yaw rates, and lateral accelerations), the planning network becomes aware of dynamic constraints. Gradients are backpropagated along a differentiable path: "control sequence → kinematic trajectory → controller solution → dynamic state → control loss," enabling collaborative learning between planning and control. This mechanism ensures that the trajectory generated by the planning network inherently satisfies dynamic executability, reducing the intensity of controller compensation.
[0058] S5. In the process of jointly training the Transformer-based planning network and the model predictive control-based neural network controller, the planning cost term is calculated according to the planning trajectory, and the planning cost term is subjected to constraint hierarchical processing. The safety and traffic rule related terms are treated as hard constraints, the comfort related terms are treated as soft constraints, and a multi-stage cost scheduling strategy is adopted to adjust the constraint strength or weight of different stages.
[0059] Hard constraints include collision safety distance constraints, red light stop line constraints, speed limit constraints, and lane boundary constraints. Hard constraints are approximately hardened using differentiable obstacle functions or augmented Lagrangian forms. Soft constraints include acceleration amplitude, acceleration, steering angle amplitude, and steering angle change rate, which are penalized using weighted quadratic or robust function forms to optimize driving quality while satisfying hard constraints.
[0060] By using constraint hierarchical modeling, "safety / rules priority" can be explicitly expressed, avoiding the trade-off between safety and comfort, reducing the sensitivity of weight tuning, and improving system verifiability.
[0061] The formula for calculating the joint loss function used in joint training is as follows:
[0062] ;
[0063] in, To imitate the loss, This is a soft constraint term. For the index of soft constraint terms, Hard constraint function Differentiable penalty functions (e.g., smooth hinge functions or augmented Lagrange forms). To control the loss, For the index of hard constraint terms, , , , These are the corresponding weighting coefficients.
[0064] like Figure 3 As shown, the multi-stage cost scheduling strategy includes: the first stage trains a Transformer-based planning network with imitation loss as the main method; the second stage gradually strengthens the hard constraints related to road geometry and traffic rules in the safety and traffic rule related terms; the third stage gradually strengthens the safety-related hard constraints in the safety and traffic rule related terms, and introduces the soft constraints of the comfort-related terms and the control loss in S4 for joint optimization; the constraint strength or weight is smoothly adjusted between each stage using linear annealing or cosine annealing; the multi-stage cost scheduling strategy avoids early gradient instability and getting trapped in local optima, and improves the training convergence speed.
[0065] like Figure 4 As shown, an autonomous driving planning and control collaborative training system includes:
[0066] The environmental information acquisition module outputs the current status information of the autonomous vehicle, the status information of surrounding traffic participants, and reference road information.
[0067] The planning module has a built-in Transformer-based planning network and a vehicle kinematics model. The input of the planning module is connected to the output of the environmental information acquisition module. The planning module outputs the future time-domain control sequence from the Transformer-based planning network, and the vehicle kinematics model converts the control sequence into a planned trajectory.
[0068] The control module incorporates a neural network controller based on model predictive control. The input of the control module is connected to the output of the planning module. The control module performs dynamic tracking optimization on the planned trajectory and outputs the control quantity and vehicle dynamic state prediction. The neural network controller based on model predictive control includes a differentiable optimization layer, a learnable iterative solver, and a neural network approximation solver.
[0069] The constraint layering module is used to layer the constraints of the planning cost items corresponding to the planning trajectory, treating safety and traffic rule-related items as hard constraints and comfort-related items as soft constraints.
[0070] The training scheduling module is used to execute multi-stage cost scheduling strategies and adjust the constraint strength or weight of different stages.
[0071] The parameter update module is used to backpropagate the gradient of the control loss relative to the control sequence along the differentiable link to the Transformer-based planning network in the planning module, so as to update the parameters of the Transformer-based planning network.
[0072] The modules work collaboratively to form a closed-loop training system. During training, the environmental information acquisition module sends data to the planning module, which generates control sequences and planned trajectories; the control module performs dynamic tracking optimization and outputs dynamic states; the constraint hierarchical module and the training scheduling module adjust the weights of the loss terms; and the parameter update module calculates gradients and updates the planning network parameters. Through this process, collaborative training of planning and control is achieved.
[0073] Therefore, the present invention adopts the above-mentioned autonomous driving planning and control collaborative training method and system, which solves problems such as planning and control decoupling, dynamic infeasibility and training instability by using constraint layering, multi-stage cost scheduling and dynamic gradient backpropagation, thereby improving the safety and comfort of autonomous driving.
[0074] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.
Claims
1. An automatic driving planning and control collaborative training method, characterized in that, Includes the following steps: S1. Obtain the current state information of the autonomous vehicle, the state information of surrounding traffic participants, and reference road information; S2. Input the acquired information into the Transformer-based planning network, output the control sequence in the future time domain, and input the control sequence into the vehicle kinematics model to generate the planned trajectory; S3. Input the planned trajectory into a neural network controller based on model predictive control for dynamic tracking optimization to obtain the executed control quantity and vehicle dynamic state prediction; S4. Construct the control loss based on the vehicle dynamics state prediction, and backpropagate the gradient of the control loss with respect to the control sequence along the differentiable link to the Transformer-based planning network to update the parameters of the Transformer-based planning network. S5. In the process of jointly training the Transformer-based planning network and the model predictive control-based neural network controller, the planning cost term is calculated according to the planning trajectory, and the planning cost term is subjected to constraint hierarchical processing. The safety and traffic rule related terms are treated as hard constraints, the comfort related terms are treated as soft constraints, and a multi-stage cost scheduling strategy is adopted to adjust the constraint strength or weight of different stages. The formula for calculating the joint loss function used in joint training is as follows: ； in, To imitate the loss, This is a soft constraint term. For the index of soft constraint terms, Hard constraint function Differentiable penalty function, To control the loss, For the index of hard constraint terms, , , , These are the corresponding weight coefficients; Hard constraints include collision safety distance constraints, red light stop line constraints, speed limit constraints, and lane boundary constraints. Hard constraints are approximately hardened using differentiable obstacle functions or augmented Lagrangian forms. Soft constraints include acceleration amplitude, acceleration, steering angle amplitude, and steering angle change rate. The multi-stage cost scheduling strategy includes: the first stage trains a Transformer-based planning network with imitation loss as the main method; the second stage gradually strengthens the hard constraints related to road geometry and traffic rules in the safety and traffic rule related terms; the third stage gradually strengthens the safety-related hard constraints in the safety and traffic rule related terms, and introduces the soft constraints related to comfort and the control loss in S4 for joint optimization; the constraint strength or weight is smoothly adjusted between each stage using linear annealing or cosine annealing.
2. The autonomous driving planning and control collaborative training method according to claim 1, characterized in that: The control sequence in S2 includes longitudinal acceleration and steering angle, where the steering angle is the front wheel angle or equivalent steering angle.
3. The autonomous driving planning and control collaborative training method according to claim 1, characterized in that: The vehicle dynamics prediction in S3 includes the center of gravity sideslip angle, yaw rate, and lateral acceleration.
4. The autonomous driving planning and control collaborative training method according to claim 1, characterized in that: The control loss in S4 includes stability penalties for the center of mass sideslip angle, yaw rate, and lateral acceleration.
5. An autonomous driving planning and control collaborative training system, used in the autonomous driving planning and control collaborative training method as described in any one of claims 1-4, characterized in that, include: The environmental information acquisition module outputs the current status information of the autonomous vehicle, the status information of surrounding traffic participants, and reference road information. The planning module has a built-in Transformer-based planning network and vehicle kinematics model. The input of the planning module is connected to the output of the environmental information acquisition module. The planning module outputs the future time domain control sequence from the Transformer-based planning network, and the vehicle kinematics model converts the control sequence into a planned trajectory. The control module has a built-in neural network controller based on model predictive control. The input of the control module is connected to the output of the planning module. The control module performs dynamic tracking optimization on the planned trajectory and outputs the execution control quantity and vehicle dynamic state prediction. The constraint layering module is used to layer the constraints of the planning cost items corresponding to the planning trajectory, treating safety and traffic rule-related items as hard constraints and comfort-related items as soft constraints. The training scheduling module is used to execute multi-stage cost scheduling strategies and adjust the constraint strength or weight of different stages. The parameter update module is used to backpropagate the gradient of the control loss relative to the control sequence along the differentiable link to the Transformer-based planning network in the planning module, so as to update the parameters of the Transformer-based planning network.
6. The autonomous driving planning and control collaborative training system according to claim 5, characterized in that: The neural network controller based on model predictive control includes a differentiable optimization layer, a learnable iterative solver, and a neural network approximation solver.