An Autonomous Driving Trajectory Predictive Control Method Oriented to Control Executability

Abstract

This invention discloses an autonomous driving trajectory prediction control method oriented towards control executability, belonging to the field of autonomous driving trajectory prediction and vehicle motion control technology, to address the problems of insufficient predictive trajectory continuity, physical feasibility, and control executability. The method employs a modular end-to-end architecture including a prediction module, a model predictive control nominal solver unit, and a residual control correction network. Based on vehicle driving samples, it constructs a locally vectorized scene representation, generates discrete multimodal predicted trajectories, Bézier curve parameterized trajectories, and modal probabilities, and combines nominal control solving, bounded residual correction, and joint loss constraints to generate the final control sequence for autonomous vehicle trajectory prediction and motion control.

Description

Technical Field

[0001] This invention relates to the field of autonomous vehicle trajectory prediction and motion control technology, and in particular to an autonomous driving trajectory prediction control method, electronic device and computer-readable storage medium oriented towards controllability. Background Technology

[0002] With the development of autonomous driving and intelligent connected vehicle technologies, vehicles in complex traffic environments need to predict their future trajectories based on their own historical motion states, the motion states of surrounding traffic participants, and road structure and semantic information, and further generate reference trajectories for vehicle controllers to track. Existing trajectory prediction methods primarily optimize the positional error of future discrete trajectory points. While these methods achieve good results in terms of average displacement error and endpoint displacement error, they lack sufficient constraints on the continuity, curvature variation, heading smoothness, and controllability of the predicted trajectory, easily leading to problems such as local jitter, abrupt curvature changes, or heading discontinuities. For autonomous vehicles, the predicted trajectory not only needs to approximate the actual future trajectory but should also be able to be converted into executable control commands that satisfy vehicle kinematic constraints and control input constraints. If the predicted trajectory itself is discontinuous or unsmooth, even with small discrete positional errors, it will cause the controller to frequently adjust the control input, thereby increasing the rate of control change and reducing closed-loop tracking stability.

[0003] Model predictive control (MDC) is a crucial control method for trajectory tracking in autonomous vehicles, capable of solving control sequences within a finite prediction time domain by combining vehicle kinematics models and control input boundary constraints. However, MDC typically uses a reference trajectory as an external input, assuming that this trajectory possesses good kinematic rationality and controllability. When the reference trajectory is directly provided by a discrete trajectory prediction model, local jitter, curvature abrupt changes, or heading discontinuities in the reference trajectory increase the difficulty of control solution and exacerbate changes in control input. Furthermore, existing prediction and control modules are often relatively independent, with the prediction module only providing the reference trajectory to the control module unidirectionally. Tracking errors, control change rates, and residual compensation requirements during the control process are difficult to feed back into the prediction and control co-optimization process through unified loss constraints. Therefore, how to maintain multimodal trajectory prediction capabilities while ensuring better motion continuity, physical feasibility, and control executability of the predicted trajectory is a key technical challenge that needs to be addressed in the field of autonomous driving trajectory predictive control. Summary of the Invention

[0004] This invention provides an autonomous driving trajectory prediction control method, electronic device, and computer-readable storage medium oriented towards control executability, which addresses the problems of insufficient constraints on trajectory continuity, physical feasibility, and control executability in existing trajectory prediction methods, as well as the lack of unified optimization constraints among the prediction module, the model predictive control nominal solution module, and the residual control correction module.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: A vectorized scene representation is constructed based on the historical motion state of the autonomous vehicle to be controlled, the motion state of surrounding traffic participants, and road structure and semantic information; the vectorized scene representation is input into a prediction module to generate multiple prediction mode results, including discrete multimodal prediction trajectories, parameterized continuous reference trajectories, and corresponding mode probabilities; a control reference mode is determined based on the mode probabilities in the prediction mode results, and the parameterized continuous reference trajectory corresponding to the control reference mode is sampled into a continuous reference trajectory sampling sequence; the continuous reference trajectory sampling sequence is input into a model predictive control nominal solution module to solve for a nominal control sequence and a nominal state sequence based on the vehicle kinematics model and control input constraints; a residual control sequence is generated through a residual control correction module based on the continuous reference trajectory sampling sequence, the nominal control sequence, and the nominal state sequence; the nominal control sequence is corrected using the residual control sequence to obtain a final control sequence, and longitudinal acceleration / deceleration control commands and lateral steering control commands for the autonomous vehicle to be controlled are generated based on the final control sequence. Furthermore, the autonomous driving trajectory prediction and control model utilizes a residual control correction mechanism and employs a total loss function that includes trajectory physical constraints and control executability constraints to jointly optimize the prediction module and the residual control correction module.

[0006] In the modular end-to-end joint optimization process, this invention constructs a total loss function consisting of trajectory prediction loss, discrete curve alignment loss, curve physics loss, and control closed-loop loss. Specifically, the trajectory prediction loss constrains the deviations between the discrete multimodal predicted trajectory and the parameterized continuous reference trajectory and the true future trajectory; the discrete curve alignment loss constrains the positional deviations between the discrete multimodal predicted trajectory and the parameterized continuous reference trajectory under the same prediction mode; the curve physics loss, as a trajectory physics constraint, constrains the motion continuity, curvature smoothness, and physical rationality of the parameterized continuous reference trajectory; and the control closed-loop loss, as a control executability constraint, constrains the tracking error of the executed trajectory, the control change rate of the final control sequence, and the magnitude of the residual control sequence.

[0007] Compared with existing technologies, this invention has at least the following beneficial effects: First, by co-predicting discrete multimodal predicted trajectories and parameterized trajectories using Bézier curves, this invention enables the prediction module to simultaneously retain the high degree of freedom of expression of discrete trajectories and the good continuity of parameterized curves. Second, this invention combines nominal solution of model predictive control with residual control correction, so that the control sequence has both nominal control priors under vehicle kinematic constraints and can reduce model errors or local control abrupt changes through bounded residual compensation. Third, this invention feeds back tracking error, control rate of change, and residual magnitude to the modular end-to-end joint optimization process through control closed-loop loss, so that the quality of predicted trajectory is no longer constrained only by discrete position error, but also by control executability. Fourth, this invention can generate continuous, smooth, and sampleable control reference trajectories, which is beneficial for reducing the control rate of change and improving the closed-loop tracking stability of autonomous vehicles. Attached Figure Description

[0008] Figure 1 This is a schematic diagram of a general framework for trajectory prediction and control co-optimization oriented towards control executability provided by an embodiment of the present invention. It shows the relationship between input and scene construction, trajectory parameterization co-prediction, model predictive control nominal solution, residual control correction and unified loss backpropagation.

[0009] Figure 2 This is a schematic diagram of an autonomous driving trajectory prediction control method for control executability provided in an embodiment of the present invention. It shows the relationship between information acquisition and vectorized scene representation, trajectory parameterized collaborative prediction, control reference trajectory determination, model predictive control nominal solution, residual control correction, control command generation, and modular end-to-end joint optimization.

[0010] Figure 3 This is a schematic diagram of an onboard electronic device structure for trajectory prediction and control collaborative optimization provided in an embodiment of the present invention. It shows the connection relationship between the external input unit, processor, memory, communication interface, vehicle actuator and computer-readable storage medium.

[0011] Figure 4 This is a schematic diagram of the cumulative distribution of the improvement in the maximum control rate of change in an embodiment of the present invention. The horizontal axis represents the difference in the maximum control rate of change between the model-predicted nominal control sequence and the final control sequence after residual correction, and the vertical axis represents the cumulative sample proportion.

[0012] Figure 5 This is a schematic diagram of the cumulative distribution of the improvement in the average control rate of change in an embodiment of the present invention. The horizontal axis represents the difference in the average control rate of change between the model-predicted nominal control sequence and the final control sequence after residual correction, and the vertical axis represents the cumulative sample proportion. Detailed Implementation

[0013] The technical solution of the present invention will be further described below with reference to specific embodiments. It should be understood that the following embodiments are only used to illustrate the present invention and are not intended to limit the scope of protection of the present invention. Without departing from the concept of the present invention, those skilled in the art can adjust the sensor type, network structure, prediction step size, control time domain, vehicle kinematics model, loss weight, control constraint range, residual limiting parameters and solver form, and all such adjustments should fall within the scope of protection of the present invention.

[0014] This invention provides an autonomous driving trajectory prediction control method, electronic device, and computer-readable storage medium oriented towards control executability. The method addresses the trajectory prediction and motion control problem of autonomous vehicles in complex traffic environments. It generates multiple prediction mode results through a prediction module, including discrete multimodal predicted trajectories, parameterized continuous reference trajectories, and corresponding mode probabilities; wherein the parameterized continuous reference trajectory is preferably a Bézier curve parameterized trajectory. A control reference mode is determined based on the mode probabilities in the prediction mode results, and the parameterized continuous reference trajectory corresponding to the control reference mode is sampled into a continuous reference trajectory sampling sequence. This continuous reference trajectory sampling sequence is used as the reference input of the model predictive control nominal solution module to solve for the nominal control sequence and nominal state sequence. Then, a residual control correction module performs bounded compensation on the nominal control sequence to obtain the final control sequence. Finally, longitudinal acceleration / deceleration control commands and lateral steering control commands for the autonomous vehicle to be controlled are generated based on the final control sequence. The autonomous driving trajectory prediction control model utilizes a residual control correction mechanism and employs a total loss function containing trajectory physical constraints and control executability constraints to jointly optimize the prediction module and the residual control correction module.

[0015] like Figure 1 As shown, the overall technical process of this invention includes input and scene construction, trajectory parameterized collaborative prediction, reference trajectory tracking control, and unified loss backpropagation. Input and scene construction is used to uniformly convert the motion state of the autonomous vehicle to be controlled, the motion state of surrounding traffic participants, and road structure and road semantic information into a vectorized scene representation; trajectory parameterized collaborative prediction is used to simultaneously output discrete multimodal predicted trajectories and continuous parameterized trajectories; reference trajectory tracking control is used to solve nominal control based on continuous reference trajectories and perform residual correction; unified loss backpropagation is used to jointly introduce prediction errors, curve physical constraints, and control execution feedback into a modular end-to-end joint optimization process. Example 1: A Predictive Control Method for Automated Driving with Controllability in Mind

[0016] The autonomous driving trajectory prediction and control model includes a prediction module, a model predictive control nominal solution module, and a residual control correction module. The prediction module includes a local motion coding unit, a global interaction modeling unit, a discrete multimodal prediction branch, a curve parameterization prediction branch, and a modal probability prediction branch. The model predictive control nominal solution module solves for a nominal control sequence and a nominal state sequence based on the vehicle kinematics model and a continuous reference trajectory sampling sequence. The residual control correction module generates a residual control sequence based on the continuous reference trajectory sampling sequence, the nominal control sequence, and the nominal state sequence, and uses the residual control sequence to correct the nominal control sequence to obtain the final control sequence. In one embodiment, the residual control correction module includes a residual control correction network. 1. Vehicle driving sample acquisition and data sources

[0017] The autonomous vehicle to be controlled can be equipped with an onboard perception unit, a positioning unit, and a mapping unit, and can supplement traffic environment information through a vehicle-to-infrastructure (V2I) unit that communicates with the autonomous vehicle. The onboard perception unit may include one or more of cameras, millimeter-wave radar, and lidar; the positioning unit may include a navigation system, an inertial measurement unit, or a satellite positioning system; the mapping unit can provide information such as lane center lines, road boundaries, lane directions, road topology, steering attributes, and traffic rule semantics; the V2I unit may include roadside perception devices, vehicle-to-everything (V2X) communication devices, or cloud-based traffic information dissemination devices.

[0018] In this embodiment, the motion state of the autonomous vehicle to be controlled can be obtained by the positioning unit and / or the on-board perception unit; road structure and road semantic information can be obtained by the map unit; the motion state of surrounding traffic participants can be obtained by the on-board perception unit, or it can be obtained jointly by the on-board perception unit and the vehicle-road cooperative unit. The vehicle-road cooperative unit is not a necessary component of this invention, but serves as an optional supplementary source of information on the motion state of surrounding traffic participants or road traffic state.

[0019] The vehicle driving samples include the motion state of the autonomous vehicle to be controlled within the historical time domain, the motion state of surrounding traffic participants, road structure and road semantic information, and the true future trajectory within the prediction time domain. The historical time domain represents several sampling moments before the current moment, and the prediction time domain represents several prediction moments after the current moment. Let the historical observation step size be... The predicted step size is For any traffic participant Its historical observation sequence is represented as follows: in, For traffic participants At any moment Two-dimensional position, Indicates the current moment.

[0020] The motion state of the autonomous vehicle to be controlled may include at least one of the following: vehicle position, speed, heading angle, and acceleration; the motion state of surrounding traffic participants may include at least one of the following: position, speed, and heading information of surrounding vehicles, pedestrians, or non-motorized vehicles; and the road structure and road semantic information may include at least one of the following: lane centerline, lane boundary, road topology, lane direction, steering attribute, and traffic rule constraints. 2. Vehicle Local Coordinate System and Vectorized Scene Representation

[0021] To reduce the impact of differences between global coordinate position and vehicle heading on model learning, this embodiment uses the current position of the autonomous vehicle to be controlled. The origin of the coordinate system is used as the current heading angle of the autonomous vehicle to be controlled. As a reference direction for the local coordinate system, the vehicle's motion state, the motion states of surrounding traffic participants, and road structure and semantic information are transformed into the vehicle's local coordinate system. For any global coordinate point... Its local coordinates Represented as:

[0022] Through this transformation, the current heading of the autonomous vehicle to be controlled is compared with the local coordinate system. The axis directions are consistent. Road map points are also converted to the vehicle's local coordinate system and denoted as follows: and .

[0023] A vectorized scene representation is constructed based on the vehicle's local coordinate system. , can be represented as: in, This is a set of historical motion vectors of the vehicle. It is a set of relative interaction vectors among traffic participants. It is a set of road structure and road semantic vectors.

[0024] Vehicle historical motion vector set Used to describe the short-term movement trends of the autonomous vehicle to be controlled and surrounding traffic participants in the historical time domain. For traffic participants The motion vectors between adjacent historical moments are represented as follows: in, The set of vehicle historical motion vectors is represented as: in, This is a set of indexes for traffic participants in the current traffic scenario.

[0025] Set of relative interaction vectors among traffic participants This describes the relative position, relative speed, or relative heading relationship of surrounding traffic participants relative to the autonomous vehicle being controlled. (For surrounding traffic participants) Its relative interaction vector is represented as: in, To control autonomous vehicles at any time The location. When the distance between surrounding traffic participants and the autonomous vehicle to be controlled is less than a preset neighborhood threshold. When this traffic participant is included in the interaction modeling scope, the following conditions must be met: .

[0026] Road structure and road semantic vector set Used to describe the road geometry and semantic information within a local area of ​​the autonomous vehicle to be controlled. For the first A map vector, represented as: in, and These are the start and end points of the map vector in the vehicle's local coordinate system, respectively. These are road semantic attributes, including lane direction, steering attributes, road boundaries, lane connectivity, and traffic rule constraints. Therefore, the road structure and the set of road semantic vectors can be represented as: in, This represents the number of map vectors within a local area of ​​the autonomous vehicle to be controlled. 3. Trajectory parameterization and collaborative prediction in the prediction module

[0027] Vectorized scene representation Input prediction module. The local motion coding unit in the prediction module is used to extract historical motion features of the autonomous vehicle to be controlled and surrounding traffic participants; the global interaction modeling unit is used to fuse the interaction relationships between the autonomous vehicle to be controlled, surrounding traffic participants, and road structure to obtain scene interaction features. The process is represented as: in, For scene encoding functions; These are learnable parameters. The local motion coding unit and the global interaction modeling unit can be implemented using a multilayer perceptron, recurrent neural network, convolutional network, attention network, graph neural network, or Transformer structure. This embodiment does not limit the specific network form.

[0028] To characterize the uncertainty of future motion, this embodiment sets The prediction mode. For the _th A prediction mode, introducing mode embedding. to By fusing the data, trajectory prediction features in this mode can be obtained. The process is represented as: in, For feature fusion function, These are learnable parameters.

[0029] Discrete multimodal prediction branch based on Output the first Discrete multimodal prediction trajectory under each prediction mode The trajectory is represented as: in, For the first The prediction mode is in the _th prediction mode in the _th ... Discrete prediction positions at each prediction time, with superscript Represents a discrete trajectory.

[0030] The prediction module also outputs the modal probabilities corresponding to each predicted mode. Modal probabilities can be obtained using the softmax function: in, For the first The score of each prediction mode, and satisfying .

[0031] Curve parameterized prediction branch based on Output the first A set of Bézier control points under each prediction mode is generated, and the parameterized trajectory of the Bézier curve is further generated. Superscript This represents a continuous parameterized trajectory.

[0032] In one implementation, the parameterized trajectory of the Bézier curve is composed of a third-order Bézier curve. For the third... The prediction mode, the first Curve experts and the first The set of control points for a given curve segment is represented as follows: The corresponding third-order Bézier curve is represented as: In the formula, , , and These are the starting point, two shape control points, and the ending point of the curve segment, respectively. Normalization parameters The corresponding trajectory position.

[0033] To reduce abrupt changes in trajectory at segment connections, adjacent curve segments must satisfy both positional continuity and tangential continuity conditions. The positional continuity condition is expressed as: The tangential continuity condition is expressed as: With the above constraints, adjacent Bézier curve segments under the same autonomous vehicle to be controlled, the same prediction mode, and the same curve expert are not only positionally continuous at the connection point, but also tangentially continuous, which is beneficial to improving the motion continuity and control trackability of the reference trajectory.

[0034] In one alternative implementation, the curve parametric prediction branch includes multiple curve experts, each corresponding to a different number of curve segments. Fewer curve segments are suitable for near-straight or low-curvature motion modes, while more curve segments are suitable for turning, lane changing, or complex interactive motion modes. Let there be a total... Curve expert, number The number of curve segments corresponding to the curve expert is For the first The prediction mode and the first prediction mode For curve experts, their set of Bézier control points is represented as: in, For the first The th prediction mode The set of Bézier control points output by the curve expert. For the first Control point generation function for curve experts, These are its learnable parameters.

[0035] For the The first in the curve expert The sampling positions of the candidate Bézier curves at each prediction time are represented as follows: in, Let be the sampling position of the k-th type of curve expert at the t-th prediction time; This indicates the curve segment to which the i-th prediction time belongs; The symbol represents the normalized parameter within the corresponding curve segment. The specific sampling rules are determined by the number of curve segments. It is determined together with the predicted step size.

[0036] No. Curve-like expert output Candidate Bézier curve sampling trajectories under each prediction mode Curve expert weights Adaptive generation based on modal features: in, For the first The th prediction mode The score of the curve expert, and meets the requirements. .

[0037] After weighted fusion of the candidate Bézier curve sampling trajectories generated by each curve expert, the first one is obtained. Sampled sequences of parameterized trajectories of Bézier curves under each prediction mode: Therefore, the first The continuous parameterized trajectory under each prediction mode is represented as follows:

[0038] The final output of the prediction module is represented as follows: in, For the first Discrete multimodal prediction trajectories under each prediction mode The parameterized trajectory of the Bézier curve in the f-th prediction mode. For the first The modal probability corresponding to each predicted modality. 4. Determination of Supervisory Mode and Control Reference Mode

[0039] To calculate the trajectory prediction loss, the supervised mode needs to be determined based on the true future trajectory. Let the true future trajectory of the b-th training sample be:

[0040] For the There are several prediction modes, and their discrete multimodal prediction trajectories are as follows: Supervisory modality Defined as the prediction mode that minimizes the average positional error between the predicted and true future trajectories:

[0041] The supervised mode is used to calculate the loss for discrete trajectory regression, curved trajectory regression, and mode classification. This supervised mode is determined by the true future trajectory, rather than by directly maximizing the mode probability, thus avoiding the ambiguity of the supervised branch caused by using the maximum mode probability as the basis for supervision during the training phase.

[0042] During the determination of the control reference trajectory, the control reference mode is determined based on the mode probabilities corresponding to each predicted mode. Defined as:

[0043] Control reference mode The corresponding parameterized trajectory of the Bézier curve The control reference trajectory was determined and sampled as a continuous reference trajectory sampling sequence:

[0044] The continuous reference trajectory sampling sequence serves as the input to the nominal solver of the model predictive control (MMDC) module. The supervisory mode is used to calculate the trajectory prediction loss, while the control reference mode is used to determine the continuous reference trajectory sampling sequence input to the MDC nominal solver module. The trajectory prediction loss is calculated based on the supervisory mode, and the control closed-loop loss is calculated based on the continuous reference trajectory sampling sequence corresponding to the control reference mode. 5. Nominal solution for model predictive control

[0045] The model predictive control nominal solver module receives a continuous reference trajectory sampling sequence. Current state of the autonomous vehicle to be controlled And the vehicle kinematics model, and solve the nominal control sequence. and nominal state sequence .

[0046] In one implementation, the vehicle state vector is represented as: in, and Let this be the position of the vehicle in the local coordinate system. For the vehicle's heading angle, For vehicle speed.

[0047] The control input is represented as: in, For longitudinal acceleration, This refers to the steering angle of the front wheels.

[0048] The vehicle state transition uses a discrete-time kinematic bicycle model. Let the sampling time interval be... The vehicle wheelbase is Then the state recursion is expressed as:

[0049] MPC solves for the nominal control sequence in the prediction time domain: The nominal state sequence is recursively derived from the vehicle kinematics model:

[0050] In one implementation, the MPC objective function includes a reference trajectory tracking error term, a control input magnitude term, and a control input rate of change term. It is expressed as: in, This is the position vector in the nominal state; , and These are weight matrices representing the trajectory tracking error, control input amplitude, and control input rate of change, respectively.

[0051] The MPC solution is expressed as: The constraints include vehicle kinematic state transition constraints, longitudinal acceleration boundary constraints, and front wheel steering angle boundary constraints. The vehicle kinematic state transition constraints are expressed as follows: .in, This is the state transition function corresponding to the two-degree-of-freedom model of the vehicle. and These are the lower and upper limits of longitudinal acceleration, respectively. and These are the lower and upper limits of the front wheel steering angle, respectively.

[0052] The nominal solver module for model predictive control is mainly used to provide nominal control priors that satisfy vehicle kinematic constraints and control input boundaries. Its solver parameters are not updated as trainable parameters of the neural network. 6. Residual control correction and final state recursion

[0053] The nominal control sequence of MPC can provide control priors that satisfy vehicle kinematic constraints and control input boundaries. However, since there may be deviations between the vehicle kinematic model and the actual vehicle response, relying solely on nominal control may lead to large control change rates or local control abrupt changes. Therefore, this embodiment introduces a residual control correction module to perform bounded compensation on the nominal control sequence; in one implementation, the residual control correction module includes a residual control correction network.

[0054] First, calculate the position tracking error of the nominal state sequence relative to the continuous reference trajectory sampling sequence. For the... At each predicted time, the position tracking error is expressed as: in, The vehicle position is in the nominal state sequence. This refers to the corresponding reference position in the continuous reference trajectory sampling sequence.

[0055] The input to the residual control correction network includes a continuous reference trajectory sampling sequence. Nominal control sequence Nominal state sequence and position tracking error sequence Let the above input features be denoted as Then the residual control correction network outputs the residual control sequence: in, For residual control correction network, These are its learnable parameters.

[0056] To avoid excessive residual correction leading to control instability, this embodiment limits the residual control value. Let... To predict discrete time indexes in the time domain, The residual control correction network in the first... Output the original value of the longitudinal acceleration residual at each discrete time point. Original value of front wheel steering angle residual .

[0057] No. The bounded residual control quantity at each discrete time satisfies: in, , For longitudinal acceleration residuals, For the front wheel steering angle residual, This represents the upper limit of the longitudinal acceleration residual. This represents the upper limit of the front wheel steering angle residual.

[0058] The bounded residual control sequence is represented as: The final control sequence satisfies: in, For nominal control sequence, This is the final control sequence.

[0059] To calculate the control closed-loop loss, the vehicle kinematic state is recursively derived from the final control sequence to obtain the final state recursive sequence. Specifically, the final state recursive sequence is... Input the vehicle's kinematics model and the current state of the autonomous vehicle to be controlled. As the initial state for the recursion, we obtain the final recursion sequence. The process is represented as: in, Let be the state recursion function constructed based on the vehicle kinematics model, and , The first step under the action of the final control sequence The vehicle recursive state at each predicted time.

[0060] Furthermore, from the recursive sequence of the final state Extracting the vehicle position component yields the execution trajectory corresponding to the final control sequence: From the nominal state sequence Extracting the vehicle position component yields the nominal recursive trajectory corresponding to the MPC nominal control: .

[0061] The execution trajectory and nominal recursive trajectory Both are used to calculate the tracking performance retention loss in the control closed-loop loss. The tracking performance retention loss is used to constrain the execution trajectory. Relative to continuous reference trajectory sampling sequence The tracking error is not significantly greater than the nominal recursive trajectory. Relative to continuous reference trajectory sampling sequence Tracking error. 7. Joint Optimization Objective

[0062] This embodiment uses a unified total loss function to jointly constrain the discrete multimodal predicted trajectory and parameterized continuous reference trajectory output by the prediction module, the nominal control sequence and nominal state sequence output by the model predictive control nominal solver module, and the residual control sequence output by the residual control correction module. It also jointly optimizes the trainable parameters in the prediction module and the residual control correction module. The total loss function is expressed as: in, For trajectory prediction loss, For discrete curve alignment loss, For curve physical loss, To control closed-loop losses, and These are the weighting coefficients for the corresponding loss terms. 7.1 Trajectory Prediction Loss

[0063] The trajectory prediction loss is used to maintain the multimodal trajectory prediction capability of the prediction module. The trajectory prediction loss includes discrete trajectory regression loss, curvilinear trajectory regression loss, and modality classification loss, expressed as: in, For discrete trajectory regression loss, For the regression loss of the curve trajectory, For modal classification loss, and These are the corresponding weighting coefficients.

[0064] For the For each sample, its true future trajectory is: Supervisory modality Determined by the actual future trajectory: .

[0065] Discrete trajectory regression loss is expressed as: The regression loss of the curve trajectory is expressed as: in, For training batch size, For smoothing Loss function; and These represent the discrete prediction position and the curve trajectory sampling position under the supervised mode, respectively.

[0066] Modality classification loss is expressed as: in, For the first The predicted probability corresponding to each sample supervision mode. 7.2 Discrete Curve Alignment Loss

[0067] Discrete curve alignment loss is used to constrain the positional deviation between the discrete multimodal prediction trajectory and the parameterized trajectory of the Bézier curve under the same prediction mode, so that the parameterized trajectory of the Bézier curve inherits the spatial distribution and motion trend of the discrete multimodal prediction trajectory.

[0068] In one implementation, a discrete multimodal predicted trajectory is used as a shape reference for the curve trajectory, and a stopping gradient operation is employed to make the discrete branches serve as reference objects for the curve branches. The discrete curve alignment loss is expressed as: in, This indicates that the gradient operation is stopped. For the first Sampling locations of Bézier curves under each prediction mode For the first Discrete predicted positions under each prediction mode.

[0069] In another alternative implementation, instead of stopping gradient operations, the positional consistency between the discrete multimodal prediction trajectory and the Bézier curve parameterized trajectory is constrained. 7.3 Curve Physical Loss

[0070] Curve physics loss is used to constrain the kinematic rationality of the parametric trajectory of a Bézier curve. Curve physics loss includes acceleration loss, rate of change of acceleration loss, and curvature loss, expressed as: in, For acceleration loss; Loss due to the rate of change of acceleration; For curvature loss; , and These are the corresponding weighting coefficients.

[0071] In the vehicle's local coordinate system, suppose the autonomous vehicle to be controlled is currently located at the origin of the local coordinate system, that is: For the sampled trajectory of the Bézier curve, its differential velocity, differential acceleration, and rate of change of acceleration are expressed as follows: .in, The sampling time interval, It is a differential velocity vector; It is a differential acceleration vector; This is the differential acceleration rate of change vector. The effective time index of the velocity is... The effective time index for acceleration is The effective time index for the rate of change of acceleration is .

[0072] Acceleration loss is expressed as: The loss due to the rate of change of acceleration is expressed as:

[0073] Curvature is calculated based on velocity and acceleration. Let... , These are the components of the velocity vector in the x and y directions, respectively; and Let x and y be the components of the acceleration vector in the x and y directions, respectively. Then the curvature is expressed as: in, This is the minimum velocity threshold in curvature calculation. To prevent extremely small constants with a denominator of zero.

[0074] Curvature loss is expressed as: in, A preset curvature threshold is set. Through the aforementioned curve physical loss, local jitter, abrupt curvature changes, and motion discontinuities in the Bézier curve sampling trajectory can be reduced, making it more suitable as a control reference trajectory. 7.4 Controlling Closed-Loop Losses

[0075] The control closed-loop loss is used to introduce control execution feedback into a modular end-to-end joint optimization process. The calculation of this loss depends on the nominal state sequence, the final state recursive sequence, the continuous reference trajectory sampling sequence, the nominal control sequence, the final control sequence, and the bounded residual control sequence. The final state recursive sequence is derived from the final control sequence. The trajectory is derived recursively from the vehicle kinematics model and serves as the source of the execution trajectory for calculating the loss of tracking performance.

[0076] The control closed-loop loss includes tracking performance retention loss, control rate of change constraint loss, and residual magnitude regularization loss, expressed as: in, To track performance retention loss, To control the rate of change constraint loss, For residual amplitude regularization loss, and These are the corresponding weighting coefficients.

[0077] set up For the nominal recursive trajectory corresponding to the nominal control of MPC, The execution trajectory is obtained by recursively calculating the final control sequence. It is a continuous reference trajectory sampling sequence.

[0078] The average tracking errors of the nominal recursive trajectory and the executed trajectory relative to the reference trajectory are expressed as follows: .

[0079] The tracking performance retention loss is expressed as: in, This represents the allowable tracking error margin. This loss is used to avoid significantly compromising the tracking performance of the MPC nominal control on the reference trajectory after residual correction.

[0080] The rate of change of control is used to evaluate the smoothness of the control sequence. For the first... The sample at the th Control input at each predicted time The rate of change is expressed as: in, Let be the control scale weight matrix. Assume the control change rate after residual correction is . The nominal control rate of change of MPC is The target control change rate is ,in, To control the rate of change to the target; To smooth the target scaling factor, and .

[0081] The loss due to the control rate of change constraint is expressed as: in, This means selecting the sample with the largest error over the time dimension. Proportional time steps and average; This represents the weight of the peak smoothing term. This loss can simultaneously constrain the overall level of the control rate of change and local peak variations, thereby suppressing drastic local control changes.

[0082] The residual amplitude regularization loss is used to limit the amplitude of the residual control correction, preventing the residual control correction module from excessively replacing the MPC nominal control. The residual amplitude regularization loss is expressed as: in, This is the correction amount for acceleration residuals; This is the correction amount for the front wheel steering angle residual; The corner residual penalty weight.

[0083] Through the aforementioned total loss function, the prediction module is not only supervised by the true future trajectory, but also constrained by the physical smoothness of the Bézier curve, MPC tracking error, control rate of change, and residual correction magnitude. Therefore, while maintaining prediction accuracy, the model can generate a continuous reference trajectory more suitable for vehicle controller tracking. 8. Modular end-to-end joint optimization

[0084] Based on the above total loss function The discrete multimodal predicted trajectory and parameterized continuous reference trajectory output by the prediction module, the nominal control sequence and nominal state sequence output by the model predictive control nominal solution module, and the residual control sequence output by the residual control correction module are jointly optimized in a modular end-to-end manner under a unified loss constraint.

[0085] In one implementation, the trajectory prediction loss constraint prediction module can be used first to enable the discrete multimodal prediction branch to have a stable multimodal trajectory prediction capability; then, discrete curve alignment loss and curve physics loss can be introduced to enable the curve parameterization prediction branch to output a continuous, smooth and sampleable curve trajectory; then, control closed-loop loss can be introduced to enable the residual control correction module to perform bounded compensation on the control sequence based on the MPC nominal control; alternatively, a modular end-to-end joint optimization method can be directly adopted to constrain the above process as a whole under a unified total loss function.

[0086] The MPC nominal solver module is mainly used to generate the nominal control sequence and nominal state sequence. Its solver parameters are not updated as trainable parameters of the neural network. The residual control correction module and the vehicle kinematics state recursion process constitute the calculation branch of the control closed-loop loss, enabling the control tracking error, control input rate of change, and residual magnitude to participate in the modular end-to-end joint optimization of the prediction module and the residual control correction module.

[0087] In one exemplary implementation, such as Figure 4 and Figure 5 As shown, the improvement in control rate of change can be used to evaluate the effect of the residual control correction module on the smoothness of the control sequence. The improvement in control rate of change can include the maximum improvement in control rate of change and the average improvement in control rate of change. Figure 4 The cumulative distribution of the improvement in the maximum rate of change of control is shown. Figure 5 The cumulative distribution of the improvement in average control rate of change is shown. The improvement in maximum control rate of change represents the difference between the maximum control rate of change of the model-predicted nominal control sequence and the maximum control rate of change of the final control sequence after residual correction in the same sample; the improvement in average control rate of change represents the difference between the average control rate of change of the model-predicted nominal control sequence and the average control rate of change of the final control sequence after residual correction in the same sample. When the improvement in control rate of change is greater than zero, it indicates that after residual control correction, the control rate of change of the final control sequence is lower than the control rate of change of the nominal control sequence, and the smoothness of the control sequence is improved. Figure 4 and Figure 5 The dashed line in the diagram represents the position where the improvement is zero, and the samples to the right of the dashed line correspond to the situation where the rate of change of control decreases. This demonstrates that the residual control correction module can reduce the rate of change of control based on the nominal control of model predictive control, and is beneficial to improving the smoothness of the control sequence and the stability of closed-loop execution.

[0088] As an example of verification, the execution results of the model predictive control after nominal control and residual control after correction can be compared using a verification sample. The results are shown in the table below: Table 1 Examples of control executability indicators under different control strategies MPC nominal control 0.116972 0.343980 4.261599 1.108808 — — MPC + Residual Control Correction 0.117239 0.345309 4.253408 1.106587 0.000881 0.004203

[0089] As shown in Table 1, after adding residual control correction, the reference ADE and reference FDE only change slightly, while the maximum control change rate and average control change rate both decrease. This indicates that the residual control correction module can slightly compensate for the nominal control sequence of the model predictive control while basically maintaining the reference trajectory tracking performance, thereby improving the smoothness of the control sequence and the stability of closed-loop execution. The above results are only an exemplary verification under one embodiment and are not intended to limit the scope of protection of this invention. Example 2: Vehicle Control Command Generation Process Based on Final Control Sequence

[0090] In one application scenario, based on the above-mentioned autonomous driving trajectory prediction control method oriented towards control executability, the on-board computing platform or vehicle controller of the autonomous vehicle to be controlled can generate vehicle control commands according to the final control sequence.

[0091] First, the motion state of the autonomous vehicle to be controlled is obtained through its onboard perception unit and positioning unit. Road structure and semantic information are obtained through the map unit. The motion state of surrounding traffic participants is also obtained through the onboard perception unit, or jointly through the vehicle-road cooperative unit that communicates with the autonomous vehicle to be controlled. Based on the current pose of the autonomous vehicle to be controlled, a local coordinate system is constructed to obtain a vectorized scene representation.

[0092] Then, the vectorized scene representation is input into the prediction module to generate multiple prediction mode results. These prediction mode results include discrete multimodal prediction trajectories, parameterized continuous reference trajectories, and corresponding mode probabilities; wherein the parameterized continuous reference trajectory is preferably a Bézier curve parameterized trajectory. A control reference mode is determined based on the mode probabilities in the prediction mode results, and the parameterized continuous reference trajectory corresponding to the control reference mode is sampled into a continuous reference trajectory sampling sequence.

[0093] Subsequently, the continuous reference trajectory sampling sequence is input into the model predictive control nominal solution module to solve for the nominal control sequence and nominal state sequence based on the vehicle kinematics model. The position tracking error between the vehicle position in the nominal state sequence and the corresponding reference position in the continuous reference trajectory sampling sequence is calculated. The continuous reference trajectory sampling sequence, nominal control sequence, nominal state sequence, and position tracking error are then input into the residual control correction module to generate a bounded residual control sequence. The nominal control sequence is corrected based on the bounded residual control sequence to obtain the final control sequence.

[0094] Finally, longitudinal acceleration / deceleration control commands and lateral steering control commands are generated for the vehicle actuators based on the final control sequence. The vehicle actuators may include the drive system, braking system, and steering system. The longitudinal acceleration / deceleration control commands are used to control the vehicle's driving or braking, while the lateral steering control commands are used to control the front wheel angle or steering actuators. Through this process, the autonomous vehicle under control can convert the continuous Bezier reference trajectory generated by the prediction module into executable vehicle control commands. Example 3: Electronic devices and computer-readable storage media

[0095] This embodiment also provides an electronic device. The electronic device includes a processor, a memory, and a computer program stored in the memory and executable by the processor. When the processor executes the computer program, it can implement an autonomous driving trajectory prediction control method oriented towards control executability as described in any of the above embodiments.

[0096] Electronic devices can be in-vehicle computing platforms, vehicle domain controllers, autonomous driving controllers, servers, edge computing devices, or other computing devices with data processing capabilities. When electronic devices are installed in the vehicle, they can communicate with in-vehicle perception units, positioning units, mapping units, vehicle controllers, and vehicle actuators. When electronic devices are installed on the server side, they can be used to perform modular end-to-end joint optimization based on vehicle driving samples and apply the optimized autonomous driving trajectory prediction and control model to the vehicle.

[0097] This embodiment also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, it can implement the autonomous driving trajectory prediction control method for controllability as described in any of the above embodiments. The computer-readable storage medium may include a read-only memory, random access memory, flash memory, hard disk, solid-state drive, memory card, optical disk, or other media capable of storing a computer program.

[0098] In the above embodiments, each module can be implemented through software, hardware, or a combination of both. The division of modules is only for illustrating the technical solutions of this invention and does not constitute a limitation on the actual hardware structure or software architecture. In practical applications, multiple modules can be integrated into the same processor or the same computing platform, or some modules can be deployed on the vehicle side and some on the server side. Any module that employs the discrete multimodal prediction and parameterized continuous reference trajectory prediction, model predictive control nominal solution, bounded residual control correction, and the modular end-to-end joint optimization mechanism based on trajectory physical constraints and control executability constraints described in this invention should be considered to fall within the protection scope of this invention.

Claims

1. An autonomous driving trajectory prediction control method oriented towards controllability, characterized in that, The method employs an autonomous driving trajectory prediction and control model, which includes a prediction module, a model prediction and control nominal solution module, and a residual control correction module. The autonomous driving trajectory prediction and control model utilizes a residual control correction mechanism and is obtained by jointly optimizing the prediction module and the residual control correction module using a total loss function that includes trajectory physical constraints and control executability constraints. The method includes the following steps: S1. Obtain the motion state of the autonomous vehicle to be controlled, the motion state of surrounding traffic participants, and road structure and road semantic information, and construct a vectorized scene representation input to the prediction module based on the motion state of the autonomous vehicle to be controlled, the motion state of surrounding traffic participants, and road structure and road semantic information. S2. The vectorized scene representation is input into the prediction module, which generates multiple prediction modal results. These prediction modal results include a discrete multimodal prediction trajectory, a parameterized continuous reference trajectory, and corresponding modal probabilities. The prediction module includes a local motion coding unit, a global interaction modeling unit, a discrete multimodal prediction branch, a curve parameterization prediction branch, and a modal probability prediction branch. The local motion coding unit is used to extract historical motion features of the vehicle, and the global interaction modeling unit is used to fuse the data between the autonomous vehicle to be controlled, surrounding traffic participants, and the road structure. The interaction relationship is as follows: the discrete multimodal prediction branch is used to output discrete multimodal prediction trajectories under multiple prediction modes; the curve parameterization prediction branch is used to output parameterized continuous reference trajectories under the corresponding prediction modes; and the mode probability prediction branch is used to output the mode probabilities corresponding to each prediction mode. The parameterized continuous reference trajectory is a Bézier curve parameterized trajectory characterized by Bézier control points, and the Bézier curve parameterized trajectory is composed of a third-order Bézier curve or a piecewise third-order Bézier curve. For any third-order Bézier curve segment, the third-order Bézier curve segment is determined by four control points, and its curve position satisfies: ,in, and These are the start and end points of the third-order Bézier curve segment, respectively. and These are the shape control points for the third-order Bézier curve segment; when a piecewise third-order Bézier curve is used, adjacent curve segments satisfy the positional continuity condition and the tangential continuity condition. S3. Determine the control reference mode based on the mode probabilities in the predicted mode results, sample the parameterized continuous reference trajectory corresponding to the control reference mode into a continuous reference trajectory sampling sequence, and input the continuous reference trajectory sampling sequence into the model prediction control nominal solution module to obtain the nominal control sequence and the nominal state sequence; S4. Based on the continuous reference trajectory sampling sequence, the nominal control sequence, and the nominal state sequence, a residual control sequence is generated through the residual control correction module, and the nominal control sequence is used to correct the nominal control sequence to obtain the final control sequence for controlling the motion of the autonomous vehicle to be controlled; wherein, the residual control correction module includes a residual control correction network, which generates a bounded residual control sequence based on the continuous reference trajectory sampling sequence, the nominal control sequence, the nominal state sequence, and the position tracking error; the position tracking error is the error between the vehicle position in the nominal state sequence and the corresponding reference position in the continuous reference trajectory sampling sequence; let i be the mode index of the control reference mode, t be the discrete time index in the prediction time domain, and: The residual control correction network in the first... Output the original value of the longitudinal acceleration residual at each discrete time point. Original value of front wheel steering angle residual The bounded longitudinal acceleration residual and the bounded front wheel steering angle residual are obtained respectively through the hyperbolic tangent function and the preset upper limit of residual; the bounded residual control quantity at the t-th discrete time satisfies: ;in, This represents the upper limit of the longitudinal acceleration residual. The upper limit of the front wheel steering angle residual; the bounded residual control sequence is The final control sequence satisfies: ;in, For nominal control sequence, This is the final control sequence; S5. Generate longitudinal acceleration / deceleration control commands and lateral steering control commands for the autonomous vehicle to be controlled based on the final control sequence.

2. The autonomous driving trajectory prediction control method based on controllability according to claim 1, characterized in that, In step S1, the motion state of the autonomous vehicle to be controlled is obtained by the vehicle-mounted perception unit and positioning unit of the autonomous vehicle to be controlled; the road structure and road semantic information is obtained by the map unit; and the motion state of the surrounding traffic participants is obtained by the vehicle-mounted perception unit, or jointly by the vehicle-mounted perception unit and the vehicle-road cooperative unit that is communicatively connected to the autonomous vehicle to be controlled. A local coordinate system of the vehicle is constructed based on the current pose of the autonomous vehicle to be controlled, and the motion state of the autonomous vehicle to be controlled, the motion state of the surrounding traffic participants, and the road structure and road semantic information are transformed into the local coordinate system of the vehicle to be controlled to obtain the vectorized scene representation. The local coordinate system of the vehicle to be controlled takes the current position of the autonomous vehicle to be controlled as the origin and the current heading of the autonomous vehicle to be controlled as the reference direction. The vectorized scene representation includes the vehicle's historical motion vector, the relative interaction vector between traffic participants, and the road structure and road semantic vector.

3. The autonomous driving trajectory prediction control method based on controllability according to claim 1, characterized in that, In step S3, the control reference mode is the prediction mode with the largest modal probability among all predicted modes; the model predictive control nominal solution module constructs a model predictive control optimization problem based on the continuous reference trajectory sampling sequence, the current state of the autonomous vehicle to be controlled, and the vehicle kinematics model. The objective function of the model predictive control optimization problem includes a reference trajectory tracking error term, a control input amplitude term, and a control input rate of change term. The constraints include vehicle kinematics state transition constraints, longitudinal acceleration boundary constraints, and front wheel steering angle boundary constraints.

4. The autonomous driving trajectory prediction control method based on controllability according to claim 1, characterized in that, The total loss function includes trajectory prediction loss, discrete curve alignment loss, curve physics loss, and control closed-loop loss. The curve physics loss serves as the trajectory physics constraint term, used to constrain the differential acceleration, rate of change of acceleration, and curvature of the parameterized continuous reference trajectory. The control closed-loop loss serves as the control executability constraint term, used to constrain the reference trajectory tracking error, the control rate of change of the final control sequence, and the magnitude of the residual control sequence. The trajectory prediction loss constrains the deviation between the discrete multimodal predicted trajectory and the parameterized continuous reference trajectory and the true future trajectory. The discrete curve alignment loss constrains the positional deviation between the discrete multimodal predicted trajectory and the parameterized continuous reference trajectory under the same prediction mode. Based on the total loss function, the discrete multimodal predicted trajectory and parameterized continuous reference trajectory output by the prediction module, the nominal control sequence and nominal state sequence output by the model predictive control nominal solution module, and the residual control sequence output by the residual control correction module are jointly constrained, and the trainable parameters in the prediction module and the residual control correction module are jointly optimized.

5. An electronic device, characterized in that, The system includes a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein the processor executes the computer program to implement an autonomous driving trajectory prediction control method oriented towards control executability as described in any one of claims 1 to 4.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements an autonomous driving trajectory prediction control method oriented towards control executability as described in any one of claims 1 to 4.

Images