Automatic driving method and system based on deep learning and preference cost learning
By employing deep learning and preference cost learning methods, interactive trajectory planning and decision-making in complex traffic environments were achieved, improving the decision-making reliability and human simulation of autonomous driving systems and addressing the shortcomings of existing technologies in modeling the mutual influence of traffic participants.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-01-16
- Publication Date
- 2026-04-14
AI Technical Summary
Existing autonomous driving systems struggle to effectively model the interactions between traffic participants in complex traffic environments, and existing decision-making methods lack adaptability and reliability in dynamic and complex scenarios.
Using deep learning and preference cost learning methods, this study simulates the future state of vehicles and surrounding traffic participants through candidate trajectory generation, interactive prediction, and human-like decision-making. It evaluates the traffic efficiency and anthropomorphism of the trajectories and selects the planning trajectory that best conforms to human values.
It improves the rationality of behavior and the level of interactive intelligence of autonomous driving systems in complex traffic environments, enhances the safety and adaptability of decision-making, imitates human driving behavior, and overcomes the limitations of traditional methods.
Smart Images
Figure CN121849181A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of autonomous driving. In particular, it relates to an autonomous driving method and system based on deep learning and preference cost learning. Background Technology
[0002] In the field of autonomous driving, prediction-based planning methods have become one of the mainstream strategies. These systems use behavior prediction modules to assist in decision-making, aiming to achieve safe and efficient autonomous driving. However, most current mainstream systems adopt a modular design that separates prediction and planning, achieving obstacle avoidance solely through passive avoidance. This makes it difficult to effectively model the interactions between traffic participants, especially in complex traffic environments with close interactions. Furthermore, existing rule-based or reinforcement learning (RL) decision-making methods also have significant limitations in practical deployments. While rule-based methods are simple to implement, they suffer from poor adaptability and limited scalability, making it difficult to cope with dynamic changes and complex scenarios. Reinforcement learning methods face challenges such as low data utilization efficiency, complex safety reward and penalty design, and difficulty in handling rare cases, affecting their practicality and stability. Therefore, how to incorporate the impact of the future behavior of autonomous vehicles on surrounding traffic participants into the planning framework for modeling, thereby enhancing the system's interactive modeling capabilities and decision reliability, is an urgent problem to be solved. Summary of the Invention
[0003] This invention provides an autonomous driving method and system based on deep learning and preference cost learning to address the problems of low interactive modeling capability and decision reliability of the system, making it difficult to cope with dynamic changes and complex scenarios.
[0004] To achieve the above objectives, in a first aspect, the present invention relates to an autonomous driving method based on deep learning and preference cost learning for autonomous driving of intelligent vehicles, comprising: Candidate trajectory generation: Lane reference lines, navigation, and surrounding traffic participant information are extracted from the current high-precision map and upstream perception of the autonomous driving system. Multiple candidate planning trajectories are sampled based on the lane reference lines, navigation, surrounding traffic participant information, and traffic rules. Interactive prediction: Based on the candidate planning trajectory, the historical status of surrounding traffic participants, and map topology information, the future status of surrounding vehicles is predicted under each candidate planning trajectory condition; Human-like decision-making: By analyzing the potential conflicts between candidate planning trajectories and the corresponding future states of surrounding vehicles to avoid risks, the preference cost network evaluates the traffic efficiency and anthropomorphism of the interaction process in the simulated future of multiple candidate planning trajectories plus the corresponding future states of surrounding vehicles, and compares and scores the candidate planning trajectories to select the planning trajectory that best fits human values.
[0005] To achieve the above objectives, in a second aspect, the present invention relates to an autonomous driving system based on deep learning and preference cost learning, a candidate trajectory generation module, used to extract lane reference lines, navigation and surrounding traffic participant information from the current high-precision map and upstream perception of the autonomous driving system, and to sample multiple candidate planning trajectories based on the lane reference lines, navigation, surrounding traffic participant information and traffic rules; An interactive prediction module is used to predict the future state of surrounding vehicles under each candidate planning trajectory based on the candidate planning trajectory, the historical state of surrounding traffic participants, and map topology information. The human-like decision-making module is used to avoid risks by analyzing the potential conflicts between candidate planning trajectories and the corresponding future states of surrounding vehicles. The preference cost network evaluates the traffic efficiency and anthropomorphism of the interaction process in the simulated future of multiple candidate planning trajectories plus the corresponding future states of surrounding vehicles, and compares and scores the candidate planning trajectories to select the planning trajectory that best fits human values.
[0006] To achieve the above objectives, in a third aspect, the present invention also relates to a computer-readable storage medium storing instructions that, when executed, perform the aforementioned autonomous driving method based on deep learning and preference cost learning.
[0007] The present invention relates to an autonomous driving method and system based on deep learning and preference cost learning, which has the following advantages compared with the prior art: This method can uncover finer-grained semantic differences and potential interaction attributes between trajectories, enabling a modeling process that more closely resembles human driving strategies. The method proposed in this patent can improve the behavioral rationality and interactive intelligence level of autonomous driving systems in complex traffic environments, providing a technological foundation for highly reliable and human-simulation-level autonomous driving decision-making systems.
[0008] This invention aims to achieve interactive autonomous vehicle trajectory planning and decision-making in complex traffic environments, and its main advantages include: Interactivity: This method considers the interaction between the vehicle and surrounding traffic participants by incorporating the planning process into a conditional prediction framework. This interactive perspective is particularly important in highly interactive scenarios (such as intersections without traffic lights) and helps improve the safety and effectiveness of decision-making.
[0009] Decision-making aligns with human values: This method captures human driving preferences using preference cost learning, making decisions based on underlying trajectory attribute preferences rather than absolute ratings, thus better mimicking the human decision-making process. Compared to traditional methods, this approach captures and understands driving behavior with greater precision.
[0010] Robustness: Human driving decision-making methods based on preference cost learning can make decisions that best align with human values based on underlying trajectory attribute preferences. This makes the decision-making process more closely resemble real-world driving behavior, enhancing the model's adaptability and robustness.
[0011] Data-driven: Compared with rule-based or reinforcement learning-based methods, this method can overcome the limitations of manual feature extraction and reveal potential trajectory feature differences and interaction attributes, providing a richer and more detailed understanding of driving behavior and avoiding the limitations of linear cost functions. Attached Figure Description
[0012] Figure 1 This is a flowchart illustrating an autonomous driving method based on deep learning and preference cost learning in Example 1.
[0013] Figure 2 The flowchart of the interactive planning and human-like decision-making method of Example 1 of an autonomous driving method based on deep learning and preference cost learning is shown in Embodiment 1.
[0014] Figure 3 The diagram shows the transformer-based interactive prediction network structure of an autonomous driving method based on deep learning and preference cost learning in Example 1.
[0015] Figure 4 Example 1 shows a decision network structure diagram based on preference cost learning for an autonomous driving method based on deep learning and preference cost learning.
[0016] Figure 5 A schematic diagram of the Frenet coordinate system for an autonomous driving method based on deep learning and preference cost learning in Example 1.
[0017] Figure 6 The simulation results of an autonomous driving method based on deep learning and preference cost learning in Example 1 are shown in the figure.
[0018] Figure 7 A schematic diagram of the structure of an autonomous driving system based on deep learning and preference cost learning in Embodiment 2 of the present invention. Detailed Implementation
[0019] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention and not the entire structure.
[0020] Example 1 For an autonomous driving method based on deep learning and preference cost learning, please refer to [link / reference]. Figure 1-6 As shown, the present invention provides an autonomous driving method based on deep learning and preference cost learning for autonomous driving of intelligent vehicles, comprising the following steps: S101 to S103.
[0021] S101 Candidate Trajectory Generation: Extract lane reference lines, navigation, and surrounding traffic participant information from the current high-precision map and upstream perception of the autonomous driving system, and sample multiple candidate planning trajectories based on the lane reference lines, navigation, surrounding traffic participant information, and traffic rules.
[0022] Step S101 specifically includes: representing the candidate trajectory as a polynomial trajectory in the Frenet coordinate system established based on the reference path, and transforming the polynomial trajectory in the Frenet coordinate system to the Cartesian coordinate system.
[0023] In this step, we generate a series of autonomous vehicle behavior trajectories by sampling different lateral and longitudinal offsets based on the current road structure (lane reference lines) and traffic rules. All trajectories must be dynamically feasible, conform to vehicle kinematics, and not violate traffic rules. Candidate trajectories are represented using polynomial curves. To adapt to complex urban road structures, we represent the trajectory as a polynomial trajectory in the Frenet coordinate system established based on the reference path, and then transform the trajectory to a Cartesian coordinate system after calculation. The Frenet coordinate system (also known as the SD coordinate system) is a local coordinate system referenced to road reference lines, where the direction along the road reference lines is defined as the s-axis, and the direction perpendicular to the reference lines is defined as the d-axis. Figure 4 As shown, in the Frenet coordinate system, the road reference line uses the arc length parameter because... The description states that the reference line is located at this point. Its tangent vector Indicates the direction of travel, normal vector This represents the lateral direction. The tangent vector of the vehicle's actual trajectory is... and normal vector This describes the vehicle's direction of motion and lateral direction in the global coordinate system. Therefore, the vehicle's actual position relative to the reference line is represented as:
[0024] In the Frenet coordinate system, the s-axis is parallel to the lane lines, and the d-axis is perpendicular to them. This representation provides a more intuitive description of the relative position of a vehicle to the road. Compared to the Cartesian coordinate system, the Frenet coordinate system simplifies the calculation of the distance a vehicle deviates from the lane center and the distance traveled along the lane, without needing to explicitly consider road curvature.
[0025] In this embodiment, S101 specifically refers to: S1011 uses a decoupled representation of the longitudinal and lateral trajectories in the Frenet coordinate system, employing a fifth-order polynomial to fit the longitudinal position of the autonomous vehicle at time t in the Frenet coordinate system. Given the lateral position d(t), the velocity and acceleration of the longitudinal position s, and the velocity and acceleration of the lateral position, the initial state and the sampled final state are given according to the lateral and longitudinal positions, respectively, to solve the fifth-order polynomial coefficients of the longitudinal trajectory and the fifth-order polynomial coefficients of the lateral trajectory. The longitudinal sampling method is to sample in increments of 0.1 m / s² within the acceleration range from -4 m / s² to 2.5 m / s², using a fixed time window T = 8 seconds. S1012 calculates the vehicle's position and velocity in the S direction after 8 seconds of acceleration using the sampled acceleration values. Based on the vehicle's current state and the sampled final state, polynomial parameters are calculated. S1013 samples the polynomial every 0.1 seconds over 8 seconds, obtaining 80 trajectory points in the S direction, and samples three states in the D direction: , and The polynomial is sampled every 0.1 seconds over 8 seconds to obtain 80 trajectory points in the D direction; The sampling results of S1014 in the S and D directions are combined to obtain 120 to 180 sampling trajectory points in the Frenet coordinate system. The trajectory is represented as a series of states. The geometric information of the reference path is used to transform it back into the Cartesian coordinate system. The geometric information includes at least: coordinates, heading angle, and curvature.
[0026] Specifically, we use a decoupled representation of the longitudinal and lateral trajectories in the Frenet coordinate system. We use a fifth-order polynomial to fit the longitudinal position of the autonomous vehicle at time t in the SD coordinate system. :
[0027] Similarly, the lateral position d(t) is also represented by a fifth-degree polynomial:
[0028] Therefore, the velocity and acceleration at the longitudinal position s can be expressed as:
[0029]
[0030] Given an initial state and final state We can solve for the coefficients of the fifth-degree polynomial of the longitudinal trajectory:
[0031] Similar to the representation of the longitudinal trajectory, the coefficients of the transverse trajectory are calculated as follows:
[0032] In the planning problem of autonomous driving, a fixed time window of 8 seconds is typically chosen. This is because this time range can cover typical braking distances and mid-range driving behaviors such as lane changes and overtaking on highways or urban roads, while maintaining sufficient foresight regarding potential risks. An excessively long prediction interval reduces reliability due to accumulated uncertainty, while an excessively short interval makes it difficult to capture interactions in a timely manner. Therefore, 8 seconds achieves a reasonable balance between safety, computational complexity, and prediction accuracy. For longitudinal velocity sampling, an acceleration range of -4 m / s² to 2.5 m / s² with a step size of 0.1 m / s² is commonly used. The design of this range conforms to dynamic constraints and driving comfort requirements: the lower limit of -4 m / s² corresponds to the safety limit of emergency braking, the upper limit of 2.5 m / s² meets the requirements of ride comfort and dynamic accessibility, and the 0.1 m / s² increment ensures the smoothness of the sampling space and the diversity of trajectory generation. Based on this, we calculate the vehicle's position and velocity in the S-direction after 8 seconds of acceleration using the sampled acceleration values. Accelerations that cause the vehicle to exceed the current speed limit or decelerate to a negative speed are filtered out (i.e., trajectories corresponding to accelerations that cause the vehicle to roll backward are classified as "stopping trajectories"). This process yields approximately 40 to 60 valid acceleration choices and their endpoints. Based on the vehicle's current state and the sampled final state, we calculate the polynomial parameters. By sampling the polynomial every 0.1 seconds over 8 seconds, 80 trajectory points in the S direction are obtained.
[0033] S102 Interactive Prediction: Based on the candidate planning trajectory, the historical status of surrounding traffic participants, and map topology information, predict the future status of surrounding vehicles under each candidate planning trajectory condition.
[0034] In this embodiment, S102 specifically includes: S1021 establishes an interactive prediction network for candidate planning trajectory prediction: the predicted future state is modeled as a Gaussian mixture model, where each mixture component corresponds to the joint future state sequence of all surrounding vehicles. At each time step, the joint future state of the surrounding vehicles is represented by a Gaussian distribution.
[0035] in Represents the average and Represents covariance, This represents the physical state of N-1 vehicles surrounding the autonomous vehicle at time t. This represents the possible future motion state of autonomous vehicles; The training process of the interactive prediction network includes steps S1022 to S1024. Step S1022 is a feature processing step; the input data includes vectorized map information. ,in This represents the shape of the map information tensor, where R is the real number field, and the superscript Nm×Np×dp indicates the tensor's dimension shape. Overall, it means the set of all tensors with real numbers as elements and a shape of Nm×Np×dp. It includes the area around the current autonomous vehicle. Each map element consists of [number] map elements. Each feature has 1 attribute feature, and each feature has 12 attributes. Key points: EV future candidate trajectory is the future candidate trajectory of the current autonomous vehicle.
[0036] The S1023 feature encoding step uses an LSTM network to process historical states. Encoding yields features representing the intelligent vehicle and surrounding vehicles. The candidate planned trajectory is encoded by another LSTM as follows: in represent It is a feature set containing the intelligent vehicle and N-1 surrounding vehicles. These represent the feature dimensions of the intelligent vehicle and each surrounding vehicle. The total number of intelligent vehicles and surrounding vehicles; candidate planned trajectories, among which The shape representing the candidate planning trajectory code. The number of candidate planning trajectories. The feature length after encoding the candidate planned trajectory; processed by MLP as Then, it is aggregated into a map representation through max pooling. During the scene interaction and fusion phase, the historical features of all intelligent agents, map features, future plans of intelligent vehicles, and their corresponding map features are stitched together into a unified scene tensor. A three-layer Transformer encoder is used to model the interaction dependencies between elements, resulting in a fused scene code. The future planning for intelligent vehicles refers to the future planning for current autonomous vehicles. represent It is a shape of tensor, This represents the number of lane lines / polylines on the map. This represents the number of attribute features contained in each lane line. D m This represents the uniform number of feature embedding dimensions in the model. This represents the number of map elements after pooling and aggregation.
[0037] S1023 Decoding Steps: Define a learnable modality embedding , Represents the number of modes. The representative decoding feature dimension is concatenated with the scene code of the corresponding intelligent vehicle or surrounding intelligent vehicle agent to form the query, and the query features are obtained through a cross-modal attention mechanism. Then, an MLP is used to decode the Gaussian distribution parameters of each mode at each time step, and another MLP is used to predict the probability distribution of each mode. Finally, the system outputs the trajectory prediction results of the surrounding vehicles in a multimodal manner. The S1024 supervised learning step is for the training process of the interactive prediction network. It uses large-scale real driving data for supervised learning. The input includes a vectorized map of the traffic scene, historical state sequences and candidate planned trajectories. The output is the multimodal distribution of future trajectories of surrounding vehicles.
[0038] The training data comes from the nuPlan dataset and its simulation platform, which contains approximately 1300 hours of real-world vehicle motion data. During the prediction model training, approximately 120,000 scenarios were extracted, with 90% used for training and 10% for validation. The network input includes a vectorized map of the traffic scene, historical state sequences, and candidate vehicle trajectories. The output is a multimodal distribution of future trajectories of surrounding traffic participants. The training objective is to make the prediction results as close as possible to the real trajectories demonstrated by experts. Specifically, a Gaussian Mixture Model (GMM) is used to model the future state distribution, and a negative log-likelihood loss is used to constrain the difference between the predicted and real trajectories. A mode probability loss is also introduced to ensure that the network correctly assigns a high probability to the optimal mode in multimodal prediction. The final total loss function consists of two parts: the alignment error of trajectory point accuracy and the probability error of mode selection, which together optimize prediction accuracy and uncertainty modeling ability. The AdamW optimizer is used during training, with an initial learning rate that is gradually decayed to ensure convergence stability and generalization performance.
[0039] S103 Human-like Decision Making: By analyzing the potential conflicts between candidate planning trajectories and the corresponding future states of surrounding vehicles to avoid risks, the preference cost network evaluates the traffic efficiency and anthropomorphism of the interaction process in the simulated future of multiple candidate planning trajectories plus the corresponding future states of surrounding vehicles, and compares and scores the candidate planning trajectories to select the planning trajectory that best fits human values.
[0040] S103 Human decision-making specifically includes: S1031-S1034 S1031 obtains the status input of surrounding vehicles through an LSTM network. Encode, obtain , This represents the number of surrounding vehicles in the current scene. The length of the hidden state feature vector output by the LSTM network after encoding each surrounding vehicle is [length missing]. Map information M is obtained after being processed using the same structure as the prediction network. , This represents the number of aggregated map elements (such as lane line segments) in the map, and the length of the feature vector mapped to each map element is [length missing]. The two are combined to form a scene context representation. , This represents the total number of nodes after concatenating the features of surrounding vehicles with map features along the line direction. The feature dimension of the concatenated version is... Then, the interaction information is extracted through a two-layer, four-head attention Transformer encoder to obtain... For the autonomous vehicle portion, two LSTMs are used to plan candidate trajectories. and trajectory attributes Encode, obtain and ,here, This represents the number of candidate trajectories generated by the planner, and 128 indicates the length of the feature vector after the geometric path (xy coordinate sequence) of each candidate trajectory is encoded. The number of attribute points is represented; then, the trajectory attributes are weighted and converged through an attention mechanism: first, MLP projection is performed, followed by softmax normalization to obtain the attention weights. Then, the aggregated features are obtained by weighted summation. ; S1032 will and Concatenation for query As keys and values, the inputs to the multi-head cross-attention Transformer are denoted as the query-aware features output. Using MLP to interpret query-aware features C sp Decoding into trajectory scoring , representing the quality score of each candidate planning trajectory; S1033 checks the collision risk of candidate planned trajectories in order of quality score from low to high; S1034 applies lateral and longitudinal perturbations to the expert trajectory in terms of spatial distribution, generating diverse but suboptimal perturbation planning trajectories. The lateral perturbation involves sampling a preset number of lateral perturbation trajectories at preset length intervals to the left and right, centered on the expert trajectory. The longitudinal perturbation involves interpolating between the starting point of the expert trajectory and its path length to generate a preset number of longitudinal perturbation trajectories to simulate early or delayed arrival. Samples exhibiting significant deviation or collision behavior are selected as negative samples and used to train the trajectory evaluation network together with the expert trajectory.
[0041] In this embodiment, specifically, the lateral perturbation is generated by offsetting 0.75m to the left and right from the expert trajectory as the center, sampling 3 trajectories at intervals, and generating a total of 6 lateral perturbation trajectories; the longitudinal perturbation is generated by interpolating between the starting point of the expert trajectory and 30%, 50%, 70%, 90%, and 100% of its path length to generate 5 trajectories.
[0042] S1035 sets the preference cost function The candidate planned trajectories and their corresponding future states of surrounding vehicles are scored, ranked, and selected. These are the parameters to be learned.
[0043] In this embodiment, S1035 further includes: a preference cost function for trajectory sorting and selection. To optimize, the following optimization loss function is defined, including the interval sorting loss. And pairwise logical ordering loss : , ,in Indicates the expert trajectory score. This represents the noise trajectory score, and margin represents the preset interval. . represents the set of positive samples. Represents the set of negative samples; Represents the score of the positive sample. Represents the negative sample score; the overall optimization objective is:
[0044] in and For the weight parameters, minimize Update parameter 𝜃 to achieve optimal sorting of candidate planned trajectories.
[0045] The preference cost function is used to score the candidate trajectories and their corresponding scene information. The parameters to be learned are set as follows: "The score of the expert demonstration trajectory should be higher than that of the noisy trajectory" in order to make the network more in line with human driving preferences.
[0046] To better illustrate the solution of the present invention, an example is given below, such as... Figure 6 As shown, it includes the following steps: This example uses the nuPlan dataset and nuPlan Simulation for experiments. In the experiments, the nuPlan Simulation closed-loop traffic simulation software was used to replace the deployment of the actual scenario, simulating complex real-world traffic situations and acquiring relevant information such as lane information and real-time and historical trajectory information of surrounding vehicles.
[0047] nuPlan, launched by Motional, is the first large-scale autonomous driving planning benchmark designed to advance research in machine learning-based path planning. This dataset encompasses over 1,300 hours of real-world driving data collected from four cities: Las Vegas, Boston, Pittsburgh, and Singapore, providing rich traffic scenarios and behavioral diversity. nuPlan not only includes high-quality automatically labeled information such as dynamic target trajectories, traffic light states, and scene labels, but also provides a closed-loop simulation and evaluation platform, enabling researchers to test and compare the performance of traditional, learning-based, and hybrid planning algorithms in a simulated environment. Through nuPlan, users can train, simulate, and visualize path planning models, deeply exploring the synergy between prediction and planning, as well as safety and generalization capabilities in complex urban traffic environments.
[0048] Using a simulation platform, we experimented with our method in multiple traffic scenarios. The effectiveness of our method was comprehensively evaluated based on indicators such as planning comfort, safety, traffic compliance, and prediction accuracy. The specific implementation steps are as follows: 1) Network Training: We selected 90% of the nuPlan val14 dataset as the training set and 10% as the validation set. Training of the interactive prediction network was performed on an NVIDIA RTX 4080 GPU with a batch size of 256. The AdamW optimizer was used with an initial learning rate of 1e-4, decreasing by a factor of 0.5 every 3 epochs starting from the 15th epoch. The entire training process lasted 30 epochs. The decision network also used the AdamW optimizer with an initial learning rate of 5e-5, decreasing by a factor of 0.5 every 5 epochs, for a total of 15 epochs. During training, various model metrics were monitored until convergence. After convergence, multiple evaluation metrics and methods were used to assess the training effect, such as accuracy and recall, to determine if the model met expectations. If the model's accuracy or other metrics did not meet expectations, the training epochs, batches, or datasets needed to be modified and retrained.
[0049] 2) Closed-Loop Simulation Verification: After model training, the trained weight parameters are loaded into the interactive prediction network and the human-like decision network. Information about the current autonomous vehicle is obtained by calling the nuPlan simulation API, including the current speed, heading, and historical status of the autonomous vehicle and surrounding vehicles, as well as lane information around the autonomous vehicle: lane lines, zebra crossings, traffic lights, etc. The candidate trajectory generation module generates corresponding potential trajectories based on this information, the interactive prediction network generates corresponding interactive trajectories based on the above information, and finally, the human-like decision network determines the optimal planned trajectory based on the interactive prediction results corresponding to each candidate planned trajectory and the attributes of the trajectory itself. The final planning result is fed into nuPlan simulation, where the simulation software uses a realistic vehicle physics model and the LQR tracking control algorithm to track the final planned trajectory. After multiple iterations of simulation, nuPlan simulation provides the simulation results for each frame and the final planning performance score.
[0050] Preliminary qualitative simulation results are as follows: Figure 6 As shown, in scenarios including intersections and multi-lane roads, our method can plan safe and comfortable trajectories that conform to human decision-making. Quantitative results are shown in Table 1, where the score represents the simulation score of the vehicle's planned trajectory by Nuplan, reflecting the quality of the trajectory under real-world conditions.
[0051] Table 1 Nuplan quantitative simulation results
[0052] Example 2 An autonomous driving system based on deep learning and preference cost learning is implemented in electronic device hardware with a central processing unit, such as a personal computer, smart terminal, local area network, or server. For implementation details in this example, please refer to [link to relevant documentation]. Figure 7 It includes a candidate trajectory generation module 61, an interactive prediction module 62, and a human-like decision-making module 63.
[0053] The candidate trajectory generation module 61 is used to extract lane reference lines, navigation and surrounding traffic participant information from the current high-precision map and upstream perception of the autonomous driving system, and to sample multiple candidate planning trajectories based on the lane reference lines, navigation, surrounding traffic participant information and traffic rules. Interactive prediction module 62 is used to predict the future state of surrounding vehicles under each candidate planning trajectory based on the candidate planning trajectory, the historical state of surrounding traffic participants, and map topology information. The human-like decision-making module 63 is used to avoid risks by analyzing the potential conflicts between candidate planning trajectories and the corresponding future states of surrounding vehicles. The preference cost network evaluates the traffic efficiency and anthropomorphism of the interaction process in the simulated future of multiple candidate planning trajectories plus the corresponding future states of surrounding vehicles, and compares and scores the candidate planning trajectories to select the planning trajectory that best fits human values.
[0054] The autonomous driving system based on deep learning and preference cost learning in this embodiment is implemented in the same way and with the same effect as the autonomous driving method based on deep learning and preference cost learning described in Embodiment 1, and will not be repeated here.
[0055] Example 3 This invention relates to a computer-readable storage medium storing instructions that, when executed, perform an autonomous driving method based on deep learning and preference cost learning according to Embodiment 1. The execution process and effects are the same as those of the autonomous driving method based on deep learning and preference cost learning described in Embodiment 1, and will not be repeated here.
[0056] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.
[0057] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.
Claims
1. An autonomous driving method based on deep learning and preference cost learning, characterized in that, Autonomous driving for smart vehicles, including: Candidate trajectory generation: Lane reference lines, navigation, and surrounding traffic participant information are extracted from the current high-precision map and upstream perception of the autonomous driving system. Multiple candidate planning trajectories are sampled based on the lane reference lines, navigation, surrounding traffic participant information, and traffic rules. Interactive prediction: Based on the candidate planning trajectory, the historical status of surrounding traffic participants, and map topology information, the future status of surrounding vehicles is predicted under each candidate planning trajectory condition; Human-like decision-making: By analyzing the potential conflicts between candidate planning trajectories and the corresponding future states of surrounding vehicles to avoid risks, the preference cost network evaluates the traffic efficiency and anthropomorphism of the interaction process in the simulated future of multiple candidate planning trajectories plus the corresponding future states of surrounding vehicles, and compares and scores the candidate planning trajectories to select the planning trajectory that best fits human values.
2. The autonomous driving method based on deep learning and preference cost learning according to claim 1, characterized in that, The candidate trajectory generation step specifically includes: representing the candidate trajectory as a polynomial trajectory in the Frenet coordinate system established based on the reference path, and transforming the polynomial trajectory in the Frenet coordinate system to the Cartesian coordinate system.
3. The autonomous driving method based on deep learning and preference cost learning according to claim 2, characterized in that, The process of representing the candidate planning trajectory as a polynomial trajectory in the Frenet coordinate system established based on the reference path, and then transforming the candidate planning trajectory to the Cartesian coordinate system after calculation, specifically involves: Using a decoupled representation of the longitudinal and lateral trajectories in the Frenet coordinate system, a fifth-order polynomial is used to fit the longitudinal position of the autonomous vehicle at time t in the Frenet coordinate system. Given the lateral position d(t), the velocity and acceleration of the longitudinal position s, and the velocity and acceleration of the lateral position, the initial state and the sampled final state are given according to the lateral and longitudinal positions, respectively, to solve the fifth-order polynomial coefficients of the longitudinal trajectory and the fifth-order polynomial coefficients of the lateral trajectory. The longitudinal sampling method is to sample in increments of 0.1 m / s² within the acceleration range from -4 m / s² to 2.5 m / s², using a fixed time window T = 8 seconds. The vehicle's position and velocity in the S direction after 8 seconds of acceleration are calculated using the sampled acceleration values. Based on the vehicle's current state and the sampled final state, polynomial parameters are calculated. The polynomial is sampled every 0.1 seconds over 8 seconds to obtain 80 trajectory points in the S direction, and three states are sampled in the D direction: , and The polynomial is sampled every 0.1 seconds over 8 seconds to obtain 80 trajectory points in the D direction; By combining the sampling results in the S and D directions, 120 to 180 sampling trajectory points in the Frenet coordinate system are obtained. The trajectory is represented as a series of states. The geometric information of the reference path is used to transform it back into the Cartesian coordinate system. .
4. The autonomous driving method based on deep learning and preference cost learning according to claim 1, characterized in that, The step of predicting the future state of surrounding vehicles under each candidate planning trajectory based on the candidate planning trajectory, the historical state of surrounding traffic participants, and map topology information includes: An interactive prediction network is established for candidate planning trajectory prediction: the predicted future states are modeled as a Gaussian mixture model, where each mixture component corresponds to the joint future state sequence of all surrounding vehicles. At each time step, the future states of the surrounding vehicles are represented by a Gaussian distribution. in Represents the average and Represents covariance, This represents the physical state of N-1 vehicles surrounding the autonomous vehicle at time t. This represents the possible future motion state of autonomous vehicles; The training process of an interactive prediction network includes: The feature processing step involves input data including vectorized map information. ,in It represents the shape of the map information tensor, containing the area around the current autonomous vehicle. Each map element consists of [number] map elements. Each feature has 1 attribute feature, and each feature has 12 attributes. D p A key point. The feature encoding step uses an LSTM network to encode historical states, obtaining features representing the intelligent vehicle and surrounding vehicles. ,in represent It is a feature set containing the intelligent vehicle and N-1 surrounding vehicles. These represent the feature dimensions of the intelligent vehicle and each surrounding vehicle. The total number of intelligent vehicles and surrounding vehicles; the candidate planned trajectory is encoded by another LSTM as ,in The shape representing the candidate planning trajectory code. The number of candidate planning trajectories. The feature length after encoding the candidate planned trajectory; processed by MLP as , represent It is a shape of tensor, This represents the number of lane lines on the map. This represents the number of attribute features contained in each lane line. The model represents a uniform number of feature embedding dimensions; these are then aggregated using max pooling to form a map representation. , The number of map elements after aggregation; during the scene interaction fusion phase, the historical features of all intelligent agents, map features, future plans of intelligent vehicles, and their corresponding map features are stitched together into a unified scene tensor. ,here This represents the number of map elements after pooling and aggregation, where 1 represents the intelligent vehicle itself. Representing the unified aligned feature dimensions; a three-layer Transformer encoder is used to model the interaction dependencies between elements, resulting in the fused scene code. ; Decoding steps: Define a learnable modality embedding , Represents the number of modes. The feature dimension is represented by the decoding feature, which is concatenated with the scene code of the corresponding intelligent vehicle or surrounding vehicles to form the query. The query feature is obtained through a cross-modal attention mechanism. Then, the MLP is used to decode the Gaussian distribution parameters of each mode at each time step, and another MLP is used to predict the probability distribution of each mode, finally outputting the trajectory prediction results of the vehicles around the multi-mode. The supervised learning step, for the training process of the interactive prediction network, uses large-scale real driving data for supervised learning. The input includes a vectorized map of the traffic scene, historical state sequences and candidate planned trajectories, and the output is the multimodal distribution of future trajectories of surrounding vehicles.
5. The autonomous driving method based on deep learning and preference cost learning according to claim 1, characterized in that, The process involves analyzing potential conflicts between candidate planned trajectories and the corresponding future states of surrounding vehicles to mitigate risks. A preference cost network evaluates the traffic efficiency and anthropomorphism of the interaction process across multiple candidate planned trajectories and simulated future states of surrounding vehicles. This evaluation compares and scores the candidate planned trajectories, ultimately selecting the one that best aligns with human values. This includes: The status of surrounding vehicles is input via an LSTM network. Encode, obtain , This represents the number of surrounding vehicles in the current scene. The length of the hidden state feature vector output by the LSTM network after encoding each surrounding vehicle is [length missing]. The map information is obtained after being processed using the same structure as the prediction network. , This represents the number of aggregated map elements (such as lane line segments) in the map, and the length of the feature vector mapped to each map element is [length missing]. The two are combined to form a scene context representation. , This represents the total number of nodes after concatenating the surrounding vehicle features with the map features along the row direction. The dimension of the resulting tensor is... Then, the interaction information is extracted through a two-layer, four-head attention Transformer encoder to obtain... ; For the autonomous vehicle portion, two LSTMs are used to plan candidate trajectories. and trajectory attributes Encode, obtain and ,here, This represents the number of candidate trajectories generated by the planner, and 128 indicates the length of the feature vector after the geometric path (xy coordinate sequence) of each candidate trajectory is encoded. The number of attribute points is represented; then, the trajectory attributes are weighted and converged through an attention mechanism: first, MLP projection is performed, followed by softmax normalization to obtain the attention weights. Then, the aggregated features are obtained by weighted summation. ; Will and Concatenation for query As keys and values, the inputs to the multi-head cross-attention Transformer are denoted as the query-aware features output. Using MLP to interpret query-aware features C sp Decoding into trajectory scoring , representing the quality score of each candidate planning trajectory; The collision risk of candidate planned trajectories is checked in order of quality score from low to high. The expert trajectory is perturbed laterally and longitudinally in terms of spatial distribution to generate diverse but suboptimal perturbed trajectories. The lateral perturbation is to sample a preset number of lateral perturbed trajectories at preset length intervals to the left and right, centered on the expert trajectory. The longitudinal perturbation is to interpolate between the starting point of the expert trajectory and its path length to generate a preset number of longitudinal perturbed trajectories to simulate early or delayed arrival. Samples that show obvious deviation or collision behavior are selected as negative samples and used to train the trajectory evaluation network together with the expert trajectory. Set the preference cost function The candidate planned trajectories and their corresponding future states of surrounding vehicles are scored, ranked, and selected. These are the parameters to be learned.
6. The autonomous driving method based on deep learning and preference cost learning according to claim 5, characterized in that, The evaluation of the efficiency and anthropomorphism of the interaction process to compare and score the candidate planning trajectories and select the planning trajectory that best aligns with human values also includes: evaluating the cost function. To optimize, the following optimization loss function is defined, including the interval sorting loss. And pairwise logical ordering loss : , .,in Indicates the expert trajectory score. This represents the noise trajectory score, and margin represents the preset interval. . represents the set of positive samples. Represents the set of negative samples; Represents the score of the positive sample. Represents the negative sample score; the overall optimization objective is: in and For the weight parameters, minimize Update parameter 𝜃 to achieve optimal sorting of candidate planned trajectories.
7. An autonomous driving system based on deep learning and preference cost learning, characterized in that, include: The candidate trajectory generation module is used to extract lane reference lines, navigation, and surrounding traffic participant information from the current high-precision map and upstream perception of the autonomous driving system, and to sample multiple candidate planning trajectories based on the lane reference lines, navigation, surrounding traffic participant information, and traffic rules. An interactive prediction module is used to predict the future state of surrounding vehicles under each candidate planning trajectory based on the candidate planning trajectory, the historical state of surrounding traffic participants, and map topology information. The human-like decision-making module is used to avoid risks by analyzing the potential conflicts between candidate planning trajectories and the corresponding future states of surrounding vehicles. The preference cost network evaluates the traffic efficiency and anthropomorphism of the interaction process in the simulated future of multiple candidate planning trajectories plus the corresponding future states of surrounding vehicles, and compares and scores the candidate planning trajectories to select the planning trajectory that best fits human values.
8. A computer-readable storage medium, characterized in that: The storage medium stores instructions that, when executed, perform an autonomous driving method based on deep learning and preference cost learning as described in any one of claims 1-6.