A model training method, electronic equipment, vehicle and readable storage medium

Abstract

Embodiments of the present application provide a model training method, an electronic device, a vehicle and a readable storage medium, wherein the method comprises: obtaining a plurality of groups of training samples, wherein each group of training samples comprises real-time scene perception data; taking the real-time scene perception data in the plurality of groups of training samples as model input, and taking a standard trajectory corresponding to the real-time scene perception data as a true value to train an initial trajectory prediction model to obtain a target trajectory prediction model. The present application solves the technical problem of low training efficiency of the trajectory prediction model in the related art.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a model training method, electronic device, vehicle, and readable storage medium. Background Technology

[0002] In the field of autonomous driving technology, existing trajectory prediction model training methods typically rely on manually designed reward functions. These manually designed reward functions are determined based on pre-defined rules, such as safe distance, speed limits, and adherence to traffic signals. While these rules are fundamental to ensuring driving safety, overly simplified designs cannot fully reflect complex driving environments and various possible driving behaviors, resulting in low training efficiency for existing trajectory prediction models.

[0003] No effective solution has yet been proposed to address the above issues. Summary of the Invention

[0004] This application provides a model training method, an electronic device, a vehicle, and a readable storage medium, aiming to improve the problem of low training efficiency of trajectory prediction models in related technologies.

[0005] According to one embodiment of this application, a model training method is provided, comprising: acquiring multiple sets of training samples, wherein each set of training samples includes: real-time scene perception data, the real-time scene perception data including: navigation target, state of any traffic participant, map data and road rule information; using the real-time scene perception data from the multiple sets of training samples as model input, and using the standard trajectory corresponding to the real-time scene perception data as ground truth to train an initial trajectory prediction model to obtain a target trajectory prediction model, wherein the reward function of the target trajectory prediction model includes: a performance reward function, the performance reward function including: a safety utility function, a comfort utility function and an efficiency utility function, the safety utility function being used to evaluate the safety compliance of the driving trajectory, the comfort utility function being used to evaluate the ride comfort of the driving trajectory, the efficiency utility function being used to evaluate the traffic efficiency of the driving trajectory, the safety utility function being determined by nonlinear mapping of safe collision time, safe lateral distance and scene risk level, the comfort utility function being determined by weighted normalization scoring of the vehicle's longitudinal acceleration, lateral acceleration and jerk, and the efficiency utility function being determined based on the ratio of actual travel time to ideal travel time.

[0006] The initial trajectory prediction model is trained by multiple sets of real-world scenario data to obtain the target trajectory prediction model. The safety utility function, comfort utility function, and efficiency utility function are integrated to enable the target trajectory prediction model to adaptively evaluate trajectory quality, significantly improve prediction accuracy and human-likeness, and achieve an intelligent balance between safety compliance, riding comfort, and efficient passage.

[0007] Optionally, determining the standard trajectory corresponding to the real-time scene perception data includes: obtaining compliant driving trajectories corresponding to the real-time scene perception data based on a hard rule base, wherein the hard rule base includes traffic regulations, safe driving guidelines, and expert experience; obtaining multiple anthropomorphic driving trajectories corresponding to the real-time scene perception data based on a historical driving experience database, wherein the historical driving experience database includes various real driving strategy data; removing driving trajectories that conflict with compliant driving trajectories from the multiple anthropomorphic driving trajectories to obtain a set of driving trajectories; in response to the existence of at least one anthropomorphic driving trajectory in the set of driving trajectories, using the anthropomorphic driving trajectory whose score meets a preset standard as the standard trajectory; in response to the existence of no anthropomorphic driving trajectory in the set of driving trajectories, using the compliant driving trajectory as the standard trajectory.

[0008] Optionally, the reward function of the target trajectory prediction model further includes: a basic survival reward function, a constraint violation penalty function, and a round-end reward function. The model training method further includes: determining a basic survival reward signal using the basic survival reward function at each training step; determining a round-end reward signal using the round-end reward function by determining whether the initial trajectory prediction model has completed the entire training round; determining a constraint violation penalty signal using the constraint violation penalty function and a performance reward signal using the performance reward function in response to the standard trajectory satisfying the preset safety constraints and preset comfort constraints, and the initial driving trajectory output by the initial trajectory prediction model not satisfying the preset safety constraints or preset comfort constraints; determining a performance reward signal using the performance reward function in response to the initial driving trajectory satisfying the preset safety constraints and preset comfort constraints; obtaining a reward signal by summing the basic survival reward signal, the round-end reward signal, the constraint violation penalty signal, and the performance reward signal; and training the initial trajectory prediction model using the reward signal to obtain the target trajectory prediction model.

[0009] Optionally, the performance reward signal is determined through the performance reward function, including: determining a first comprehensive utility value of the initial driving trajectory and a second comprehensive utility value of the standard trajectory based on the safety utility function, comfort utility function, and efficiency utility function, respectively; determining the utility difference between the first comprehensive utility value and the second comprehensive utility value; and determining the performance reward signal through the performance reward function and the utility difference.

[0010] Optionally, determining the first comprehensive utility value of the initial driving trajectory and the second comprehensive utility value of the standard trajectory based on the safety utility function, comfort utility function, and efficiency utility function respectively includes: determining the first safety utility value of the initial driving trajectory and the second safety utility value of the standard trajectory using the safety utility function; determining the first comfort utility value of the initial driving trajectory and the second comfort utility value of the standard trajectory using the comfort utility function; determining the first efficiency utility value of the initial driving trajectory and the second efficiency utility value of the standard trajectory using the efficiency utility function; weighting the first safety utility value, the first comfort utility value, and the first efficiency utility value using a first preset weight to obtain the first comprehensive utility value, wherein the first preset weight is dynamically adjusted based on the risk categories of multiple types of test scenarios; and weighting the second safety utility value, the second comfort utility value, and the second efficiency utility value using a second preset weight to obtain the second comprehensive utility value, wherein the second preset weight is dynamically adjusted based on the risk categories of multiple types of test scenarios.

[0011] Optionally, the performance reward signal is determined by the performance reward function and the utility difference, including: determining the performance reward signal as zero according to the performance reward function in response to a utility difference equal to zero; determining the performance reward signal as a positive reward according to the performance reward function in response to a utility difference greater than zero; and determining the performance reward signal as a negative reward according to the performance reward function in response to a utility difference less than zero.

[0012] Optionally, the initial trajectory prediction model is trained using the reward signal to obtain the target trajectory prediction model, including: determining experience units based on the reward signal, wherein the experience units are used to update the initial driving trajectory; updating the initial driving trajectory according to the experience units in response to the completion of multiple training samples or the total number of training steps reaching a preset value, to obtain an updated driving trajectory; and continuously training the initial trajectory prediction model based on the updated driving trajectory to obtain the target trajectory prediction model.

[0013] According to one embodiment of this application, a model training apparatus is also provided, comprising: an acquisition module for acquiring multiple sets of training samples, wherein each set of training samples includes: real-time scene perception data, the real-time scene perception data including: navigation target, state of any traffic participant, map data and road rule information; and a training module for using the real-time scene perception data from the multiple sets of training samples as model input, and using the standard trajectory corresponding to the real-time scene perception data as ground truth to train an initial trajectory prediction model to obtain a target trajectory prediction model, wherein the reward function of the target trajectory prediction model includes: a performance reward function. The performance reward function includes: a safety utility function, a comfort utility function, and an efficiency utility function. The safety utility function is used to evaluate the safety compliance of the driving trajectory; the comfort utility function is used to evaluate the ride comfort of the driving trajectory; and the efficiency utility function is used to evaluate the traffic efficiency of the driving trajectory. The safety utility function is determined by nonlinear mapping of safe collision time, safe lateral distance, and scenario risk level; the comfort utility function is determined by weighted normalization scoring of the vehicle's longitudinal acceleration, lateral acceleration, and jerk; and the efficiency utility function is determined based on the ratio of actual travel time to ideal travel time.

[0014] Optionally, the model training device further includes: a first determining module, used to obtain compliant driving trajectories corresponding to real-time scene perception data based on a hard rule base, wherein the hard rule base includes traffic regulations, safe driving guidelines, and expert experience; to obtain multiple anthropomorphic driving trajectories corresponding to real-time scene perception data based on a historical driving experience database, wherein the historical driving experience database includes various real driving strategy data; to remove driving trajectories that conflict with compliant driving trajectories from the multiple anthropomorphic driving trajectories to obtain a set of driving trajectories; in response to the existence of at least one anthropomorphic driving trajectory in the set of driving trajectories, to use the anthropomorphic driving trajectory whose score meets a preset standard as the standard trajectory; in response to the existence of no anthropomorphic driving trajectory in the set of driving trajectories, to use the compliant driving trajectory as the standard trajectory.

[0015] Optionally, the reward function of the target trajectory prediction model further includes: a basic survival reward function, a constraint violation penalty function, and a round-end reward function. The training module is also used to determine the basic survival reward signal at each training step using the basic survival reward function; determine the round-end reward signal by judging whether the initial trajectory prediction model has completed the entire training round using the round-end reward function; determine the constraint violation penalty signal and the performance reward signal by the constraint violation penalty function in response to the standard trajectory satisfying the preset safety constraints and preset comfort constraints, and the initial driving trajectory output by the initial trajectory prediction model not satisfying the preset safety constraints or preset comfort constraints; determine the performance reward signal by the performance reward function in response to the initial driving trajectory satisfying the preset safety constraints and preset comfort constraints; obtain the reward signal by summing the basic survival reward signal, the round-end reward signal, the constraint violation penalty signal, and the performance reward signal; and train the initial trajectory prediction model using the reward signal to obtain the target trajectory prediction model.

[0016] Optionally, the model training device further includes: a second determining module, used to determine a first comprehensive utility value of the initial driving trajectory and a second comprehensive utility value of the standard trajectory based on the safety utility function, the comfort utility function, and the efficiency utility function, respectively; determine the utility difference between the first comprehensive utility value and the second comprehensive utility value; and determine a performance reward signal through the performance reward function and the utility difference.

[0017] Optionally, the second determining module is further configured to determine a first safety utility value of the initial driving trajectory and a second safety utility value of the standard trajectory using a safety utility function; determine a first comfort utility value of the initial driving trajectory and a second comfort utility value of the standard trajectory using a comfort utility function; determine a first efficiency utility value of the initial driving trajectory and a second efficiency utility value of the standard trajectory using an efficiency utility function; weight the first safety utility value, the first comfort utility value, and the first efficiency utility value using a first preset weight to obtain a first comprehensive utility value, wherein the first preset weight is dynamically adjusted based on the risk categories of multiple types of test scenarios; and weight the second safety utility value, the second comfort utility value, and the second efficiency utility value using a second preset weight to obtain a second comprehensive utility value, wherein the second preset weight is dynamically adjusted based on the risk categories of multiple types of test scenarios.

[0018] Optionally, the second determining module is further configured to determine the performance reward signal as zero based on the performance reward function in response to a utility difference equal to zero; to determine the performance reward signal as a positive reward based on the performance reward function in response to a utility difference greater than zero; and to determine the performance reward signal as a negative reward based on the performance reward function in response to a utility difference less than zero.

[0019] Optionally, the training module is also used to determine experience units based on reward signals, wherein the experience units are used to update the initial driving trajectory; in response to the completion of multiple training samples or the total number of training steps reaching a preset value, the initial driving trajectory is updated according to the experience units to obtain the updated driving trajectory; the initial trajectory prediction model is continuously trained based on the updated driving trajectory to obtain the target trajectory prediction model.

[0020] According to one embodiment of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the model training method described in any of the preceding claims.

[0021] According to one embodiment of this application, a vehicle is also provided, including: a memory storing an executable program; and a processor for running the executable program, wherein the executable program executes the model training method described in any of the above claims when running on the processor.

[0022] According to one embodiment of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer-readable storage medium, and the computer program is configured to execute the model training method described in any of the above claims when run on a computer or processor.

[0023] In this embodiment, multiple sets of training samples are acquired, each set including real-time scene perception data, which includes navigation target, state of any traffic participant, map data, and road rule information. The real-time scene perception data from the multiple sets of training samples is used as model input, and the standard trajectory corresponding to the real-time scene perception data is used as ground truth to train the initial trajectory prediction model to obtain the target trajectory prediction model. The reward function of the target trajectory prediction model includes a performance reward function, which includes a safety utility function, a comfort utility function, and an efficiency utility function. The safety utility function is used to evaluate the safety compliance of the driving trajectory, and the comfort utility function is used to evaluate the safety compliance of the driving trajectory. The suitability utility function is used to evaluate the ride comfort of the driving trajectory, the efficiency utility function is used to evaluate the traffic efficiency of the driving trajectory, the safety utility function is determined by nonlinear mapping of safe collision time, safe lateral distance and scenario risk level, the comfort utility function is determined by weighted normalization scoring of vehicle longitudinal acceleration, lateral acceleration and jerk, and the efficiency utility function is determined based on the ratio of actual travel time to ideal travel time. This achieves the goal of multi-dimensional quantification of the merits of driving trajectories, thereby achieving the technical effect of training a human-like trajectory prediction model that combines safety, comfort and efficiency, and thus solving the technical problem of low training efficiency of trajectory prediction models in related technologies. Attached Figure Description

[0024] Figure 1 This is a flowchart of a model training method provided in an embodiment of this application;

[0025] Figure 2 This is a scenario risk level comparison diagram provided in one embodiment of this application;

[0026] Figure 3 This is a flowchart of a model training method based on a virtual expert system provided in an embodiment of this application;

[0027] Figure 4 This is a structural diagram of a model training device provided in an embodiment of this application;

[0028] Figure 5 This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0029] To make the technical problems, technical solutions, and beneficial effects solved by this application clearer, the following detailed description is provided in conjunction with embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0030] One embodiment of this application provides a model training method, comprising: acquiring multiple sets of training samples, wherein each set of training samples includes: real-time scene perception data, which includes: navigation target, state of any traffic participant, map data and road rule information; using the real-time scene perception data from the multiple sets of training samples as model input, and using the standard trajectory corresponding to the real-time scene perception data as ground truth to train an initial trajectory prediction model to obtain a target trajectory prediction model, wherein the reward function of the target trajectory prediction model includes: a performance reward function, which includes: a safety utility function, a comfort utility function and an efficiency utility function. The safety utility function is used to evaluate the safety compliance of the driving trajectory, the comfort utility function is used to evaluate the ride comfort of the driving trajectory, and the efficiency utility function is used to evaluate the traffic efficiency of the driving trajectory. The safety utility function is determined by nonlinear mapping of safe collision time, safe lateral distance and scene risk level, the comfort utility function is determined by weighted normalization scoring of vehicle longitudinal acceleration, lateral acceleration and jerk, and the efficiency utility function is determined based on the ratio of actual travel time to ideal travel time.

[0031] Through the above steps, the goal of multi-dimensional quantification of driving trajectory quality is achieved, thereby enabling the training of a human-like trajectory prediction model that combines safety, comfort, and efficiency, and thus solving the technical problem of low training efficiency of trajectory prediction models in related technologies.

[0032] Figure 1 This is a flowchart of a model training method provided in an embodiment of this application, as shown below. Figure 1 As shown, it includes the following steps:

[0033] S11: Obtain multiple sets of training samples, where each set of training samples includes: real-time scene perception data, which includes: navigation target, state of any traffic participant, map data and road rule information;

[0034] In this embodiment of the application, multiple training samples refer to a large set of driving scenario data collected in a real environment. Each set of samples corresponds to an independent driving scenario and is used to train the trajectory prediction model to learn reasonable behavior patterns under different scenarios.

[0035] Real-time scene perception data refers to a complete set of information describing the current driving environment that is acquired in real time by sensors or simulation systems at a certain moment.

[0036] Navigation targets refer to the final destination of an autonomous vehicle or a series of key waypoints in the route planning, used to guide the vehicle to complete its task.

[0037] The state of any traffic participant refers to the position, speed, acceleration, heading angle, and direction of motion of other dynamic entities (such as the vehicle in front, oncoming vehicles, pedestrians, non-motorized vehicles, etc.) in the same scene as the vehicle at the current moment, and there are no restrictions here.

[0038] Map data refers to the static environmental information provided by high-definition maps, including lane geometry, road boundaries, right-of-way information, traffic sign locations, intersection topology, etc., which are not limited here. Map data also refers to the ability of high-definition maps to constrain vehicle drivable areas and the legality of vehicle behavior.

[0039] Road rule information refers to the dynamic or static traffic rules applicable in the current scenario, such as traffic light status (red / green), speed limits, no-lane-changing zones, yield signs, pedestrian right-of-way, etc., which are not restricted here. Road rule information can be used to determine whether vehicle behavior is compliant.

[0040] It can be seen that by acquiring multiple sets of training samples containing navigation targets, traffic participant states, map data, and road rule information, multidimensional input data that closely resembles the real driving environment is constructed, enabling the trajectory prediction model to simultaneously perceive intent, interaction, and environmental constraints, significantly improving the scenario adaptability and decision rationality of trajectory prediction.

[0041] S12: Using real-time scene perception data from multiple training samples as model input and the standard trajectory corresponding to the real-time scene perception data as ground truth, the initial trajectory prediction model is trained to obtain the target trajectory prediction model. The reward function of the target trajectory prediction model includes a performance reward function, which includes a safety utility function, a comfort utility function, and an efficiency utility function. The safety utility function is used to evaluate the safety compliance of the driving trajectory, the comfort utility function is used to evaluate the ride comfort of the driving trajectory, and the efficiency utility function is used to evaluate the traffic efficiency of the driving trajectory. The safety utility function is determined by nonlinear mapping of safe collision time, safe lateral distance, and scene risk level. The comfort utility function is determined by weighted normalization scoring of the vehicle's longitudinal acceleration, lateral acceleration, and jerk. The efficiency utility function is determined based on the ratio of actual travel time to ideal travel time.

[0042] In this embodiment of the application, the standard trajectory refers to a reference driving trajectory that conforms to safety and comfort specifications, which is actually executed by a human driver or generated by a simulation system in a given real-time scenario, and is used as the true value for model training.

[0043] An initial trajectory prediction model refers to a deep learning model that has not been fully trained. It has the basic structure to output a predicted trajectory after inputting scene data, but has not yet learned human driving behavior patterns.

[0044] The target trajectory prediction model refers to the trajectory prediction model that has been trained and optimized through multiple sets of samples, and can predict trajectories that meet multiple requirements such as safety, comfort, and efficiency.

[0045] The performance reward function is a comprehensive scoring function used to quantify the gap between the predicted trajectory and the ideal driving behavior. It is composed of three weighted sub-utility functions: safety, comfort, and efficiency.

[0046] The safety utility function is used to evaluate the ability of a trajectory to avoid collisions by combining the minimum of the collision time and lateral distance with the scene risk level and through an exponential nonlinear mapping.

[0047] The comfort utility function is used to perform weighted normalization of longitudinal or lateral acceleration and jerk to evaluate driving smoothness.

[0048] The efficiency utility function is used to calculate the ratio of actual travel time to ideal travel time. The closer the ratio is to 1, the higher the efficiency.

[0049] The safe collision time is the time it is expected to take for a collision to occur between the vehicle and the vehicle in front or an obstacle at the current relative speed. It is used to measure the risk of a collision.

[0050] Safe lateral distance represents the closest lateral distance between a vehicle and an adjacent obstacle (such as a lane-changing vehicle or a guardrail), and is used to assess lateral safety margin.

[0051] Figure 2 This is a scenario risk level comparison diagram provided in one embodiment of this application, such as... Figure 2 As shown, the scenario risk level is a quantitative basic risk value determined by a comprehensive assessment of environmental type (such as highways, urban roads, intersections, school areas), traffic density (such as low, medium, and high), and weather (such as sunny / cloudy, rainy, and snowy), which is used to dynamically adjust the risk sensitivity coefficient.

[0052] Longitudinal acceleration is used to represent the rate of change of a vehicle's velocity along the direction of travel, reflecting the intensity of acceleration or braking.

[0053] Lateral acceleration is used to represent the rate of change of a vehicle's velocity in the direction perpendicular to its travel, reflecting the smoothness of steering or lane changing.

[0054] Jerk is the derivative of acceleration with respect to time, used to measure the degree of abrupt change in acceleration, and directly affects the discomfort of passengers.

[0055] The actual travel time is the real time it takes for the vehicle to complete the specified path under the current trajectory.

[0056] Ideal travel time is the theoretical shortest travel time under conditions of no traffic interference, no rule constraints, and optimal route and speed, and serves as a benchmark for efficiency evaluation.

[0057] Using real-time scene perception data from multiple training samples as model input and standard trajectories corresponding to the real-time scene perception data as ground truth to train the initial trajectory prediction model and obtain the target trajectory prediction model can be understood as using multi-dimensional perception information of real driving scenarios as input and high-quality standard trajectories as the "golden reference standard" to guide the initial trajectory prediction model to continuously adjust parameters and obtain the target trajectory prediction model, thereby improving the training efficiency of the trajectory prediction model.

[0058] It can be seen that by quantifying safety, comfort and efficiency through multi-dimensional utility functions, the trajectory prediction model internalizes complex driving principles during supervised training, avoiding bias caused by single-index optimization, thereby improving the compliance of the model's output trajectory and the robustness of the system.

[0059] Optionally, in step S12, determining the standard trajectory corresponding to the real-time scene perception data includes the following steps:

[0060] Step S121: Obtain the compliant driving trajectory corresponding to the real-time scene perception data based on the hard rule base, which includes: traffic regulations, safe driving guidelines and expert experience;

[0061] Step S122: Obtain multiple anthropomorphic driving trajectories corresponding to real-time scene perception data based on the historical driving experience database, wherein the historical driving experience database includes: various real driving strategy data;

[0062] Step S123: Remove driving trajectories that conflict with compliant driving trajectories from multiple anthropomorphic driving trajectories to obtain a set of driving trajectories;

[0063] Step S124: In response to the existence of at least one anthropomorphic driving trajectory in the driving trajectory set, the anthropomorphic driving trajectory whose score meets the preset standard among the at least one anthropomorphic driving trajectory is taken as the standard trajectory.

[0064] Step S125: In response to the absence of any anthropomorphic driving trajectory in the driving trajectory set, the compliant driving trajectory is taken as the standard trajectory.

[0065] In this embodiment, the hard rule base is a set of deterministic rules formed by the structured coding of traffic regulations, safe driving guidelines, and expert experience, which is used to define the boundaries of safe driving behavior.

[0066] The compliant driving trajectory is a deterministic driving path generated based on a hard rule base that meets all safety and regulatory constraints, prioritizing absolute safety.

[0067] Traffic regulations are mandatory driving rules of a country or region, such as stopping at red lights, speed limits, and right-of-way rules, and are not restricted here.

[0068] Safe driving guidelines are defensive driving principles defined by industry standards or the International Organization for Standardization (ISO) 21448, such as minimum following distance and obstacle avoidance buffer zones, and are not restricted here.

[0069] Expert experience refers to widely recognized operating habits summarized by experienced drivers or safety engineers, such as "slowing down and yielding in advance" and "avoiding lane changes in blind spots," which are not restricted here.

[0070] The historical driving experience database contains a large collection of real human driving data, covering diverse scenarios and driving styles, which are not limited here.

[0071] Human-like driving trajectories are candidate trajectories generated by historical data-driven models that simulate natural human driving behavior, and are characterized by fluency and efficiency.

[0072] The driving trajectory set is a set of feasible trajectories that are retained after being removed, while simultaneously satisfying compliance and anthropomorphism.

[0073] The preset criteria are used to select the optimal anthropomorphic trajectory. For example, the trajectory with the highest comprehensive score in the set of driving trajectories is used as the standard trajectory, but there is no restriction here.

[0074] Obtaining compliant driving trajectories corresponding to real-time scene perception data based on a rigid rule base can be understood as generating a safe benchmark driving trajectory without any violation risks, based on traffic regulations, safety guidelines, and expert experience.

[0075] The multiple anthropomorphic driving trajectories corresponding to real-time scene perception data obtained from historical driving experience databases can be understood as using real human driving data to output multiple potential driving modes that conform to human habits, are smooth, but may not be entirely compliant.

[0076] The process of removing driving trajectories that conflict with compliant driving trajectories from multiple anthropomorphic driving trajectories can be understood as deleting all anthropomorphic driving trajectories that violate hard safety constraints (such as running red lights or collision risks) from multiple anthropomorphic driving trajectories, and retaining only candidate paths that are both safe and in line with human behavioral characteristics.

[0077] In response to the existence of at least one anthropomorphic driving trajectory in the set of driving trajectories, taking the anthropomorphic driving trajectory whose score meets the preset standard as the standard trajectory can be understood as prioritizing the anthropomorphic driving trajectory with the best overall performance as the standard trajectory under the premise of safety.

[0078] Since no anthropomorphic driving trajectory exists in the set of driving trajectories, using the compliant driving trajectory as the standard trajectory can be understood as using the compliant driving trajectory as the standard trajectory when all anthropomorphic driving trajectories pose safety hazards.

[0079] As can be seen, the above steps establish a safety-first, human-optimized standard trajectory generation logic: first, hard rules ensure a safety baseline; then, real human driving data is integrated to generate multiple human-like driving trajectories, and conflicting elements are intelligently eliminated, retaining compliant and natural driving trajectories. When a human-like driving trajectory that conforms to safety rules exists, the one with the best overall performance is selected. If no human-like driving trajectory that conforms to safety rules exists, the compliant driving trajectory is used as the standard trajectory. This effectively balances safety and practicality, avoiding the rigidity of pure rules and the uncontrollable risks of purely human driving data. It provides high-reliability, scenario-adaptive standard trajectories for model training, significantly improving model training efficiency.

[0080] Optionally, the reward function of the target trajectory prediction model also includes: a basic survival reward function, a constraint violation penalty function, and a round-end reward function. The model training method further includes the following steps:

[0081] Step S13: At each training step, determine the basic survival reward signal through the basic survival reward function;

[0082] Step S14: Determine the end-of-round reward signal by judging whether the initial trajectory prediction model has completed the entire training round and using the end-of-round reward function.

[0083] Step S15: In response to the standard trajectory satisfying the preset safety constraints and preset comfort constraints, and the initial driving trajectory output by the initial trajectory prediction model not satisfying the preset safety constraints or preset comfort constraints, a constraint violation penalty signal is determined through the constraint violation penalty function, and a performance reward signal is determined through the performance reward function.

[0084] Step S16: In response to the initial driving trajectory satisfying the preset safety constraints and preset comfort constraints, a performance reward signal is determined through the performance reward function.

[0085] Step S17: The reward signal is obtained by summing the basic survival reward signal, the round end reward signal, the constraint violation penalty signal, and the performance reward signal.

[0086] Step S18: Train the initial trajectory prediction model using the reward signal to obtain the target trajectory prediction model.

[0087] In this embodiment, the basic survival reward function is used to provide a small positive incentive to the initial trajectory prediction model at each step length, so as to prevent the model from crashing due to frequent negative rewards in the early stage of training.

[0088] The constraint violation penalty function is used to output a fixed negative reward when the trajectory output by the initial trajectory prediction model violates the safety hard threshold or the comfort hard threshold, so as to constrain the bottom-line behavior of the model.

[0089] The end-of-round reward function is used to output a positive or negative reward based on whether the training round was successfully completed.

[0090] The basic survival reward signal is a scalar positive value output by the basic survival reward function at each time step, used to maintain training stability.

[0091] The round-end reward signal is a positive or negative reward signal output by the round-end reward function in each training round, used to indicate whether the task was successful or failed in the current round.

[0092] The preset safety constraints are safety boundaries defined by indicators such as safe collision time and minimum lateral distance, such as TTC≥2.5s and lateral distance≥3.0m, which are not restricted here.

[0093] The preset comfort constraints are defined by the upper limit of comfort based on longitudinal acceleration, lateral acceleration, and jerk threshold.

[0094] The constraint violation penalty signal is the negative reward signal output by the constraint violation penalty function when the model breaks through the above-mentioned preset safety constraints or preset comfort constraints.

[0095] Performance reward signals are continuous positive or negative reward signals generated by comparing the utility functions of safety, comfort, and efficiency with the standard trajectory, provided that no violations have occurred.

[0096] The reward signal is the final feedback signal obtained by summing the basic survival reward signal, the round end reward signal, the constraint violation penalty signal, and the performance reward signal.

[0097] At each training step, determining the basic survival reward signal through the basic survival reward function can be understood as giving the model a small positive incentive at each step, avoiding continuous penalties due to improper behavior in the early stages that could lead to training collapse, while dynamically reducing the reward according to the scenario risk to guide the model to make prudent decisions.

[0098] Determining whether the initial trajectory prediction model has completed the entire training round and using the round-end reward function to determine the round-end reward signal can be understood as providing positive or negative feedback to the model based on whether the current training task has reached its endpoint.

[0099] In response to the standard trajectory satisfying preset safety constraints and preset comfort constraints, and the initial driving trajectory output by the initial trajectory prediction model not satisfying the preset safety constraints or preset comfort constraints, the constraint violation penalty signal is determined through the constraint violation penalty function, and the performance reward signal is determined through the performance reward function. This can be understood as follows: when there is no dangerous driving behavior (such as collision risk or sudden braking) on ​​the standard trajectory, but there is dangerous driving behavior (such as collision risk or sudden braking) on ​​the initial driving trajectory, the constraint violation penalty signal is determined through the constraint violation penalty function, and the performance reward signal is determined through the performance reward function.

[0100] In response to the initial driving trajectory meeting the preset safety constraints and preset comfort constraints, the performance reward signal is determined through the performance reward function. This can be understood as determining the performance reward signal through the performance reward function when there is no dangerous driving behavior (such as collision risk or sudden braking) in the initial driving trajectory.

[0101] By summing the basic survival reward signal, the round end reward signal, the constraint violation penalty signal, and the performance reward signal, the reward signal can be understood as the comprehensive reward signal of the initial trajectory prediction model obtained by comprehensively processing the basic survival reward signal, the round end reward signal, the constraint violation penalty signal, and the performance reward signal.

[0102] Training the initial trajectory prediction model using the reward signal to obtain the target trajectory prediction model can be understood as training the initial trajectory prediction model based on the comprehensive reward signal to finally obtain the target trajectory prediction model.

[0103] As can be seen, through the above steps, the basic survival reward function maintains training stability, the end-of-round reward function guides task completion, the constraint violation penalty function guides the model to not violate safety rules, and the performance reward function incentivizes the model's output trajectory to conform to safety, comfort, and efficiency under the premise of compliance. This avoids the rigidity and sparsity of traditional manual reward functions, enabling the model's output trajectory to autonomously learn efficient, comfortable, and human-like driving strategies while ensuring safety, thereby improving training efficiency.

[0104] Optionally, in step S16, determining the performance reward signal through the performance reward function includes the following steps:

[0105] Step S161: Determine the first comprehensive utility value of the initial driving trajectory and the second comprehensive utility value of the standard trajectory based on the safety utility function, comfort utility function, and efficiency utility function, respectively.

[0106] Step S162: Determine the utility difference between the first comprehensive utility value and the second comprehensive utility value;

[0107] Step S163: Determine the performance reward signal through the performance reward function and the utility difference.

[0108] In this embodiment of the application, the first comprehensive utility value refers to the comprehensive score calculated based on the utility functions of three dimensions—safety, comfort, and efficiency—of the initial driving trajectory in the current scenario, reflecting the comprehensive performance level of the initial driving trajectory relative to the ideal target in terms of safety, smoothness, and efficiency.

[0109] The second comprehensive utility value refers to the comprehensive score calculated based on the utility functions of three dimensions—safety, comfort, and efficiency—of the standard trajectory in the current scenario. It represents the baseline level where safety, comfort, and efficiency are optimally balanced in this scenario.

[0110] Determining the first comprehensive utility value of the initial driving trajectory and the second comprehensive utility value of the standard trajectory based on the safety utility function, comfort utility function, and efficiency utility function can be understood as comparing the performance of the initial driving trajectory and the standard trajectory in three dimensions: safety, comfort, and efficiency. The current trajectory quality of the initial driving trajectory and the expert ideal level of the standard trajectory are then calculated using a weighted fusion algorithm.

[0111] Determining the utility difference between the first comprehensive utility value and the second comprehensive utility value can be understood as quantifying the utility difference between the initial driving trajectory and the standard trajectory. If the utility difference is positive, it means that the performance of the initial driving trajectory exceeds the performance of the standard trajectory. If the utility difference is negative, it means that the performance of the initial driving trajectory does not exceed the performance of the standard trajectory.

[0112] Determining the performance reward signal using the performance reward function and utility difference can be understood as using the Sigmoid function to transform the utility difference into a continuous, symmetrical reward value. If the utility difference is zero, the performance reward signal is zero; if the utility difference is positive, the performance reward signal is a positive incentive; if the utility difference is negative, the performance reward signal is a negative penalty.

[0113] As can be seen, based on the above steps, the overall performance of the initial driving trajectory compared to the standard trajectory is quantified by a multidimensional utility function, and then the utility difference is mapped by the Sigmoid function to generate a continuous and symmetrical performance reward signal. This achieves the refinement of the reward and the ability to adapt to different scenarios, effectively guiding the model to significantly improve the anthropomorphism and generalization ability of the strategy while ensuring safety.

[0114] Optionally, in step S161, determining the first comprehensive utility value of the initial driving trajectory and the second comprehensive utility value of the standard trajectory based on the safety utility function, comfort utility function, and efficiency utility function respectively includes the following steps:

[0115] Step S1611: Determine the first safety utility value of the initial driving trajectory and the second safety utility value of the standard trajectory respectively through the safety utility function;

[0116] Step S1612: Determine the first comfort utility value of the initial driving trajectory and the second comfort utility value of the standard trajectory respectively through the comfort utility function;

[0117] Step S1613: Determine the first efficiency utility value of the initial driving trajectory and the second efficiency utility value of the standard trajectory respectively through the efficiency utility function;

[0118] Step S1614: The first safety utility value, the first comfort utility value, and the first efficiency utility value are weighted by the first preset weight to obtain the first comprehensive utility value. The first preset weight is dynamically adjusted based on the risk categories of various test scenarios.

[0119] Step S1615: The second safety utility value, the second comfort utility value, and the second efficiency utility value are weighted by the second preset weight to obtain the second comprehensive utility value. The second preset weight is dynamically adjusted based on the risk categories of various test scenarios.

[0120] In this embodiment of the application, the first safety utility value is the safety score of the initial driving trajectory calculated based on collision risk (such as safe collision time, minimum collision distance) in the current scenario, reflecting the risk avoidance capability.

[0121] The second safety utility value is the safety score corresponding to the standard trajectory.

[0122] The primary comfort utility value is a score for the smoothness of acceleration and jerk during the initial driving trajectory, reflecting the comfort experience of the occupants.

[0123] The second comfort utility value is the comfort score corresponding to the standard trajectory.

[0124] The first efficiency utility value is the efficiency score for completing the task on the initial driving trajectory, reflecting the responsiveness.

[0125] The second efficiency utility value is the efficiency score corresponding to the standard trajectory.

[0126] The first preset weight is a dynamic weight used to integrate the first safety utility value, the first comfort utility value, and the first efficiency utility value of the initial driving trajectory, and the first preset weight is automatically adjusted according to the risk level of the scenario.

[0127] The second preset weight is a dynamic weight used to fuse the second safety utility value, the second comfort utility value, and the second efficiency utility value of the standard trajectory, and the second preset weight is automatically adjusted according to the risk level of the scenario.

[0128] Determining the first safety utility value of the initial driving trajectory and the second safety utility value of the standard trajectory through the safety utility function can be understood as follows: based on hard safety indicators such as safe collision time and minimum lateral distance, an exponential nonlinear mapping function is used to transform the actual risk level of the initial driving trajectory and the standard trajectory into a safety score between 0 and 1, thereby obtaining the first safety utility value and the second safety utility value.

[0129] Determining the first comfort utility value of the initial driving trajectory and the second comfort utility value of the standard trajectory through the comfort utility function can be understood as follows: by comprehensively considering indicators such as longitudinal and lateral acceleration and jerk, a logarithmic function is used to convert the comfort performance of the initial driving trajectory and the standard trajectory into comfort scores, thereby obtaining the first comfort utility value and the second comfort utility value.

[0130] Determining the first efficiency utility value of the initial driving trajectory and the second efficiency utility value of the standard trajectory through the efficiency utility function can be understood as taking the ratio of task completion time to ideal time as input, and using a linear mapping method to generate efficiency scores for the initial driving trajectory and the standard trajectory respectively, thereby obtaining the first efficiency utility value and the second efficiency utility value.

[0131] The first comprehensive utility value is obtained by weighting the first safety utility value, the first comfort utility value, and the first efficiency utility value with the first preset weight. This can be understood as dynamically allocating the weight ratios of safety, comfort, and efficiency according to the risk level of the current test scenario. The first comprehensive utility value of the initial driving trajectory in the current scenario is generated by weighted summation of the safety score, comfort score, and efficiency score.

[0132] The second comprehensive utility value is obtained by weighting the second safety utility value, the second comfort utility value, and the second efficiency utility value using the second preset weight. This can be understood as using the same scenario adaptation mechanism as the first preset weight to weight and fuse the safety score, comfort score, and efficiency score of the standard trajectory to generate the second comprehensive utility value of the standard trajectory in the current scenario.

[0133] It can be seen that by using a multi-dimensional utility function to quantify driving behavior based on the initial driving trajectory and dynamically adjusting the weights based on the scenario risk level, the evaluation criteria can be adaptively adjusted according to the environment. Furthermore, the same weighting system is used to calculate the comprehensive utility of the initial driving trajectory and the standard trajectory, ensuring consistency in the comparison benchmark and improving model training efficiency and policy generalization ability.

[0134] Optionally, in step S163, the performance reward signal is determined through the performance reward function and the utility difference, including the following steps:

[0135] Step S1631: In response to the utility difference being equal to zero, the performance reward signal is determined to be zero based on the performance reward function;

[0136] Step S1632: In response to a utility difference greater than zero, determine the performance reward signal as a positive reward based on the performance reward function;

[0137] Step S1633: In response to the utility difference being less than zero, determine the performance reward signal as a negative reward based on the performance reward function.

[0138] In this embodiment of the application, the determination that the performance reward signal is zero in response to the utility difference being equal to zero can be understood as follows: when the overall utility of the initial driving trajectory is completely consistent with the overall utility of the standard trajectory, the system determines that the performance of the initial driving trajectory has reached the baseline level and does not provide additional incentives to avoid over-rewarding and causing the strategy to stagnate.

[0139] In response to a utility difference greater than zero, determining a positive performance reward signal based on the performance reward function can be understood as follows: when the overall utility of the initial driving trajectory exceeds the overall utility of the standard trajectory, the system outputs a positive reward through the Sigmoid function, reinforcing the decision-making behavior of the initial driving trajectory and guiding the strategy to continuously optimize.

[0140] In response to a utility difference of less than zero, the performance reward signal determined by the performance reward function is a negative reward. This can be understood as follows: when the overall utility of the initial driving trajectory is inferior to the overall utility of the standard trajectory, the system outputs a negative penalty to explicitly suppress inefficient or dangerous behaviors and drive the strategy to converge toward a better solution.

[0141] It can be seen that by dynamically determining the reward signal through the utility difference, reinforcement learning can autonomously identify superior and inferior behaviors under unsupervised conditions, avoid sparse rewards and misleading signals, significantly improve the policy convergence speed and anthropomorphism level, and ensure that the training process is efficient, stable and has a clear objective.

[0142] Optionally, in step S18, the initial trajectory prediction model is trained using the reward signal to obtain the target trajectory prediction model, including the following steps:

[0143] Step S181: Determine experience units based on reward signals, wherein experience units are used to update the initial driving trajectory;

[0144] Step S182: In response to the completion of multiple training samples or the total number of training steps reaching a preset value, the initial driving trajectory is updated based on the experience unit to obtain the updated driving trajectory.

[0145] Step S183: The initial trajectory prediction model is continuously trained based on the updated driving trajectory to obtain the target trajectory prediction model.

[0146] In this embodiment, the experience unit refers to the quadruple data (S, A, R, S') used in reinforcement learning to record the training process. That is, under a specific environmental state (S), the action (A) performed by the algorithm under test, the obtained comprehensive reward signal (R), and the next state (S') after execution. This experience unit completely records the interaction trajectory of "state-action-reward-next state" and is the original data sample for training the policy network.

[0147] Updating the driving trajectory refers to generating a higher-performance autonomous driving decision trajectory by iteratively optimizing the parameters of the policy network based on experience units using reinforcement learning algorithms.

[0148] Determining experience units based on reward signals can be understood as constructing reusable training samples from the environmental state, the action performed, the reward obtained, and the next state at each step to obtain experience units.

[0149] In response to the completion of multiple training samples or the total number of training steps reaching a preset value, the initial driving trajectory is updated based on the empirical unit. The updated driving trajectory can be understood as follows: after multiple training samples have been completed or the total number of training steps has reached a preset step size, the decision model parameters are optimized using the empirical unit to obtain the updated driving trajectory.

[0150] The process of continuously training the initial trajectory prediction model based on the updated driving trajectory to obtain the target trajectory prediction model can be understood as iteratively simulating and learning the initial trajectory prediction model using the updated driving trajectory, so that the initial trajectory prediction model eventually converges into the target trajectory prediction model.

[0151] As can be seen, firstly, experience units are constructed through reward signals to achieve closed-loop recording of behavior and feedback. Then, the initial driving trajectory is updated based on reinforcement learning algorithm to obtain the updated driving trajectory. Finally, the initial trajectory prediction model is repeatedly trained based on the updated driving trajectory to obtain the target trajectory prediction model, thereby improving the training efficiency and generalization of the trajectory prediction model.

[0152] Figure 3This is a flowchart of a model training method based on a virtual expert system provided in an embodiment of this application, as shown below. Figure 3 As shown, the scene perception information is first extracted, including: running the initial trajectory prediction model in the simulation test platform to obtain complete dynamic information of the test scene in real time. The scene perception information includes: the vehicle's status (position, speed, heading angle, etc.), the status of surrounding traffic participants (position, speed, type of vehicles, pedestrians, non-motorized vehicles, etc.), and high-precision map and road rule information (lane lines, traffic signs, traffic light status, right-of-way, etc.). Then, the extracted scene perception information is simultaneously input into two parallel computing modules to generate the expected behavior under the current test scene: a) Expected behavior generation module based on rule model: This module has a built-in deterministic rule base generated by formalizing traffic regulations, safe driving guidelines (such as ISO 21448 SOTIF), and expert experience. For example: "stop at a red light", "maintain a safe following distance", etc. This module outputs expected behaviors and spatiotemporal constraints based on hard rules that must be followed, reflecting the compliance of driving behavior and ensuring safety. b) Expected Behavior Generation Module Based on Prediction Model: This module uses a deep learning model trained on a large amount of human driving data to predict the most likely behavioral trajectories of multiple rational and safe human drivers in the current scenario, reflecting the comfort, efficiency and anthropomorphism of driving behavior.

[0153] The aforementioned prediction model can be a target-conditional hierarchical spatiotemporal graph neural network. The nodes of the constructed graph structure represent the vehicle and surrounding traffic participants (vehicles, pedestrians, etc.). The edges of the graph structure represent the spatial relationships between traffic participants, and the weights are dynamically calculated by a learnable graph attention network to capture the interaction intensity. The target-conditional hierarchical spatiotemporal graph neural network includes a spatial layer, a temporal layer, and a target injection layer. The spatial layer uses a graph attention network to aggregate the interaction information of nodes within the same time slice. The temporal layer uses gated recurrent units to capture the historical movement trends of individual nodes. The target injection layer encodes the vehicle's future navigation point sequence (from a high-precision map and route planning module) into a target context vector G. Specifically, a lightweight Transformer encoder is used to encode 5-10 future key path points (Frenet coordinates) to obtain a fixed-dimensional target context vector G. In terms of fusion, the target vector G is concatenated with the vehicle node features encoded by the spatiotemporal graph. Finally, the output layer inputs the final graph state, which incorporates the target information, into the decoder. The decoder outputs multiple probabilistic trajectories and their confidence levels for the vehicle node over a future period, as well as discrete behavioral intentions (such as: turn left, follow, stop, change lanes to the right to prepare to enter the ramp).

[0154] The training data for the aforementioned prediction model uses a large-scale human driving dataset, and this training data needs to include trajectory history as well as the vehicle's current final destination or path planning information. A negative log-likelihood loss function is used to optimize the probabilities of multiple trajectories, making the predicted trajectory distribution match real human trajectories. The prediction model is first pre-trained on a public dataset to obtain general prior knowledge of driving behavior. Subsequently, driving data from specific regions can be used for fine-tuning to conform to desired driving styles and cultural habits.

[0155] Next, the expected behaviors generated by the rule-based model and the prediction-based model are fused and arbitrated hierarchically. When the rule-based expectation and the prediction-based expectation conflict, the output of the rule-based model is taken as the highest criterion, that is, safety is given priority. For example, the prediction model may give the expectation of "passing slowly", but the rule model requires "stopping", then the final comprehensive expected behavior is determined to be "stop". If there is no conflict, an optimal expected behavior sequence that takes into account safety, comfort and efficiency is generated.

[0156] For example, the core of hierarchical arbitration is to combine data-driven probabilistic trajectories (from predictive models) with knowledge-driven deterministic constraints (from rule models). The specific algorithm steps are as follows:

[0157] a) Constraint Validation: First, check whether all probabilistic trajectories generated by the prediction model, Trajectory_Set={τi|i=1,…,N}, satisfy all spatiotemporal constraints Constraint_Set={Cj|j=1,…,M} output by the rule model. Eliminate all trajectories that violate any hard safety constraints (e.g., trajectories that would lead to a collision or running a red light).

[0158] b) Optimal Trajectory Selection: If, after constraint verification in step a), there are no remaining trajectories generated by the prediction model, it means that all trajectories of the prediction model are unsafe. In this case, the arbitrator uses the rule-based model as the highest criterion and ultimately outputs the rule-based model behavior. Otherwise, from the set of feasible trajectories generated by the remaining prediction models, the trajectory with the highest comprehensive score is selected as the final expected behavior based on the scoring function S(τ).

[0159] S(τ)=w_comfort J_comfort(τ)+w_efficiency J_efficiency(τ)+w_conformance J_conformance(τ), where J_comfort(τ) is the comfort cost, inversely proportional to the integral of the acceleration (a) and jerk (jerk) of the current trajectory, i.e., J_comfort∝(|a(t)|+|jerk(t)|)dt. J_efficiency(τ) is the efficiency cost, inversely proportional to the time (T) required to complete the current trajectory, i.e., J_efficiency∝T. J_conformance(τ) is the conformity cost, directly proportional to the similarity between the current trajectory τk and the highest confidence trajectory τtop of the prediction model, i.e., J_conformance∝distance(τk, τtop). These are the weighting coefficients for each cost, which can be adjusted according to evaluation preferences (for example, if the evaluation focuses on safety, the weight of comfort can be increased to avoid sudden braking).

[0160] Next, the generated comprehensive expected behavior is defined as the baseline reference trajectory for the current driving scenario. This baseline reference trajectory represents the optimal behavioral paradigm across multiple dimensions, including safety, comfort, and efficiency. Safety is assessed using core indicators such as collision time and minimum lateral distance to evaluate the algorithm's ability to avoid collisions. Comfort is assessed using core indicators such as longitudinal acceleration, longitudinal jerk, lateral acceleration, and lateral jerk to quantify the vehicle's smoothness and stability, reflecting the occupants' subjective experience. Efficiency is assessed using core indicators such as travel time to evaluate the algorithm's agility in completing driving tasks.

[0161] This application unifies safety constraints, comfort requirements, and efficiency goals into a reward signal for reinforcement learning, clearly guiding the algorithm's optimization direction through a positive and negative reward mechanism. The reward signal consists of three parts: Reward Signal = Basic Survival Reward + Constraint Violation Penalty + Round End Reward + Performance Reward, i.e., R = R_baseline + R_penalty + R_final + R_performance. Where R_baseline = R_baseline + 0.01 (1-Risk_Level). A small positive reward is given at each step to address the issue of the trajectory prediction model continuously receiving negative rewards due to insufficient intelligence in the early stages of training, which could lead to training failure. However, the base reward value is reduced in high-risk scenarios to encourage cautious behavior. To ensure absolute safety and relative comfort, strict safety and comfort boundaries are established, with severe penalties for violations: when the expected behavior of the baseline reference trajectory is within the safety / comfort threshold, but the key performance indicators of the tested algorithm (such as collision time, longitudinal acceleration, lateral acceleration, etc.) exceed the safety / comfort threshold, the behavior of the tested algorithm is deemed unacceptable, R_penalty is recorded as -10, R_performance is recorded as 0, and the round is terminated. If the endpoint is reached, the training round ends successfully, and R_final=10 is recorded; if the endpoint is not reached, but the preset maximum time has been reached, the task of the round fails, and R_final=-10 is recorded; if neither of the above conditions is met, the round continues, and R_final=0 is recorded.

[0162] When R_penalty ≠ -10, calculate R_performance. R_performance is determined by a combination of the safety utility function, comfort utility function, and efficiency utility function. Among them, the safety utility function considers the increasing marginal utility in the low-safety region, strengthening the sensitivity to dangerous behaviors; the utility saturation in the high-safety region, avoiding over-conservatism; and the automatic enhancement of the penalty in risky scenarios, using an exponential utility function: f_s(R_safety) = 1 - exp(-λ × R_safety). Where: R_safety: safety score, R_safety=min(TTC_score, Distance_score), where TTC_score=min(1.0, TTC / TTC_safe)#TTC_safe=5.0, Distance_score=min(1.0, Min_Distance / Distance_safe)#Distance_safe=3.0m; λ=3.0+2.0×Risk_Level (dynamic risk sensitivity coefficient); Risk_Level∈[0,1] is provided by the scene recognition module.

[0163] The comfort utility function adopts a logarithmic function: f_c(R_comfort)=log(1+k×R_comfort) / log(1+k), R_comfort=max(0, 1-[w1×|a_lon| / a_lon_max+w2×|j_lon| / j_lon_max+w3×|a_lat| / a_lat_max+w4×|j_lat| / j_lat_max]), where a_lon_max=3.0m / s², j_lon_max=10.0m / s³, a_lat_max=2.0m / s², j_lat_max=8.0m / s³, weights: w1=0.25, w2=0.35, w3=0.20, w4=0.20; k: comfort sensitivity coefficient, generally taken as 8.0.

[0164] The efficiency utility function adopts a linear utility function to encourage continuous optimization: f_e(R_efficiency)=R_efficiency, and the efficiency score R_efficiency=min(1,T_intersection_ideal / T_intersection).

[0165] Next, a generalized weighted utility function is used to perform multi-objective fusion and calculate the comprehensive utility:

[0166] U = [w_s × f_s(R_safety)^p + w_c × f_c(R_comfort)^p + w_e × f_e(R_efficiency)^p]^(1 / p), where p = 1.8. The weights are dynamically adjusted based on scenario risk: High-risk scenario: [w_s, w_c, w_e] = [0.8, 0.15, 0.05], Normal scenario: [w_s, w_c, w_e] = [0.5, 0.3, 0.2], High-efficiency scenario: [w_s, w_c, w_e] = [0.4, 0.3, 0.3]. Risk_Level ≥ 0.8 indicates a high-risk scenario; 0.5 ≤ Risk_Level < 0.8 indicates a normal scenario; Risk_Level < 0.5 indicates a high-efficiency scenario.

[0167] Then, the Sigmoid function is used to calculate positive and negative rewards based on relative performance to the gold standard: R_performance = 2.0 × [2 / (1+exp(-3×ΔU))-1], where ΔU = (U_sut - U_golden) / U_golden. Reward characteristics: ΔU = 0 → R_performance = 0, i.e., reaching the benchmark; ΔU > 0 → R_performance > 0, i.e., exceeding the benchmark, positive reward; ΔU < 0 → R_performance < 0, i.e., falling below the benchmark, negative reward.

[0168] Finally, once all test scenarios have been completed, or the total number of steps has reached a preset value (i.e., the training batch is finished), the series of empirical units (S, A, R, S') containing the comprehensive reward R are input into the reinforcement learning algorithm. The policy network is updated through importance sampling, and training continues. This process is repeated for multiple rounds until the average reward no longer increases significantly in consecutive training rounds and stabilizes at a high level. At this point, the model can be considered converged, and training ends. Where S = (s0, s1, ..., s...). t The current state represents the state of the environment at time step t; A = (a0, a1, ..., a...). t The trajectory prediction model in state s t The action to be taken; R = (r0, r1, ..., r t The environment executes action a in the trajectory prediction model. t The immediate reward given afterward; S'=(s'0, s'1, ..., s' t The environment executes action a in the trajectory prediction model. t The state is then transitioned to.

[0169] This application utilizes virtual experts to dynamically generate multi-dimensional reward signals, automating and intelligentizing the reward function, thereby significantly improving training efficiency and avoiding manual parameter tuning. Hard constraint penalties ensure a safety baseline, ultimately enabling the trajectory prediction model to learn a human-like driving strategy that combines safety, comfort, and efficiency, adapting to different scenarios and thus improving the driving performance of autonomous vehicles.

[0170] Figure 4 This is a structural diagram of a model training device provided in an embodiment of this application, as shown below. Figure 4As shown, the model training device 400 includes: an acquisition module 401, used to acquire multiple sets of training samples, wherein each set of training samples includes: real-time scene perception data, which includes: navigation target, state of any traffic participant, map data, and road rule information; and a training module 402, used to use the real-time scene perception data from the multiple sets of training samples as model input, and to train the initial trajectory prediction model using the standard trajectory corresponding to the real-time scene perception data as ground truth to obtain a target trajectory prediction model, wherein the reward function of the target trajectory prediction model includes: a performance reward function, and a performance reward function. The reward functions include: a safety utility function, a comfort utility function, and an efficiency utility function. The safety utility function is used to evaluate the safety compliance of the driving trajectory; the comfort utility function is used to evaluate the ride comfort of the driving trajectory; and the efficiency utility function is used to evaluate the traffic efficiency of the driving trajectory. The safety utility function is determined by nonlinear mapping of safe collision time, safe lateral distance, and scenario risk level; the comfort utility function is determined by weighted normalization scoring of the vehicle's longitudinal acceleration, lateral acceleration, and jerk; and the efficiency utility function is determined based on the ratio of actual travel time to ideal travel time.

[0171] Optionally, the model training device further includes: a first determining module, used to obtain compliant driving trajectories corresponding to real-time scene perception data based on a hard rule base, wherein the hard rule base includes traffic regulations, safe driving guidelines, and expert experience; to obtain multiple anthropomorphic driving trajectories corresponding to real-time scene perception data based on a historical driving experience database, wherein the historical driving experience database includes various real driving strategy data; to remove driving trajectories that conflict with compliant driving trajectories from the multiple anthropomorphic driving trajectories to obtain a set of driving trajectories; in response to the existence of at least one anthropomorphic driving trajectory in the set of driving trajectories, to use the anthropomorphic driving trajectory whose score meets a preset standard as the standard trajectory; in response to the existence of no anthropomorphic driving trajectory in the set of driving trajectories, to use the compliant driving trajectory as the standard trajectory.

[0172] Optionally, the reward function of the target trajectory prediction model further includes: a basic survival reward function, a constraint violation penalty function, and a round-end reward function. The training module 402 is also used to determine a basic survival reward signal at each training step using the basic survival reward function; determine a round-end reward signal by judging whether the initial trajectory prediction model has completed the entire training round using the round-end reward function; determine a constraint violation penalty signal by the constraint violation penalty function and a performance reward signal by the performance reward function in response to the standard trajectory satisfying the preset safety constraints and preset comfort constraints, and the initial driving trajectory output by the initial trajectory prediction model not satisfying the preset safety constraints or preset comfort constraints; determine a performance reward signal by the performance reward function in response to the initial driving trajectory satisfying the preset safety constraints and preset comfort constraints; obtain a reward signal by summing the basic survival reward signal, the round-end reward signal, the constraint violation penalty signal, and the performance reward signal; and train the initial trajectory prediction model using the reward signal to obtain the target trajectory prediction model.

[0173] Optionally, the model training device further includes: a second determining module, used to determine a first comprehensive utility value of the initial driving trajectory and a second comprehensive utility value of the standard trajectory based on the safety utility function, the comfort utility function, and the efficiency utility function, respectively; determine the utility difference between the first comprehensive utility value and the second comprehensive utility value; and determine a performance reward signal through the performance reward function and the utility difference.

[0174] Optionally, the second determining module is further configured to determine a first safety utility value of the initial driving trajectory and a second safety utility value of the standard trajectory using a safety utility function; determine a first comfort utility value of the initial driving trajectory and a second comfort utility value of the standard trajectory using a comfort utility function; determine a first efficiency utility value of the initial driving trajectory and a second efficiency utility value of the standard trajectory using an efficiency utility function; weight the first safety utility value, the first comfort utility value, and the first efficiency utility value using a first preset weight to obtain a first comprehensive utility value, wherein the first preset weight is dynamically adjusted based on the risk categories of multiple types of test scenarios; and weight the second safety utility value, the second comfort utility value, and the second efficiency utility value using a second preset weight to obtain a second comprehensive utility value, wherein the second preset weight is dynamically adjusted based on the risk categories of multiple types of test scenarios.

[0175] Optionally, the second determining module is further configured to determine the performance reward signal as zero based on the performance reward function in response to a utility difference equal to zero; to determine the performance reward signal as a positive reward based on the performance reward function in response to a utility difference greater than zero; and to determine the performance reward signal as a negative reward based on the performance reward function in response to a utility difference less than zero.

[0176] Optionally, the training module 402 is further configured to determine experience units based on reward signals, wherein the experience units are used to update the initial driving trajectory; in response to the completion of multiple training samples or the total number of training steps reaching a preset value, the initial driving trajectory is updated according to the experience units to obtain the updated driving trajectory; the initial trajectory prediction model is continuously trained based on the updated driving trajectory to obtain the target trajectory prediction model.

[0177] This application also provides an electronic device 500, please refer to... Figure 5 It includes a memory 501 and a processor 502, wherein the memory 501 is used to store computer programs; the processor 502 is used to execute the programs stored in the memory 501 to implement the model training method described in any embodiment of this application.

[0178] This application also provides a vehicle, including: a memory storing an executable program; and a processor for running the executable program, wherein the executable program executes the model training method described above when running on the processor.

[0179] This application also provides a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the model training method described above when run on a computer or processor.

[0180] In this application, "multiple" refers to two or more.

[0181] In this application, unless otherwise expressly defined, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0182] The terms “first,” “second,” “third,” “fourth,” etc., in this application (if present) are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0183] In this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, in this application, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0184] Unless otherwise specified, all steps in this application may be performed sequentially or randomly. For example, if the method includes steps A and B, it means that the method may include steps A and B performed sequentially, or it may include steps B and A performed sequentially. For example, if the method may also include step C, it means that step C may be added to the method in any order. For example, the method may include steps A, B, and C, or it may include steps A, C, and B, or it may include steps C, A, and B, etc.

[0185] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A model training method, characterized in that, include: Multiple sets of training samples are obtained, wherein each set of training samples includes: real-time scene perception data, which includes: navigation target, state of any traffic participant, map data and road rule information; The real-time scene perception data from the multiple sets of training samples is used as model input, and the standard trajectory corresponding to the real-time scene perception data is used as ground truth to train the initial trajectory prediction model to obtain the target trajectory prediction model. The reward function of the target trajectory prediction model includes a performance reward function, which comprises a safety utility function, a comfort utility function, and an efficiency utility function. The safety utility function evaluates the safety compliance of the driving trajectory, the comfort utility function evaluates the ride comfort of the driving trajectory, and the efficiency utility function evaluates the traffic efficiency of the driving trajectory. The safety utility function is determined by nonlinear mapping of safe collision time, safe lateral distance, and scene risk level. The comfort utility function is determined by weighted normalization scoring of the vehicle's longitudinal acceleration, lateral acceleration, and jerk. The efficiency utility function is determined based on the ratio of actual travel time to ideal travel time.

2. The method according to claim 1, characterized in that, Determining the standard trajectory corresponding to the real-time scene perception data includes: The compliant driving trajectory corresponding to the real-time scene perception data is obtained based on a hard rule base, wherein the hard rule base includes: traffic regulations, safe driving guidelines and expert experience; Multiple anthropomorphic driving trajectories corresponding to the real-time scene perception data are obtained from the historical driving experience database, wherein the historical driving experience database includes: various real driving strategy data; The driving trajectories that conflict with the compliant driving trajectories among the multiple anthropomorphic driving trajectories are removed to obtain a set of driving trajectories; In response to the existence of at least one anthropomorphic driving trajectory in the set of driving trajectories, the anthropomorphic driving trajectory whose score meets the preset standard among the at least one anthropomorphic driving trajectory is taken as the standard trajectory; In response to the absence of any anthropomorphic driving trajectory in the set of driving trajectories, the compliant driving trajectory is taken as the standard trajectory.

3. The method according to claim 1, characterized in that, The reward function of the target trajectory prediction model further includes: a basic survival reward function, a constraint violation penalty function, and a round-end reward function; the method further includes: At each training step, the basic survival reward signal is determined using the basic survival reward function; By determining whether the initial trajectory prediction model has completed the entire training round, the round-end reward signal is determined using the round-end reward function. In response to the standard trajectory satisfying the preset safety constraints and preset comfort constraints, and the initial driving trajectory output by the initial trajectory prediction model not satisfying the preset safety constraints or the preset comfort constraints, a constraint violation penalty signal is determined by the constraint violation penalty function, and a performance reward signal is determined by the performance reward function. In response to the initial driving trajectory satisfying the preset safety constraints and the preset comfort constraints, the performance reward signal is determined through the performance reward function; The reward signal is obtained by summing the basic survival reward signal, the round end reward signal, the constraint violation penalty signal, and the performance reward signal. The initial trajectory prediction model is trained using the reward signal to obtain the target trajectory prediction model.

4. The method according to claim 3, characterized in that, Determining the performance reward signal through the performance reward function includes: The first comprehensive utility value of the initial driving trajectory and the second comprehensive utility value of the standard trajectory are determined according to the safety utility function, the comfort utility function and the efficiency utility function, respectively. Determine the utility difference between the first comprehensive utility value and the second comprehensive utility value; The performance reward signal is determined by the performance reward function and the utility difference.

5. The method according to claim 4, characterized in that, The step of determining the first comprehensive utility value of the initial driving trajectory and the second comprehensive utility value of the standard trajectory based on the safety utility function, the comfort utility function, and the efficiency utility function respectively includes: The first safety utility value of the initial driving trajectory and the second safety utility value of the standard trajectory are determined by the safety utility function. The first comfort utility value of the initial driving trajectory and the second comfort utility value of the standard trajectory are determined by the comfort utility function. The efficiency utility function is used to determine the first efficiency utility value of the initial driving trajectory and the second efficiency utility value of the standard trajectory, respectively. The first comprehensive utility value is obtained by weighting the first safety utility value, the first comfort utility value, and the first efficiency utility value with a first preset weight. The first preset weight is dynamically adjusted based on the risk categories of multiple types of test scenarios. The second comprehensive utility value is obtained by weighting the second safety utility value, the second comfort utility value, and the second efficiency utility value with a second preset weight. The second preset weight is dynamically adjusted based on the risk categories of the various test scenarios.

6. The method according to claim 4, characterized in that, The step of determining the performance reward signal using the performance reward function and the utility difference includes: In response to the utility difference being equal to zero, the performance reward signal is determined to be zero according to the performance reward function; In response to the utility difference being greater than zero, the performance reward signal is determined to be a positive reward based on the performance reward function; In response to the utility difference being less than zero, the performance reward signal is determined to be a negative reward based on the performance reward function.

7. The method according to claim 3, characterized in that, The step of training the initial trajectory prediction model using the reward signal to obtain the target trajectory prediction model includes: An experience unit is determined based on the reward signal, wherein the experience unit is used to update the initial driving trajectory; In response to the completion of the multiple sets of training samples or the total number of training steps reaching a preset value, the initial driving trajectory is updated according to the experience unit to obtain the updated driving trajectory; The initial trajectory prediction model is continuously trained based on the updated driving trajectory to obtain the target trajectory prediction model.

8. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the model training method as described in any one of claims 1 to 7.

9. A vehicle, characterized in that, include: Memory, which stores executable programs; A processor for running the executable program, wherein the executable program, when running on the processor, performs the model training method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program is configured to execute the model training method described in any one of claims 1 to 7 when run on a computer or processor.

Images