Training methods and electronic devices for autonomous driving planning models

Abstract

This application discloses a training method and electronic device for an autonomous driving planning model, relating to the field of autonomous driving technology. After training a first planning model based on imitation learning and a second planning model based on reinforcement learning in each training round, the inference results of autonomous driving trajectory planning for each planning model are evaluated, and the model parameters of each planning model are updated based on the evaluation results until the model convergence condition is met. Finally, the model parameters of each planning model in the converged state are fused as the model parameters of the final autonomous driving planning model. This enables the trained autonomous driving planning model to effectively combine the stable human-like prior of imitation learning and the exploratory generalization ability of reinforcement learning, thereby facilitating robust trajectory prediction and safe planning for complex and long-tailed scenarios in real autonomous driving trajectory planning, and improving the generalization ability of the autonomous driving planning model in long-tailed scenarios at intersections.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a training method and electronic device for an autonomous driving planning model. Background Technology

[0002] As a cutting-edge field at the intersection of modern transportation and artificial intelligence, autonomous driving technology aims to achieve safe, efficient, and human-like decision-making and control in complex open traffic environments without human intervention.

[0003] In related technologies, autonomous driving strategies based on imitation learning (IL) are highly dependent on the coverage of expert data. The complexity of traffic interaction at intersections makes it difficult to fully represent "long-tail scenarios" in the training data. Long-tail scenarios usually refer to driving events that occur very infrequently in daily driving, but once they occur, they have extremely high risks and uncertainties. Once the strategy deviates from the expert trajectory in actual driving, it will fall into the distribution offset problem, leading to an explosion of cumulative errors and loss of control over decision-making behavior. Summary of the Invention

[0004] This application provides a training method and electronic device for an autonomous driving planning model, which at least solves the problem of insufficient generalization ability of IL strategies in autonomous driving in long-tail intersection scenarios in related technologies.

[0005] This application provides a method for training an autonomous driving planning model, including: Multiple planning models are trained in parallel using an autonomous driving training set, including a first planning model based on imitation learning and a second planning model based on reinforcement learning. In response to the completion of the current training round, each planning model performs autonomous driving trajectory planning and inference based on the autonomous driving validation set. Based on the reasoning results, the compliance of trajectory planning for each planning model is evaluated, and the evaluation results for each planning model are obtained. Based on the evaluation results of each planning model, the model parameters of each planning model are updated, and the model enters the next training round. In response to the fact that the evaluation results of each planning model in multiple rounds of training meet the preset model convergence conditions, the latest model parameters of each planning model are fused to obtain the model parameters of the autonomous driving planning model.

[0006] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program to implement the training method of any of the above-described autonomous driving planning models.

[0007] According to the training method of the autonomous driving planning model in this application, after training a first planning model based on imitation learning and a second planning model based on reinforcement learning in each training round, the inference results of autonomous driving trajectory planning of each planning model are evaluated, and the model parameters of each planning model are updated based on the evaluation results until the model convergence condition is met. Finally, the model parameters of each planning model in the converged state are fused as the model parameters of the final autonomous driving planning model. This enables the trained autonomous driving planning model to effectively combine the stable human-like prior of imitation learning and the exploratory generalization ability of reinforcement learning. This is beneficial for achieving robust trajectory prediction and safe planning for complex and long-tailed scenarios in real autonomous driving trajectory planning, and improves the generalization ability of the autonomous driving planning model in autonomous driving in long-tailed scenarios at intersections. Attached Figure Description

[0008] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 A flowchart illustrating a training method for an autonomous driving planning model provided in an embodiment of this application; Figure 2 This is a schematic diagram of the training architecture of an autonomous driving planning model in an embodiment of this application; Figure 3 This is a flowchart illustrating an evaluation method for a planning model in an embodiment of this application; Figure 4 This is a schematic diagram of the competitive learning strategy in each training round of an embodiment of this application; Figure 5 A block diagram illustrating the composition of a training device for an autonomous driving planning model provided in this application embodiment; Figure 6 This is a schematic diagram of the composition structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0010] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0011] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0012] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0013] In related technologies, autonomous driving decision-making and planning methods mainly rely on two major artificial intelligence learning paradigms: imitation learning (IL) and reinforcement learning (RL).

[0014] Imitation learning (IL) aims to learn a mapping policy from state space to action space directly from a large number of human drivers' control behaviors under various sensor inputs through supervised learning. The advantages of IL lie in its intuitiveness, relatively low implementation difficulty, stable initial policy performance, and ability to efficiently embed human drivers' experience and safety priors into the model, ensuring the human-likeness and interpretability of vehicle behavior.

[0015] Reinforcement learning (RL) learns an optimal policy through trial and error with the environment to maximize cumulative rewards. The theoretical potential of RL lies in its ability to discover globally optimal control policies that surpass human expert experience through exploration mechanisms, making it particularly suitable for optimizing non-human intuitive performance indicators such as traffic efficiency and energy consumption. Furthermore, RL optimizes from the perspective of long-term rewards, essentially solving the credit allocation problem in complex decision-making.

[0016] However, in real-world applications of autonomous driving, the end-to-end pure imitation learning (IL) planning scheme represents the end-to-end learning paradigm in the field of autonomous driving decision-making. Its core lies in constructing a unified deep neural network that directly receives raw sensor inputs (such as images, point clouds, and map information) and directly outputs control commands (such as steering angle, acceleration, and braking pressure) or local driving trajectories for the vehicle in the future time domain. Its learning process is entirely based on imitation learning, fitting the mapping relationship between the state and actions of human expert drivers recorded in historical data through a maximum likelihood estimation objective function.

[0017] However, the shortcomings of simply imitating IL's planning schemes are significantly amplified in complex scenarios such as intersections, mainly manifested in the following two problems: (1) Distribution shift and lack of generalization: The robustness of the IL strategy depends heavily on the coverage of the expert's training data. For common "long-tail scenarios" in intersection areas (such as sudden lane cutting, abnormal parking, complex traffic light interactions, etc.), due to the lack of sufficient coverage of these extreme and novel states in the expert data, once the model enters the state space outside of these data distributions during testing, its decision output will become unreliable, the strategy cannot generalize to unseen complex interaction patterns, resulting in decision failure. When the strategy deviates from the expert trajectory in actual driving, it will fall into the distribution shift problem.

[0018] (2) Explosive amplification of accumulated error: The IL model only imitates the expert's single-step action at each time step. When the decision-making strategy deviates slightly at a certain time t, the vehicle will be guided to a new state on a non-expert trajectory. Since it is not within the training data distribution, the error of the strategy at time t+1 will be larger and will accumulate exponentially over time. In intersection areas with extremely high safety requirements, this error accumulation will lead to uncontrollable dangerous behaviors, such as the path planning rapidly deviating from the lane when turning, or failing to make timely safe braking when facing obstacles.

[0019] The planning scheme combining IL and RL includes a two-stage learning paradigm. In the first stage, IL is used for pre-training, employing expert data for imitation learning to initialize the policy to a stable state with basic driving capabilities, effectively addressing the low sample efficiency problem when training RL from random initialization. In the second stage, RL is used for fine-tuning. The IL-pretrained policy is used as a starting point and then deployed to a simulation environment or a constrained real-world environment. Reward signals (such as safety rewards and efficiency rewards) are collected through interaction with the environment, and RL algorithms such as PPO and SAC are used to fine-tune the policy.

[0020] While combining IL and RL in planning is a natural approach, it suffers from fundamental flaws in knowledge fusion and optimization efficiency, failing to achieve deep complementarity between the strengths of IL and RL. Specifically, this is reflected in the following aspects: (1) Knowledge Isolation and Conflict in Optimization Objectives: The optimization objectives of the two stages are isolated and asynchronous. The IL stage aims to minimize the gap with expert data (behavioral fit), while the RL stage aims to maximize cumulative rewards (long-term value). In the fine-tuning stage, the policy gradient of RL must overcome the strong prior knowledge and fixed weights formed in the IL stage. This optimization pattern leads to gradient inconsistency; the gradient direction of IL tends to maintain the conservatism of the policy, while the gradient direction of RL tends to be exploratory, and the two will conflict in most cases.

[0021] (2) Policy weight solidification and getting stuck in local optima: The IL stage uses a large amount of high-confidence expert data for training, which makes the policy network weights form a strong, low-variance prior, and the policy weights are prone to solidification. In the RL fine-tuning stage, the policy is difficult to effectively change its weight distribution, and the exploration effect is not significant. The optimization process of RL is often limited to local optima near the IL policy, lacking the ability to conduct in-depth exploration and discover new and better driving behaviors. Especially in complex intersection long-tail scenarios, this "shallow" RL fine-tuning cannot provide sufficient improvement in generalization ability.

[0022] (3) Dependence on online interaction and sample efficiency issues: Although IL provides efficient initialization, the RL fine-tuning stage still requires a large amount of online (or simulated) environment interaction to collect rewards and experience. For high-risk applications such as autonomous driving, the high cost and high risk of real-world exploration cannot be afforded, and existing simulators are difficult to fully simulate the complex interactions and physical details of real intersection scenarios. This makes the sample efficiency and safety of the RL fine-tuning stage of the two-stage method an unresolved problem.

[0023] To effectively improve the insufficient generalization ability of the strategy in the long-tail scenario of autonomous driving at intersections in the above-mentioned related technologies based on imitation learning (IL), and the problems of inefficient knowledge fusion and fixed policy weights in the autonomous driving planning scheme that combines IL and RL, this application provides a training method and electronic device for an autonomous driving planning model.

[0024] This application provides a method for training an autonomous driving planning model. Figure 1 A flowchart illustrating a training method for an autonomous driving planning model provided in this application embodiment is shown below. Figure 1 As shown, the training method for this autonomous driving planning model includes, but is not limited to, the following steps S101 to S105.

[0025] Step S101: Train multiple planning models in parallel based on the autonomous driving training set. The multiple planning models include a first planning model based on imitation learning and a second planning model based on reinforcement learning.

[0026] In autonomous driving scenarios, each planning model is used to perform autonomous driving trajectory planning and reasoning based on the input autonomous driving data, and predict and output autonomous driving trajectory planning data.

[0027] Figure 2 This is a schematic diagram of the training architecture of an autonomous driving planning model in an embodiment of this application, as shown below. Figure 2 As shown, in the training architecture of the autonomous driving planning model, the first planning model Planner (IL) based on imitation learning is a module that learns from expert demonstration data and outputs action sequences or trajectory planning strategies that conform to the expert's behavioral logic. The core objective is to enable the first planning model to replicate the expert's trajectory planning ability without a reward function. Using the first planning model Planner (IL), the autonomous driving trajectory is learned from a large amount of autonomous driving data from human drivers through imitation learning.

[0028] The second planning model Planner (RL) based on reinforcement learning is a module / algorithm responsible for reasoning and searching for optimal action sequences based on known / learned environmental models. Its core function is to simulate the decision-making process in the model space and generate optimal or near-optimal action strategies for the agent. Using the second planning model Planner (RL), the optimal trajectory planning decision is learned through interaction and trial and error with the driving environment via reinforcement learning, aiming to generate an autonomous driving trajectory with the goal of maximizing cumulative rewards.

[0029] An autonomous driving training set is a collection of autonomous driving training data. The autonomous driving training set includes multiple autonomous driving training data sets, which can be pre-collected vehicle driving data. Autonomous driving training data includes, but is not limited to, environmental features, vehicle state features, and driving commands extracted from driving environment data perceived by the vehicle's environmental perception modules (such as surround-view camera sensors, radar, etc.) through perception models.

[0030] Combination Figure 2 As shown, the perceptual model refers to the perceptual backbone, a core structure used for feature extraction in deep learning. In deep learning, the backbone typically refers to the core structure used for feature extraction, such as ResNet and VGG. The perceptual backbone adds more perceptual capabilities, such as modules for processing multimodal inputs or enhancing feature extraction, and can be used to extract multi-level, multi-scale features. For example, perceptual models can employ open-source pre-trained models such as BEVFormer, PETRv2, and BEVDepth.

[0031] Driving environment data includes visual images of the vehicle's surroundings. Environmental features can be high-dimensional, semantically rich image features, which can be used to accurately identify road structures (lane lines, curbs) and the precise location, speed, orientation, and bounding boxes of traffic participants (other vehicles, pedestrians, non-motorized vehicles). Especially at complex intersections, accurate identification of these dynamic elements is a prerequisite for safe decision-making.

[0032] Ego state features can be the kinematic state characteristics of a vehicle, providing accurate real-time status information, including but not limited to the vehicle's position, speed, acceleration, yaw rate, and historical trajectory information. These features are crucial, serving not only as starting constraints for subsequent trajectory planning but also as the basis for trajectory planning strategies to assess the current level of risk and predict their own inertial behavior.

[0033] Driving commands are high-level goals and constraints from the upper-level task planning system, such as "go straight at the next intersection", "turn left into the target lane", or "stop at the designated location". Driving commands elevate the strategy's goal from simply avoiding collisions to efficiently completing specific tasks.

[0034] Combination Figure 2 As shown, in step S101, the environmental features, vehicle state features, and driving commands in the above-mentioned autonomous driving training data are spliced ​​together to form the input features of the planning model. These features are then input into each planning model for autonomous driving trajectory planning training. Each planning model is trained and iterated in parallel. Each training round may include multiple training iterations, such as k training iterations. The value of k can be set according to actual needs. After each training iteration is completed, the model parameters of the planning model are updated iteratively.

[0035] Among them, such as Figure 2 As shown, the training objective of the first planning model is to ensure the stability and human-like conservatism of the trajectory. It is trained through supervised learning. During training, based on the difference between the autonomous driving trajectory output by the first planning model and the expert trajectory, the model parameters are optimized through backpropagation, allowing the first planning model to gradually learn the expert trajectory generation logic. The supervision signal of the first planning model is the expert trajectory. The objective of supervised learning, such as mean squared error (MSE) or maximum likelihood estimation, is used to minimize the deviation between its predicted autonomous driving trajectory and the expert trajectory, ensuring high reliability and smoothness on common and high-frequency driving behaviors.

[0036] For the second-order planning model, the training objective is to maximize the reward to improve the robustness and generalization of the trajectory planning strategy. It is trained using reinforcement learning. During training, the second-order planning model generates autonomous driving trajectories based on the input autonomous driving training data and obtains corresponding rewards through a defined reward function. Based on the accumulated rewards, reinforcement learning algorithms (such as Proximal Policy Optimization) are used to iteratively update the model parameters. The supervision signal for the second-order planning model is the reward function, which aims to quantify the quality of the predicted autonomous driving trajectory, including safety scores, efficiency scores, and comfort scores.

[0037] Step S102: In response to the completion of the current training round, each planning model performs autonomous driving trajectory planning and inference based on the autonomous driving validation set.

[0038] After each training round is completed, autonomous driving trajectory planning and reasoning are performed using the first planning model and the second planning model respectively based on the autonomous driving verification set.

[0039] The autonomous driving validation dataset is a collection of autonomous driving validation data used to validate / evaluate the performance of autonomous driving trajectory planning inference of each planning model after training in the current training round. It includes multiple autonomous driving validation datasets, including but not limited to autonomous driving environmental features, vehicle state features, driving commands, and trajectory planning annotations.

[0040] In step S102 above, the autonomous driving verification data of the autonomous driving verification set is input into each planning model. Each planning model performs autonomous driving trajectory planning inference based on the current model parameters to obtain the inference results of each planning model. The inference results include the trajectory planning data (such as trajectory point sequences) output by each planning model.

[0041] Step S103: Based on the reasoning results, conduct a compliance assessment of trajectory planning for each planning model to obtain the assessment results for each planning model.

[0042] The compliance of each planning model in trajectory planning is evaluated by assessing the inference results of each planning model, such as safety, comfort, and efficiency, and the evaluation results of each planning model are obtained.

[0043] Step S104: Update the model parameters of each planning model according to the evaluation results of each planning model, and proceed to the next training round.

[0044] Combination Figure 2As shown, after each training round is completed, a competitive learning strategy is used to determine the update method of the model parameters of each planning model based on the deviation of the evaluation results of each planning model, and the model parameters of each planning model are updated. After the update is completed, the next training round is entered, and the planning models are trained in parallel using the autonomous driving training set.

[0045] Step S105: In response to the fact that the evaluation results of each planning model in the continuous multi-round training meet the preset model convergence conditions, the latest model parameters of each planning model are fused to obtain the model parameters of the autonomous driving planning model.

[0046] After multiple training rounds, in response to the evaluation results of each planning model obtained in each training round of continuous training satisfying the preset model convergence conditions, the latest model parameters of each planning model are fused to obtain the fused model parameters, which are then used as the model parameters of the final autonomous driving planning model.

[0047] According to the training method of the autonomous driving planning model in the embodiments of this application, after training a first planning model based on imitation learning and a second planning model based on reinforcement learning in each training round, the inference results of autonomous driving trajectory planning of each planning model are evaluated, and the model parameters of each planning model are updated according to the evaluation results until the model convergence condition is met. Finally, the model parameters of each planning model in the convergence state are fused as the model parameters of the final autonomous driving planning model. This enables the trained autonomous driving planning model to effectively combine the stable human-like prior of imitation learning and the exploratory generalization ability of reinforcement learning, which is conducive to achieving robust trajectory prediction and safe planning for complex and long-tailed scenarios in real autonomous driving trajectory planning, and improves the generalization ability of autonomous driving planning model in long-tailed scenarios at intersections.

[0048] In some embodiments, before training multiple planning models in parallel based on an autonomous driving training set, the training method further includes: initializing model parameters for multiple planning models, wherein the initial model parameters for each planning model are the same.

[0049] In other words, each planning model is initialized using the same model parameters, usually through random initialization or pre-training with a small amount of expert data, to ensure that they have the same starting point.

[0050] After initializing the model parameters, the planning models are trained and iterated in parallel. Each planning model independently calculates its own loss and gradient on the same batch of training data to achieve synchronous learning.

[0051] In some embodiments, each planning model adopts the same model structure. For example, an end-to-end autonomous driving planning model such as Transfuser or DiffusionDrive can be selected as the initial planning model.

[0052] In some embodiments, in step S102 above, the reasoning result includes trajectory planning data output by each planning model. The trajectory planning data describes the sequence of trajectory points of the autonomous driving trajectory planned and output by the planning model. The trajectory planning data includes vehicle position data and vehicle speed data for each planned trajectory point (i.e., planned position point).

[0053] Figure 3 This is a flowchart illustrating an evaluation method for a planning model in an embodiment of this application. In some embodiments, such as... Figure 3 As shown, in step S103 above, the compliance assessment of trajectory planning for each planning model is performed based on the reasoning results, and the assessment results for each planning model are obtained, including: Step S301: For each planning model, evaluate the speed deviation cost of the autonomous driving trajectory planned by the planning model based on the vehicle speed data output by the planning model.

[0054] Speed ​​deviation cost is an efficiency metric for evaluating the deviation between the vehicle speed and the target speed at each trajectory point in the autonomous driving trajectory planned by the planning model. It is a penalty score for the deviation between the vehicle speed and the target speed at each trajectory point in the autonomous driving trajectory planned by the planning model. The target speed is the expected speed of the vehicle at the trajectory point, such as road speed limit, preset cruise speed, following speed, curve safety speed, etc.

[0055] In some embodiments, in step S301, based on the deviation between the vehicle speed and the target speed at each trajectory point planned by the planning model, a preset speed deviation cost function is used to evaluate the speed deviation cost of the autonomous driving trajectory planned by the planning model.

[0056] The speed deviation between the vehicle speed and the target speed at each trajectory point planned by the planning model is quantified into an optimizable numerical cost by using a preset speed deviation cost function. The larger the deviation, the higher the cost.

[0057] In some embodiments, the speed deviation cost of the autonomous driving trajectory planned by the planning model is evaluated based on the vehicle speed data output by the planning model. This includes: constructing a speed deviation relationship based on the deviation between the vehicle speed and the target speed at each time point in the trajectory planning time, wherein the speed deviation relationship is used to characterize the difference between the vehicle speed and the target speed at each time point; performing integration processing on the speed deviation relationship to obtain the cumulative speed deviation within the trajectory planning time; and obtaining the speed deviation cost based on the ratio of the cumulative speed deviation to the trajectory planning time, wherein the speed deviation cost characterizes the average cumulative speed deviation per unit time.

[0058] The trajectory planning time refers to the total duration of the autonomous driving trajectory planned and output by the planning model. Within the trajectory planning time, each time point corresponds to a planned trajectory point, and each trajectory point corresponds to a vehicle speed and vehicle position.

[0059] The speed deviation relationship is used to characterize the difference between the vehicle speed and the target speed at each time point. The speed deviation relationship can be the difference between the vehicle speed and the target speed at each time point or the square of the difference.

[0060] In some embodiments, the integration process is a definite integral process, which obtains the cumulative velocity deviation within the trajectory planning time by performing definite integral processing on the velocity deviation relationship within the trajectory planning time.

[0061] In some embodiments, the velocity deviation cost function can be expressed as:

[0062] Where T represents the trajectory planning time of the autonomous driving trajectory output by the planning model, v(t) represents the vehicle speed at time t, and v ref The target speed.

[0063] By using the aforementioned method to evaluate the speed deviation cost, a squared penalty is applied to the speed deviation between the vehicle speed and the target speed at each trajectory point output by the planning model. By accumulating the speed deviation penalties for each trajectory point on the entire autonomous driving trajectory, the speed deviation cost of the autonomous driving trajectory output by the planning model can be evaluated. This helps to reduce the overspeed / underspeed phenomena in the autonomous driving trajectory planned by the planning model.

[0064] Step S302: Based on the vehicle position data output by the planning model, evaluate the collision risk cost of the autonomous driving trajectory planned by the planning model.

[0065] Collision risk cost is a safety indicator that assesses the collision risk between each trajectory point (i.e., the planned position point) and surrounding obstacles in the autonomous driving trajectory planned by the planning model. It is a penalty score for the collision risk between each trajectory point and surrounding obstacles in the autonomous driving trajectory planned by the planning model. Surrounding obstacles can be static obstacles, such as guardrails, buildings, construction zones, and curbs, or dynamic obstacles, such as vehicles, pedestrians, and non-motorized vehicles.

[0066] In some embodiments, in step S302, based on the distance between each planned location point output by the planning model and the location points of surrounding obstacles, a preset collision risk cost function is used to evaluate the collision risk cost of the autonomous driving trajectory planned by the planning model.

[0067] By using a preset collision risk cost function, the collision risk between each planned location point and surrounding obstacles is quantified into an optimizable numerical cost based on the distance between the planned location point and surrounding obstacles according to the planning model. The closer the planned location point is to surrounding obstacles, the higher the risk of the vehicle colliding with surrounding obstacles at that planned location point, and therefore the greater the collision risk cost.

[0068] In some embodiments, the collision risk cost of the autonomous driving trajectory planned by the planning model is evaluated based on the vehicle position data output by the planning model, including: obtaining the distance between each planned position point and each surrounding obstacle position point in the vehicle position data; calculating the collision penalty value of each planned position point based on the distance between each planned position point and each surrounding obstacle position point, wherein the collision penalty value represents the sum of the similarity between the planned position point and each surrounding obstacle position point; and summing the collision penalty values ​​of each planned position point in the autonomous driving trajectory planned by the planning model to obtain the collision risk cost.

[0069] In some embodiments, the distance between each planned location point and each surrounding obstacle location point can be calculated using the Euclidean distance algorithm.

[0070] In some embodiments, the distance between the planned location point and each surrounding obstacle location point is converted into the similarity between the planned location point and each surrounding obstacle location point. The closer the distance, the higher the similarity and the greater the collision risk. The similarity can be used to quantify the collision risk. The higher the similarity, the more collision penalty is obtained.

[0071] Around each planned location point, there may be one or more obstacles. For each planned location point, the similarity between the planned location point and each surrounding obstacle location point can be used as the collision penalty score between the planned location point and each surrounding obstacle location point. Then, the sum of the similarity between the planned location point and each surrounding obstacle location point is the total collision penalty value of the planned location point.

[0072] In some embodiments, a collision penalty value for each planned location point is calculated based on an exponential distance penalty according to the distance between each planned location point and each surrounding obstacle location point, in order to assess the collision risk between each planned location point and each surrounding obstacle location point. The closer the planned location point is to the surrounding obstacle location point, the higher the collision risk, and the penalty increases exponentially. The above similarity can also be represented by an exponential distance penalty. Then, the collision penalty value for each planned location point is the sum of the exponential distance penalties between the planned location point and each surrounding obstacle location point.

[0073] In some embodiments, the collision risk cost function can be expressed as:

[0074] Where, x i This represents the i-th planned position point on the autonomous driving trajectory output by the planning model; o j This represents the location of the j-th surrounding obstacle of the i-th planned location point; β is the attenuation factor used to control distance sensitivity; This represents the square of the distance between the i-th planned location and the j-th location of the surrounding obstacles; The exponential distance penalty is represented by M, where M is the number of surrounding obstacle locations around the i-th planned location, and N is the number of planned locations on the autonomous driving trajectory output by the planning model.

[0075] It should be noted that the squared distance in the above collision risk cost function With area dimension m 2 To make the independent variable of the exponential function exp() dimensionless, the dimension of β is m. -2 The specific value of β can be determined based on the vehicle size, safety buffer distance requirements, and the accuracy of the sensing grid; this application embodiment does not impose any special restrictions on this.

[0076] In some embodiments, the preferred value range for β is 0.1~5m. -2 For example, β is 0.5m -2 or 1m -2 Before actual model training, the optimal value of β can be determined experimentally on the validation set using conventional hyperparameter tuning methods such as grid search or empirical value setting.

[0077] By using the above-mentioned collision risk cost assessment method, an exponential distance penalty is applied to the distance between each planned location point output by the planning model and each surrounding obstacle location point. By accumulating the exponential distance penalties of each planned location point on the entire autonomous driving trajectory, the collision risk cost of the autonomous driving trajectory output by the planning model can be assessed. This is conducive to encouraging the autonomous driving trajectory planned by the planning model to stay as far away from surrounding obstacles as possible.

[0078] Step S303: Based on the vehicle position data output by the planning model, evaluate the lane departure cost of the autonomous driving trajectory planned by the planning model.

[0079] Lane departure cost is a safety indicator that evaluates the degree of deviation between each planned position point and the lane centerline in the autonomous driving trajectory planned by the planning model. It is the penalty score for the lateral offset of each planned position point relative to the lane centerline in the autonomous driving trajectory planned by the planning model. The lateral offset of the planned position point relative to the lane centerline is the vertical distance between the planned position point and the lane centerline.

[0080] In some embodiments, in step S303, based on the vertical distance between each planned location point output by the planning model and the nearest lane centerline, a preset lane departure cost function is used to evaluate the lane departure cost of the autonomous driving trajectory planned by the planning model.

[0081] The lane departure cost function quantifies the degree of deviation between the planned location point and the nearest lane centerline based on the vertical distance between each planned location point and the nearest lane centerline. The larger the vertical distance, the larger the lateral offset, that is, the greater the degree of deviation, and therefore the higher the cost.

[0082] In some embodiments, based on the vehicle position data output by the planning model, the lane departure cost of the autonomous driving trajectory planned by the planning model is evaluated, including: obtaining the vertical distance between each planned position point in the vehicle position data and the nearest lane centerline; and summing the squares of the vertical distances of each planned position point in the autonomous driving trajectory planned by the planning model to obtain the cumulative lane distance deviation. The lane deviation cost is obtained by the ratio of the cumulative lane distance deviation to the number of planned location points. The lane deviation cost represents the average lane distance deviation of each planned location point.

[0083] The lane departure cost is the ratio of the cumulative lane distance deviation to the number of planned location points.

[0084] In some embodiments, the lane departure cost function can be expressed as:

[0085] Where, d i The vertical distance between the i-th planned location point on the autonomous driving trajectory planned by the planning model and the nearest lane centerline is represented by N, where N is the number of planned location points on the autonomous driving trajectory planned by the planning model.

[0086] By using the above-mentioned method for evaluating lane departure costs, a squared penalty is applied to the vertical distance between each planned position point output by the planning model and the nearest lane centerline. By accumulating the vertical distance penalties between each planned position point and the nearest lane centerline throughout the entire autonomous driving trajectory, the lane departure cost of the autonomous driving trajectory output by the planning model is evaluated. This method is more sensitive to large lateral deviations. The larger the vertical distance between the planned position point and the lane centerline, the larger the lateral deviation and the greater the degree of deviation from the lane centerline, thus resulting in a higher cost. This encourages the planning model to plan autonomous driving trajectories that are closer to the lane centerline.

[0087] Step S304: Based on the vehicle position data output by the planning model, evaluate the cost of illegal driving of the autonomous driving trajectory planned by the planning model.

[0088] The cost of illegal driving is a safety indicator that assesses the degree of compliance with traffic rules by each trajectory point (i.e., the planned location point) in the autonomous driving trajectory planned by the planning model. It is a penalty score for illegal driving behavior at each trajectory point in the autonomous driving trajectory planned by the planning model.

[0089] In some embodiments, in step S304, based on the traffic rule compliance status of each planned location point in the autonomous driving trajectory planned by the planning model, a preset violation driving cost function is used to evaluate the violation driving cost of the autonomous driving trajectory planned by the planning model.

[0090] By using a preset cost function for illegal driving, the traffic rule compliance status of each planning location point in the planning model is quantified into an optimizable numerical cost. If illegal driving exists at a planning location point, the cost of illegal driving is high; otherwise, the cost of illegal driving is low.

[0091] In some embodiments, the cost of illegal driving of the autonomous driving trajectory planned by the planning model is evaluated based on the vehicle location data output by the planning model, including: determining whether the vehicle meets the conditions for illegal driving at each planned location point in the vehicle location data; binarizing the determination result of each planned location point to obtain the penalty value for illegal driving at each planned location point; and weighted summing the penalty values ​​for illegal driving at each planned location point in the autonomous driving trajectory planned by the planning model to obtain the cost of illegal driving.

[0092] Among them, the conditions for illegal driving can include whether the vehicle crosses the line, whether it crosses the stop line at a red light, whether it speeds in a speed-limited area, and whether it drives in a prohibited area. If a vehicle meets the conditions for illegal driving at a certain planned location, it means that the vehicle has violated traffic rules at that planned location, such as crossing the line, crossing the stop line at a red light, speeding in a speed-limited area, or driving in a prohibited area.

[0093] The judgment result for each planned location point is binarized. For example, if a vehicle meets the conditions for illegal driving at a planned location point, a higher penalty value is assigned, which is the first value; if a vehicle does not meet the conditions for illegal driving at a planned location point, a lower penalty value is assigned, which is the second value. For example, the first value is 1 and the second value is 0.

[0094] In the weighted summation of the penalty values ​​for violations at each planned location point in the autonomous driving trajectory planned by the planning model, the weight of the penalty value for each planned location point is the weight of each planned location point. The specific weight can be configured according to the actual situation. For example, planned location points near intersections can be given a larger weight.

[0095] In some embodiments, the cost function for illegal driving can be expressed as:

[0096] Where, x i Let N represent the i-th planned location point on the autonomous driving trajectory planned by the planning model, and N be the number of planned location points on the autonomous driving trajectory planned by the planning model. This represents the weight of the i-th planning location point. Indicates conditions for illegal driving. Let be the indicator function; if the i-th planned location point meets the conditions for illegal driving, i.e. If the value is 1, the indicator function will have a value of 1; otherwise, the indicator function will have a value of 0.

[0097] By using the aforementioned method to assess the cost of illegal driving, penalties are imposed on each planned location point output by the planning model. By accumulating the penalties for illegal driving at each planned location point along the entire autonomous driving trajectory, the cost of illegal driving on the autonomous driving trajectory output by the planning model is assessed. The fewer illegal driving behaviors at the planned location points output by the planning model, the lower the cost of illegal driving, which encourages the autonomous driving trajectory output by the planning model to better comply with traffic rules and improve driving safety.

[0098] Step S305: Based on the speed deviation cost, collision risk cost, lane departure cost, and illegal driving cost, obtain the evaluation results of the planning model.

[0099] By comprehensively evaluating the speed deviation cost, collision risk cost, lane departure cost, and illegal driving cost of the autonomous driving trajectory planned by the planning model, the evaluation result of the planning model is obtained. The evaluation result is used to measure the current autonomous driving trajectory planning capability of the planning model.

[0100] In some embodiments, the evaluation result of the planning model is obtained based on the speed deviation cost, collision risk cost, lane departure cost, and illegal driving cost, including: normalizing the speed deviation cost, collision risk cost, lane departure cost, and illegal driving cost respectively; weighting and summing the normalized speed deviation cost, collision risk cost, lane departure cost, and illegal driving cost to obtain the comprehensive planning cost; and determining the evaluation result of the planning model based on the comprehensive planning cost.

[0101] Normalization is a data preprocessing technique that maps data with different dimensions and numerical ranges to a specified uniform interval through mathematical transformation, such as the numerical interval [0, 1].

[0102] In some embodiments, the autonomous driving verification set may include autonomous driving verification data of multiple expert trajectories. Therefore, when each planning model infers, it will also output multiple predicted autonomous driving trajectories. When evaluating the planning model, the compliance assessment of the multiple autonomous driving trajectories planned by the planning model can be performed to obtain the speed deviation cost, collision risk cost, lane departure cost, and illegal driving cost of the multiple autonomous driving trajectories.

[0103] Taking the speed deviation cost of an autonomous driving trajectory as an example, the speed deviation cost of each trajectory can be normalized using the maximum and minimum values ​​among multiple autonomous driving trajectories. The normalization process involves calculating the first difference between the speed deviation cost and the minimum speed deviation cost, and the second difference between the maximum and minimum speed deviation costs. The normalized speed deviation cost is obtained by calculating the ratio of the first difference to the second difference. Similarly, the normalization process for other costs of autonomous driving trajectories can refer to the above method for speed deviation cost normalization, which will not be elaborated upon here.

[0104] Furthermore, when normalizing the aforementioned costs, to avoid the calculation error where the maximum value of each cost (such as the speed deviation cost) equals the minimum value, resulting in a zero normalization denominator (i.e., the second difference), a very small positive constant э can be added to the normalization denominator, for example, э=10. -6That is, the normalized denominator is (second difference + э) to ensure the stability of numerical calculation. Alternatively, a conditional judgment logic can be set so that when the maximum cost is detected to equal the minimum cost, the normalized cost of that term is directly set to a preset default value (e.g., 0). The above mechanism to prevent division by zero errors can effectively ensure the stable operation of the training process.

[0105] In the weighted summation of the normalized speed deviation cost, collision risk cost, lane departure cost, and illegal driving cost, the weights of each cost are configured to be the same, and the sum of the weights of each cost is 1. For example, the weight of each cost is 0.25. Alternatively, different weights can be configured for different costs according to actual needs, and the sum of the weights of each cost is 1.

[0106] In some embodiments, the evaluation result of the autonomous driving trajectory planned by the planning model is the calculated comprehensive planning cost of the autonomous driving trajectory planned by the planning model, or the comprehensive planning reward is calculated based on the comprehensive planning cost of the autonomous driving trajectory planned by the planning model. The evaluation result of the autonomous driving trajectory planned by the planning model is the calculated comprehensive planning reward, and the comprehensive planning reward is the difference between 1 and the comprehensive planning cost.

[0107] In some embodiments, the evaluation result of the planning model is the sum of the evaluation results of multiple autonomous driving trajectories obtained after multiple autonomous driving planning by the planning model. That is, the comprehensive planning cost obtained from the evaluation of each autonomous driving trajectory planned by the planning model is summed to obtain the evaluation result of the planning model, or the comprehensive planning reward obtained from the evaluation of each autonomous driving trajectory planned by the planning model is summed to obtain the evaluation result of the planning model.

[0108] In some embodiments, the evaluation result of the planning model is the average of the evaluation results of multiple autonomous driving trajectories obtained after multiple autonomous driving planning by the planning model. That is, the comprehensive planning cost obtained by evaluating each autonomous driving trajectory planned by the planning model is averaged to obtain the evaluation result of the planning model, or the comprehensive planning reward obtained by evaluating each autonomous driving trajectory planned by the planning model is averaged to obtain the evaluation result of the planning model.

[0109] In some embodiments, updating the model parameters of each planning model based on the evaluation results of each planning model includes: keeping the current model parameters of each planning model unchanged in response to the evaluation result deviation of each planning model being less than a first preset value; updating the current model parameters of each planning model to the model parameters of the planning model with the best evaluation result in response to the evaluation result deviation of each planning model being greater than a second preset value; fusing the current model parameters of each planning model in response to the evaluation result deviation of each planning model being between the first preset value and the second preset value; and updating the current model parameters of each planning model to the fused model parameters.

[0110] The evaluation result deviation of each planning model is the absolute value of the difference between the evaluation result A of the first planning model and the evaluation result B of the second planning model, |AB|. After each training round, through a competitive learning strategy, the evaluation result deviation between the evaluation results of the first planning model and the second planning model is compared with the first preset value M and the second preset value N, triggering an adaptive knowledge transfer mechanism between the first planning model and the second planning model.

[0111] The first preset value M is less than the second preset value N. The first preset value M and the second preset value N are used to distinguish between small perturbations and significant advantages in model performance.

[0112] In some embodiments, the preferred range of the first preset value M is 0.01 to 0.05, used to filter random disturbances; the preferred range of the second preset value N is 0.1 to 0.3, used to identify planning models with significant advantages. In practical applications, the specific values ​​of M and N can be determined by observing the convergence speed and performance oscillation amplitude of the model in the initial training rounds, and fine-tuning within the above preferred ranges.

[0113] Figure 4 This is a schematic diagram of the competitive learning strategy in each training round of an embodiment of this application, as shown below. Figure 4As shown, when the evaluation deviation |AB| of each planning model is less than the first preset value M, i.e., |AB| < M, the difference in inference performance between the first and second planning models is small. This indicates that the cumulative performance difference between the first and second planning models in the past training round is very small. This state usually occurs in the early stage of training, or when the two planning models reach a temporary equilibrium after sufficient fusion. In this case, although the training objectives of the two planning models are different, their behavioral outputs on the current batch of autonomous driving validation sets are very similar. To avoid introducing unnecessary computational overhead and small gradient perturbations, the evaluation results of the two planning models are cleared, i.e., all accumulated evaluation results A and B are cleared, keeping the model parameters of each planning model unchanged, and the next training round is continued to re-evaluate the first and second planning models.

[0114] When the evaluation deviation |AB| of each planning model lies between the first preset value M and the second preset value N, i.e., N>|AB|>M, the difference in inference performance between the first and second planning models is moderate. This indicates that there is a significant difference between the two planning models, but one does not significantly outperform the other. In this state, one planning model exhibits a moderate advantage in a specific scenario or indicator. This advantage is worth learning from by the other planning model, but it is not enough to completely replace the other. Therefore, the current model parameters of each planning model are fused, and the current model parameters of each planning model are updated to the fused model parameters.

[0115] When the deviation of the evaluation results of each planning model |AB| is greater than the second preset value N, that is, |AB|>N, the reasoning performance of the first planning model and the second planning model is significantly different. This indicates that the reasoning performance of one of the first planning model and the second planning model is significantly superior to that of the other. The planning model with the significantly superior reasoning performance is determined as the winner. That is, the planning model with the better evaluation result (the lowest comprehensive planning cost or the highest comprehensive planning reward) is taken as the winner of the competitive learning. The model parameters of the winner are directly used to replace the model parameters of each planning model to update the model parameters of each planning model. That is, the current model parameters of each planning model are updated to the model parameters of the planning model with the best evaluation result, thereby realizing knowledge transfer.

[0116] By updating the model parameters of the first and second planning models using the competitive learning strategy described above after each training round, the model parameter knowledge of the first and second planning models is dynamically transferred during the training phase. This allows the stable knowledge of the first planning model and the exploratory knowledge of the second planning model to learn from and integrate with each other dynamically and adaptively. This achieves real-time and dynamic fusion of the advantages of the two learning paradigms, thereby improving the knowledge isolation problem, breaking the knowledge solidification, and mitigating the sharp conflict between the goals of the two planning models (fitting expert behavior and maximizing long-term rewards), thus achieving a dynamic balance between efficient exploration and stability.

[0117] In some embodiments, the current model parameters of each planning model are fused, including: The current model parameters of each planning model are weighted and summed to obtain the merged model parameters; the weights of the current model parameters of each planning model are determined based on the proportion of the evaluation results of each planning model.

[0118] The weights used in parameter fusion are determined by the evaluation result A of the first planning model and the evaluation result B of the second planning model. In some embodiments, the evaluation result of each planning model is a comprehensive planning reward; the higher the comprehensive planning reward, the better the evaluation result. The parameter fusion weight of the first planning model is A / (A+B), and the parameter fusion weight of the second planning model is B / (A+B).

[0119] In some embodiments, the evaluation result of each planning model is the comprehensive planning cost. The lower the comprehensive planning cost, the better the evaluation result. To ensure that the planning model with a better evaluation result (i.e., a lower comprehensive planning cost) receives a larger parameter fusion weight, the parameter fusion weight of each planning model can be determined through dual transformation or cross-inverse ratio mechanism: 1) Dual transformation mechanism: The normalized comprehensive planning cost is converted into an equivalent comprehensive planning reward (i.e., 1-A and 1-B). Then the parameter fusion weight of the first planning model is (1-A) / (2-AB), and the parameter fusion weight of the second planning model is (1-B) / (2-AB). 2) Cross-inverse ratio mechanism: The cross-ratio is directly used for inverse ratio allocation, that is, the proportion of the comprehensive planning cost of the other party is used as its own weight. The higher the comprehensive planning cost of the other party, the greater the parameter fusion weight allocated to itself. Then the parameter fusion weight of the first planning model is determined to be B / (A+B), and the parameter fusion weight of the second planning model is determined to be A / (A+B).

[0120] Thus, through the parameter fusion weight allocation method of the above-mentioned planning models, the planning model with better evaluation results is given a larger parameter fusion weight to ensure that its "advantageous knowledge" plays a dominant role in the planning model after fusion of model parameters. The fused model parameters will replace the original model parameters of the first and second planning models, realizing knowledge transfer, and will serve as the initial model parameters for the next training round.

[0121] In some embodiments, the first planning model, the second planning model, and the autonomous driving planning model have the same model structure and are all implemented using end-to-end autonomous driving planning models such as Transfuser and DiffusionDrive.

[0122] In some embodiments, the model convergence condition includes: the deviation of the evaluation results of each planning model obtained in consecutive training rounds is less than a first preset value.

[0123] If the evaluation results of each planning model evaluated in multiple consecutive training rounds all meet the model convergence condition that the evaluation result deviation |AB| of each planning model is less than the first preset value M, then the iteration stops, the latest model parameters of the two planning models are fused, and the model parameters of the final autonomous driving planning model are output.

[0124] There is no special limit to the number of consecutive rounds; it can be set according to actual needs, such as 5 consecutive training rounds.

[0125] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0126] This application provides a training device for an autonomous driving planning model. Figure 5 A block diagram of a training device for an autonomous driving planning model provided in this application embodiment is shown below. Figure 5 As shown, the training device includes: Parallel training module 501 is used to train multiple planning models in parallel based on the autonomous driving training set. The multiple planning models include a first planning model based on imitation learning and a second planning model based on reinforcement learning. The inference module 502 is used to perform autonomous driving trajectory planning inference based on the autonomous driving validation set by each planning model in response to the completion of the current training round. Evaluation module 503 is used to evaluate the compliance of trajectory planning for each planning model based on the reasoning results, and obtain the evaluation results for each planning model. The parameter update module 504 is used to update the model parameters of each planning model according to the evaluation results of each planning model, and then proceed to the next training round. The parameter fusion module 505 is used to fuse the latest model parameters of each planning model in response to the evaluation results of each planning model in multiple rounds of training meeting the preset model convergence conditions, so as to obtain the model parameters of the autonomous driving planning model.

[0127] In some embodiments, the inference result includes trajectory planning data output by each planning model, the trajectory planning data including planned vehicle position data and vehicle speed data, and the evaluation module 503 includes: The speed deviation evaluation submodule is used to evaluate the speed deviation cost of the autonomous driving trajectory planned by the planning model for each planning model, based on the vehicle speed data output by the planning model. The collision risk assessment submodule is used to assess the collision risk cost of the autonomous driving trajectory planned by the planning model based on the vehicle position data output by the planning model. The lane departure assessment submodule is used to assess the lane departure cost of the autonomous driving trajectory planned by the planning model based on the vehicle position data output by the planning model. The illegal driving assessment submodule is used to assess the cost of illegal driving on the autonomous driving trajectory planned by the planning model based on the vehicle position data output by the planning model. The comprehensive evaluation submodule is used to obtain the evaluation results of the planning model based on the speed deviation cost, collision risk cost, lane departure cost, and illegal driving cost.

[0128] In some embodiments, the comprehensive evaluation submodule is used to: normalize the speed deviation cost, collision risk cost, lane departure cost, and the illegal driving cost respectively; weight and sum the normalized speed deviation cost, collision risk cost, lane departure cost, and illegal driving cost to obtain the comprehensive planning cost; and determine the evaluation result of the planning model based on the comprehensive planning cost.

[0129] In some embodiments, the speed deviation evaluation submodule is used to: construct a speed deviation relationship based on the deviation between the vehicle speed and the target speed at each time point in the vehicle speed data during the trajectory planning time, the speed deviation relationship being used to characterize the difference between the vehicle speed and the target speed at each time point; perform integration processing on the speed deviation relationship to obtain the cumulative speed deviation during the trajectory planning time; and obtain the speed deviation cost based on the ratio of the cumulative speed deviation to the trajectory planning time, the speed deviation cost being used to characterize the average cumulative speed deviation per unit time.

[0130] In some embodiments, the collision risk assessment submodule is used to: obtain the distance between each planned location point and each surrounding obstacle location point in the vehicle location data; calculate the collision penalty value of each planned location point based on the distance between each planned location point and each surrounding obstacle location point, wherein the collision penalty value represents the sum of the similarity between the planned location point and each surrounding obstacle location point; and accumulate the collision penalty values ​​of each planned location point in the autonomous driving trajectory planned by the planning model to obtain the collision risk cost.

[0131] In some embodiments, the lane departure assessment submodule is used to: obtain the vertical distance between each planned location point in the vehicle location data and the nearest lane centerline; accumulate the squared values ​​of the vertical distances of each planned location point in the autonomous driving trajectory planned by the planning model to obtain the cumulative lane distance deviation; and obtain the lane departure cost based on the ratio of the cumulative lane distance deviation to the number of planned location points, wherein the lane departure cost represents the average lane distance deviation of each planned location point.

[0132] In some embodiments, the violation driving assessment submodule is used to: determine whether the vehicle meets the violation driving conditions at each planned location point in the vehicle location data; binarize the judgment result of each planned location point to obtain the violation driving penalty value of each planned location point; and perform weighted summation of the violation driving penalty values ​​of each planned location point in the autonomous driving trajectory planned by the planning model to obtain the violation driving cost.

[0133] In some embodiments, the parameter update module 504 is configured to: keep the current model parameters of each planning model unchanged in response to the evaluation result deviation of each planning model being less than a first preset value; and update the current model parameters of each planning model to the model parameters of the planning model with the best evaluation result in response to the evaluation result deviation of each planning model being greater than a second preset value.

[0134] In some embodiments, the parameter update module 504 is used to: in response to the deviation of the evaluation results of each planning model being between a first preset value and a second preset value, perform parameter fusion on the current model parameters of each planning model; and update the current model parameters of each planning model to the fused model parameters.

[0135] In some embodiments, the parameter fusion module 505 is used to perform a weighted summation of the current model parameters of each planning model to obtain the fused model parameters; wherein, the weighting of the current model parameters of each planning model is determined according to the proportion of the evaluation results of each planning model.

[0136] In some embodiments, the first planning model, the second planning model, and the autonomous driving planning model have the same model structure.

[0137] In some embodiments, the model convergence condition includes: the deviation of the evaluation results of each planning model obtained in consecutive training rounds is less than a first preset value.

[0138] In the embodiments of this application, the description of the features corresponding to the training device for the autonomous driving planning model can be found in the relevant description of the embodiments corresponding to the training method for the autonomous driving planning model described above, and will not be repeated here.

[0139] Figure 6 This is a schematic diagram of the composition structure of an electronic device provided in an embodiment of this application. An embodiment of this application also provides an electronic device, such as... Figure 6 As shown, the electronic device includes a memory 10 and a processor 20. The memory 10 stores a computer program, and the processor 20 is configured to run the computer program to perform the steps in any of the above-described embodiments of the training method for an autonomous driving planning model.

[0140] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described embodiments of the training method for autonomous driving planning models at runtime.

[0141] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0142] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in the training method embodiments of any of the above-described autonomous driving planning models.

[0143] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described embodiments of the training method for an autonomous driving planning model.

[0144] Any of the components, modules, units, parts, methods, and operations described herein can be implemented using software, firmware, hardware (e.g., fixed logic circuitry), manual processing, or any combination thereof. Alternatively or additionally, any functionality described herein can be performed at least in part by one or more hardware logic components, such as, but not limited to, a central processing unit (CPU), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), an application-specific standard product (ASSP), a system-on-a-chip (SoC), a complex programmable logic device (CPLD), a microprocessor (MCU), etc. The terms "system," "computing device," or "apparatus" as used herein encompass various means, devices, and machines for processing data, including, for example, one or more programmable processors, computers, SoCs, or combinations thereof. The apparatus may also include code that creates an execution environment for the computer program in question, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or one or more combinations thereof. The aforementioned computer program (also known as a program, software, software application, app, script, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and can be deployed in any form, including as a standalone program or as a module, component, subroutine, object, or other unit suitable for a computing environment.

[0145] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0146] The training method for an autonomous driving planning model provided in this application has been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only intended to help understand the method and core ideas of this application. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A training method for an autonomous driving planning model, characterized in that, include: Multiple planning models are trained in parallel using an autonomous driving training set, including a first planning model based on imitation learning and a second planning model based on reinforcement learning. In response to the completion of the current training round, each planning model performs autonomous driving trajectory planning and inference based on the autonomous driving validation set. Based on the reasoning results, the compliance of trajectory planning for each planning model is evaluated, and the evaluation results for each planning model are obtained. Based on the evaluation results of each planning model, the model parameters of each planning model are updated, and the model enters the next training round. In response to the fact that the evaluation results of each planning model in multiple rounds of training meet the preset model convergence conditions, the latest model parameters of each planning model are fused to obtain the model parameters of the autonomous driving planning model.

2. The training method according to claim 1, characterized in that, The inference results include trajectory planning data output by each planning model. The trajectory planning data includes planned vehicle position data and vehicle speed data. The compliance assessment of each planning model based on the inference results yields the assessment results for each planning model, including: For each planning model, based on the vehicle speed data output by the planning model, evaluate the speed deviation cost of the autonomous driving trajectory planned by the planning model; Based on the vehicle position data output by the planning model, assess the collision risk cost of the autonomous driving trajectory planned by the planning model; Based on the vehicle position data output by the planning model, evaluate the lane departure cost of the autonomous driving trajectory planned by the planning model; Based on the vehicle location data output by the planning model, evaluate the cost of illegal driving for the autonomous driving trajectory planned by the planning model; The evaluation result of the planning model is obtained based on the speed deviation cost, the collision risk cost, the lane departure cost, and the illegal driving cost.

3. The training method according to claim 2, characterized in that, The evaluation result of the planning model, obtained based on the speed deviation cost, the collision risk cost, the lane departure cost, and the violation driving cost, includes: The speed deviation cost, the collision risk cost, the lane departure cost, and the illegal driving cost are all normalized respectively. The normalized speed deviation cost, collision risk cost, lane departure cost, and illegal driving cost are weighted and summed to obtain the comprehensive planning cost; The evaluation result of the planning model is determined based on the comprehensive planning cost.

4. The training method according to claim 2, characterized in that, The step of evaluating the speed deviation cost of the autonomous driving trajectory planned by the planning model based on the vehicle speed data output by the planning model includes: Based on the deviation between the vehicle speed and the target speed at each time point within the trajectory planning time, a speed deviation relationship is constructed, which is used to characterize the difference between the vehicle speed and the target speed at each time point. The velocity deviation relationship is integrated to obtain the cumulative velocity deviation within the trajectory planning time. The speed deviation cost is obtained by the ratio of the cumulative speed deviation to the trajectory planning time, and the speed deviation cost represents the average cumulative speed deviation per unit time.

5. The training method according to claim 2, characterized in that, The step of evaluating the collision risk cost of the autonomous driving trajectory planned by the planning model based on the vehicle position data output by the planning model includes: Obtain the distance between each planned location point and each location point of surrounding obstacles in the vehicle location data; Based on the distance between each planned location point and each surrounding obstacle location point, a collision penalty value is calculated for each planned location point; the collision penalty value represents the sum of the similarities between the planned location point and each surrounding obstacle location point. The collision penalty value of each planned location point in the autonomous driving trajectory planned by the planning model is accumulated to obtain the collision risk cost.

6. The training method according to claim 2, characterized in that, The step of evaluating the lane departure cost of the autonomous driving trajectory planned by the planning model based on the vehicle position data output by the planning model includes: Obtain the vertical distance between each planned location point in the vehicle location data and the nearest lane centerline; The cumulative lane distance deviation is obtained by summing the squares of the vertical distances to each planned location point in the autonomous driving trajectory planned by the planning model. The lane deviation cost is obtained based on the ratio of the cumulative lane distance deviation to the number of planned location points. The lane deviation cost represents the average lane distance deviation for each planned location point.

7. The training method according to claim 2, characterized in that, The step of evaluating the cost of illegal driving based on the vehicle position data output by the planning model includes: Determine whether the vehicle in the vehicle location data meets the conditions for illegal driving at each planned location point; The judgment result of each planned location point is binarized to obtain the penalty value for illegal driving at each planned location point; The cost of violating driving is obtained by weighted summing of the penalty values ​​for each planned location point in the autonomous driving trajectory planned by the planning model.

8. The training method according to claim 1, characterized in that, The step of updating the model parameters of each planning model based on the evaluation results of each planning model includes: In response to the fact that the deviation of the evaluation results of each planning model is less than the first preset value, the current model parameters of each planning model are kept unchanged; In response to the deviation of the evaluation results of each planning model being greater than the second preset value, the current model parameters of each planning model are updated to the model parameters of the planning model with the best evaluation results; In response to the fact that the deviation of the evaluation results of each planning model is between the first preset value and the second preset value, the current model parameters of each planning model are fused. Update the current model parameters of each planning model to the merged model parameters.

9. The training method according to claim 1 or 8, characterized in that, The first planning model, the second planning model and the autonomous driving planning model have the same model structure. The model convergence condition includes: the evaluation result deviation of each planning model obtained in multiple consecutive training rounds is less than a first preset value. The step of fusing the current model parameters of each planning model includes: weighting and summing the current model parameters of each planning model to obtain the fused model parameters; wherein the weighting of the current model parameters of each planning model is determined based on the proportion of the evaluation results of each planning model.

10. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the training method for the autonomous driving planning model as described in any one of claims 1 to 9.

Images