Automatic driving method, device, electronic device, storage medium and program product

Abstract

This disclosure presents an autonomous driving method, device, electronic device, storage medium, and program product. It guides the predicted driving trajectory of a target vehicle based on anchor point trajectories and uses this predicted driving trajectory as a basic driving strategy to provide a fundamental reference trajectory for the vehicle's autonomous driving. This addresses the distribution offset problem of behavior clones in closed-loop deployment in related technologies. Furthermore, it predicts multiple second driving parameters (including but not limited to predicted bird's-eye view state, predicted operation actions, and action reward signals) using offline driving parameters and first driving parameters. These second driving parameters serve as candidate driving strategies. A preset regularization penalty algorithm, a critic algorithm, and the basic driving strategy (predicted driving trajectory) are then used to determine the quality of the candidate driving strategies, ultimately identifying a safer driving strategy. This achieves offline reinforcement learning, solving problems such as high computational cost, low training efficiency, and insufficient policy robustness inherent in reinforcement learning.

Description

Technical Field

[0001] This disclosure relates to intelligent driving technology, and in particular to an autonomous driving method, apparatus, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] In the field of autonomous driving, Behavior Cloning (BC) and Reinforcement Learning (RL) are common application technologies. Behavior Cloning, as a mainstream imitation learning strategy, focuses on using log data generated during human driving to construct a mapping relationship from sensor inputs (such as camera images and LiDAR point clouds) to the vehicle's future trajectory or control commands (such as steering wheel angle and accelerator / brake commands). By training the model with a large amount of labeled data, it learns the driving decision-making patterns of human drivers in various scenarios, thus mimicking human driving behavior in practical applications. For example, the unified framework proposed by UniAD integrates perception, prediction, and planning, simplifying the architecture of autonomous driving systems. Reinforcement learning, on the other hand, involves an agent interacting with the environment and optimizing its own strategy based on reward signals from environmental feedback to maximize long-term rewards. In autonomous driving scenarios, reinforcement learning can be used to learn the vehicle's decision-making strategies in complex traffic environments, adjusting control commands through continuous trial and error to adapt the vehicle to different road conditions.

[0003] However, both behavioral cloning and reinforcement learning have significant drawbacks in related technologies: behavioral cloning faces severe distribution shift problems when deployed in a closed loop, and models that perform well in open-loop training accumulate prediction errors in actual driving, causing the vehicle to enter a state different from the training data distribution. Moreover, the training data is difficult to cover all extreme or rare scenarios, and the vehicle's ability to recover from errors is weak. In addition, behavioral cloning lacks a reasoning mechanism for long-term consequences and cannot effectively weigh decisions in complex driving scenarios. Reinforcement learning relies on interactive environments or simulators, which consumes large amounts of computational resources, has low training efficiency, and the generated driving strategies are prone to deviating from natural human driving behavior. Offline reinforcement learning is limited by the data coverage, making it difficult to effectively handle distribution shift problems, unable to fully utilize offline data to learn robust driving strategies, and unable to meet the stringent safety and reliability requirements of autonomous driving. Summary of the Invention

[0004] In order to solve the technical problems in the related art, embodiments of this disclosure provide an autonomous driving method, apparatus, electronic device, computer-readable storage medium, and computer program product.

[0005] According to a first aspect of the present disclosure, an autonomous driving method is provided, the method comprising:

[0006] In response to receiving the first driving parameters of the target vehicle, an anchor point trajectory is determined, and a predicted driving trajectory is determined based on the anchor point trajectory and the first driving parameters of the target vehicle. The first driving parameters of the target vehicle include the first-moment bird's-eye view state of the target vehicle, the first-moment driving speed, and the historical driving trajectory relative to the first moment. The anchor point trajectory is a set of coordinates of reference trajectory points estimated according to a preset estimation method.

[0007] Based on preset offline driving parameters, the first driving parameters of the target vehicle, and the predicted driving trajectory, multiple second driving parameters of the target vehicle are determined. Each second driving parameter includes the predicted bird's-eye view state of the target vehicle, the predicted operation action, and the action reward signal. The predicted bird's-eye view state is the bird's-eye view state of the target vehicle relative to the first time step predicted by the world model. The preset offline driving parameters include the bird's-eye view state of the target vehicle relative to the first time step.

[0008] Based on a preset regularization penalty algorithm, a preset number of critic algorithms, and the predicted driving trajectory, the target driving parameters are determined from multiple second driving parameters of the target vehicle.

[0009] The target driving parameters are used to determine the target operation action of the target vehicle, and the target vehicle is controlled to perform the target operation action in order to achieve the autonomous driving of the target vehicle.

[0010] As an optional embodiment, in response to receiving first driving parameters of the target vehicle, determining the anchor point trajectory, and determining the predicted driving trajectory based on the anchor point trajectory and the first driving parameters of the target vehicle, wherein the first driving parameters of the target vehicle include the target vehicle's first-moment bird's-eye view state, first-moment driving speed, and historical driving trajectory relative to the first moment, and the anchor point trajectory is a set of coordinates of reference trajectory points estimated according to a preset estimation method, including:

[0011] Based on the first driving parameters of the target vehicle, the trajectory points of the uniform speed driving path of the target vehicle are estimated according to a preset estimation method to obtain the anchor point trajectory, which is a set of coordinates of reference trajectory points.

[0012] The first future trajectory of the target vehicle is determined based on the first driving parameters of the target vehicle. The first future trajectory is a set of predicted driving trajectory points within a preset time period, with the first moment of the target vehicle as the starting point.

[0013] Determine the offset parameter between the first future trajectory point and the anchor point trajectory;

[0014] The predicted driving trajectory is obtained based on the offset parameters and the anchor point trajectory;

[0015] The first driving parameters of the target vehicle include the first-moment bird's-eye view state of the target vehicle, the first-moment driving speed, and the historical driving trajectory relative to the first moment. The anchor point trajectory is a set of coordinates of reference trajectory points estimated according to a preset estimation method.

[0016] As an optional embodiment, the anchor point trajectory is obtained by estimating the trajectory points of the uniform speed driving path of the target vehicle according to a preset estimation method based on the first driving parameters of the target vehicle. The anchor point trajectory is a set of coordinates of reference trajectory points, including:

[0017] Obtain the target vehicle's current speed and position coordinates at the first moment;

[0018] Based on the current position and the current speed, the trajectory points of the uniform speed driving path of the target vehicle are calculated at preset time intervals within a preset time range, and the coordinate set of the trajectory points of the uniform speed driving path is the anchor point trajectory.

[0019] As an optional embodiment, obtaining the predicted driving trajectory based on the offset parameter and the anchor point trajectory includes:

[0020] The predicted lateral and longitudinal offsets are obtained based on the offset parameters.

[0021] The predicted lateral offset is superimposed on the lateral coordinates of the anchor point trajectory, and the predicted longitudinal offset is superimposed on the longitudinal coordinates of the anchor point trajectory. The trajectory formed by the superimposed coordinates of the anchor point trajectory is the predicted driving trajectory.

[0022] As an optional embodiment, the step of determining multiple second driving parameters of the target vehicle based on preset offline driving parameters, the first driving parameters of the target vehicle, and the predicted driving trajectory, wherein any one of the second driving parameters includes the predicted bird's-eye view state of the target vehicle, the predicted operation action, and the action reward signal, the predicted bird's-eye view state being the next bird's-eye view state of the target vehicle relative to the first time step predicted by the world model, and the preset offline driving parameters including the previous bird's-eye view state of the target vehicle relative to the first time step, including:

[0023] Extract historical bird's-eye view features of the preset offline driving parameters;

[0024] The historical bird's-eye view features are transformed into semantic feature vectors with spatial location encoding;

[0025] The first predicted action is converted into an action-encoded feature vector.

[0026] The action encoding feature vector and the semantic feature vector are concatenated to output spatiotemporal correlation features;

[0027] Based on the spatiotemporal correlation features, the cross-attention between the action encoding feature vector and the semantic feature vector is calculated to simulate the temporal and spatial changes of the target vehicle and obtain multiple second driving parameters of the target vehicle.

[0028] Wherein, any one of the second driving parameters includes the predicted bird's-eye view state of the target vehicle, the predicted operation action and the action reward signal, the predicted bird's-eye view state is the next bird's-eye view state of the target vehicle relative to the first time, as predicted by the world model, and the preset offline driving parameter includes the previous bird's-eye view state of the target vehicle relative to the first time.

[0029] As an optional embodiment, the step of converting the historical bird's-eye view features into a semantic feature vector with spatial location encoding includes:

[0030] The historical bird's-eye view features are compressed using the attention mechanism of the pre-trained world model to obtain the semantic feature vector with spatial location encoding.

[0031] As an optional embodiment, the step of determining the target driving parameters from multiple second driving parameters of the target vehicle based on a preset regularization penalty algorithm, a preset number of critic algorithms, and the predicted driving trajectory includes:

[0032] The value function of any second driving parameter is calculated by an algorithm with a preset number of critics. The value function represents the expected cumulative reward of the predicted bird's-eye view state and the predicted driving action in the second driving parameter.

[0033] For all the second driving parameters, sort them according to the size of the value function to determine the second driving parameter with the smallest value function;

[0034] The loss function between the second driving parameter that minimizes the value function and the predicted driving trajectory is calculated based on a preset regularization penalty algorithm.

[0035] In response to the loss function being less than a preset loss threshold, the second driving parameter that minimizes the loss function is determined as the target driving parameter.

[0036] As an optional embodiment, the step of determining the target operation action of the target vehicle based on the target driving parameters and controlling the target vehicle to execute the target operation action to achieve autonomous driving of the target vehicle includes:

[0037] The predicted driving action in the target driving parameters is determined as the target driving action of the target vehicle;

[0038] Change the first predicted operation action to the target operation action;

[0039] The steering angle and acceleration of the target vehicle are controlled by the target operation actions to achieve autonomous driving of the target vehicle.

[0040] According to a second aspect of the present disclosure, an autonomous driving device is provided, the device comprising:

[0041] The first determining module is used to determine the anchor point trajectory in response to receiving the first driving parameters of the target vehicle, and to determine the predicted driving trajectory based on the anchor point trajectory and the first driving parameters of the target vehicle. The first driving parameters of the target vehicle include the first-moment bird's-eye view state of the target vehicle, the first-moment driving speed, and the historical driving trajectory relative to the first moment. The anchor point trajectory is a set of coordinates of reference trajectory points estimated according to a preset estimation method.

[0042] The second determining module is used to determine multiple second driving parameters of the target vehicle based on preset offline driving parameters, the first driving parameters of the target vehicle, and the predicted driving trajectory. Each second driving parameter includes the predicted bird's-eye view state of the target vehicle, the predicted operation action, and the action reward signal. The predicted bird's-eye view state is the bird's-eye view state of the target vehicle at the next moment relative to the first moment, as predicted by the world model. The preset offline driving parameters include the bird's-eye view state of the target vehicle at the previous moment relative to the first moment.

[0043] The third determining module is used to determine the target driving parameters from multiple second driving parameters of the target vehicle based on a preset regularization penalty algorithm, a preset number of critic algorithms and the predicted driving trajectory.

[0044] The fourth determining module is used to determine the target operation action of the target vehicle based on the target driving parameters, and control the target vehicle to perform the target operation action in order to achieve the autonomous driving of the target vehicle.

[0045] As an optional embodiment, the first determining module includes:

[0046] Anchor point determination unit is used to estimate the trajectory points of the uniform speed driving path of the target vehicle according to the first driving parameters of the target vehicle and according to a preset estimation method, so as to obtain the anchor point trajectory, wherein the anchor point trajectory is a set of coordinates of reference trajectory points.

[0047] The future trajectory determination unit is used to determine the first future trajectory of the target vehicle based on the first driving parameters of the target vehicle. The first future trajectory is a set of predicted driving trajectory points within a preset time period, with the first moment of the target vehicle as the starting point.

[0048] An offset parameter determination unit is used to determine the offset parameter between the first future trajectory point and the anchor point trajectory.

[0049] A driving trajectory prediction unit is used to obtain a predicted driving trajectory based on the offset parameters and the anchor point trajectory.

[0050] An operation prediction unit is used to determine a first predicted operation action of the target vehicle based on the predicted driving trajectory.

[0051] The first driving parameters of the target vehicle include the first-moment bird's-eye view state of the target vehicle, the first-moment driving speed, and the historical driving trajectory relative to the first moment. The anchor point trajectory is a set of coordinates of reference trajectory points estimated according to a preset estimation method.

[0052] As an optional embodiment, the anchor point determination unit is further configured to:

[0053] Obtain the target vehicle's current speed and position coordinates at the first moment;

[0054] Based on the current position and the current speed, the trajectory points of the uniform speed driving path of the target vehicle are calculated at preset time intervals within a preset time range, and the coordinate set of the trajectory points of the uniform speed driving path is the anchor point trajectory.

[0055] As an optional embodiment, the driving trajectory prediction unit is further used for:

[0056] The predicted lateral and longitudinal offsets are obtained based on the offset parameters.

[0057] The predicted lateral offset is superimposed on the lateral coordinates of the anchor point trajectory, and the predicted longitudinal offset is superimposed on the longitudinal coordinates of the anchor point trajectory. The trajectory formed by the superimposed coordinates of the anchor point trajectory is the predicted driving trajectory.

[0058] As an optional embodiment, the second determining module includes:

[0059] The feature extraction unit is used to extract historical bird's-eye view features of the preset offline driving parameters;

[0060] The first conversion unit is used to convert the historical bird's-eye view features into semantic feature vectors with spatial location encoding;

[0061] The second conversion unit is used to convert the predicted driving trajectory into a motion coding feature vector.

[0062] The feature concatenation unit is used to perform feature concatenation processing on the action encoding feature vector and the semantic feature vector, and output spatiotemporal correlation features;

[0063] A cross-attention calculation unit is used to calculate the cross-attention between the action encoding feature vector and the semantic feature vector based on the spatiotemporal correlation features, so as to simulate the temporal and spatial changes of the target vehicle and obtain multiple second driving parameters of the target vehicle.

[0064] Wherein, any one of the second driving parameters includes the predicted bird's-eye view state of the target vehicle, the predicted operation action and the action reward signal, the predicted bird's-eye view state is the next bird's-eye view state of the target vehicle relative to the first time, as predicted by the world model, and the preset offline driving parameter includes the previous bird's-eye view state of the target vehicle relative to the first time.

[0065] As an optional embodiment, the first conversion unit is further used for

[0066] The historical bird's-eye view features are compressed using the attention mechanism of the pre-trained world model to obtain the semantic feature vector with spatial location encoding.

[0067] As an optional embodiment, the third determining module includes:

[0068] The value function calculation unit is used to calculate the value function of any second driving parameter by using a preset number of critic algorithms. The value function represents the expected cumulative reward of the predicted bird's-eye view state and the predicted operation action in the second driving parameter.

[0069] The value function sorting unit is used to sort all the second driving parameters according to the size of the value function in order to determine the second driving parameter with the smallest value function.

[0070] The loss function determination unit is used to calculate the loss function between the second driving parameter that minimizes the value function and the predicted driving trajectory based on a preset regularization penalty algorithm.

[0071] The driving parameter prediction unit is used to determine the second driving parameter that minimizes the loss function as the target driving parameter in response to the loss function being less than a preset loss threshold.

[0072] As an optional embodiment, the fourth determining module includes:

[0073] A determining unit is used to determine the predicted operation action in the target driving parameters as the target operation action of the target vehicle.

[0074] The control unit is used to control the steering angle and acceleration values ​​of the target vehicle through the target operation actions, so as to realize the autonomous driving of the target vehicle.

[0075] According to a third aspect of the present disclosure, an electronic device is provided, comprising:

[0076] Memory, used to store computer program products;

[0077] A processor is configured to execute a computer program product stored in the memory, and when the computer program product is executed, to implement the method described in the first aspect above.

[0078] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the method described in the first aspect.

[0079] According to a fifth aspect of the present disclosure, a computer program product is provided, including computer program instructions that, when executed by a processor, implement the method described in the first aspect above.

[0080] In this embodiment, by determining anchor point trajectories and then guiding the predicted driving trajectory of the target vehicle based on these trajectories, and using the predicted driving trajectory as a basic driving strategy, this method of determining the basic driving strategy based on anchor point trajectories provides a basic reference trajectory for the autonomous driving of the target vehicle. This solves the problem of distribution offset of behavior clones in closed-loop deployment in related technologies. Furthermore, by predicting multiple second driving parameters (including but not limited to predicted bird's-eye view state, predicted operation actions, and action reward signals) using offline driving parameters and first driving parameters, these multiple second driving parameters are used as candidate driving strategies. Then, by using a preset regularization penalty algorithm, a preset number of critic algorithms, and the basic driving strategy (predicted driving trajectory), the quality or feasibility of the candidate driving strategies is determined, thereby determining a safer driving strategy. This achieves offline reinforcement learning, solving the problems of high computational cost, low training efficiency, and insufficient policy robustness in reinforcement learning, and enabling more accurate trajectory prediction.

[0081] The technical solutions of this disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0082] The accompanying drawings, which form part of this specification, illustrate embodiments of this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0083] This disclosure will become clearer with reference to the accompanying drawings and the following detailed description, wherein:

[0084] Figure 1 This is an architectural block diagram of an embodiment of an autonomous driving method disclosed herein.

[0085] Figure 2 This is one of the flowcharts for an embodiment of the autonomous driving method disclosed herein.

[0086] Figure 3 This is a second flowchart of an embodiment of the autonomous driving method of this disclosure.

[0087] Figure 4 This is a structural block diagram of an autonomous driving device according to an embodiment of the present disclosure.

[0088] Figure 5 This is a structural block diagram of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0089] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this disclosure.

[0090] To enable those skilled in the art to clearly and accurately understand the technical solutions of the embodiments of this disclosure, the following will first be combined with... Figure 1 The architecture diagram shown is used for illustration. The technical solution of this disclosure embodiment can be composed of three parts: anchor-guided behavior cloning, reinforcement learning (or offline reinforcement learning), and a world model. First, the driving parameters of the target vehicle (or self-vehicle) are obtained. The driving parameters may include historical driving parameters (such as the driving parameters of the self-vehicle at the previous moment) and first driving parameters (the driving parameters of the self-vehicle at the first moment). The driving parameters can be derived from historical bird's-eye view and current bird's-eye view. Some or all of the bird's-eye view features extracted from the historical bird's-eye view and current bird's-eye view are used as driving parameters, including but not limited to: ① the position coordinates (x, y), heading angle, and speed of the self-vehicle and surrounding vehicles in the past N frames; ② the self-vehicle control actions (such as the steering angle and acceleration at historical moments); ③ environmental perception information (such as lane topology, traffic light status, obstacle bounding boxes, drivable area segmentation masks), etc.

[0091] The anchor point guidance behavior cloning component uses the anchor point trajectory as a reference frame to clone the guided behavior, resulting in a relative predicted driving trajectory as the base driving strategy. For example, assuming the target vehicle is traveling at 60 km / h, the anchor point trajectory will generate a straight path at the current speed, while the actual trajectory may be curved. The anchor point guidance behavior cloning component predicts the offset of this curved trajectory relative to the straight path. This addresses the stability issue when speed changes.

[0092] The world model, a simulation engine, is a core technological framework in the field of artificial intelligence used to simulate environmental dynamics and predict future states. It constructs internal system representations through training with multimodal data. In this embodiment, it can simulate the driving situation of the target vehicle (or self-driving vehicle) based on its driving parameters, predicting future driving trajectories, driving states, environmental perceptions, etc. (e.g., a preset time period relative to the first moment), and setting reward predictions to determine whether the future driving trajectory, state, and other outcomes are positive (goal-oriented) or harmful (unsafe). In this way, the pre-trained world model replaces the real environment for policy planning and evaluation, eliminating the need for the agent to interact with the real environment, effectively solving the problems of high computational cost, low training efficiency, and insufficient policy robustness inherent in reinforcement learning.

[0093] However, the anchor-guided behavior cloning part still suffers from distribution offset issues. That is, during actual driving of the target vehicle, various new scenarios are encountered, and the provided predicted driving trajectory may not be able to handle some complex new scenarios, leading to distribution offset problems. To address this, this embodiment combines anchor-guided behavior cloning, a world model, and reinforcement learning. The reinforcement learning algorithm can employ a dual-delay mechanism (e.g., using a preset number of critic algorithms (e.g., two critic networks) to mutually correct the strategy, plus behavior cloning regularization (i.e., a preset regularization penalty algorithm) to prevent deviation from the dataset, and then adjusting and optimizing the basic driving strategy based on the final results of the critic algorithms, without excessively deviating from the basic driving strategy, thus obtaining a safe and reliable final driving strategy.

[0094] The following is combined with Figure 2 The technical solutions of the embodiments of this disclosure will now be described in detail. Figure 2 This is one of the flowcharts for an autonomous driving method according to an embodiment of the method disclosed herein. Figure 2 As shown, an autonomous driving method may include the following steps:

[0095] Step 201: In response to receiving the first driving parameters of the target vehicle, determine the anchor point trajectory, and determine the predicted driving trajectory based on the anchor point trajectory and the first driving parameters of the target vehicle.

[0096] The target vehicle is the autonomous vehicle whose driving trajectory (or driving strategy) needs to be acquired. The first driving parameter is the target vehicle's driving parameters at the first moment. Specifically, the first driving parameter includes the target vehicle's first-moment bird's-eye view state, its first-moment driving speed, and its historical driving trajectory relative to the first moment. The anchor point trajectory is the set of coordinates of reference trajectory points estimated using a preset estimation method. The first driving parameter can come from images captured by the target vehicle's cameras, point cloud data collected by LiDAR, the target vehicle's speed sensors, etc.

[0097] In some embodiments, the trajectory points of the uniform speed driving path of the target vehicle can be estimated according to the first driving parameters of the target vehicle and a preset estimation method to obtain the anchor point trajectory. The anchor point trajectory is a set of coordinates of the reference trajectory points. The first future trajectory of the target vehicle can be determined according to the first driving parameters of the target vehicle. The first future trajectory is a set of predicted driving trajectory points within a preset time period with the first moment of the target vehicle as the time starting point. Then, the offset parameter between the first future trajectory point and the anchor point trajectory is determined. Finally, the predicted driving trajectory is obtained according to the offset parameter and the anchor point trajectory.

[0098] Furthermore, based on the target vehicle's first driving parameters, the trajectory points of the target vehicle's uniform speed driving path are estimated according to a preset estimation method to obtain the anchor point trajectory. This can be achieved by: obtaining the target vehicle's current speed and position coordinates at the first moment; and, based on the current position and speed, calculating the trajectory points of the target vehicle's uniform speed driving path within a preset time range at preset time intervals. The set of coordinates of the trajectory points of the uniform speed driving path is the anchor point trajectory. For example, a coordinate system is established with the autonomous vehicle (target vehicle) itself as the center. Assuming the autonomous vehicle (target vehicle) maintains a constant current speed, a planned time range (i.e., a preset time range) and a time interval (preset time interval) are set. For example, the planned time range (i.e., the preset time range) is the set of trajectory points at 6 time points (preset time intervals) within 3 seconds and 2Hz from the first moment. These trajectory points can be represented by coordinates (x, y), and the set of coordinates of these trajectory points is determined as the anchor point trajectory.

[0099] Furthermore, the predicted driving trajectory can be obtained based on the offset parameters and the anchor point trajectory. This can be achieved by obtaining the predicted lateral and longitudinal offsets based on the offset parameters, then superimposing the predicted lateral offsets onto the lateral coordinates of the anchor point trajectory, and superimposing the predicted longitudinal offsets onto the longitudinal coordinates of the anchor point trajectory. The trajectory formed by the coordinates of the superimposed anchor point trajectory is the predicted driving trajectory.

[0100] For example, firstly, a virtual reference trajectory is generated by driving at a constant speed with the target vehicle as the center, for example, a path point is collected every 0.5 seconds within 10 seconds, and the coordinates of these path points are collected to obtain the anchor point trajectory. Then, the offset between the real future trajectory (i.e., the first future trajectory; for example, the recorded real trajectory of the target vehicle, the one for the next 5 seconds is used as the real future trajectory) and the anchor point trajectory is calculated (for example, a lateral offset of 2 meters, i.e., a lateral offset of 2 meters). Finally, the predicted driving trajectory is the offset (lateral offset of 2 meters) plus the anchor point trajectory.

[0101] Among them, step 201 can be achieved through Figure 1 The anchor-point guided behavior cloning component combines anchor-point trajectories with behavior clones to construct a mapping relationship from target vehicle sensor inputs (e.g., camera images, LiDAR point clouds) to the target vehicle's future trajectory or control commands (e.g., steering wheel angle, accelerator / brake commands), thus achieving anchor-point trajectory guided behavior cloning. In this way, the anchor-point trajectory dynamically changes at different speeds, avoiding learning biases in a fixed coordinate system. Learning relative offsets (rather than absolute positions) reduces prediction jitter in high-speed scenarios, solves the distribution offset problem of behavior clones in closed-loop deployment, and enables more accurate trajectory prediction, improving the generalization capability of this embodiment in different driving scenarios.

[0102] Step 202: Based on the preset offline driving parameters, the first driving parameters of the target vehicle, and the predicted driving trajectory, determine multiple second driving parameters of the target vehicle. Each second driving parameter includes the predicted bird's-eye view state of the target vehicle, the predicted operation action, and the action reward signal. The predicted bird's-eye view state is the bird's-eye view state of the target vehicle relative to the first moment predicted by the world model. The preset offline driving parameters include the bird's-eye view state of the target vehicle relative to the first moment.

[0103] In this embodiment of the disclosure, the second driving parameter is the predicted driving parameter of the target vehicle at a second moment, and the second moment is after the first moment.

[0104] World models are a core technical framework in the field of artificial intelligence used to simulate environmental dynamics and predict future states. They are constructed through training with multimodal data to build internal representations of the system. In this embodiment, the world model is used to simulate and predict a bird's-eye view of the future moment based on preset offline driving parameters, the first driving parameters of the target vehicle, and the predicted driving trajectory. The reward label corresponding to the bird's-eye view of the future moment, i.e., the second driving parameter, is then used in subsequent reinforcement training. This supports offline reinforcement learning and solves the problems of high computational cost, low training efficiency, and insufficient policy robustness in reinforcement learning. By using the learned world model to support offline reinforcement learning, policy learning can be performed without interaction with the environment, enhancing learning efficiency and stability. In closed-loop simulation benchmark tests, compared with a baseline using only behavioral clones, the safety and robustness of autonomous driving are effectively improved.

[0105] In this embodiment, the world model is essentially a neural network that can learn to predict changes in the target vehicle's state and the rewards obtained from offline data (preset offline driving parameters). As an example, step 202 can be implemented using the world model as follows:

[0106] Step A: Extract historical bird's-eye view features with preset offline driving parameters, and convert the historical bird's-eye view features into semantic feature vectors with spatial location encoding.

[0107] In this step, the world model spatially encodes features of preset offline driving parameters (e.g., historical bird's-eye view or the target vehicle's state in the bird's-eye view relative to the previous moment) to generate a compressed feature representation that preserves spatial semantic relationships. As one implementation, the attention mechanism of the pre-trained world model is used to compress the feature dimensions of the historical bird's-eye view features, resulting in the semantic feature vector with spatial location encoding. Specifically, convolutional neural networks can be used to fuse features such as historical bird's-eye view features, and a convolutional block attention mechanism can be used to extract key spatial features, generating a low-dimensional feature representation containing object position and semantic information. Position encoding is added to mark spatial coordinate relationships during the feature combination stage, thereby obtaining a semantic feature vector with spatial location encoding.

[0108] Step B: Convert the predicted driving trajectory into an action coding feature vector, perform feature concatenation processing on the action coding feature vector and the semantic feature vector, and output the spatiotemporal correlation feature.

[0109] In this step, the spatiotemporal fusion of features is achieved. The predicted driving trajectory is transformed into an action-encoded feature vector and projected onto the latent space. This vector is then concatenated with the flattened semantic feature vector. A Transformer encoder with a multi-head attention mechanism is used to process the combined features, and position encoding is added to preserve the spatial coordinates of the BEV features, resulting in spatiotemporally related features. The Transformer encoder can process the combined features by, for example, calculating the interaction attention weights between the semantic feature vector and the action-encoded features, then establishing the spatiotemporal feature association between the two through a multi-head attention mechanism, and outputting a feature vector that fuses the spatiotemporally changing features—this is the spatiotemporally related feature.

[0110] Step C: Based on the spatiotemporal correlation features, calculate the cross-attention between the action encoding feature vector and the semantic feature vector to simulate the temporal and spatial changes of the target vehicle and obtain multiple second driving parameters of the target vehicle.

[0111] The spatiotemporal features of the Transformer's output are reshaped, and spatial resolution is improved through convolutional blocks with CBAM (Convolutional Block Attention Module) attention. This predicts the future BEV state (e.g., the position, speed (and sometimes orientation), road structure, and traffic signal status of all objects around the target vehicle), and uses a lightweight reward head composed of convolutional blocks and a spatial attention mechanism to estimate the reward based on the predicted future BEV state, determining whether the outcome is positive (goal-oriented) or harmful (unsafe). Specifically, the lightweight reward head composed of convolutional blocks and a spatial attention mechanism is trained with supervised reward labels, supporting value learning under a world model. For example, the predicted future BEV state is input into the lightweight reward prediction network (the lightweight reward head composed of convolutional blocks and a spatial attention mechanism). This network extracts features through convolutional layers and spatial attention layers, outputting a scalar reward value reflecting driving safety risk and route progress completion. Then, in this way, the pre-trained world model replaces the real environment for policy planning and evaluation, eliminating the need for the agent to interact with the real environment. This effectively solves the problems of high computational cost, low training efficiency, and insufficient policy robustness in reinforcement learning.

[0112] Step 203: Based on a preset regularization penalty algorithm, a preset number of critic algorithms, and a predicted driving trajectory, determine the target driving parameters from multiple second driving parameters of the target vehicle.

[0113] As one implementation, step 203 can be implemented as follows: Calculate the value function of any second driving parameter using a preset number of critic algorithms. The value function represents the expected cumulative reward of the predicted bird's-eye view state and the predicted driving action in the second driving parameter. Sort all second driving parameters according to the size of the value function to determine the second driving parameter with the smallest value function. Calculate the loss function between the second driving parameter with the smallest value function and the predicted driving trajectory based on a preset regularization penalty algorithm. In response to the loss function being less than a preset loss threshold, determine the second driving parameter with the smallest value function as the target driving parameter.

[0114] In this step, it can be done by Figure 1 Offline reinforcement learning is implemented. An offline reinforcement learning method is used in conjunction with a pre-trained single-step world model to simulate environmental dynamics. The output second driving parameters optimize the predicted driving trajectory (basic driving policy) output by the anchor-guided behavior clone. Specifically, the Twin Delayed DDPG (TD3) algorithm is used, combined with a behavior clone regularizer (TD3+BC) to ensure the policy conforms to the offline data distribution. A preset number of critic algorithms (e.g., two critic networks) are set. The critic networks are trained using the second driving parameters (current BEV state and reward) predicted by the world model based on the current BEV state (second driving parameters) and candidate vehicle actions (predicted driving trajectory), and through a specific loss calculation method. In some embodiments, the update optimization of the anchor-guided behavior clone can be delayed relative to the update of the critic networks (preset number of critic algorithms). This maximizes the value of its own actions while approximating the behavior policy generated from the offline data, effectively solving the distribution offset problem of behavior clones in closed-loop deployment and improving the overall performance of the policy.

[0115] In this embodiment of the disclosure, a preset number of critic algorithms can estimate in parallel, simultaneously receiving the current BEV state (second driving parameter) and candidate vehicle actions (predicted driving trajectory), each calculating and outputting a value function, and finally selecting the current BEV state (second driving parameter) with the smallest value function, and then calculating the loss function of the current BEV state (second driving parameter) with the smallest value function and the candidate vehicle actions (predicted driving trajectory).

[0116] Step 204: Determine the target operation action of the target vehicle based on the target driving parameters, and control the target vehicle to execute the target operation action in order to achieve autonomous driving of the target vehicle.

[0117] Specifically, the predicted operation actions in the target driving parameters are determined as the target operation actions of the target vehicle. The steering angle and acceleration values ​​of the target vehicle are controlled through the target operation actions to achieve autonomous driving of the target vehicle.

[0118] In summary, by determining anchor point trajectories and then guiding the predicted driving trajectory of the target vehicle based on these trajectories, and using the predicted driving trajectory as the basic driving strategy, this method of determining the basic driving strategy using anchor point trajectories provides a basic reference trajectory for the autonomous driving of the target vehicle. This solves the distribution offset problem of behavior clones in closed-loop deployment in related technologies. Furthermore, by predicting multiple second driving parameters (including but not limited to predicting bird's-eye view state, predicting operation actions, and action reward signals) using offline driving parameters and first driving parameters, these multiple second driving parameters are used as candidate driving strategies. Then, by using a preset regularization penalty algorithm, a preset number of critic algorithms, and the basic driving strategy (predicted driving trajectory), the merits or feasibility of the candidate driving strategies are determined, thereby identifying a safer driving strategy. This achieves offline reinforcement learning, solving the problems of high computational cost, low training efficiency, and insufficient policy robustness in reinforcement learning, and enabling more accurate trajectory prediction.

[0119] In order to enable those skilled in the art to fully understand the technical solution disclosed herein, and in combination with Figure 3 The illustrated embodiments further describe the technical solution. Figure 3 This is a second flowchart of an embodiment of the autonomous driving method of this disclosure.

[0120] like Figure 3 As shown, firstly, the target vehicle (or self-driving vehicle) can obtain initial driving parameters through sensors such as cameras, lidar, and other sensors. Historical driving parameters can also be obtained through databases or storage servers. Taking the multi-view images in the figure as an example, they can represent the target vehicle's current or historical trajectory, current or historical bird's-eye view, etc. As the initial input, the VAD (Vectorized Autonomous Driving, an end-to-end autonomous driving paradigm that improves planning efficiency and safety by modeling the driving scene as a fully vectorized representation) bird's-eye view encoder can extract bird's-eye view features or bird's-eye view states (including but not limited to the position, speed (and sometimes orientation) of all objects around the timber vehicle, road structure, traffic signal status, etc.).

[0121] Then, the input is processed by the behavior cloning module. The behavior cloning module guides behavior cloning by using anchor points. Based on the dynamic perception anchor points (anchor point trajectories), it predicts and determines the prediction deviation (offset parameters), thereby obtaining the predicted driving trajectory as the basic driving strategy (including but not limited to acceleration, deceleration, turning left, turning right, driving speed, etc.).

[0122] Simultaneously, the data is input into a world model. This world model is based on the bird's-eye view features (current or first moment) and historical bird's-eye view features (which can come from historical logs). It also performs behavior embedding (which can be the operation behavior in the basic driving strategy based on the current bird's-eye view state, or the operation behavior in historical log data) to obtain fused features of bird's-eye view features and action encoding features for temporal and spatial simulation. Then, it is encoded by an encoder (e.g., a Transformer encoder with a multi-head attention mechanism) and the spatial resolution is improved by convolutional blocks with CBAM attention to predict the future (second moment or the next moment relative to the first moment) bird's-eye view state (i.e., the future bird's-eye view in the figure). Then, a lightweight reward head composed of convolutional blocks and spatial attention mechanism is used to estimate the reward based on the predicted future (second moment or the next moment relative to the first moment) bird's-eye view state (i.e., the future bird's-eye view in the figure), and the result is judged as positive (safe) or harmful (unsafe). The second driving parameters are output (including but not limited to the predicted bird's-eye view state, predicted operation action, and action reward signal).

[0123] Secondly, the output of the world model (the second driving parameter) is input into the offline reinforcement learning module, and the basic driving policy is also input into the offline reinforcement learning module for initialization. The reinforcement learning (offline reinforcement learning) uses the Twin Delayed DDPG (TD3) algorithm (e.g., the critic algorithm) to calculate the value function (Q-value) of the second driving parameter, and combines the output of the world model (the second driving parameter) with a regularizer (pre-defined regularization penalty algorithm) for anchor trajectory guidance behavior cloning to determine the target driving parameter. Then, the target driving parameter is used to optimize the basic driving policy, and finally, the final driving policy is output.

[0124] Correspondingly, this disclosure also provides device embodiments corresponding to the foregoing method embodiments. The device embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. Figure 4 This is a structural block diagram of an autonomous driving device according to one embodiment of the present disclosure. Figure 4 The aforementioned autonomous driving device may include:

[0125] The first determining module 401 is used to determine the anchor point trajectory in response to receiving the first driving parameters of the target vehicle, and to determine the predicted driving trajectory based on the anchor point trajectory and the first driving parameters of the target vehicle. The first driving parameters of the target vehicle include the first-moment bird's-eye view state of the target vehicle, the first-moment driving speed, and the historical driving trajectory relative to the first moment. The anchor point trajectory is a set of coordinates of reference trajectory points estimated according to a preset estimation method.

[0126] The second determining module 402 is used to determine multiple second driving parameters of the target vehicle based on preset offline driving parameters, the first driving parameters of the target vehicle, and the predicted driving trajectory. Each second driving parameter includes the predicted bird's-eye view state of the target vehicle, the predicted operation action, and the action reward signal. The predicted bird's-eye view state is the bird's-eye view state of the target vehicle at the next moment relative to the first moment, as predicted by the world model. The preset offline driving parameters include the bird's-eye view state of the target vehicle at the previous moment relative to the first moment.

[0127] The third determining module 403 is used to determine the target driving parameters from multiple second driving parameters of the target vehicle based on a preset regularization penalty algorithm, a preset number of critic algorithms and the predicted driving trajectory.

[0128] The fourth determining module 404 is used to determine the target operation action of the target vehicle based on the target driving parameters, and control the target vehicle to perform the target operation action in order to achieve the autonomous driving of the target vehicle.

[0129] In this embodiment, by determining anchor point trajectories and then guiding the predicted driving trajectory of the target vehicle based on these trajectories, and using the predicted driving trajectory as a basic driving strategy, this method of determining the basic driving strategy based on anchor point trajectories provides a basic reference trajectory for the autonomous driving of the target vehicle. This solves the problem of distribution offset of behavior clones in closed-loop deployment in related technologies. Furthermore, by predicting multiple second driving parameters (including but not limited to predicted bird's-eye view state, predicted operation actions, and action reward signals) using offline driving parameters and first driving parameters, these multiple second driving parameters are used as candidate driving strategies. Then, by using a preset regularization penalty algorithm, a preset number of critic algorithms, and the basic driving strategy (predicted driving trajectory), the quality or feasibility of the candidate driving strategies is determined, thereby determining a safer driving strategy. This achieves offline reinforcement learning, solving the problems of high computational cost, low training efficiency, and insufficient policy robustness in reinforcement learning, and enabling more accurate trajectory prediction.

[0130] As an optional embodiment, the first determining module includes:

[0131] Anchor point determination unit is used to estimate the trajectory points of the uniform speed driving path of the target vehicle according to the first driving parameters of the target vehicle and according to a preset estimation method, so as to obtain the anchor point trajectory, wherein the anchor point trajectory is a set of coordinates of reference trajectory points.

[0132] The future trajectory determination unit is used to determine the first future trajectory of the target vehicle based on the first driving parameters of the target vehicle. The first future trajectory is a set of predicted driving trajectory points within a preset time period, with the first moment of the target vehicle as the starting point.

[0133] An offset parameter determination unit is used to determine the offset parameter between the first future trajectory point and the anchor point trajectory.

[0134] A driving trajectory prediction unit is used to obtain a predicted driving trajectory based on the offset parameters and the anchor point trajectory.

[0135] An operation prediction unit is used to determine a first predicted operation action of the target vehicle based on the predicted driving trajectory.

[0136] The first driving parameters of the target vehicle include the first-moment bird's-eye view state of the target vehicle, the first-moment driving speed, and the historical driving trajectory relative to the first moment. The anchor point trajectory is a set of coordinates of reference trajectory points estimated according to a preset estimation method.

[0137] As an optional embodiment, the anchor point determination unit is further configured to:

[0138] Obtain the target vehicle's current speed and position coordinates at the first moment;

[0139] Based on the current position and the current speed, the trajectory points of the uniform speed driving path of the target vehicle are calculated at preset time intervals within a preset time range, and the coordinate set of the trajectory points of the uniform speed driving path is the anchor point trajectory.

[0140] As an optional embodiment, the driving trajectory prediction unit is further used for:

[0141] The predicted lateral and longitudinal offsets are obtained based on the offset parameters.

[0142] The predicted lateral offset is superimposed on the lateral coordinates of the anchor point trajectory, and the predicted longitudinal offset is superimposed on the longitudinal coordinates of the anchor point trajectory. The trajectory formed by the superimposed coordinates of the anchor point trajectory is the predicted driving trajectory.

[0143] As an optional embodiment, the second determining module includes:

[0144] The feature extraction unit is used to extract historical bird's-eye view features of the preset offline driving parameters;

[0145] The first conversion unit is used to convert the historical bird's-eye view features into semantic feature vectors with spatial location encoding;

[0146] The second conversion unit is used to convert the predicted driving trajectory into a motion coding feature vector.

[0147] The feature concatenation unit is used to perform feature concatenation processing on the action encoding feature vector and the semantic feature vector, and output spatiotemporal correlation features;

[0148] A cross-attention calculation unit is used to calculate the cross-attention between the action encoding feature vector and the semantic feature vector based on the spatiotemporal correlation features, so as to simulate the temporal and spatial changes of the target vehicle and obtain multiple second driving parameters of the target vehicle.

[0149] Wherein, any one of the second driving parameters includes the predicted bird's-eye view state of the target vehicle, the predicted operation action and the action reward signal, the predicted bird's-eye view state is the next bird's-eye view state of the target vehicle relative to the first time, as predicted by the world model, and the preset offline driving parameter includes the previous bird's-eye view state of the target vehicle relative to the first time.

[0150] As an optional embodiment, the first conversion unit is further used for

[0151] The historical bird's-eye view features are compressed using the attention mechanism of the pre-trained world model to obtain the semantic feature vector with spatial location encoding.

[0152] As an optional embodiment, the third determining module includes:

[0153] The value function calculation unit is used to calculate the value function of any second driving parameter by using a preset number of critic algorithms. The value function represents the expected cumulative reward of the predicted bird's-eye view state and the predicted operation action in the second driving parameter.

[0154] The value function sorting unit is used to sort all the second driving parameters according to the size of the value function in order to determine the second driving parameter with the smallest value function.

[0155] The loss function determination unit is used to calculate the loss function between the second driving parameter that minimizes the value function and the predicted driving trajectory based on a preset regularization penalty algorithm.

[0156] The driving parameter prediction unit is used to determine the second driving parameter that minimizes the loss function as the target driving parameter in response to the loss function being less than a preset loss threshold.

[0157] As an optional embodiment, the fourth determining module includes:

[0158] A determining unit is used to determine the predicted operation action in the target driving parameters as the target operation action of the target vehicle.

[0159] The control unit is used to control the steering angle and acceleration values ​​of the target vehicle through the target operation actions, so as to realize the autonomous driving of the target vehicle.

[0160] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this disclosure. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0161] Below, for reference Figure 5 This describes an electronic device according to embodiments of the present disclosure. The electronic device may be either or both of a first device and a second device, or a standalone device independent of them, which may communicate with the first device and the second device to receive acquired input signals from them.

[0162] Figure 5 A block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0163] like Figure 5 As shown, the electronic device includes one or more processors and memory.

[0164] A processor can be a central processing unit (CPU) or other form of processing unit with data processing and / or instruction execution capabilities, and can control other components in an electronic device to perform desired functions.

[0165] The memory can store one or more computer program products, and the memory can include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program products can be stored on the computer-readable storage medium, and the processor can run the computer program products to implement the autonomous driving methods of the various embodiments of this disclosure described above and / or other desired functions.

[0166] In one example, the electronic device may also include input devices and output devices, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0167] In addition, the input device may also include, for example, a keyboard, a mouse, etc.

[0168] This output device can output various information to the outside, including determined distance information, direction information, etc. The output device may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0169] Of course, to simplify, Figure 5 Only some of the components of the electronic device relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device may include any other suitable components depending on the specific application.

[0170] In addition to the methods and apparatus described above, embodiments of this disclosure may also be computer program products comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the autonomous driving methods according to various embodiments of this disclosure as described in the foregoing portions of this specification.

[0171] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this disclosure. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0172] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions that, when executed by a processor, cause the processor to perform the steps in the autonomous driving methods according to various embodiments of this disclosure as described in the foregoing portion of this specification.

[0173] The computer-readable storage medium may be any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0174] The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in this disclosure are merely illustrative and not restrictive, and should not be construed as necessarily possessed by each embodiment of the present disclosure. Furthermore, the specific details disclosed above are provided for illustrative purposes and to facilitate understanding, rather than as limitations. These details do not limit the present disclosure to necessarily being implemented using these specific details.

[0175] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0176] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0177] The methods and apparatus of this disclosure may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the methods is for illustrative purposes only, and the steps of the methods of this disclosure are not limited to the order specifically described above unless otherwise specifically stated. Furthermore, in some embodiments, this disclosure may also be implemented as a program recorded on a recording medium, the program including machine-readable instructions for implementing the methods according to this disclosure. Thus, this disclosure also covers recording media storing programs for performing the methods according to this disclosure.

[0178] It should also be noted that in the apparatus, devices, and methods of this disclosure, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions to this disclosure.

[0179] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0180] The above description has been provided for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. An autonomous driving method, characterized in that, The method includes: In response to receiving the first driving parameters of the target vehicle, an anchor point trajectory is determined, and a predicted driving trajectory is determined based on the anchor point trajectory and the first driving parameters of the target vehicle. The first driving parameters of the target vehicle include the first-moment bird's-eye view state of the target vehicle, the first-moment driving speed, and the historical driving trajectory relative to the first moment. The anchor point trajectory is a set of coordinates of reference trajectory points estimated according to a preset estimation method. Based on preset offline driving parameters, the first driving parameters of the target vehicle, and the predicted driving trajectory, multiple second driving parameters of the target vehicle are determined. Each second driving parameter includes the predicted bird's-eye view state of the target vehicle, the predicted operation action, and the action reward signal. The predicted bird's-eye view state is the bird's-eye view state of the target vehicle at the next moment relative to the first moment, as predicted by the world model. The preset offline driving parameters include the bird's-eye view state of the target vehicle at the previous moment relative to the first moment. Based on a preset regularization penalty algorithm, a preset number of critic algorithms, and the predicted driving trajectory, the target driving parameters are determined from multiple second driving parameters of the target vehicle. The target driving parameters are used to determine the target operation action of the target vehicle, and the target vehicle is controlled to perform the target operation action in order to achieve the autonomous driving of the target vehicle.

2. The method according to claim 1, characterized in that, The method responds to receiving first driving parameters of the target vehicle, determines anchor point trajectories, and determines predicted driving trajectories based on the anchor point trajectories and the first driving parameters of the target vehicle. The first driving parameters of the target vehicle include the target vehicle's first-moment bird's-eye view state, first-moment driving speed, and historical driving trajectory relative to the first moment. The anchor point trajectory is a set of coordinates of reference trajectory points estimated according to a preset estimation method, including: Based on the first driving parameters of the target vehicle, the trajectory points of the uniform speed driving path of the target vehicle are estimated according to a preset estimation method to obtain the anchor point trajectory, which is a set of coordinates of reference trajectory points. The first future trajectory of the target vehicle is determined based on the first driving parameters of the target vehicle. The first future trajectory is a set of predicted driving trajectory points within a preset time period, with the first moment of the target vehicle as the starting point. Determine the offset parameter between the first future trajectory point and the anchor point trajectory; The predicted driving trajectory is obtained based on the offset parameters and the anchor point trajectory; The first driving parameters of the target vehicle include the first-moment bird's-eye view state of the target vehicle, the first-moment driving speed, and the historical driving trajectory relative to the first moment. The anchor point trajectory is a set of coordinates of reference trajectory points estimated according to a preset estimation method.

3. The method according to claim 2, characterized in that, The anchor point trajectory is obtained by estimating the trajectory points of the uniform speed driving path of the target vehicle according to the first driving parameters of the target vehicle and a preset estimation method. The anchor point trajectory is a set of coordinates of reference trajectory points, including: Obtain the target vehicle's current speed and position coordinates at the first moment; Based on the current position and the current speed, the trajectory points of the uniform speed driving path of the target vehicle are calculated at preset time intervals within a preset time range, and the coordinate set of the trajectory points of the uniform speed driving path is the anchor point trajectory.

4. The method according to claim 2 or 3, characterized in that, The step of obtaining the predicted driving trajectory based on the offset parameter and the anchor point trajectory includes: The predicted lateral and longitudinal offsets are obtained based on the offset parameters. The predicted lateral offset is superimposed on the lateral coordinates of the anchor point trajectory, and the predicted longitudinal offset is superimposed on the longitudinal coordinates of the anchor point trajectory. The trajectory formed by the superimposed coordinates of the anchor point trajectory is the predicted driving trajectory.

5. The method according to claim 1, characterized in that, The method involves determining multiple second driving parameters of the target vehicle based on preset offline driving parameters, the first driving parameters of the target vehicle, and the predicted driving trajectory. Each second driving parameter includes the predicted bird's-eye view state of the target vehicle, the predicted operational action, and the action reward signal. The predicted bird's-eye view state is the bird's-eye view state of the target vehicle relative to the first time step predicted by the world model. The preset offline driving parameters include the bird's-eye view state of the target vehicle relative to the first time step, including: Extract historical bird's-eye view features of the preset offline driving parameters; The historical bird's-eye view features are transformed into semantic feature vectors with spatial location encoding; The predicted driving trajectory is converted into an action-encoded feature vector; The action encoding feature vector and the semantic feature vector are concatenated to output spatiotemporal correlation features; Based on the spatiotemporal correlation features, the cross-attention between the action encoding feature vector and the semantic feature vector is calculated to simulate the temporal and spatial changes of the target vehicle and obtain multiple second driving parameters of the target vehicle. Wherein, any one of the second driving parameters includes the predicted bird's-eye view state of the target vehicle, the predicted operation action and the action reward signal, the predicted bird's-eye view state is the next bird's-eye view state of the target vehicle relative to the first time, as predicted by the world model, and the preset offline driving parameter includes the previous bird's-eye view state of the target vehicle relative to the first time.

6. The method according to claim 5, characterized in that, The process of converting the historical bird's-eye view features into a semantic feature vector with spatial location encoding includes: The historical bird's-eye view features are compressed using the attention mechanism of the pre-trained world model to obtain the semantic feature vector with spatial location encoding.

7. The method according to claim 1, characterized in that, The method, based on a preset regularization penalty algorithm, a preset number of critic algorithms, and the predicted driving trajectory, determines the target driving parameters from multiple second driving parameters of the target vehicle, including: The value function of any second driving parameter is calculated by an algorithm with a preset number of critics. The value function represents the expected cumulative reward of the predicted bird's-eye view state and the predicted driving action in the second driving parameter. For all the second driving parameters, sort them according to the size of the value function to determine the second driving parameter with the smallest value function; The loss function between the second driving parameter that minimizes the value function and the predicted driving trajectory is calculated based on a preset regularization penalty algorithm. In response to the loss function being less than a preset loss threshold, the second driving parameter that minimizes the loss function is determined as the target driving parameter.

8. The method according to claim 7, characterized in that, The step of determining the target operation action of the target vehicle based on the target driving parameters, and controlling the target vehicle to execute the target operation action to achieve autonomous driving of the target vehicle includes: The predicted driving action in the target driving parameters is determined as the target driving action of the target vehicle; The steering angle and acceleration of the target vehicle are controlled by the target operation actions to achieve autonomous driving of the target vehicle.

9. An automatic driving device, characterized in that, The device includes: The first determining module is used to determine the anchor point trajectory in response to receiving the first driving parameters of the target vehicle, and to determine the predicted driving trajectory based on the anchor point trajectory and the first driving parameters of the target vehicle. The first driving parameters of the target vehicle include the first-moment bird's-eye view state of the target vehicle, the first-moment driving speed, and the historical driving trajectory relative to the first moment. The anchor point trajectory is a set of coordinates of reference trajectory points estimated according to a preset estimation method. The second determining module is used to determine multiple second driving parameters of the target vehicle based on preset offline driving parameters, the first driving parameters of the target vehicle, and the predicted driving trajectory. Each second driving parameter includes the predicted bird's-eye view state of the target vehicle, the predicted operation action, and the action reward signal. The predicted bird's-eye view state is the bird's-eye view state of the target vehicle at the next moment relative to the first moment, as predicted by the world model. The preset offline driving parameters include the bird's-eye view state of the target vehicle at the previous moment relative to the first moment. The third determining module is used to determine the target driving parameters from multiple second driving parameters of the target vehicle based on a preset regularization penalty algorithm, a preset number of critic algorithms and the predicted driving trajectory. The fourth determining module is used to determine the target operation action of the target vehicle based on the target driving parameters, and control the target vehicle to perform the target operation action in order to achieve the autonomous driving of the target vehicle.

10. An electronic device, characterized in that, include: Memory, used to store computer program products; A processor for executing a computer program product stored in the memory, wherein when the computer program product is executed, it implements the method described in any one of claims 1-8.

11. A computer-readable storage medium storing computer program instructions thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1-8.

12. A computer program product comprising computer program instructions, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1-8.

Images