Automatic driving method and device, electronic equipment, storage medium and program product

Abstract

The embodiment of the invention discloses an automatic driving method and device, electronic equipment, a storage medium and a program product, a predicted driving track of a target vehicle is guided based on an anchor point track, the predicted driving track is used as a basic driving strategy, a basic reference track is provided for self-driving of the target vehicle, and the driving accuracy of the target vehicle is improved. The problem of distribution deviation of behavior clone in closed-loop deployment in related technologies is solved, multiple second driving parameters (including but not limited to prediction of aerial view states, prediction of operation actions and action reward signals) are predicted through the offline driving parameters and the first driving parameters, the multiple second driving parameters are used as alternative driving strategies, and the driving speed of the driving strategy is improved. And determining whether the alternative driving strategy is good or not through a preset regular penalty algorithm, a commentator algorithm and a basic driving strategy (predicted driving track) so as to determine a safer driving strategy, so that offline reinforcement learning is realized, and the problems of high calculation cost, low training efficiency, insufficient strategy robustness and the like existing in reinforcement learning are solved.

Description

Technical Field

[0001] The present disclosure relates to intelligent driving technology, and in particular to an autonomous driving method, device, electronic device, computer-readable storage medium, and computer program product. Background Art

[0002] In the field of autonomous driving, behavior cloning (BC) and reinforcement learning (RL) are commonly applied technologies. Behavior cloning, a mainstream imitation learning strategy, focuses on leveraging log data generated by human driving to construct a mapping from sensor inputs (such as camera images and lidar point clouds) to the vehicle's future trajectory or control commands (such as steering wheel angles and accelerator and brake commands). Models are trained using large amounts of labeled data to learn the driving decision-making patterns of human drivers in various scenarios, enabling them to mimic human driving behavior in real-world applications. A unified framework, such as the UniAD framework, integrates perception, prediction, and planning, simplifying the autonomous driving system architecture. Reinforcement learning, on the other hand, involves an intelligent agent interacting with the environment and optimizing its strategy based on reward signals from the environment to maximize long-term rewards. In autonomous driving scenarios, reinforcement learning can be used to learn a vehicle's decision-making strategies in complex traffic environments, adapting control commands through trial and error to adapt to varying road conditions.

[0003] However, in related technologies, both behavioral cloning and reinforcement learning have obvious defects: behavioral cloning will face serious distribution shift problems when deployed in a closed loop. Models that perform well in open-loop training will continue to accumulate prediction errors in actual driving, causing the vehicle to enter a state different from the distribution of the training data. Moreover, the training data is difficult to cover all extreme or rare scenarios, and the vehicle's ability to recover from errors is weak; in addition, behavioral cloning lacks a reasoning mechanism for long-term consequences and cannot effectively weigh decisions in complex driving scenarios; reinforcement learning relies on interactive environments or simulators, consumes a lot of computing resources, has low training efficiency, and the generated driving strategies easily deviate from natural human driving behavior; offline reinforcement learning is limited by data coverage, and it is difficult to effectively deal with the distribution shift problem. It cannot fully utilize offline data to learn robust driving strategies, and it is difficult to meet the strict requirements of autonomous driving for safety and reliability. Summary of the Invention

[0004] In order to solve the technical problems in the related art, the embodiments of the present disclosure provide an autonomous driving method, device, electronic device, computer-readable storage medium and computer program product.

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided an autonomous driving method, the method comprising: In response to receiving a first driving parameter of a target vehicle, determining an anchor trajectory, and determining a predicted driving trajectory based on the anchor trajectory and the first driving parameter of the target vehicle, wherein the first driving parameter of the target vehicle includes a bird's-eye view state of the target vehicle at a first moment, a driving speed at the first moment, and a historical driving trajectory relative to the first moment, and the anchor trajectory is a set of coordinates of reference trajectory points estimated according to a preset estimation method; determining, based on preset offline driving parameters, the first driving parameters of the target vehicle, and the predicted driving trajectory, a plurality of second driving parameters of the target vehicle, wherein any of the second driving parameters includes a predicted bird's-eye view state of the target vehicle, a predicted operation action, and an action reward signal, the predicted bird's-eye view state being a bird's-eye view state of the target vehicle at a next moment relative to the first moment predicted by the world model, and the preset offline driving parameters including a bird's-eye view state of the target vehicle at a previous moment relative to the first moment; determining a target driving parameter from a plurality of second driving parameters of the target vehicle based on a preset regularized penalty algorithm, a preset number of critic algorithms, and the predicted driving trajectory; A target operating action of the target vehicle is determined according to the target driving parameter, and the target vehicle is controlled to perform the target operating action to achieve automatic driving of the target vehicle.

[0006] As an optional embodiment, in response to receiving a first driving parameter of the target vehicle, an anchor point trajectory is determined, and a predicted driving trajectory is determined based on the anchor point trajectory and the first driving parameter of the target vehicle, wherein the first driving parameter of the target vehicle includes a bird's-eye view state of the target vehicle at a first moment, a driving speed at the first moment, and a historical driving trajectory relative to the first moment, and the anchor point trajectory is a set of coordinates of reference trajectory points estimated according to a preset estimation method, including: According to the first driving parameter of the target vehicle, the trajectory points of the uniform speed driving path of the target vehicle are estimated according to a preset estimation method to obtain the anchor point trajectory, where the anchor point trajectory is a coordinate set of the reference trajectory points; determining a first future trajectory of the target vehicle based on a first driving parameter of the target vehicle, wherein the first future trajectory is a set of predicted driving trajectory points within a preset time period starting from a first moment of the target vehicle; determining an offset parameter between the first future trajectory point and the anchor trajectory; Obtaining a predicted driving trajectory according to the offset parameter and the anchor point trajectory; Among them, the first driving parameters of the target vehicle include the bird's-eye view state of the target vehicle at the first moment, the driving speed at the first moment and the historical driving trajectory relative to the first moment, and the anchor point trajectory is a coordinate set of reference trajectory points estimated according to a preset estimation method.

[0007] As an optional embodiment, based on the first driving parameter of the target vehicle, the trajectory points of the uniform speed driving path of the target vehicle are estimated according to a preset estimation method to obtain the anchor point trajectory, where the anchor point trajectory is a coordinate set of reference trajectory points according to the first driving parameter of the target vehicle, including: Obtaining the current speed and position coordinates of the target vehicle at a first moment; Based on the current position and the current speed, trajectory points of the uniform speed driving path of the target vehicle are calculated at preset time intervals within a preset time range, and the coordinate set of the trajectory points of the uniform speed driving path is the anchor point trajectory.

[0008] As an optional embodiment, obtaining the predicted driving trajectory according to the offset parameter and the anchor point trajectory includes: Obtaining predicted lateral and longitudinal offsets according to the offset parameters; The predicted lateral offset is superimposed on the lateral coordinate of the anchor point trajectory, and the predicted longitudinal offset is superimposed on the longitudinal coordinate of the anchor point trajectory. The trajectory formed by the coordinates of the superimposed anchor point trajectory is the predicted driving trajectory.

[0009] As an optional embodiment, the multiple second driving parameters of the target vehicle are determined based on the preset offline driving parameters, the first driving parameters of the target vehicle, and the predicted driving trajectory. Any of the second driving parameters includes a predicted bird's-eye view state of the target vehicle, a predicted operation action, and an action reward signal. The predicted bird's-eye view state is a bird's-eye view state of the target vehicle at a next moment relative to the first moment predicted by the world model. The preset offline driving parameters include a bird's-eye view state of the target vehicle at a previous moment relative to the first moment, including: extracting historical bird's-eye view features of the preset offline driving parameters; Converting the historical bird's-eye view features into a semantic feature vector with spatial position encoding; Converting the first predicted operation action into an action encoding feature vector; Performing feature splicing processing on the action coding feature vector and the semantic feature vector to output spatiotemporal correlation features; Based on the spatiotemporal correlation features, calculating a cross attention between the action encoding feature vector and the semantic feature vector to simulate temporal and spatial changes of the target vehicle to obtain a plurality of second driving parameters of the target vehicle; Among them, any one of the second driving parameters includes the predicted bird's-eye view state, predicted operation action and action reward signal of the target vehicle, the predicted bird's-eye view state is the bird's-eye view state of the target vehicle at the next moment relative to the first moment predicted by the world model, and the preset offline driving parameters include the bird's-eye view state of the target vehicle at the previous moment relative to the first moment.

[0010] As an optional embodiment, converting the historical bird's-eye view features into a semantic feature vector with spatial position encoding includes: The attention mechanism of the pre-trained world model is used to perform feature dimension compression processing on the historical bird's-eye view features to obtain the semantic feature vector with spatial position encoding.

[0011] As an optional embodiment, determining the target driving parameter from the plurality of second driving parameters of the target vehicle based on a preset regularization penalty algorithm, a preset number of critic algorithms, and the predicted driving trajectory includes: Calculating a value function of any second driving parameter using a preset number of critic algorithms, wherein the value function represents an expected cumulative reward of a predicted bird's-eye view state and a predicted operation action in the second driving parameter; sorting all the second driving parameters according to the magnitude of their value functions to determine the second driving parameter with the smallest value function; Calculating a loss function between the second driving parameter with the minimum value function and the predicted driving trajectory based on a preset regularized penalty algorithm; In response to the loss function being less than a preset loss threshold, the second driving parameter with the minimum value function is determined as the target driving parameter.

[0012] As an optional embodiment, determining a target operation action of the target vehicle according to the target driving parameter, and controlling the target vehicle to perform the target operation action to achieve automatic driving of the target vehicle includes: determining the predicted operating action in the target driving parameter as the target operating action of the target vehicle; Changing the first predicted operation action to the target operation action; The steering angle value and acceleration value of the target vehicle are controlled through the target operation action to achieve automatic driving of the target vehicle.

[0013] According to a second aspect of an embodiment of the present disclosure, there is provided an automatic driving device, the device comprising: a first determination module, configured to determine an anchor point trajectory in response to receiving a first driving parameter of a target vehicle, and determine a predicted driving trajectory based on the anchor point trajectory and the first driving parameter of the target vehicle, wherein the first driving parameter of the target vehicle includes a bird's-eye view state of the target vehicle at a first moment, a driving speed at the first moment, and a historical driving trajectory relative to the first moment, and the anchor point trajectory is a set of coordinates of reference trajectory points estimated according to a preset estimation method; a second determination module, configured to determine a plurality of second driving parameters of the target vehicle based on preset offline driving parameters, the first driving parameters of the target vehicle, and the predicted driving trajectory, wherein any of the second driving parameters includes a predicted bird's-eye view state of the target vehicle, a predicted operation action, and an action reward signal, the predicted bird's-eye view state being a bird's-eye view state of the target vehicle at a next moment relative to the first moment predicted by the world model, and the preset offline driving parameters including a bird's-eye view state of the target vehicle at a previous moment relative to the first moment; a third determination module, configured to determine a target driving parameter from the plurality of second driving parameters of the target vehicle based on a preset regularized penalty algorithm, a preset number of critic algorithms, and the predicted driving trajectory; The fourth determination module is used to determine the target operation action of the target vehicle according to the target driving parameter, and control the target vehicle to perform the target operation action to achieve automatic driving of the target vehicle.

[0014] As an optional embodiment, the first determining module includes: an anchor point determination unit, configured to estimate, based on the first driving parameter of the target vehicle and in accordance with a preset estimation method, a trajectory point of a uniform speed driving path of the target vehicle to obtain the anchor point trajectory, wherein the anchor point trajectory is a coordinate set of reference trajectory points; a future trajectory determination unit, configured to determine a first future trajectory of the target vehicle based on a first driving parameter of the target vehicle, wherein the first future trajectory is a set of predicted driving trajectory points within a preset time period starting from a first moment of the target vehicle; an offset parameter determining unit, configured to determine an offset parameter between the first future trajectory point and the anchor point trajectory; a driving trajectory prediction unit, configured to obtain a predicted driving trajectory based on the offset parameter and the anchor point trajectory; an operation action prediction unit, configured to determine a first predicted operation action of the target vehicle according to the predicted driving trajectory; Among them, the first driving parameters of the target vehicle include the bird's-eye view state of the target vehicle at the first moment, the driving speed at the first moment and the historical driving trajectory relative to the first moment, and the anchor point trajectory is a coordinate set of reference trajectory points estimated according to a preset estimation method.

[0015] As an optional embodiment, the anchor point determination unit is further configured to: Obtaining the current speed and position coordinates of the target vehicle at a first moment; Based on the current position and the current speed, trajectory points of the uniform speed driving path of the target vehicle are calculated at preset time intervals within a preset time range, and the coordinate set of the trajectory points of the uniform speed driving path is the anchor point trajectory.

[0016] As an optional embodiment, the driving trajectory prediction unit is further configured to: Obtaining predicted lateral and longitudinal offsets according to the offset parameters; The predicted lateral offset is superimposed on the lateral coordinate of the anchor point trajectory, and the predicted longitudinal offset is superimposed on the longitudinal coordinate of the anchor point trajectory. The trajectory formed by the coordinates of the superimposed anchor point trajectory is the predicted driving trajectory.

[0017] As an optional embodiment, the second determining module includes: a feature extraction unit, configured to extract historical bird's-eye view features of the preset offline driving parameters; A first conversion unit is used to convert the historical bird's-eye view features into a semantic feature vector with spatial position encoding; a second conversion unit, configured to convert the predicted driving trajectory into an action coding feature vector; A feature splicing unit, configured to perform feature splicing processing on the action coding feature vector and the semantic feature vector, and output a spatiotemporal correlation feature; a cross-attention calculation unit, configured to calculate a cross-attention between the action coding feature vector and the semantic feature vector based on the spatiotemporal correlation feature, so as to simulate the temporal and spatial changes of the target vehicle and obtain a plurality of second driving parameters of the target vehicle; Among them, any one of the second driving parameters includes the predicted bird's-eye view state, predicted operation action and action reward signal of the target vehicle, the predicted bird's-eye view state is the bird's-eye view state of the target vehicle at the next moment relative to the first moment predicted by the world model, and the preset offline driving parameters include the bird's-eye view state of the target vehicle at the previous moment relative to the first moment.

[0018] As an optional embodiment, the first conversion unit is further used to The attention mechanism of the pre-trained world model is used to perform feature dimension compression processing on the historical bird's-eye view features to obtain the semantic feature vector with spatial position encoding.

[0019] As an optional embodiment, the third determining module includes: a value function calculation unit, configured to calculate a value function of any second driving parameter by using a preset number of critic algorithms, wherein the value function represents an expected cumulative reward of a predicted bird's-eye view state and a predicted operation action in the second driving parameter; a value function sorting unit, configured to sort all the second driving parameters according to the magnitude of their value functions, so as to determine the second driving parameter having the smallest value function; a loss function determining unit, configured to calculate a loss function between the second driving parameter with the minimum value function and the predicted driving trajectory based on a preset regularized penalty algorithm; The driving parameter prediction unit is configured to, in response to the loss function being less than a preset loss threshold, determine the second driving parameter with the minimum value function as a target driving parameter.

[0020] As an optional embodiment, the fourth determining module includes: a determining unit, configured to determine the predicted operating action in the target driving parameter as a target operating action of the target vehicle; A control unit is used to control the steering angle value and acceleration value of the target vehicle through the target operation action to achieve automatic driving of the target vehicle.

[0021] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including: a memory for storing a computer program product; The processor is configured to execute the computer program product stored in the memory, and when the computer program product is executed, the method described in the first aspect above is implemented.

[0022] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described in the first aspect above is implemented.

[0023] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising computer program instructions, which implement the method described in the first aspect when executed by a processor.

[0024] In an embodiment of the present disclosure, an anchor trajectory is determined, and then a predicted driving trajectory of a target vehicle is guided based on the anchor trajectory. The predicted driving trajectory is used as a basic driving strategy. This anchor trajectory-guided method for determining the basic driving strategy provides a basic reference trajectory for the target vehicle's self-driving, thereby resolving the distribution offset problem of behavioral cloning in closed-loop deployment in related technologies. Multiple second driving parameters (including but not limited to predicted bird's-eye view states, predicted operational actions, and action reward signals) are predicted using offline driving parameters and first driving parameters. These multiple second driving parameters are used as alternative driving strategies. The quality or feasibility of the alternative driving strategies is then determined using a preset regularized penalty algorithm, a preset number of critic algorithms, and the basic driving strategy (predicted driving trajectory). This results in a safer driving strategy, enabling offline reinforcement learning. This addresses the problems of high computational cost, low training efficiency, and insufficient strategy robustness inherent in reinforcement learning, and enables more accurate trajectory prediction.

[0025] The technical solution of the present disclosure is further described in detail below through the accompanying drawings and examples. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0027] The present disclosure can be more clearly understood from the following detailed description with reference to the accompanying drawings, in which: Figure 1 This is an architectural block diagram of a method for implementing autonomous driving according to an embodiment of the present disclosure.

[0028] Figure 2 This is one of the flowcharts of the autonomous driving method of an embodiment of the method disclosed herein.

[0029] Figure 3 This is the second flowchart of the autonomous driving method according to an embodiment of the method disclosed herein.

[0030] Figure 4 This is a structural block diagram of an automatic driving device according to an embodiment of the present invention.

[0031] Figure 5 This is a structural block diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] The following will be combined with the accompanying drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the embodiments described are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of the present disclosure.

[0033] In order to enable those skilled in the art to clearly and accurately understand the technical solutions of the embodiments of the present disclosure, Figure 1 The architecture block diagram shown is used for introduction. The technical solution of the embodiment of the present disclosure can be composed of three parts: anchor-guided behavior cloning, reinforcement learning (or offline reinforcement learning), and world model. First, the driving parameters of the target vehicle (or ego vehicle) are obtained, where the driving parameters can include historical driving parameters (such as the driving parameters of the ego vehicle at the previous moment) and first driving parameters (the driving parameters of the ego vehicle at the first moment). The driving parameters can be derived from the historical bird's-eye view map and the current bird's-eye view map. Some or all bird's-eye view features extracted from the historical bird's-eye view map and the current bird's-eye view map are used as driving parameters, including but not limited to: ① the position coordinates (x, y), heading angle, and speed of the ego vehicle and surrounding vehicles in the past N frames; ② the control action of the ego vehicle (such as the steering angle and acceleration at the historical moment); ③ environmental perception information (such as lane line topology, traffic light status, obstacle bounding box, drivable area segmentation mask), etc.

[0034] The anchor-guided behavior cloning component uses the anchor trajectory as a reference to guide behavior cloning, generating a relative predicted driving trajectory that serves as the basis for the driving strategy. For example, if the target vehicle is traveling at 60 km / h, the anchor trajectory will generate a straight path based on the current speed. The actual trajectory may be curved, so the anchor-guided behavior cloning component predicts the offset of this curved trajectory relative to the straight path. This solves the stability issue during speed changes.

[0035] The world model, a simulation engine, is a core technical framework in the field of artificial intelligence used to simulate environmental dynamics and predict future states. It constructs a system's internal representation through multimodal data training. In the disclosed embodiment, it can simulate the target vehicle's (or ego vehicle's) driving conditions based on its driving parameters, predicting its future driving trajectory (e.g., a preset time period relative to the first moment), driving state, environmental perception, and so on. It also sets reward predictions to determine whether the future driving trajectory, state, and other outcomes will be positive (goal-oriented) or negative (unsafe). In this way, the pre-trained world model replaces the real environment for strategy planning and evaluation, eliminating the need for the agent to interact with the real environment. This effectively addresses the problems of reinforcement learning, such as high computational cost, low training efficiency, and insufficient strategy robustness.

[0036] However, anchor-guided behavior cloning still suffers from distribution shift issues. Specifically, during actual driving, the target vehicle encounters a variety of new scenarios, and the predicted driving trajectory may not be adequate for some complex new scenarios, leading to distribution shift. Combined with world model simulation to obtain more driving strategies, the disclosed embodiments combine anchor-guided behavior cloning, world modeling, and reinforcement learning. The reinforcement learning algorithm can employ a dual-delay mechanism (for example, using a preset number of critic algorithms (e.g., two critic networks) to perform mutual error correction on the strategies, combined with behavior cloning regularization (i.e., a preset regularization penalty algorithm) to prevent deviation from the dataset. The final results of the critic algorithms are then used to adjust and optimize the base driving strategy without excessively biasing the base driving strategy, resulting in a safe and reliable final driving strategy.

[0037] The following combination Figure 2 The technical solutions of the embodiments of the present disclosure are further introduced in detail. Figure 2 This is one of the flow charts of the automatic driving method according to an embodiment of the disclosed method. Figure 2 As shown, an automatic driving method may include the following steps: Step 201 : In response to receiving a first driving parameter of a target vehicle, determining an anchor point trajectory, and determining a predicted driving trajectory based on the anchor point trajectory and the first driving parameter of the target vehicle.

[0038] The target vehicle is the autonomous vehicle whose driving trajectory (or driving strategy) is currently being acquired. The first driving parameters are the driving parameters of the target vehicle at a first moment. These first driving parameters include the target vehicle's bird's-eye view state at the first moment, its driving speed at the first moment, and its historical driving trajectory relative to the first moment. The anchor trajectory is the coordinate set of reference trajectory points estimated using a preset estimation method. The first driving parameters can be derived from images captured by the target vehicle's camera, point cloud data collected by a lidar (lidar), or the target vehicle's speed sensor, among other sources.

[0039] In some embodiments, the trajectory points of the uniform driving path of the target vehicle can be estimated according to a preset estimation method based on the first driving parameter of the target vehicle to obtain an anchor point trajectory, which is a coordinate set of reference trajectory points according to the target vehicle, and the first future trajectory of the target vehicle can be determined according to the first driving parameter of the target vehicle, and the first future trajectory is a set of predicted driving trajectory points within a preset time period with the first moment of the target vehicle as the time starting point, and then the offset parameter between the first future trajectory point and the anchor point trajectory is determined, and finally the predicted driving trajectory is obtained based on the offset parameter and the anchor point trajectory.

[0040] Furthermore, based on the first driving parameter of the target vehicle, the trajectory points of the target vehicle's uniform driving path are estimated according to a preset estimation method to obtain an anchor trajectory. This can be achieved by obtaining the current speed and position coordinates of the target vehicle at a first moment, and then, based on the current position and current speed, calculating the trajectory points of the target vehicle's uniform driving path at preset time intervals within a preset time range. The coordinate set of the trajectory points of the uniform driving path is the anchor trajectory. For example, a coordinate system is established with the autonomous driving vehicle (target vehicle) itself as the center. Assuming that the autonomous driving vehicle (target vehicle) maintains its current speed, a planning time range (i.e., a preset time range) and a time interval (preset time interval) are set. For example, the planning time range (i.e., a preset time range) is a set of trajectory points for six time points (preset time intervals) at 2 Hz in the future, 3 seconds from the first moment. These trajectory points can be represented by coordinates (x, y), and the set of coordinates of these trajectory points is determined as the anchor trajectory.

[0041] Furthermore, the predicted driving trajectory is obtained based on the offset parameters and the anchor point trajectory. This can be achieved by obtaining predicted lateral and longitudinal offsets based on the offset parameters, and then superimposing the predicted lateral offset onto the lateral coordinates of the anchor point trajectory, and superimposing the predicted longitudinal offset onto the longitudinal coordinates of the anchor point trajectory. The trajectory formed by the coordinates of the superimposed anchor point trajectory is the predicted driving trajectory.

[0042] For example, first, with the ego vehicle (target vehicle) as the center, a virtual reference trajectory is generated by driving at a constant speed according to the ego vehicle's current speed, for example, one path point every 0.5 seconds within 10 seconds. The coordinates of these path points are collected to obtain the anchor point trajectory. Then, the offset between the real future trajectory (i.e., the first future trajectory; for example, the current real trajectory of the recorded target vehicle, using the next 5 seconds as the real future trajectory) and the anchor point trajectory is calculated (for example, a lateral offset of 2 meters, that is, a lateral offset of 2 meters). The final predicted driving trajectory is the offset (lateral offset of 2 meters) plus the anchor point trajectory.

[0043] Here, step 201 can be performed by Figure 1 The anchor-guided behavioral cloning portion of the system is implemented by combining anchor trajectories with behavioral cloning to construct a mapping from target vehicle sensor inputs (e.g., camera images, LiDAR point clouds, etc.) to the target vehicle's future trajectory or control commands (e.g., steering wheel angle, accelerator and brake commands), enabling anchor-trajectory-guided behavioral cloning. This allows the dynamic change of anchor trajectories at different speeds, avoiding learning bias associated with a fixed coordinate system. Learning relative offsets (rather than absolute positions) reduces prediction jitter in high-speed scenarios, addressing the distributional offset issue of behavioral cloning in closed-loop deployments. This allows for more accurate trajectory predictions, improving the generalization capabilities of the disclosed embodiments across diverse driving scenarios.

[0044] In step 202, a plurality of second driving parameters of the target vehicle are determined based on the preset offline driving parameters, the first driving parameters of the target vehicle, and the predicted driving trajectory. Any second driving parameter includes a predicted bird's-eye view state of the target vehicle, a predicted operation action, and an action reward signal. The predicted bird's-eye view state is a bird's-eye view state of the target vehicle at the next moment relative to the first moment predicted by the world model. The preset offline driving parameters include a bird's-eye view state of the target vehicle at the previous moment relative to the first moment.

[0045] In the embodiment of the present disclosure, the second driving parameter is a predicted driving parameter of the target vehicle at a second moment, and the second moment is located after the first moment.

[0046] The world model is a core technical framework in the field of artificial intelligence for simulating environmental dynamics and predicting future states. It constructs the internal representation of the system through multimodal data training. In the embodiment of the present disclosure, the world model is used to simulate and predict the bird's-eye view of the future moment based on preset offline driving parameters, the first driving parameters of the target vehicle, and the predicted driving trajectory, and output the reward label corresponding to the bird's-eye view of the future moment, that is, the second driving parameter, which is then used in subsequent reinforcement training to support offline reinforcement learning and solve the problems of high computational cost, low training efficiency, and insufficient strategy robustness in reinforcement learning. The learned world model is used to support offline reinforcement learning, which can perform strategy learning without interacting with the environment, thereby enhancing learning efficiency and stability. In the closed-loop simulation benchmark test, the safety and robustness of autonomous driving were effectively improved compared with the baseline that only used behavior cloning.

[0047] The world model in the disclosed embodiment is essentially a neural network that can learn and predict target vehicle state changes and rewards from offline data (preset offline driving parameters). As an example, implementing step 202 using the world model can be implemented as follows: Step A: Extract historical bird's-eye view features of preset offline driving parameters and convert them into semantic feature vectors with spatial position encoding.

[0048] In this step, the world model spatially encodes the features of preset offline driving parameters (e.g., historical bird's-eye view images or the target vehicle's bird's-eye view state at the previous moment relative to the first moment) to generate a compressed feature representation that preserves spatial semantic relationships. As an implementation method, the attention mechanism of the pre-trained world model is used to perform feature dimension compression processing on the historical bird's-eye view features to obtain the semantic feature vector with spatial position encoding. Specifically, for example, historical bird's-eye view features can be fused through a convolutional neural network, and a convolutional block attention mechanism can be used to extract key spatial features to generate a low-dimensional feature representation that contains object position and semantic information. Position encoding is then added to mark spatial coordinate relationships during the feature combination stage, thereby obtaining a semantic feature vector with spatial position encoding.

[0049] Step B: Convert the predicted driving trajectory into an action coding feature vector, perform feature concatenation on the action coding feature vector and the semantic feature vector, and output the spatiotemporal correlation features.

[0050] This step achieves spatiotemporal fusion of features. The predicted driving trajectory is converted into an action encoding feature vector and projected into the latent space. It is then concatenated with the flattened semantic feature vector. The combined features are processed using a Transformer encoder with a multi-head attention mechanism. Position encoding is added to preserve the spatial coordinates of the BEV features, resulting in spatiotemporal correlation features. For example, the Transformer encoder can be used to process the combined features by calculating the interaction attention weights between the semantic feature vector and the action encoding feature. The multi-head attention mechanism then establishes a spatiotemporal correlation between the two features. The output is a feature vector that integrates the spatiotemporal variation features, which is the spatiotemporal correlation feature.

[0051] Step C: Based on the spatiotemporal correlation features, the cross attention between the action encoding feature vector and the semantic feature vector is calculated to simulate the temporal and spatial changes of the target vehicle and obtain multiple second driving parameters of the target vehicle.

[0052] The Transformer output is reshaped to include spatiotemporal features, and its spatial resolution is increased through convolutional blocks with CBAM (Convolutional Block Attention Module) attention. This predicts the future BEV state (including, but not limited to, the position, speed (and sometimes orientation), road structure, and traffic signal status of all objects surrounding the target vehicle). A lightweight reward head, comprised of convolutional blocks and a spatial attention mechanism, estimates a reward based on the predicted future BEV state, determining whether the outcome is positive (goal-oriented) or negative (unsafe). Specifically, the lightweight reward head, comprised of convolutional blocks and a spatial attention mechanism, is trained using supervised reward labels to support value learning within a world model. For example, the predicted future BEV state is fed into a lightweight reward prediction network (comprising convolutional blocks and a spatial attention mechanism). This network extracts features through convolutional and spatial attention layers and outputs a scalar reward value reflecting driving safety risk and route completion. Then, in this way, the pre-trained world model replaces the real environment for strategy planning and evaluation, without the need for the intelligent agent to interact with the real environment, effectively solving the problems of reinforcement learning such as high computational cost, low training efficiency, and insufficient strategy robustness.

[0053] Step 203 : determining a target driving parameter from a plurality of second driving parameters of the target vehicle based on a preset regularized penalty algorithm, a preset number of critic algorithms, and the predicted driving trajectory.

[0054] As an implementation method, step 203 can be implemented as follows: calculating the value function of any second driving parameter through a preset number of critic algorithms, where the value function represents the expected cumulative reward of the predicted bird's-eye view state and the predicted operation action in the second driving parameter; sorting all second driving parameters according to the size of the value function to determine the second driving parameter with the smallest value function; calculating the loss function between the second driving parameter with the smallest value function and the predicted driving trajectory based on a preset regularized penalty algorithm; and in response to the loss function being less than a preset loss threshold, determining the second driving parameter with the smallest value function as the target driving parameter (target driving parameter).

[0055] In this step, you can Figure 1 Offline reinforcement learning is implemented in this framework. Offline reinforcement learning methods are combined with a trained single-step world model to simulate environmental dynamics. The output second driving parameters are used to optimize the predicted driving trajectory (base driving policy) output by anchor-guided behavioral cloning. Specifically, the Twin Delayed DDPG (TD3) algorithm is used, combined with a behavioral cloning regularizer (TD3+BC) to ensure that the policy conforms to the offline data distribution. A preset number of critic algorithms (e.g., two critic networks) are set up. The critic network uses the second driving parameters (current BEV state and reward) predicted by the world model based on the current BEV state (second driving parameter) and candidate vehicle actions (predicted driving trajectory) and is trained using a specific loss calculation method. In some embodiments, the update optimization of anchor-guided behavioral cloning can be delayed relative to that of the critic network (preset number of critic algorithms). This allows the anchor-guided behavioral cloning to maintain a close approximation to the behavioral policy generated from offline data while maximizing the value of its own actions. This effectively addresses the distribution shift issue of behavioral cloning in closed-loop deployment and improves the overall performance of the policy.

[0056] In the disclosed embodiment, a preset number of critic algorithms can be estimated in parallel, simultaneously receiving the current BEV state (second driving parameter) and candidate vehicle actions (predicted driving trajectory), each calculating and outputting a value function, and ultimately selecting the current BEV state (second driving parameter) with the minimum value function. The loss function between the current BEV state (second driving parameter) with the minimum value function and the candidate vehicle action (predicted driving trajectory) is then calculated.

[0057] Step 204 , determining a target operating action of the target vehicle according to the target driving parameters, and controlling the target vehicle to perform the target operating action to achieve automatic driving of the target vehicle.

[0058] Specifically, the predicted operation action in the target driving parameters is determined as the target operation action of the target vehicle, and the steering angle value and acceleration value of the target vehicle are controlled by the target operation action to achieve automatic driving of the target vehicle.

[0059] In summary, by determining the anchor point trajectory, and then guiding the predicted driving trajectory of the target vehicle based on the anchor point trajectory, and using the predicted driving trajectory as the basic driving strategy, the anchor point trajectory guides the method for determining the basic driving strategy, providing a basic reference trajectory for the target vehicle's self-driving, and solving the distribution offset problem of behavior cloning in closed-loop deployment in related technologies. The offline driving parameters and the first driving parameters are used to predict multiple second driving parameters (including but not limited to predicted bird's-eye view states, predicted operation actions, and action reward signals), and the multiple second driving parameters are used as alternative driving strategies. The quality or feasibility of the alternative driving strategies is then determined by a preset regularized penalty algorithm, a preset number of critic algorithms, and the basic driving strategy (predicted driving trajectory), thereby determining a safer driving strategy, realizing offline reinforcement learning, and solving the problems of high computational cost, low training efficiency, and insufficient strategy robustness in reinforcement learning, and being able to more accurately predict trajectories.

[0060] In order to enable those skilled in the art to fully understand the technical solution of this disclosure, Figure 3 The illustrated embodiments further describe the technical solution. Figure 3 This is the second flowchart of the autonomous driving method according to an embodiment of the method disclosed herein.

[0061] like Figure 3 As shown, first, the first driving parameters can be obtained through the target vehicle (or the self-vehicle) such as cameras, lidars and other sensors, and the historical driving parameters can be obtained through a database or storage server. Taking the multi-perspective image in the figure as an example, it can represent the current or historical trajectory, current or historical bird's-eye view, etc. of the target vehicle, which is used as the initial input. The bird's-eye view encoder can extract bird's-eye view features or bird's-eye view states (including but not limited to the position, speed (sometimes also direction), road structure, traffic signal state, etc. of all objects around the timber vehicle).

[0062] Then, the data is input into the behavior cloning module. The behavior cloning module uses anchor-guided behavior cloning to estimate the prediction deviation (offset parameter) based on the dynamic perception anchor point (anchor point trajectory), thereby obtaining the predicted driving trajectory as the basic driving strategy (at least including but not limited to acceleration, deceleration, left turn, right turn, driving speed and other operations).

[0063] At the same time, the data is also input into the world model. The world model is based on the (current or first moment) bird's-eye view features and historical bird's-eye view features (which can come from historical logs), and behavior embedding is performed (which can be the operation behavior in the basic driving strategy based on the current bird's-eye view state, or the operation behavior in historical log data). The fusion feature of the bird's-eye view features and the action encoding features is obtained for temporal and spatial simulation. The data is then encoded by an encoder (such as a Transformer encoder with a multi-head attention mechanism) and the spatial resolution is increased by a convolution block with CBAM attention. The future (second moment or the next moment relative to the first moment) bird's-eye view state (i.e., the future bird's-eye view in the figure) is predicted. A lightweight reward head, such as one composed of a convolution block and a spatial attention mechanism, is then used to estimate the reward based on the predicted future (second moment or the next moment relative to the first moment) bird's-eye view state (i.e., the future bird's-eye view in the figure). The result is determined to be positive (safe) or harmful (unsafe), and the second driving parameter is output (including at least but not limited to the predicted bird's-eye view state, predicted operation action, and action reward signal).

[0064] Next, the world model output (secondary driving parameters) is fed into the offline reinforcement learning module, along with the base driving policy for initialization. Reinforcement learning (offline reinforcement learning) uses a Twin Delayed DDPG (TD3) algorithm (such as the critic algorithm) to calculate the value function (Q-value) for the secondary driving parameters. The world model output (secondary driving parameters) is combined with the regularizer (pre-set regularized penalty algorithm) for anchor trajectory-guided behavior cloning to determine the target driving parameters. The target driving parameters are then used to optimize the base driving policy, ultimately outputting the final driving policy.

[0065] Correspondingly, the embodiments of the present disclosure also provide device embodiments corresponding to the aforementioned method embodiments. The device embodiments of the embodiments of the present disclosure are described in detail below with reference to the accompanying drawings. Figure 4 This is a structural block diagram of an automatic driving device according to an embodiment of the present invention. Figure 4 The automatic driving device may include: a first determining module 401 configured to determine an anchor point trajectory in response to receiving a first driving parameter of a target vehicle, and determine a predicted driving trajectory based on the anchor point trajectory and the first driving parameter of the target vehicle, wherein the first driving parameter of the target vehicle includes a bird's-eye view state of the target vehicle at a first moment, a driving speed at the first moment, and a historical driving trajectory relative to the first moment, and the anchor point trajectory is a set of coordinates of reference trajectory points estimated according to a preset estimation method; a second determination module 402, configured to determine a plurality of second driving parameters of the target vehicle based on preset offline driving parameters, the first driving parameters of the target vehicle, and the predicted driving trajectory, wherein any of the second driving parameters includes a predicted bird's-eye view state of the target vehicle, a predicted operation action, and an action reward signal; the predicted bird's-eye view state is a bird's-eye view state of the target vehicle at a next moment relative to the first moment predicted by the world model; and the preset offline driving parameters include a bird's-eye view state of the target vehicle at a previous moment relative to the first moment; A third determination module 403 is configured to determine a target driving parameter from the plurality of second driving parameters of the target vehicle based on a preset regularized penalty algorithm, a preset number of critic algorithms, and the predicted driving trajectory; The fourth determination module 404 is used to determine the target operation action of the target vehicle according to the target driving parameter, and control the target vehicle to perform the target operation action to achieve automatic driving of the target vehicle.

[0066] In an embodiment of the present disclosure, an anchor trajectory is determined, and then a predicted driving trajectory of a target vehicle is guided based on the anchor trajectory. The predicted driving trajectory is used as a basic driving strategy. This anchor trajectory-guided method for determining the basic driving strategy provides a basic reference trajectory for the target vehicle's self-driving, thereby resolving the distribution offset problem of behavioral cloning in closed-loop deployment in related technologies. Multiple second driving parameters (including but not limited to predicted bird's-eye view states, predicted operational actions, and action reward signals) are predicted using offline driving parameters and first driving parameters. These multiple second driving parameters are used as alternative driving strategies. The quality or feasibility of the alternative driving strategies is then determined using a preset regularized penalty algorithm, a preset number of critic algorithms, and the basic driving strategy (predicted driving trajectory). This results in a safer driving strategy, enabling offline reinforcement learning. This addresses the problems of high computational cost, low training efficiency, and insufficient strategy robustness inherent in reinforcement learning, and enables more accurate trajectory prediction.

[0067] As an optional embodiment, the first determining module includes: an anchor point determination unit, configured to estimate, based on the first driving parameter of the target vehicle and in accordance with a preset estimation method, a trajectory point of a uniform speed driving path of the target vehicle to obtain the anchor point trajectory, wherein the anchor point trajectory is a coordinate set of reference trajectory points; a future trajectory determination unit, configured to determine a first future trajectory of the target vehicle based on a first driving parameter of the target vehicle, wherein the first future trajectory is a set of predicted driving trajectory points within a preset time period starting from a first moment of the target vehicle; an offset parameter determining unit, configured to determine an offset parameter between the first future trajectory point and the anchor point trajectory; a driving trajectory prediction unit, configured to obtain a predicted driving trajectory based on the offset parameter and the anchor point trajectory; an operation action prediction unit, configured to determine a first predicted operation action of the target vehicle according to the predicted driving trajectory; Among them, the first driving parameters of the target vehicle include the bird's-eye view state of the target vehicle at the first moment, the driving speed at the first moment and the historical driving trajectory relative to the first moment, and the anchor point trajectory is a coordinate set of reference trajectory points estimated according to a preset estimation method.

[0068] As an optional embodiment, the anchor point determination unit is further configured to: Obtaining the current speed and position coordinates of the target vehicle at a first moment; Based on the current position and the current speed, trajectory points of the uniform speed driving path of the target vehicle are calculated at preset time intervals within a preset time range, and the coordinate set of the trajectory points of the uniform speed driving path is the anchor point trajectory.

[0069] As an optional embodiment, the driving trajectory prediction unit is further configured to: Obtaining predicted lateral and longitudinal offsets according to the offset parameters; The predicted lateral offset is superimposed on the lateral coordinate of the anchor point trajectory, and the predicted longitudinal offset is superimposed on the longitudinal coordinate of the anchor point trajectory. The trajectory formed by the coordinates of the superimposed anchor point trajectory is the predicted driving trajectory.

[0070] As an optional embodiment, the second determining module includes: a feature extraction unit, configured to extract historical bird's-eye view features of the preset offline driving parameters; A first conversion unit is used to convert the historical bird's-eye view features into a semantic feature vector with spatial position encoding; a second conversion unit, configured to convert the predicted driving trajectory into an action coding feature vector; A feature splicing unit, configured to perform feature splicing processing on the action coding feature vector and the semantic feature vector, and output a spatiotemporal correlation feature; a cross-attention calculation unit, configured to calculate a cross-attention between the action coding feature vector and the semantic feature vector based on the spatiotemporal correlation feature, so as to simulate the temporal and spatial changes of the target vehicle and obtain a plurality of second driving parameters of the target vehicle; Among them, any one of the second driving parameters includes the predicted bird's-eye view state, predicted operation action and action reward signal of the target vehicle, the predicted bird's-eye view state is the bird's-eye view state of the target vehicle at the next moment relative to the first moment predicted by the world model, and the preset offline driving parameters include the bird's-eye view state of the target vehicle at the previous moment relative to the first moment.

[0071] As an optional embodiment, the first conversion unit is further used to The attention mechanism of the pre-trained world model is used to perform feature dimension compression processing on the historical bird's-eye view features to obtain the semantic feature vector with spatial position encoding.

[0072] As an optional embodiment, the third determining module includes: a value function calculation unit, configured to calculate a value function of any second driving parameter by using a preset number of critic algorithms, wherein the value function represents an expected cumulative reward of a predicted bird's-eye view state and a predicted operation action in the second driving parameter; a value function sorting unit, configured to sort all the second driving parameters according to the magnitude of their value functions, so as to determine the second driving parameter having the smallest value function; a loss function determining unit, configured to calculate a loss function between the second driving parameter with the minimum value function and the predicted driving trajectory based on a preset regularized penalty algorithm; The driving parameter prediction unit is configured to, in response to the loss function being less than a preset loss threshold, determine the second driving parameter with the minimum value function as a target driving parameter.

[0073] As an optional embodiment, the fourth determining module includes: a determining unit, configured to determine the predicted operating action in the target driving parameter as a target operating action of the target vehicle; A control unit is used to control the steering angle value and acceleration value of the target vehicle through the target operation action to achieve automatic driving of the target vehicle.

[0074] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this disclosure. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0075] Below, reference Figure 5 The electronic device according to the embodiment of the present disclosure is described. The electronic device may be either or both of the first device and the second device, or a standalone device independent of them, and the standalone device may communicate with the first device and the second device to receive collected input signals from them.

[0076] Figure 5 A block diagram of an electronic device according to an embodiment of the present disclosure is illustrated.

[0077] like Figure 5 As shown, the electronic device includes one or more processors and memory.

[0078] The processor may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0079] The memory may store one or more computer program products. The memory may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), a hard disk, flash memory, etc. One or more computer program products may be stored on the computer-readable storage medium, and the processor may execute the computer program products to implement the autonomous driving methods of the various embodiments of the present disclosure described above and / or other desired functions.

[0080] In one example, the electronic device may further include an input device and an output device, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0081] In addition, the input device may also include, for example, a keyboard, a mouse, and the like.

[0082] The output device can output various information to the outside, including determined distance information, direction information, etc. The output device can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0083] Of course, to simplify, Figure 5 Only some of the components related to the present disclosure in the electronic device are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, the electronic device may further include any other appropriate components according to specific application scenarios.

[0084] In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps in the autonomous driving method according to various embodiments of the present disclosure described in the above part of this specification.

[0085] The computer program product may be written in any combination of one or more programming languages ​​to implement the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as C or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0086] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enable the processor to execute the steps of the autonomous driving method according to various embodiments of the present disclosure described in the above part of this specification.

[0087] The computer-readable storage medium may be any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0088] The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in this disclosure are merely illustrative and not restrictive, and should not be construed as necessarily possessed by each embodiment of the present disclosure. Furthermore, the specific details disclosed above are provided for illustrative purposes and to facilitate understanding, rather than as limitations. These details do not limit the present disclosure to necessarily being implemented using these specific details.

[0089] Each embodiment in this specification is described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. References to the same or similar parts between the various embodiments are sufficient. For system embodiments, since they largely correspond to method embodiments, their description is relatively simple. For relevant parts, references to the description of the method embodiments are sufficient.

[0090] The block diagrams of the devices, devices, equipment, and systems involved in this disclosure are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, or configured in any manner. Words such as "include," "comprise," "have," and the like are open-ended words, meaning "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or" and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0091] The methods and apparatus of the present disclosure may be implemented in many ways. For example, the methods and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above unless otherwise specified. In addition, in some embodiments, the present disclosure may also be implemented as programs recorded in a recording medium, which include machine-readable instructions for implementing the methods according to the present disclosure. Thus, the present disclosure also covers recording media that store programs for executing the methods according to the present disclosure.

[0092] It should also be noted that in the apparatus, device, and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0093] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0094] The above description has been provided for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. An automatic driving method, characterized in that: The method comprises: In response to receiving a first driving parameter of a target vehicle, determining an anchor trajectory, and determining a predicted driving trajectory based on the anchor trajectory and the first driving parameter of the target vehicle, wherein the first driving parameter of the target vehicle includes a bird's-eye view state of the target vehicle at a first moment, a driving speed at the first moment, and a historical driving trajectory relative to the first moment, and the anchor trajectory is a set of coordinates of reference trajectory points estimated according to a preset estimation method; determining, based on preset offline driving parameters, the first driving parameters of the target vehicle, and the predicted driving trajectory, a plurality of second driving parameters of the target vehicle, wherein any of the second driving parameters includes a predicted bird's-eye view state of the target vehicle, a predicted operation action, and an action reward signal, the predicted bird's-eye view state being a bird's-eye view state of the target vehicle at a next moment relative to the first moment predicted by the world model, and the preset offline driving parameters including a bird's-eye view state of the target vehicle at a previous moment relative to the first moment; determining a target driving parameter from a plurality of second driving parameters of the target vehicle based on a preset regularized penalty algorithm, a preset number of critic algorithms, and the predicted driving trajectory; A target operating action of the target vehicle is determined according to the target driving parameter, and the target vehicle is controlled to perform the target operating action to achieve automatic driving of the target vehicle.

2. The method according to claim 1, characterized in that In response to receiving a first driving parameter of the target vehicle, an anchor point trajectory is determined, and a predicted driving trajectory is determined based on the anchor point trajectory and the first driving parameter of the target vehicle, wherein the first driving parameter of the target vehicle includes a bird's-eye view state of the target vehicle at a first moment, a driving speed at the first moment, and a historical driving trajectory relative to the first moment, and the anchor point trajectory is a set of coordinates of reference trajectory points estimated according to a preset estimation method, including: According to the first driving parameter of the target vehicle, the trajectory points of the uniform speed driving path of the target vehicle are estimated according to a preset estimation method to obtain the anchor point trajectory, where the anchor point trajectory is a coordinate set of the reference trajectory points; determining a first future trajectory of the target vehicle based on a first driving parameter of the target vehicle, wherein the first future trajectory is a set of predicted driving trajectory points within a preset time period starting from a first moment of the target vehicle; determining an offset parameter between the first future trajectory point and the anchor trajectory; Obtaining a predicted driving trajectory according to the offset parameter and the anchor point trajectory; Among them, the first driving parameters of the target vehicle include the bird's-eye view state of the target vehicle at the first moment, the driving speed at the first moment and the historical driving trajectory relative to the first moment, and the anchor point trajectory is a coordinate set of reference trajectory points estimated according to a preset estimation method.

3. The method according to claim 2, characterized in that The method of estimating the trajectory points of the uniform speed driving path of the target vehicle according to a preset estimation method based on the first driving parameter of the target vehicle to obtain the anchor point trajectory, wherein the anchor point trajectory is a coordinate set of reference trajectory points according to the first driving parameter of the target vehicle, including: Obtaining the current speed and position coordinates of the target vehicle at a first moment; Based on the current position and the current speed, trajectory points of the uniform speed driving path of the target vehicle are calculated at preset time intervals within a preset time range, and the coordinate set of the trajectory points of the uniform speed driving path is the anchor point trajectory.

4. The method according to claim 2 or 3, characterized in that Obtaining a predicted driving trajectory according to the offset parameter and the anchor point trajectory includes: Obtaining predicted lateral and longitudinal offsets according to the offset parameters; The predicted lateral offset is superimposed on the lateral coordinate of the anchor point trajectory, and the predicted longitudinal offset is superimposed on the longitudinal coordinate of the anchor point trajectory. The trajectory formed by the coordinates of the superimposed anchor point trajectory is the predicted driving trajectory.

5. The method according to claim 1, wherein The method further comprises determining a plurality of second driving parameters of the target vehicle based on the preset offline driving parameters, the first driving parameters of the target vehicle, and the predicted driving trajectory, wherein any of the second driving parameters includes a predicted bird's-eye view state of the target vehicle, a predicted operation action, and an action reward signal. The predicted bird's-eye view state is a bird's-eye view state of the target vehicle at a next moment relative to a first moment predicted by the world model. The preset offline driving parameters include a bird's-eye view state of the target vehicle at a previous moment relative to the first moment, including: extracting historical bird's-eye view features of the preset offline driving parameters; Converting the historical bird's-eye view features into a semantic feature vector with spatial position encoding; Converting the predicted driving trajectory into an action encoding feature vector; Performing feature splicing processing on the action coding feature vector and the semantic feature vector to output spatiotemporal correlation features; Based on the spatiotemporal correlation features, calculating a cross attention between the action encoding feature vector and the semantic feature vector to simulate temporal and spatial changes of the target vehicle to obtain a plurality of second driving parameters of the target vehicle; Among them, any one of the second driving parameters includes the predicted bird's-eye view state, predicted operation action and action reward signal of the target vehicle, the predicted bird's-eye view state is the bird's-eye view state of the target vehicle at the next moment relative to the first moment predicted by the world model, and the preset offline driving parameters include the bird's-eye view state of the target vehicle at the previous moment relative to the first moment.

6. The method according to claim 5, characterized in that The step of converting the historical bird's-eye view features into a semantic feature vector with spatial position encoding includes: The attention mechanism of the pre-trained world model is used to perform feature dimension compression processing on the historical bird's-eye view features to obtain the semantic feature vector with spatial position encoding.

7. The method according to claim 1, characterized in that The determining of the target driving parameter from the plurality of second driving parameters of the target vehicle based on a preset regularized penalty algorithm, a preset number of critic algorithms, and the predicted driving trajectory includes: Calculating a value function of any second driving parameter using a preset number of critic algorithms, wherein the value function represents an expected cumulative reward of a predicted bird's-eye view state and a predicted operation action in the second driving parameter; sorting all the second driving parameters according to the magnitude of their value functions to determine the second driving parameter with the smallest value function; Calculating a loss function between the second driving parameter with the minimum value function and the predicted driving trajectory based on a preset regularized penalty algorithm; In response to the loss function being less than a preset loss threshold, the second driving parameter with the minimum value function is determined as the target driving parameter.

8. The method according to claim 7, characterized in that The determining a target operation action of the target vehicle according to the target driving parameter, and controlling the target vehicle to perform the target operation action to achieve automatic driving of the target vehicle, includes: determining the predicted operating action in the target driving parameter as the target operating action of the target vehicle; The steering angle value and acceleration value of the target vehicle are controlled through the target operation action to achieve automatic driving of the target vehicle.

9. An automatic driving device, characterized in that: The device comprises: a first determination module, configured to determine an anchor point trajectory in response to receiving a first driving parameter of a target vehicle, and determine a predicted driving trajectory based on the anchor point trajectory and the first driving parameter of the target vehicle, wherein the first driving parameter of the target vehicle includes a bird's-eye view state of the target vehicle at a first moment, a driving speed at the first moment, and a historical driving trajectory relative to the first moment, and the anchor point trajectory is a set of coordinates of reference trajectory points estimated according to a preset estimation method; a second determination module, configured to determine a plurality of second driving parameters of the target vehicle based on preset offline driving parameters, the first driving parameters of the target vehicle, and the predicted driving trajectory, wherein any of the second driving parameters includes a predicted bird's-eye view state of the target vehicle, a predicted operation action, and an action reward signal, the predicted bird's-eye view state being a bird's-eye view state of the target vehicle at a next moment relative to the first moment predicted by the world model, and the preset offline driving parameters including a bird's-eye view state of the target vehicle at a previous moment relative to the first moment; a third determination module, configured to determine a target driving parameter from the plurality of second driving parameters of the target vehicle based on a preset regularized penalty algorithm, a preset number of critic algorithms, and the predicted driving trajectory; The fourth determination module is used to determine the target operation action of the target vehicle according to the target driving parameter, and control the target vehicle to perform the target operation action to achieve automatic driving of the target vehicle.

10. An electronic device, characterized in that: include: a memory for storing a computer program product; A processor is configured to execute a computer program product stored in the memory, and when the computer program product is executed, implements the method described in any one of claims 1 to 8.

11. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method described in any one of claims 1 to 8 is implemented.

12. A computer program product comprising computer program instructions, characterized in that When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images