Unified framework for solving automatic driving track prediction and planning consistency based on world model

Abstract

The invention discloses a unified framework for solving automatic driving track prediction and planning consistency based on a world model. According to the method, through cooperative work of the automatic driving domain controller and the vehicle-mounted sensing system, end-to-end joint optimization of track prediction and planning in a complex traffic scene is realized, time sequence dependence and interaction dynamics among intelligent agents are accurately captured, and the prediction capability and robustness of a model are remarkably improved. The method comprises the following specific steps: firstly, constructing a generative world model, and generating potential future state representation by utilizing a behavior conditional and backtracking expansion technology; secondly, in combination with global modeling and a local convolutional network, multi-scale features are extracted, adaptive fusion is carried out, and a multi-modal prediction trajectory is generated; then, a multi-target planning model is adopted to integrate various driving indexes, and a track with the minimum loss function is generated; finally, path planning parameters are dynamically optimized through real-time environment perception and decision feedback, and the problems of prediction uncertainty and planning consistency of the automatic driving track are effectively solved.

Description

Technical Field

[0001] This invention relates to the field of trajectory prediction and motion planning technology in intelligent driving systems, specifically to a unified framework based on a world model to solve the consistency of trajectory prediction and planning in autonomous driving. Background Technology

[0002] With the rapid development of intelligent transportation systems, autonomous driving technology is iteratively upgrading from closed, controlled scenarios to open, complex environments. In this process, intelligent decision-making methods based on multi-objective optimization are increasingly becoming a research focus, providing efficient and adaptive solutions for uncertain dynamic environments. Motion prediction and planning is a core functional module in autonomous driving systems. By performing real-time predictive analysis of the behavior of surrounding traffic participants and generating optimal driving trajectories that satisfy multiple constraints such as safety, comfort, and operational efficiency, it plays a crucial role in improving the performance of autonomous driving systems in complex scenarios.

[0003] Traditional rule-based or heuristic methods, while widely used in commercial autonomous driving systems, exhibit significant limitations in handling high-dimensional, multi-variable dynamic traffic scenarios, struggling to effectively address the complex decision-making challenges posed by real-time multi-agent interactions and rapidly changing environments. Learning-based end-to-end decision-making techniques, such as behavior cloning and reinforcement learning, extract decision patterns from expert driving data or learn optimal strategies through interaction with the environment; however, these methods are highly dependent on the quality and distribution of training data and lack explicit safety mechanisms. Another data-driven end-to-end approach, while achieving significant progress in prediction accuracy by directly generating predicted trajectories and control signals from the scene context, still faces the problem of training and deployment environment distribution offsets in practical applications, resulting in insufficient model generalization ability and an inability to maintain stability in highly uncertain scenarios.

[0004] In recent years, world models have become an important research direction in motion prediction and planning due to their superior ability to model dynamic systems in uncertain environments. These models effectively capture temporal evolution patterns and agent interaction modes at a high-dimensional abstraction level by constructing a generative latent representation space of the environment. A typical world model architecture includes an encoder, a latent dynamics module, and a decoder, capable of learning state transition functions based on historical observation sequences and generating long-term time-domain predictions. However, existing world model methods still face two core technical challenges: first, the policy distribution shift problem, where agents make significantly different decisions in different traffic scenarios, and prediction errors accumulate over time, affecting decision stability; second, the computational complexity problem, although some studies have enhanced expressive power through complex generative models and game theory frameworks, the computational cost increases significantly, making it difficult to meet the real-time processing requirements of in-vehicle systems and limiting the feasibility of real-time deployment of such methods on resource-constrained autonomous driving computing platforms.

[0005] In intelligent transportation decision-making systems, onboard multimodal sensor networks are responsible for environmental data acquisition and real-time scene representation construction, while autonomous driving domain controllers perform tasks such as perception, prediction, and planning based on multi-source heterogeneous data. However, the inherent data uncertainty, asynchronous sampling, and information redundancy in the multi-sensor fusion process significantly increase the complexity of environmental modeling. In traditional systems, trajectory prediction and planning modules employ independent optimization frameworks, leading to insufficient information transfer between modules and inconsistent decision objectives, ultimately reducing the system's decision reliability and execution performance in complex and dynamic traffic environments. Therefore, designing an integrated framework that unifies the modeling of trajectory prediction and motion planning tasks while ensuring computational efficiency and real-time requirements, and achieving collaborative optimization and consistent decision-making between the two, is a challenging problem. Summary of the Invention

[0006] The technical problems this invention aims to solve are: inconsistencies in decision-making caused by independent optimization of trajectory prediction and motion planning modules in autonomous driving systems, and the issues of policy distribution offset and high computational complexity in world models. The purpose of this invention is to provide a unified framework based on a world model, which effectively solves the consistency problem between modules through end-to-end joint optimization, improves the system's environmental adaptability and safety in dynamic scenarios, and provides a new technical path for intelligent decision-making in high-level autonomous driving systems.

[0007] This invention is achieved through the following technical solution:

[0008] Step S1: Construct a predictive planning training model, initialize model parameters and safety error threshold, and model the optimization problem;

[0009] Step S2: Collect environmental data through the vehicle sensor system, and encode the vectorized map and vehicle historical trajectory by the processing unit;

[0010] Step S3: The autonomous driving domain controller constructs a potential future state representation based on a generative world model, where behavior conditionation is achieved by mapping perception data to the potential space through an encoder, and a differentiable backtracking unfolding mechanism dynamically adjusts the potential state through inverse coding.

[0011] Step S4: Apply global temporal modeling based on road curvature adaptive window to obtain global feature vectors;

[0012] Step S5: Using local interaction reasoning driven by a spatiotemporal graph convolutional network, extract spatial dependency features between agents to obtain local feature vectors;

[0013] Step S6: Adaptively fuse multi-scale features through a gated cross-attention mechanism to generate a unified feature representation;

[0014] Step S7: Based on the fused features, use the prediction model to predict the multimodal trajectories of each vehicle in the traffic scene;

[0015] Step S8: The planning model comprehensively considers driving indicators such as driving efficiency, comfort, traffic rules, traffic signals, and safety to generate the trajectory with the minimum loss function;

[0016] Step S9: Calculate the deviation between the actual trajectory and the predicted trajectory, adjust the real-time perception parameters through gradient backpropagation, generate the optimal driving decision, and feed it back to the training model.

[0017] Step S10: Repeat the method described in steps S3 to S9 until the training model converges.

[0018] Compared with the prior art, the present invention has the following significant advantages: (1) It effectively alleviates policy distribution shift and error accumulation through behavioral conditions and backtracking-unrolling mechanism, thereby improving policy diversity and robustness; (2) It adopts a global-local optimization strategy, combined with graph convolution and multi-scale attention, to balance computational efficiency and information fusion, thereby enhancing the adaptability in complex environments; (3) It introduces a real-time perception feedback loop to dynamically adjust decision weights, thereby significantly improving the model's response speed and stability in changing scenarios.

[0019] The present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0020] The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and form part of this application, do not constitute a limitation thereof. In the drawings:

[0021] Figure 1 This is a flowchart illustrating the unified framework for solving the consistency between autonomous driving trajectory prediction and planning based on the world model, as presented in this invention.

[0022] Figure 2 This is a diagram illustrating the application scenario of the unified framework in this invention.

[0023] Figure 3 This is a trajectory score map of three different scenarios within the unified framework of this invention embodiment.

[0024] Figure 4 This is a schematic diagram of the trajectory and speed changes of the unified framework in a lane-changing scenario in an embodiment of the present invention. Detailed Implementation

[0025] This invention proposes a unified framework for solving the consistency between trajectory prediction and planning in autonomous driving based on a world model. Specifically: First, a generative world model is constructed, utilizing behavioral conditionalization and backtracking unrolling techniques to generate representations of potential future states; second, by combining global curvature adaptive temporal modeling and a local dynamic spatiotemporal graph convolutional network, multi-scale features are extracted and adaptively fused using a gated cross-attention mechanism to generate multimodal predicted trajectories; then, a multi-objective programming model is employed to integrate various driving indicators and generate the trajectory with the minimum loss function; finally, path planning parameters are dynamically optimized through real-time environmental perception and decision feedback mechanisms. Figures 1-4 The specific steps are as follows:

[0026] Step S1: Construct a predictive planning training model, initialize model parameters and safety error threshold, and model the optimization problem;

[0027] Step S2: Collect environmental data through the vehicle sensor system, and encode the vectorized map and vehicle historical trajectory by the processing unit;

[0028] Step S3: The autonomous driving domain controller constructs a potential future state representation based on a generative world model, where behavior conditionation is achieved by mapping perception data to the potential space through an encoder, and a differentiable backtracking unfolding mechanism dynamically adjusts the potential state through inverse coding.

[0029] Step S4: Apply global temporal modeling based on road curvature adaptive window to obtain global feature vectors;

[0030] Step S5: Using local interaction reasoning driven by a spatiotemporal graph convolutional network, extract spatial dependency features between agents to obtain local feature vectors;

[0031] Step S6: Adaptively fuse multi-scale features through a gated cross-attention mechanism to generate a unified feature representation;

[0032] Step S7: Based on the fused features, use the prediction model to predict the multimodal trajectories of each vehicle in the traffic scene;

[0033] Step S8: The planning model comprehensively considers driving indicators such as driving efficiency, comfort, traffic rules, traffic signals, and safety to generate the trajectory with the minimum loss function;

[0034] Step S9: Calculate the deviation between the actual trajectory and the predicted trajectory, adjust the real-time perception parameters through gradient backpropagation, generate the optimal driving decision, and feed it back to the training model.

[0035] Step S10: Repeat the method described in steps S3 to S9 until the training model converges.

[0036] As a specific implementation method, the modeling of the optimization problem in step S1 includes the following technical elements:

[0037] Step S1.1: Establish the joint optimization objective function, whose mathematical expression is:

[0038]

[0039] Where τ represents the planned path, Y k This represents the predicted k-th possible trajectory. It corresponds to the actual trajectory, p k Let C(τ,Y) be the multimodal trajectory probability distribution output by the trajectory prediction model. k The cost function for path planning is defined as the predicted trajectory Y. k Total cost below;

[0040] Step S1.2: The path planning cost function C(τ,Y) k The calculation method for ) is as follows:

[0041]

[0042] Each of the sub-items satisfies: c collision c is the collision risk cost term calculated based on the Euclidean distance between trajectories. speed c is a penalty term for deviations from the expected speed of the planned path. comfort This represents the comfort cost term calculated jointly by acceleration and jerk. a1, a2, and a3 are learnable weighting coefficients.

[0043] In step S3, the generative world model is implemented through the following technical solutions:

[0044] Step S3.1: Behavioral Conditioning. Historical trajectories are fused with physical constraints through latent space modeling. The behavioral encoder employs feature concatenation and nonlinear transformation, represented as follows:

[0045]

[0046] Where S is the historical state sequence and U is the control command sequence. This represents a concatenation operation along the feature dimension, φ b W is a nonlinear correction function. b To learn the projection matrix, c b This is the bias term. Output behavioral feature b t As a conditional input to the potential state, it is used to constrain the dynamic feasibility of state transitions;

[0047] Furthermore, in step S3, the differentiable backtracking expansion mechanism is implemented through the following technical solution:

[0048] Step S3.2.1: Reverse temporal coding: based on the observed state o at the current time. t Gradient backpropagation is performed to correct the latent state of historical time tj, and the input features are constructed as follows:

[0049] o t-j =[s t-j ,u t-j [M]

[0050] Among them, s t-j For the original state of historical moment tj, u t-j M represents the control command at the corresponding moment, and M represents the contextual features of the vectorized map.

[0051] Step S3.2.2: Dynamic State Correction: Adjust the contribution of historical latent states based on attention weights, specifically as follows:

[0052]

[0053] Wherein, attention weight a j φ is calculated using the normalization function. r It is a nonlinear transformation network consisting of 3 layers of MLP.

[0054] Furthermore, in step S4, the global feature vector is implemented through the following technical solution;

[0055] Step S4.1: Calculate the average curvature of the historical trajectory:

[0056]

[0057] The velocity vector is defined as: v τ =[v τ cos θ τ ,v τ sin θ τ ] T , This is the acceleration vector. The curvature κ quantifies the degree of curvature of the trajectory; high curvature corresponds to frequent turns, while low curvature corresponds to straight travel.

[0058] Step S4.2: Dynamically adjust the timing window size based on curvature:

[0059]

[0060] Among them W base The base window size is ∈ = 1e -5 To prevent division by zero, W min and W max These are the lower and upper limits of the window, respectively.

[0061] Step S4.3: Divide the input feature sequence into non-overlapping windows according to window size W:

[0062]

[0063] in This represents a function for partitioning non-overlapping windows.

[0064] Step S4.4: Fuse global features through residual connections and layer normalization:

[0065]

[0066] in The representation shows layer normalization. W-MSA represents the multi-head self-attention calculation process within the window, and SW-MSA represents the displacement window self-attention calculation process. X s Features processed by window displacement;

[0067] Furthermore, in step S5, the local feature vector is implemented through the following technical solution:

[0068] Step S5.1: A spatiotemporal graph convolutional network based on Chebyshev polynomial approximation is used to process the spatial interaction relationships between agents. Its main recursive formula is:

[0069]

[0070] Where L t For Laplace matrix, This represents the feature representation of the l-th layer. These are learnable weight parameters;

[0071] Step S5.2: Local interaction features are defined as follows l represents the total number of layers in the spatiotemporal graph convolutional network.

[0072] Furthermore, in step S6, multi-scale fusion is achieved through the following technical solutions:

[0073] Step S6.1: Output the global feature Z from claim 4 g The local feature Z output by claim 5 l Projected to the query space, key space, and value space:

[0074] Q g =Z g W q ,K l =Z l W k V l =Z l W v ;

[0075] Step S6.2: Calculate the gating attention weights and fuse features:

[0076]

[0077] Where ⊙ represents element-wise multiplication, and σ(·) is the activation function.

[0078] Furthermore, in step S7, the multimodal trajectory is based on the unified feature representation Z generated according to claim 6. final A probabilistic generation model is constructed to output a multimodal trajectory distribution.

[0079] Furthermore, in step S8, the loss function is expressed as:

[0080] L=λ1L prediction +λ2L score +λ3L imitation +λ4L cost +λ5L uncertainty

[0081] in: For trajectory prediction loss, For the i-th agent, the k-th trajectory is predicted. For the corresponding true trajectory, measure the error between the predicted trajectory and the true trajectory;

[0082] To evaluate the loss probabilistically, p is used to optimize the accuracy of the trajectory probability distribution. k Let be the predicted probability of the k-th trajectory;

[0083] L imitation =smoothL1(τ-τ) gv To simulate loss, the planned trajectory τ is compared with the actual driving trajectory τ. gv The deviation between them guides the planning module to tend towards realistic driving behavior;

[0084] At the cost of loss, ensure that the generated control commands satisfy all physical and environmental constraints, where ω j It is the constraint weight, c j Indicates constraints;

[0085] Uncertainty loss is used to evaluate the reliability of the model in unseen scenarios.

[0086] Furthermore, in step S9, environmental perception and decision feedback are achieved through the following technical solutions:

[0087] Step S9.1: Calculate the deviation between the actual trajectory and the predicted trajectory;

[0088] Step S9.2: Update the model parameters based on the execution state feedback using the gradient backpropagation method:

[0089]

[0090] Where L is the comprehensive performance evaluation function, and η is the learning rate;

[0091] Step S9.3: Generate the optimal driving decision based on the updated parameters and feed the execution result back to the training model to form a closed-loop optimization mechanism.

[0092] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0093] Example

[0094] This example provides a unified framework for solving the consistency problem of autonomous driving trajectory prediction and planning based on a world model, which is described in detail below:

[0095] See Figure 2 The unified framework proposed in this invention achieves efficient decision-making through a perception-prediction-planning closed-loop mechanism, effectively solving the consistency problem between trajectory prediction and planning in autonomous driving.

[0096] The unified framework of this invention includes the following steps:

[0097] Step S1: Construct a predictive planning training model and initialize the system. This includes: initializing model parameters and safety error thresholds, setting specific parameters for road intersection turning scenarios, determining collision risk assessment thresholds, defining constraints for road planning decisions, and modeling the overall optimization problem.

[0098] Step S2: Collect environmental data through the vehicle sensor system, including static information such as traffic lights, road direction indicators, and speed limits, as well as dynamic information such as the real-time position and motion status of vehicles currently on the road. The processing unit fuses these data with the vector map and encodes them into structured feature vectors.

[0099] Step S3: The autonomous driving domain controller is implemented based on a generative world model. First, a kinematic bicycle model is used to describe the vehicle's kinematic characteristics. Then, a multi-head attention mechanism is used to explicitly model the spatial interaction patterns between vehicles and capture the mutual influence between different vehicles. At the same time, the behavior conditional encoder maps the perception data to the latent space and introduces a differentiable backtracking unfolding mechanism for long-term time-series dependent scenarios to dynamically adjust the latent state through inverse encoding.

[0100] Step S4: Global Temporal Modeling. A global temporal modeling method based on road curvature adaptive windows is applied. First, the average road curvature value is calculated, and then the temporal window size is dynamically adjusted according to the curvature magnitude. Smaller time windows are used in turning areas to capture fine trajectory changes, while larger time windows are used in straight road sections to improve efficiency. A displacement window mechanism enhances cross-window information exchange, ensuring the continuity of temporal information. Finally, through residual connectivity and layer normalization feature fusion, a global feature vector reflecting the road topology and global traffic flow characteristics is generated.

[0101] Step S5: Local Interaction Reasoning. Local interaction reasoning is performed using a spatiotemporal graph convolutional network to extract spatial dependency features between agents. When constructing the spatiotemporal graph, vehicles are represented as nodes, and the spatiotemporal interaction strength is encoded using edge weights. The adjacency matrix is ​​dynamically updated based on the spatiotemporal distance between vehicles. Subsequently, a normalized Laplacian matrix is ​​used for graph convolution operations, and local interaction features between vehicles are effectively extracted through K-order Chebyshev polynomial approximation, ultimately generating a local feature vector.

[0102] Step S6: Adaptively fuse multi-scale features through a gated cross-attention mechanism to dynamically adjust the contribution of local information to the global representation. In the left-turn scenario, the model increases the attention weight to the intersection area and oncoming vehicles, and reduces the weight to areas far from the decision point, generating a unified feature representation containing multi-scale information;

[0103] Step S7: Based on the fused features, the prediction model performs multimodal predictions of the future trajectories of each vehicle in the traffic scenario. Diverse trajectory distributions are generated through latent variable sampling, with each trajectory accompanied by a corresponding trajectory score, providing a reliable basis for the next planning decision.

[0104] Step S8: Based on the prediction results, the planning model comprehensively considers multiple driving indicators such as driving efficiency, comfort level, traffic rule compliance, traffic signal status, and safe distance, and finally generates the optimal driving trajectory that minimizes the loss function value.

[0105] Step S9: The model calculates the deviation between the actual trajectory and the predicted trajectory, adjusts the real-time perception parameters through gradient backpropagation, generates the optimal driving decision, and feeds it back to the training model. The model perceives changes in the trajectories of surrounding vehicles in real time, optimizes left-turn decisions, and feeds them back to the model, continuously optimizing parameters to adapt to the dynamic traffic environment, forming a closed-loop optimization mechanism;

[0106] Step S10: Repeat the methods described in steps S3 to S9 until the training model converges. Through continuous iteration, the model continuously improves its prediction and planning capabilities for various scenarios, forming a unified framework that ensures consistency between prediction and planning, thereby achieving safe, comfortable, and efficient autonomous driving decisions.

[0107] Simulation experiments were conducted on the unified framework for solving the consistency of autonomous driving trajectory prediction and planning based on the world model described in this invention. The results are analyzed below:

[0108] Figure 3 This invention demonstrates the ability of the unified framework described in this paper to predict and evaluate candidate behaviors in various driving scenarios. In each scenario, the model scores candidate schemes based on their proximity to the real trajectory and their feasibility in a specific traffic environment. It can be seen that the unified framework described in this invention achieves a systematic improvement in safety, trajectory smoothness, and prediction accuracy in closed-loop control scenarios, fully demonstrating decision robustness and generalization ability in highly dynamic and uncertain traffic environments, and providing a more promising solution for integrated planning and prediction in complex multi-agent scenarios.

[0109] Figure 4 A scenario of an autonomous vehicle changing lanes to the left was simulated to visualize trajectory and speed changes. The autonomous vehicle, initially in the right lane, will merge into the left lane. During this process, it can be observed that the autonomous vehicle adjusts its lane-changing behavior based on the behavior of surrounding vehicles. The speed curves show that within the unified framework described in this invention, the autonomous vehicle's speed is smoothly adjusted and a safe distance is maintained, ensuring that there are no sudden changes in speed during lane changes.

[0110] The beneficial effects of the technical solution provided by the embodiments of the present invention are as follows: (1) The problem of policy distribution offset is effectively solved by setting behavioral conditions, historical trajectories are directly encoded to generate diversified policies, and the backtracking-unrolling mechanism is used to reduce unnecessary error accumulation in the model set; (2) Based on the global-local optimization method, the historical trajectory and control variable state are dynamically updated, and combined with graph convolutional networks and multi-scale self-attention mechanisms, the deep fusion of local and global information is realized, which significantly enhances the adaptability of the model in complex scenarios while maintaining computational efficiency; (3) With the help of real-time perception feedback loop, the perception-decision-feedback cycle is successfully simulated, and the weight coefficients can be dynamically adjusted to improve the model's response capability in changing environments. This closed-loop system ensures efficient decision-making and trajectory optimization, laying a solid foundation for realizing autonomous driving in diverse and unpredictable real-world scenarios.

[0111] For those skilled in the art, the present invention is not limited to the exemplary details of the above embodiments. The scope of protection of the present invention should be defined by the scope of the appended claims and not by the description of the above embodiments. Any technical solutions based on the present invention and any equivalent modifications made thereto should fall within the scope of protection of the present invention.

Claims

1. A unified framework based on a world model for solving the consistency between trajectory prediction and planning in autonomous driving, applied to intelligent vehicle systems for intelligent decision-making tasks in complex dynamic traffic scenarios, characterized in that... Includes the following steps: Step S1: Construct a predictive planning training model, initialize model parameters and safety error threshold, and model the optimization problem; Step S2: Collect environmental data through the vehicle sensor system, and encode the vectorized map and vehicle historical trajectory by the processing unit; Step S3: The autonomous driving domain controller constructs a potential future state representation based on a generative world model, where behavior conditionation is achieved by mapping perception data to the potential space through an encoder, and a differentiable backtracking unfolding mechanism dynamically adjusts the potential state through inverse coding. Step S4: Apply global temporal modeling based on road curvature adaptive window to obtain global feature vectors; Step S5: Using local interaction reasoning driven by a spatiotemporal graph convolutional network, extract spatial dependency features between agents to obtain local feature vectors; Step S6: Adaptively fuse multi-scale features through a gated cross-attention mechanism to generate a unified feature representation; Step S7: Based on the fused features, use the prediction model to predict the multimodal trajectories of each vehicle in the traffic scene; Step S8: The planning model comprehensively considers driving indicators such as driving efficiency, comfort, traffic rules, traffic signals, and safety to generate the trajectory with the minimum loss function; Step S9: Calculate the deviation between the actual trajectory and the predicted trajectory, adjust the real-time perception parameters through gradient backpropagation, generate the optimal driving decision, and feed it back to the training model. Step S10: Repeat the method described in steps S3 to S9 until the training model converges.

2. The unified framework for solving the consistency of autonomous driving trajectory prediction and planning based on a world model as described in claim 1, characterized in that: The modeling of the optimization problem in step S1 includes the following technical elements: Step S1.1: Establish the joint optimization objective function, whose mathematical expression is: Where τ represents the planned path, Y k This represents the predicted k-th possible trajectory. It corresponds to the actual trajectory, p k Let C(τ,Y) be the multimodal trajectory probability distribution output by the trajectory prediction model. k The cost function for path planning is defined as the predicted trajectory Y. k Total cost below; Step S1.2: The path planning cost function C(τ, Y) k The calculation method for ) is as follows: Each of the sub-items satisfies: c collision c is the collision risk cost term calculated based on the Euclidean distance between trajectories. speed c is a penalty term for deviations from the expected speed of the planned path. comfort This represents the comfort cost term calculated jointly by acceleration and jerk. a1, a2, and a3 are learnable weighting coefficients.

3. The unified framework according to claim 1 or 2, characterized in that: The generative world model in step S3 is implemented through the following technical solution: Step S3.1: Behavioral Conditioning. Historical trajectories are fused with physical constraints through latent space modeling. The behavioral encoder employs feature concatenation and nonlinear transformation, represented as follows: Where S is the historical state sequence and U is the control command sequence. This represents a concatenation operation along the feature dimension, φ b W is a nonlinear correction function. b To learn the projection matrix, c b This is the bias term. Output behavioral feature b t As a conditional input to the potential state, it is used to constrain the dynamic feasibility of state transitions; Step S3.2: The execution process of the differentiable backtracking expansion mechanism includes: Step S3.2.1: Reverse temporal coding: based on the observed state o at the current time. t Gradient backpropagation is used to correct the latent state of historical time tj, and the input features are constructed as follows: o t-j =[s t-j ,u t-j ,M] Among them, s t-j For the original state of historical moment tj, u t-j M represents the control command at the corresponding moment, and M represents the contextual features of the vectorized map. Step S3.2.2: Dynamic State Correction: Adjust the contribution of historical latent states based on attention weights, specifically as follows: Wherein, attention weight a j φ is calculated using the normalization function. r It is a nonlinear transformation network consisting of 3 layers of MLP.

4. The unified framework according to claim 1 or 2, characterized in that: The global feature vector in step S4 is implemented through the following technical solution: Step S4.1: Calculate the average curvature of the historical trajectory: The velocity vector is defined as: v τ =[v τ cosθ τ v τ sinθ τ ] T , This is the acceleration vector. The curvature κ quantifies the degree of curvature of the trajectory; high curvature corresponds to frequent turns, while low curvature corresponds to straight travel. Step S4.2: Dynamically adjust the timing window size based on curvature: Among them W base The base window size, ∈1e -5 To prevent division by zero, W min and W max These are the lower and upper limits of the window, respectively. Step S4.3: Divide the input feature sequence into non-overlapping windows according to window size W: in This represents a function for partitioning non-overlapping windows. Step S4.4: Fuse global features through residual connections and layer normalization: in The representation shows layer normalization, W-MSA for multi-head self-attention calculation within the window, and SW-MSA for the calculation process of displacement window self-attention. X s These are features processed by window displacement.

5. The unified framework according to claim 1 or 2, characterized in that: The local feature vector in step S5 is achieved through the following technical solution: Step S5.1: A spatiotemporal graph convolutional network based on Chebyshev polynomial approximation is used to process the spatial interaction relationships between agents. Its main recursive formula is: Where L t For Laplace matrix, This represents the feature representation of the l-th layer. These are learnable weight parameters; Step S5.2: Local interaction features are defined as follows l represents the total number of layers in the spatiotemporal graph convolutional network.

6. The unified framework according to claim 4 or 5, characterized in that: The multi-scale fusion in step S6 is achieved through the following technical solution: Step S6.1: Output the global feature Z from claim 4 g The local feature Z output by claim 5 l Projected to the query space, key space, and value space: Q g =Z g W q ,K l =Z l W k ,V l =Z l W v Step S6.2: Calculate the gating attention weights and fuse features: Where ⊙ represents element-wise multiplication, and σ(·) is the activation function.

7. The unified framework according to claim 1 or 2, characterized in that: The multimodal trajectory in step S7 is based on the unified feature representation Z generated in claim 6. final A probabilistic generation model is constructed to output a multimodal trajectory distribution.

8. The unified framework according to claim 1 or 2, characterized in that: The loss function in step S8 is expressed as: L=λ1L prediction +λ2L score +λ3L imitation +λ4L cost +λ5L uncertainty in: For trajectory prediction loss, For the i-th agent, the k-th trajectory is predicted. For the corresponding true trajectory, measure the error between the predicted trajectory and the true trajectory; To evaluate the loss probabilistically, p is used to optimize the accuracy of the trajectory probability distribution. k Let be the predicted probability of the k-th trajectory; L imitation =smoothL1(τ-τ) gv To simulate loss, the planned trajectory τ is compared with the actual driving trajectory τ. gv The deviation between them guides the planning module to tend towards realistic driving behavior; At the cost of loss, ensure that the generated control commands satisfy all physical and environmental constraints, where ω j It is the constraint weight, c j Indicates constraints; Uncertainty loss is used to evaluate the reliability of the model in unseen scenarios.

9. The unified framework according to claim 1 or 2, characterized in that: Step S9 achieves environmental perception and decision feedback through the following process: Step S9.1: Calculate the deviation between the actual trajectory and the predicted trajectory; Step S9.2: Update the model parameters based on the execution state feedback using the gradient backpropagation method: Where L is the comprehensive performance evaluation function, and η is the learning rate; Step S9.3: Generate the optimal driving decision based on the updated parameters and feed the execution result back to the training model to form a closed-loop optimization mechanism.

Images