Space-time interaction trajectory prediction method based on trajectory lane consistency

Abstract

The invention provides a space-time interaction trajectory prediction method based on trajectory lane consistency, and the method comprises the steps: carrying out the feature extraction of an input target agent historical trajectory and a high-precision map through an embedded and space-time interaction module, and predicting the lane probability distribution of each time step through a lane section positioning module; and performing final track generation and track decoding by integrating modal information and local context in the fusion refining module, and outputting a multi-modal future track result. Historical sequence information and high-precision map elements are fully utilized in the two stages of encoding and decoding; according to the method, the subject history and the lane sequence where the subject history is located are deeply fused, and map information can be utilized more efficiently.

Description

Technical Field

[0001] This invention relates to the field of intelligent driving technology, and in particular to a spatiotemporal interactive trajectory prediction method based on trajectory-lane consistency. Background Technology

[0002] Autonomous driving systems need to generate multimodal, feasible, and stable future trajectories within a short period of time based on historical motion sequences, surrounding interactions, and the road environment. This problem is commonly referred to as motion prediction or trajectory prediction.

[0003] Existing methods for scene representation and map utilization have undergone several rounds of evolution, from rasterized bird's-eye view (BEV) to vectorized multi-polyline representation, and then to sequence / graph structure modeling with attention mechanisms at its core. Early rasterized methods were convenient for obtaining local semantics using convolutional networks, but they suffered from problems such as quantization errors and insufficient long-range receptive fields. Subsequently, vectorized representations modeled lanes and trajectories as polylines / graphs, combined with graph neural networks or Transformers to improve topological fidelity and interaction modeling capabilities. In recent years, mainstream methods have further unified the handling of multi-agent and multi-element interactions in voxelized / vectorized scenes using a query / attention paradigm.

[0004] However, the above methods still have structural deficiencies in the use of map information. They usually treat lanes as static contexts attached to the main representation, making it difficult to fully reflect the map's modulating effect on motion patterns. Summary of the Invention

[0005] This invention provides a spatiotemporal interactive trajectory prediction method based on trajectory-lane consistency, which addresses the structural deficiencies in the utilization of map information in existing technologies. It achieves a trajectory prediction method that can deeply integrate the subject's history with its lane sequence and utilize map information more efficiently.

[0006] This invention provides a spatiotemporal interactive trajectory prediction method based on trajectory-lane consistency, comprising: After initial feature encoding and spatiotemporal interaction modeling of the historical trajectory of the agent and the scene map in the driving scenario, the context features of the scene are obtained, which include lane segment features and agent features. The context features of the scene are input into the spatiotemporal semantic consistency module to obtain the refined temporal query features and the filtered scene context output by the spatiotemporal semantic consistency module. The refined temporal query features represent the future temporal state that integrates lane semantics, and the filtered scene context represents the local interactive environment composed of high-probability lane segments. The contextual features of the scene are input into the hierarchical multimodal decoder. In the first stage, the hierarchical multimodal decoder updates the pattern query and generates candidate trajectories through the interaction between the pattern query and the scene context. In the second stage, the updated pattern query is fused with the refined temporal query features and the filtered scene context to decode several refined trajectories and their corresponding probability distributions. The number of trajectories is the same as the number of pattern queries, and each pattern query represents a driving intention.

[0007] According to the spatiotemporal interaction trajectory prediction method based on trajectory lane consistency provided by the present invention, the step of inputting the contextual features of the scene into a spatiotemporal semantic consistency module to obtain the refined temporal query features output by the spatiotemporal semantic consistency module specifically includes: Generate a time-series query for a future time period, and perform bidirectional time-series modeling on the time-series query for the future time period and the context feature representation of the received scene to obtain a spatiotemporal semantic state query vector, wherein the time-series query for the future time period includes the time-series query vector for each time step of the future time period. For each time step in the future period, calculate the compatibility score between its spatiotemporal semantic state query vector and lane segment features to obtain the lane segment probability distribution, which represents the probability that the target agent belongs to each lane segment at each time step. A preset number of lane segments with the highest compatibility scores are selected to form a candidate interaction set. The candidate interaction set is used as a key and value. The spatiotemporal semantic state query vector of the corresponding time step is used as the query to perform cross-attention operation and obtain the refined query representation of the time step. By integrating the refined query representations of each time step, the refined time-series query features are obtained.

[0008] According to the spatiotemporal interaction trajectory prediction method based on trajectory lane consistency provided by the present invention, the step of performing bidirectional temporal modeling on the temporal query of the future time period and the contextual feature representation of the received scene to obtain the spatiotemporal semantic state query vector specifically includes: Using the time-series query of the future period as the query item and the contextual feature representation of the scene as the key and value, a cross-attention operation is performed to update the time-series query of the future period in order to obtain environmental semantic information; The future time-series queries obtained after cross-attention are sequentially input into the bidirectional time-series modeling module to model the dependencies between adjacent time steps, thereby obtaining the spatiotemporal semantic state query vector.

[0009] According to the spatiotemporal interaction trajectory prediction method based on trajectory lane consistency provided by the present invention, the step of inputting the context features of the scene into a spatiotemporal semantic consistency module to obtain the filtered scene context output by the spatiotemporal semantic consistency module specifically includes: The highest-scoring lane segments are combined and deduplicated to obtain locally important regions. The lane segment features and agent features corresponding to the locally important regions are then selected from the context features of the scene to form the filtered scene context.

[0010] According to the spatiotemporal interaction trajectory prediction method based on trajectory-lane consistency provided by the present invention, the step of obtaining the contextual features of the scene after initial feature encoding and spatiotemporal interaction modeling of the historical trajectory of the agent and the scene map in the driving scenario specifically includes: Initial feature encoding is performed on the historical trajectory of the agent and the scene map in the driving scenario to obtain the topological features representing the scene map and the motion features representing the historical trajectory of the agent. The topological features and motion features are concatenated and input into a scene-level Transformer for joint modeling to obtain an initial context feature representation that includes the global spatiotemporal relationships of the scene. For each historical moment, the states of all agents in the scene are associated with their nearest lane segments to construct a lane sequence, which is then extracted into a summary vector by the time sequence module. After fusing the summary vector with the residual of the agent's initial context feature representation, it is concatenated with the initial context feature representation of the lane segment and input into the Transformer for a second interaction to obtain the context feature representation of the scene.

[0011] According to the present invention, a spatiotemporal interactive trajectory prediction method based on trajectory lane consistency is provided, wherein the spatiotemporal semantic consistency module aligns the predicted future temporal state with the high-probability lane segment through a loss function during training.

[0012] The present invention also provides a spatiotemporal interactive trajectory prediction device based on trajectory lane consistency, comprising: The feature module is used to perform initial feature encoding and spatiotemporal interaction modeling on the historical trajectory of the agent and the scene map in the driving scenario to obtain the context features of the scene. The context features of the scene include lane segment features and agent features. The encoding module is used to input the context features of the scene into the spatiotemporal semantic consistency module to obtain the refined temporal query features and the filtered scene context output by the spatiotemporal semantic consistency module. The refined temporal query features represent the future temporal state that incorporates lane semantics, and the filtered scene context represents the local interactive environment composed of high-probability lane segments. The decoding module is used to input the context features of the scene into the hierarchical multimodal decoder. In the first stage, the hierarchical multimodal decoder updates the pattern query and generates candidate trajectories through the interaction between the pattern query and the scene context. In the second stage, the updated pattern query is fused with the refined temporal query features and the filtered scene context to decode several refined trajectories and their corresponding probability distributions. The number of trajectories is the same as the number of pattern queries, and each pattern query represents a driving intention.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the spatiotemporal interactive trajectory prediction method based on trajectory lane consistency as described above.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the spatiotemporal interactive trajectory prediction method based on trajectory lane consistency as described above.

[0015] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the spatiotemporal interactive trajectory prediction method based on trajectory lane consistency as described above.

[0016] The present invention provides a spatiotemporal interactive trajectory prediction method based on trajectory-lane consistency. By introducing spatiotemporal lane interaction at the encoding stage, a time-step-level association is established between the agent's historical sequence and the corresponding lane sequence, explicitly injecting dynamic memory aligned with "lane-time." This allows the agent's representation to simultaneously perceive historical movement trends and lane geometric constraints, thereby achieving a more semantically consistent scene understanding. At the decoding stage, a hierarchical semantic reasoning process is constructed, sequentially passing through trajectory proposal generation, lane segment probabilistic localization, local context filtering based on lane distribution, and multimodal fusion refinement, achieving lane semantic alignment and scene importance filtering in the prediction stage. This mechanism not only improves the model's spatial consistency and semantic interpretability at the lane level but also achieves a better balance between information utilization efficiency, prediction accuracy, and trajectory diversity, significantly enhancing prediction stability and generalization ability in complex road scenarios. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is one of the flowcharts of the spatiotemporal interactive trajectory prediction method based on trajectory lane consistency provided by the present invention; Figure 2 This is the second flowchart of the spatiotemporal interactive trajectory prediction method based on trajectory lane consistency provided by the present invention; Figure 3 This is a schematic diagram of lane segment localization in the spatiotemporal interactive trajectory prediction method based on trajectory lane consistency provided by the present invention; Figure 4 This is a schematic diagram of the spatiotemporal interactive trajectory prediction device based on trajectory lane consistency provided by the present invention; Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0020] The following is combined Figures 1 to 3 This invention introduces a spatiotemporal interactive trajectory prediction method based on trajectory-lane consistency, such as... Figure 1 As shown, it includes: Step 101: After initial feature encoding and spatiotemporal interaction modeling of the historical trajectory of the intelligent agent and the scene map in the driving scenario, the context features of the scene are obtained. The context features of the scene include lane segment features and intelligent agent features. The historical trajectory is the trajectory of an intelligent agent during a historical period, and its duration is determined based on experience.

[0021] Optionally, the historical trajectory length is set to 5 seconds (50 frames in total).

[0022] The intelligent agents in a driving scenario include the target intelligent agent that needs to perform trajectory prediction and other intelligent agents present in the scenario.

[0023] Before encoding the historical trajectory and scene map of the intelligent agent, the input high-precision map data and the historical trajectory data of each intelligent agent in the driving scenario can be uniformly normalized in terms of coordinates.

[0024] Optionally, a local coordinate system is established with the position of the target agent in the last frame of the historical time series as the origin and its orientation direction as the y-axis.

[0025] By translation and rotation, all agents and map elements are unified to this local coordinate system. The historical trajectory of each agent is vectorized, and the coordinate difference between two adjacent time steps is calculated to obtain the motion vector sequence of the agent at each time step.

[0026] After completing coordinate alignment and vectorization, feature encoding is performed on the polyline segments of the scene map and the historical trajectory of each agent.

[0027] The initial feature encoding involves using encoding modules to encode both the scene map and the agent's historical trajectory, resulting in feature codes for the scene map and the agent, respectively. Spatiotemporal interaction modeling then fuses the agent's lane sequence and historical trajectory features, embedding each agent's lane interaction history into the resulting scene context features.

[0028] Step 102: Input the context features of the scene into the spatiotemporal semantic consistency module to obtain the refined temporal query features and the filtered scene context output by the spatiotemporal semantic consistency module. The refined temporal query features represent the future temporal state that integrates lane semantics, and the filtered scene context represents the local interactive environment composed of high-probability lane segments. Furthermore, the extracted contextual features of the scene are first input into the spatiotemporal semantic consistency module.

[0029] like Figure 2 As shown, the spatiotemporal semantic consistency module first predicts the trajectory of the agent at each time step in the future period as the query item, and updates the query by interacting with the contextual features of the scene through the predicted trajectory. Then, it performs bidirectional time modeling on the updated query to generate a dynamic representation of continuous time and update the query.

[0030] Based on this, the lane segment localization module in the spatiotemporal semantic consistency module determines the most likely lane segments where the target agent will appear in each time step, and uses the lane segments to interact with the features after bidirectional time modeling to obtain refined temporal query features, which serve as input for subsequent trajectory refinement.

[0031] Furthermore, the most likely lane segments of all time steps are integrated as high-probability lane segments to represent the lanes most likely to appear for the target agent. The high-probability lane segments are used to filter the context representation of the scene, and the filtered scene context is used for subsequent trajectory refinement.

[0032] Step 103: Input the context features of the scene into the hierarchical multimodal decoder. In the first stage of the hierarchical multimodal decoder, the pattern query is updated and candidate trajectories are generated through the interaction between the pattern query and the scene context. In the second stage, the updated pattern query is fused with the refined temporal query features and the filtered scene context to decode several refined trajectories and their corresponding probability distributions. The number of trajectories is the same as the number of pattern queries, and each pattern query represents a driving intention.

[0033] During the decoding phase, the hierarchical multimodal decoder first receives the contextual features of the scene, and interacts each trained pattern query with the contextual features of the scene to capture global semantics and interaction information, which is then mapped by the decoder. k There are 10 candidate future trajectories and their probabilities. The number of mapped subsequent future trajectories is the same as the number of pre-defined pattern queries.

[0034] Each pattern query corresponds to a potential driving intention, which is a pre-trained learnable pattern query vector.

[0035] Specifically, such as Figure 2 As shown, in this process, k Each learnable pattern query is used as Q. The received context features of the scene are used as keys and values, a cross-attention operation is performed, and the pattern query is updated. Then, the... k Each pattern query performs a self-attention operation on itself and updates the pattern query. Then, the corresponding candidate future trajectories and their probabilities are obtained through the decoder mapping.

[0036] In the above manner, the trained pattern query is updated in this candidate future trajectory prediction task. In the second stage of the decoder, such as... Figure 2 As shown, the updated pattern query is first broadcast and element-level added to the refined time-series query feature representation in the time dimension to form a fused query vector.

[0037] The fused query vector is then interacted with the filtered scene context in the decoder's fusion and refinement module, and further refined through self-attention to capture spatial constraints and dynamic relationships within the local context. After the interaction and refinement are complete, the output is... k The refined trajectory and its corresponding probability distribution are used as the prediction result.

[0038] This invention introduces spatiotemporal lane interaction at the encoding stage, establishing a time-step-level association between the agent's historical sequence and the corresponding lane sequence. This injects dynamic memory aligned with "lane-time," enabling the agent's representation to simultaneously perceive historical movement trends and lane geometric constraints, thus achieving a more semantically consistent scene understanding. At the decoding stage, a hierarchical semantic reasoning process is constructed, sequentially involving trajectory proposal generation, lane segment probabilistic localization, local context filtering based on lane distribution, and multimodal fusion refinement. This achieves lane semantic alignment and scene importance filtering during the prediction phase. This mechanism not only improves the model's spatial consistency and semantic interpretability at the lane level but also achieves a better balance between information utilization efficiency, prediction accuracy, and trajectory diversity, significantly enhancing prediction stability and generalization ability in complex road scenarios.

[0039] In the spatiotemporal interactive trajectory prediction method based on trajectory lane consistency of the present invention, the step of inputting the contextual features of the scene into the spatiotemporal semantic consistency module to obtain the refined temporal query features output by the spatiotemporal semantic consistency module specifically includes: Generate a time-series query for a future time period, and perform bidirectional time-series modeling on the time-series query for the future time period and the context feature representation of the received scene to obtain a spatiotemporal semantic state query vector, wherein the time-series query for the future time period includes the time-series query vector for each time step of the future time period. like Figure 2 As shown, the spatiotemporal semantic consistency module first predicts the position of the target agent at each time step in the future period, obtains the temporal query vector for each time step in the future period, and forms the temporal query for the future period.

[0040] Then, the time-series query for future time periods interacts with the contextual features of the received scene to obtain environmental semantic information. After the interaction, the updated time-series query for future time periods is sequentially input into the bidirectional time-series modeling module to model the dependencies between adjacent time steps and generate a dynamic feature representation in continuous time, namely the spatiotemporal semantic state query vector. .

[0041] For each time step in the future period, calculate the compatibility score between its spatiotemporal semantic state query vector and lane segment features to obtain the lane segment probability distribution, which represents the probability that the target agent belongs to each lane segment at each time step. For each time step of the future prediction, the spatiotemporal semantic state query after bidirectional time series modeling is interactively calculated with the features of all lane segments in the map to obtain the compatibility score between each lane segment and the query vector at that time step: ; In the formula, This represents the compatibility score between the features of time step t and lane segment j. L j express N m A collection of lane segments; , representing the interaction features between lane segment features and spatiotemporal semantic state query vector; Represents a spatiotemporal semantic state query vector.

[0042] Then, the compatibility score is normalized to obtain the lane segment probability distribution, which is used to represent the probability that the target agent belongs to each lane segment at that time step: .

[0043] A preset number of lane segments with the highest compatibility scores are selected to form a candidate interaction set. The candidate interaction set is used as a key and value. The spatiotemporal semantic state query vector of the corresponding time step is used as the query to perform cross-attention operation and obtain the refined query representation of the time step. By integrating the refined query representations of each time step, the refined time-series query features are obtained.

[0044] For each future moment t ,according to The candidate interaction set is formed by selecting the preset number of lane segments with the highest scores. And perform a local interaction alignment with the spatiotemporal semantic state query at the corresponding time step to refine the query representation: ; ; The specific number of preset items can be set according to task requirements or computing resources; in this embodiment, it is set to 4.

[0045] By using the above method, a refined query representation updated after performing cross-attention operations at each time step is obtained. By integrating the refined query representations of all time steps, the refined temporal query features used for decoding can be obtained.

[0046] In the spatiotemporal interactive trajectory prediction method based on trajectory lane consistency of this invention, the step of performing bidirectional temporal modeling on the temporal query of the future time period and the contextual feature representation of the received scene to obtain the spatiotemporal semantic state query vector specifically includes: Using the time-series query of the future period as the query item and the contextual feature representation of the scene as the key and value, a cross-attention operation is performed to update the time-series query of the future period in order to obtain environmental semantic information; Specifically, the time-series query for future periods is used as the query term, and the contextual features of the scene are used as the key and value. A cross-attention operation is performed to update the time-series query for future periods in order to obtain environmental semantic information: ; In the formula, , T f This represents a time step in the future period. In this implementation, the predicted duration of the future period is 6 seconds, totaling 60 frames, corresponding to 60 time steps.

[0047] The future time-series queries obtained after cross-attention are sequentially input into the bidirectional time-series modeling module to model the dependencies between adjacent time steps, thereby obtaining the spatiotemporal semantic state query vector.

[0048] Updated Input the bidirectional timing modeling module to perform modeling: ; In the formula, BiMamba is a bidirectional time series modeling module.

[0049] In the spatiotemporal interactive trajectory prediction method based on trajectory lane consistency of the present invention, the step of inputting the contextual features of the scene into the spatiotemporal semantic consistency module to obtain the filtered scene context output by the spatiotemporal semantic consistency module specifically includes: The highest-scoring lane segments are combined and deduplicated to obtain locally important regions. The lane segment features and agent features corresponding to the locally important regions are then selected from the context features of the scene to form the filtered scene context.

[0050] Using the above method, at each time step, the results are filtered out. n The lane segment with the highest compatibility score will be integrated with future time periods. t At each time step, we obtain t × n Each lane segment is combined and deduplicated to obtain a local importance region composed of high-confidence lane segments.

[0051] Based on this, the context features of the scene are filtered by the local importance region, and only the lane segment features and agent features corresponding to the local importance region in the context features of the scene are retained to form the filtered scene context.

[0052] In the spatiotemporal interactive trajectory prediction method based on trajectory-lane consistency of this invention, the step of obtaining the contextual features of the scene after initial feature encoding and spatiotemporal interaction modeling of the historical trajectory of the intelligent agent and the scene map in the driving scenario specifically includes: Initial feature encoding is performed on the historical trajectory of the agent and the scene map in the driving scenario to obtain the topological features representing the scene map and the motion features representing the historical trajectory of the agent. Specifically, the historical trajectory of the intelligent agent and the polyline segment of the scene map in the driving scenario are encoded first.

[0053] Optionally, the geometric and topological features of the map polylines are extracted by the polyline encoding module and used as the topological features of the scene map. : .

[0054] Optionally, motion features are extracted from the agent's historical trajectory using a historical encoding module. : .

[0055] In the above formula, This indicates the corresponding encoding module. M Represents scene map data, A This represents the agent's historical trajectory. In this implementation, the line encoding module is PointNet, and the history encoding module is UniMamba.

[0056] The topological features and motion features are concatenated and input into a scene-level Transformer for joint modeling to obtain an initial context feature representation that includes the global spatiotemporal relationships of the scene. Based on this, the encoded topological features and motion features are concatenated and then input into a scene-level Transformer model for joint modeling, resulting in an initial contextual feature representation that includes the global spatiotemporal relationships of the scene. : ; In the formula, PE represents the position code, specifically the coordinates of each agent at the last historical time step before differential processing.

[0057] The obtained initial contextual features are used as input for subsequent lane interaction and decoding prediction.

[0058] For each historical moment, the states of all agents in the scene are associated with their nearest lane segments to construct a lane sequence, which is then extracted into a summary vector by the time sequence module. Based on the above, spatiotemporal lane interaction coding is performed. Specifically, although the basic features extracted separately can effectively describe individual entities, such features may ignore the spatiotemporal interaction between the agent and the lane. Therefore, in this embodiment, a spatiotemporal lane interaction encoder is used to associate each agent with its current lane segment, and the corresponding lane sequence representation is integrated through residual addition to obtain a richer embedding.

[0059] In one specific implementation, for each historical moment The state of all agents in the driving scenario is associated with their nearest lane segment to construct a lane sequence. The lane sequence includes the relative position of the agent from the center point of the lane and the relative orientation.

[0060] Optionally, first calculate the distance from each agent to the lane centerline, and determine the lane line closest to the agent. : ; In the formula, It is the first j The centerline of each lane segment It calculates the Euclidean distance from a point to a line.

[0061] The lane sequence is extracted into a summary vector by the time-series module: ; In the formula, Z This represents the extracted summary vector. L seq Indicates lane sequence, This indicates a timing module.

[0062] Optionally, the timing module is UniMamba.

[0063] After fusing the summary vector with the residual of the agent's initial context feature representation, it is concatenated with the initial context feature representation of the lane segment and input into the Transformer for a second interaction to obtain the context feature representation of the scene.

[0064] Based on this, the summary vector is combined with the agent's initial context features. This represents the initial contextual feature representation of the debut segment after residual fusion. The data is concatenated and fed into a Transformer for a second interaction to obtain a contextual feature representation of the scene. : ; ; In the formula, Right now , Right now .

[0065] In the spatiotemporal interactive trajectory prediction method based on trajectory lane consistency of the present invention, the spatiotemporal semantic consistency module uses a loss function during training to keep the predicted future temporal state aligned with the high-probability lane segment.

[0066] During the training process, the spatiotemporal semantic consistency module of this invention first initializes a corresponding temporal query vector for each time step of a future period, generates a time index in chronological order, and performs feature mapping through a temporal embedding multilayer perceptron to obtain a corresponding number of temporal query vectors, and makes each query correspond to a future time step.

[0067] During training, the initialized temporal query vectors interact with the scene context features obtained from modeling the training set data, and then input them into the bidirectional temporal modeling module for modeling. The modeling method is the same as in the usage stage, and will not be repeated here. After passing through the bidirectional temporal modeling module, the features are input into the decoding network. Each temporal query vector is decoded into the future position coordinates of the corresponding time step, thus obtaining the entire proposed future trajectory. The predicted trajectory is compared with the actual future trajectory, and a smoothed L1 loss is used to calculate the error. This supervises the trajectory prediction task in this stage, enabling the trained spatiotemporal semantic consistency module to generate the temporal query for the generated future time period.

[0068] In the task of predicting lane probability distribution, a label-smoothed cross-entropy loss function is used to supervise the lane prediction results. The true label is determined by the minimum distance between the target agent's true position at that time step and the centerlines of all lane segments during training; the closest lane segment corresponds to a label value of 1, and the rest are 0. For example... Figure 3 As shown, the model calculates the probability of the target agent being located in each lane segment at each future time step during the prediction process. The color intensity represents the probability magnitude, with darker colors indicating higher probabilities.

[0069] Furthermore, to maintain consistency between the predicted trajectory and lane semantic judgment, the spatial deviation between the predicted point and the corresponding lane centerline needs to be constrained during training. Specifically, during each training iteration, the weighted expected distance from the predicted point to each lane centerline is calculated using the predicted lane probability distribution as weights. This expected value is then used as a consistency loss term. By minimizing this weighted distance, the predicted trajectory (i.e., the future temporal state) is aligned with the high-probability lane segments. ; During this stage, the total loss of the spatiotemporal semantic consistency module It consists of the above three parts, including trajectory regression loss. Lane classification loss Trajectory-Lane Consistency Loss : ; In the formula, For smoothing L1 loss of trajectory position, Cross-entropy loss for lane segment classification, The weighted consistency loss is calculated between the predicted point and the lane centerline. In a specific implementation, the weights of the three losses can be set according to the data distribution and task requirements. In this implementation, , and Take values ​​of 0.7, 0.2, and 0.1 respectively.

[0070] For hierarchical multimodal decoders, during training, a pattern query branch is designed in the first stage to model trajectory pattern generation under different driving intentions and provide a global subsequent trajectory distribution for the subsequent fusion stage. Specifically, instantiating... k There are k learnable pattern query vectors, corresponding to different potential driving intentions.

[0071] During training, each pattern query interacts with the contextual features of the received scene, and is then mapped by the decoder. k The process of identifying candidate future trajectories and their probabilities is the same as the usage process, and will not be elaborated here.

[0072] During the supervision of this task, the same loss form is used for both the candidate future trajectories generated by the pattern query branch and the refined trajectories output by the fusion and refinement module. That is, a "winner-takes-all" strategy is adopted, selecting the trajectory closest to the true trajectory from the k predicted trajectories for supervision. The loss function of this branch consists of the trajectory regression term. With pattern classification items Composition, in the form of: .

[0073] Based on this, the overall objective function of the model is derived from the loss of the spatiotemporal semantic consistency module. The loss of the first stage of the decoder in generating candidate trajectories The loss of the second stage of the decoder in generating the refined trajectory constitute: .

[0074] Building upon this, during the training phase, the optimizer employs the AdamW optimization algorithm, and the learning rate scheduler uses a warm-up cosine annealing strategy (WarmupCosLR). The learning rate is gradually increased in the early stages of training, followed by a cosine decrease to achieve a stable optimization process. Backpropagation and gradient updates are performed on the network parameters according to the total loss function until the preset number of training epochs is reached.

[0075] Through the above methods, the spatiotemporal semantic consistency module and hierarchical multimodal decoder of this invention can be trained. During the inference stage, the input historical trajectory and map features are processed sequentially through the aforementioned steps, and the future multimodal trajectory prediction results are output, including several candidate trajectories and their corresponding probability distributions. Depending on the actual application requirements, the trajectory with the highest probability can be selected as the final prediction result, or multiple high-confidence trajectories can be output for use by the downstream decision-making and planning modules.

[0076] During the encoding stage, the spatiotemporal semantic consistency module proposed in this invention extracts the sequence features of the lane segment where the agent is located at each historical moment and combines them with relative pose information for temporal aggregation, thereby establishing a dynamic memory with "lane-time" alignment, realizing explicit modeling of road constraints and semantic completion of the subject's motion trend.

[0077] In the decoding phase, this invention introduces a lane segment localization task and a trajectory-lane consistency constraint: the model predicts the probability distribution of the lane segment where the target vehicle is most likely to be located at each future time step, thereby reflecting the degree of matching between trajectory points and lane semantics. By introducing a consistency expectation constraint into the loss function, the predicted trajectory not only closely approximates the real trajectory geometrically, but also maintains semantic consistency with the road topology.

[0078] Meanwhile, by designing a local context filtering and fusion refinement mechanism, during the prediction stage, several high-confidence lane segments are selected based on the lane probability distribution at each time step and deduplicated to form a locally high-importance region. Only this region is used for focused modeling and feature fusion. This design reduces attention computational complexity, improves information utilization efficiency, and enhances the stability and interpretability of the prediction.

[0079] In one specific implementation, to verify the trajectory prediction performance of the present invention, the Argoverse 2Motion Forecasting Dataset (Argoverse 2 MF) was used. The historical trajectory length was set to 5 seconds, and the future prediction duration to 6 seconds, to train the model (Lac-Net) of the present invention. This dataset, provided by Argo AI, is a publicly available high-precision map trajectory dataset used for trajectory prediction research in autonomous driving scenarios. Data collection covers six typical cities, encompassing various road structures and traffic environments, demonstrating good diversity and representativeness. The Argoverse 2 MF dataset contains approximately 250,000 driving scenarios, each lasting approximately 11 seconds, sampled at a frequency of 10Hz, recording the temporal location, speed, orientation, and motion category information of various traffic participants (including vehicles, pedestrians, cyclists, etc.). Each scenario is accompanied by a high-definition map (HD Map), including geometric and semantic annotations of lane centerlines, lane boundaries, drivable areas, traffic signals, and pedestrian crossings, which can be used to assist the model in establishing spatial constraints and lane relationships. In the experiments of this invention, the target agent is selected from the dataset as the prediction object, and the other traffic participants are used as context input.

[0080] On the Argoverse 2 Motion Forecasting dataset, the performance of the LaC-Net of this invention was compared with that of several mainstream prediction models. The evaluation metrics included minADE (minimum mean error) 1, minFDE (minimum final error), minADE6, minFDE6, and MR (false negative rate). The experimental results are shown in Table 1 below.

[0081] Table 1

[0082] As shown in Table 1, the LaC-Net method of this invention achieves best or near-best results on all core metrics. Specifically, minADE6 and minFDE6 reach 0.63 and 1.20 respectively, significantly outperforming the baseline model DeMo (0.65 and 1.25); the false negative rate MR6 is reduced to 0.14, the lowest among the compared methods. The results demonstrate that this invention achieves higher trajectory accuracy and lane consistency within an end-to-end prediction framework, and while maintaining multimodal prediction capabilities, improves the model's stability and spatial alignment capabilities in complex scenarios.

[0083] In summary, experimental verification based on the Argoverse 2 standard benchmark shows that the LaC-Net method proposed in this invention has significant advantages in fully utilizing map semantics and lane structure information, reaching the advanced level of current end-to-end prediction methods.

[0084] The following describes the spatiotemporal interactive trajectory prediction device based on trajectory-lane consistency provided by the present invention. The spatiotemporal interactive trajectory prediction device based on trajectory-lane consistency described below and the spatiotemporal interactive trajectory prediction method based on trajectory-lane consistency described above can be referred to in correspondence.

[0085] like Figure 4 As shown, the spatiotemporal interactive trajectory prediction device based on trajectory lane consistency of the present invention includes a feature module 401, an encoding module 402 and a decoding module 403. The feature module 401 is used to perform initial feature encoding and spatiotemporal interaction modeling on the historical trajectory of the intelligent agent and the scene map in the driving scenario to obtain the context features of the scene. The context features of the scene include lane segment features and intelligent agent features. The historical trajectory is the trajectory of an intelligent agent during a historical period, and its duration is determined based on experience.

[0086] Optionally, the historical trajectory length is set to 5 seconds (50 frames in total).

[0087] The intelligent agents in a driving scenario include the target intelligent agent that needs to perform trajectory prediction and other intelligent agents present in the scenario.

[0088] Before encoding the historical trajectory and scene map of the intelligent agent, the input high-precision map data and the historical trajectory data of each intelligent agent in the driving scenario can be uniformly normalized in terms of coordinates.

[0089] Optionally, a local coordinate system is established with the position of the target agent in the last frame of the historical time series as the origin and its orientation direction as the y-axis.

[0090] By translation and rotation, all agents and map elements are unified to this local coordinate system. The historical trajectory of each agent is vectorized, and the coordinate difference between two adjacent time steps is calculated to obtain the motion vector sequence of the agent at each time step.

[0091] After completing coordinate alignment and vectorization, feature encoding is performed on the polyline segments of the scene map and the historical trajectory of each agent.

[0092] The initial feature encoding involves using encoding modules to encode both the scene map and the agent's historical trajectory, resulting in feature codes for the scene map and the agent, respectively. Spatiotemporal interaction modeling then fuses the agent's lane sequence and historical trajectory features, embedding each agent's lane interaction history into the resulting scene context features.

[0093] The encoding module 402 is used to input the context features of the scene into the spatiotemporal semantic consistency module to obtain the refined temporal query features and the filtered scene context output by the spatiotemporal semantic consistency module. The refined temporal query features represent the future temporal state that integrates lane semantics, and the filtered scene context represents the local interactive environment composed of high-probability lane segments. Furthermore, the extracted contextual features of the scene are first input into the spatiotemporal semantic consistency module.

[0094] like Figure 2 As shown, the spatiotemporal semantic consistency module first predicts the trajectory of the agent at each time step in the future period as the query item, and updates the query by interacting with the contextual features of the scene through the predicted trajectory. Then, it performs bidirectional time modeling on the updated query to generate a dynamic representation of continuous time and update the query.

[0095] Based on this, the lane segment localization module in the spatiotemporal semantic consistency module determines the most likely lane segments where the target agent will appear in each time step, and uses the lane segments to interact with the features after bidirectional time modeling to obtain refined temporal query features, which serve as input for subsequent trajectory refinement.

[0096] Furthermore, the most likely lane segments of all time steps are integrated as high-probability lane segments to represent the lanes most likely to appear for the target agent. The high-probability lane segments are used to filter the context representation of the scene, and the filtered scene context is used for subsequent trajectory refinement.

[0097] The decoding module 403 is used to input the context features of the scene into the hierarchical multimodal decoder. In the first stage of the hierarchical multimodal decoder, the pattern query is updated and candidate trajectories are generated through the interaction between the pattern query and the scene context. In the second stage, the updated pattern query is fused with the refined temporal query features and the filtered scene context to decode several refined trajectories and their corresponding probability distributions. The number of trajectories is the same as the number of pattern queries, and each pattern query represents a driving intention.

[0098] During the decoding phase, the hierarchical multimodal decoder first receives the contextual features of the scene, and interacts each trained pattern query with the contextual features of the scene to capture global semantics and interaction information, which is then mapped by the decoder. k There are 10 candidate future trajectories and their probabilities. The number of mapped subsequent future trajectories is the same as the number of pre-defined pattern queries.

[0099] Each pattern query corresponds to a potential driving intention, which is a pre-trained learnable pattern query vector.

[0100] Specifically, such as Figure 2 As shown, in this process, k Each learnable pattern query is used as Q. The received context features of the scene are used as keys and values, a cross-attention operation is performed, and the pattern query is updated. Then, the... k Each pattern query performs a self-attention operation on itself and updates the pattern query. Then, the corresponding candidate future trajectories and their probabilities are obtained through the decoder mapping.

[0101] In the above manner, the trained pattern query is updated in this candidate future trajectory prediction task. In the second stage of the decoder, such as... Figure 2 As shown, the updated pattern query is first broadcast and element-level added to the refined time-series query feature representation in the time dimension to form a fused query vector.

[0102] The fused query vector is then interacted with the filtered scene context in the decoder's fusion and refinement module, and further refined through self-attention to capture spatial constraints and dynamic relationships within the local context. After the interaction and refinement are complete, the output is... k The refined trajectory and its corresponding probability distribution are used as the prediction result.

[0103] This invention introduces spatiotemporal lane interaction at the encoding stage, establishing a time-step-level association between the agent's historical sequence and the corresponding lane sequence. This injects dynamic memory aligned with "lane-time," enabling the agent's representation to simultaneously perceive historical movement trends and lane geometric constraints, thus achieving a more semantically consistent scene understanding. At the decoding stage, a hierarchical semantic reasoning process is constructed, sequentially involving trajectory proposal generation, lane segment probabilistic localization, local context filtering based on lane distribution, and multimodal fusion refinement. This achieves lane semantic alignment and scene importance filtering during the prediction phase. This mechanism not only improves the model's spatial consistency and semantic interpretability at the lane level but also achieves a better balance between information utilization efficiency, prediction accuracy, and trajectory diversity, significantly enhancing prediction stability and generalization ability in complex road scenarios.

[0104] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute a spatiotemporal interaction trajectory prediction method based on trajectory lane consistency. This method includes: performing initial feature encoding and spatiotemporal interaction modeling on the historical trajectory of the agent and the scene map in the driving scenario to obtain the context features of the scene, which include lane segment features and agent features; inputting the context features of the scene into a spatiotemporal semantic consistency module to obtain refined temporal query features and filtered scene context output by the spatiotemporal semantic consistency module, wherein the refined temporal query features represent the future temporal state incorporating lane semantics, and the filtered scene context represents a local interaction environment composed of high-probability lane segments; inputting the context features of the scene into a hierarchical multimodal decoder, wherein the first stage of the hierarchical multimodal decoder updates the pattern query and generates candidate trajectories through interaction between the pattern query and the scene context; and the second stage fuses the updated pattern query with the refined temporal query features and the filtered scene context to decode and obtain several refined trajectories and their corresponding probability distributions, wherein the number of trajectories is the same as the number of pattern queries, and each pattern query represents a driving intention.

[0105] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0106] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the spatiotemporal interactive trajectory prediction method based on trajectory-lane consistency provided by the above methods. This method includes: performing initial feature encoding and spatiotemporal interaction modeling on the historical trajectory of the agent and the scene map in the driving scenario to obtain the context features of the scene, which include lane segment features and agent features; inputting the context features of the scene into a spatiotemporal semantic consistency module to obtain a refined time-series query output by the spatiotemporal semantic consistency module. The system comprises features and filtered scene context, wherein the refined temporal query features represent the future temporal state incorporating lane semantics, and the filtered scene context represents the local interactive environment composed of high-probability lane segments; the scene context features are input into a hierarchical multimodal decoder; in the first stage of the hierarchical multimodal decoder, the pattern query is updated and candidate trajectories are generated through the interaction between the pattern query and the scene context; in the second stage, the updated pattern query is fused with the refined temporal query features and the filtered scene context to decode several refined trajectories and their corresponding probability distributions, wherein the number of trajectories is the same as the number of pattern queries, and each pattern query represents a driving intention.

[0107] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, this computer program implements the spatiotemporal interactive trajectory prediction method based on trajectory-lane consistency provided by the methods described above. This method includes: performing initial feature encoding and spatiotemporal interaction modeling on the historical trajectory of an agent and a scene map in a driving scenario to obtain contextual features of the scene, wherein the contextual features of the scene include lane segment features and agent features; inputting the contextual features of the scene into a spatiotemporal semantic consistency module to obtain refined temporal query features and filtered scene context output by the spatiotemporal semantic consistency module, wherein... The refined temporal query features represent the future temporal state that incorporates lane semantics, and the filtered scene context represents the local interactive environment composed of high-probability lane segments. The context features of the scene are input into a hierarchical multimodal decoder. In the first stage, the hierarchical multimodal decoder updates the pattern query and generates candidate trajectories through the interaction between the pattern query and the scene context. In the second stage, the updated pattern query is fused with the refined temporal query features and the filtered scene context to decode several refined trajectories and their corresponding probability distributions. The number of trajectories is the same as the number of pattern queries, and each pattern query represents a driving intention.

[0108] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0109] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A spatiotemporal interactive trajectory prediction method based on trajectory-lane consistency, characterized in that, include: After initial feature encoding and spatiotemporal interaction modeling of the historical trajectory of the agent and the scene map in the driving scenario, the context features of the scene are obtained, which include lane segment features and agent features. The context features of the scene are input into the spatiotemporal semantic consistency module to obtain the refined temporal query features and the filtered scene context output by the spatiotemporal semantic consistency module. The refined temporal query features represent the future temporal state that integrates lane semantics, and the filtered scene context represents the local interactive environment composed of high-probability lane segments. The contextual features of the scene are input into the hierarchical multimodal decoder. In the first stage, the hierarchical multimodal decoder updates the pattern query and generates candidate trajectories through the interaction between the pattern query and the scene context. In the second stage, the updated pattern query is fused with the refined temporal query features and the filtered scene context to decode several refined trajectories and their corresponding probability distributions. The number of trajectories is the same as the number of pattern queries, and each pattern query represents a driving intention.

2. The spatiotemporal interactive trajectory prediction method based on trajectory-lane consistency according to claim 1, characterized in that, The step of inputting the contextual features of the scene into the spatiotemporal semantic consistency module to obtain the refined temporal query features output by the spatiotemporal semantic consistency module specifically includes: Generate a time-series query for a future time period, and perform bidirectional time-series modeling on the time-series query for the future time period and the context feature representation of the received scene to obtain a spatiotemporal semantic state query vector, wherein the time-series query for the future time period includes the time-series query vector for each time step of the future time period. For each time step in the future period, calculate the compatibility score between its spatiotemporal semantic state query vector and lane segment features to obtain the lane segment probability distribution, which represents the probability that the target agent belongs to each lane segment at each time step. A preset number of lane segments with the highest compatibility scores are selected to form a candidate interaction set. The candidate interaction set is used as a key and value. The spatiotemporal semantic state query vector of the corresponding time step is used as the query to perform cross-attention operation and obtain the refined query representation of the time step. By integrating the refined query representations of each time step, the refined time-series query features are obtained.

3. The spatiotemporal interactive trajectory prediction method based on trajectory lane consistency according to claim 2, characterized in that, The step of performing bidirectional temporal modeling on the temporal query and the contextual feature representation of the received scene for the future time period to obtain the spatiotemporal semantic state query vector specifically includes: Using the time-series query of the future period as the query item and the contextual feature representation of the scene as the key and value, a cross-attention operation is performed to update the time-series query of the future period in order to obtain environmental semantic information; The future time-series queries obtained after cross-attention are sequentially input into the bidirectional time-series modeling module to model the dependencies between adjacent time steps, thereby obtaining the spatiotemporal semantic state query vector.

4. The spatiotemporal interactive trajectory prediction method based on trajectory lane consistency according to claim 2, characterized in that, The step of inputting the context features of the scene into the spatiotemporal semantic consistency module to obtain the filtered scene context output by the spatiotemporal semantic consistency module specifically includes: The highest-scoring lane segments are combined and deduplicated to obtain locally important regions. The lane segment features and agent features corresponding to the locally important regions are then selected from the context features of the scene to form the filtered scene context.

5. The spatiotemporal interactive trajectory prediction method based on trajectory-lane consistency according to claim 1, characterized in that, The step of obtaining the contextual features of the scene after initial feature encoding and spatiotemporal interaction modeling of the historical trajectory and scene map of the intelligent agent in the driving scenario specifically includes: Initial feature encoding is performed on the historical trajectory of the agent and the scene map in the driving scenario to obtain the topological features representing the scene map and the motion features representing the historical trajectory of the agent. The topological features and motion features are concatenated and input into a scene-level Transformer for joint modeling to obtain an initial context feature representation that includes the global spatiotemporal relationships of the scene. For each historical moment, the states of all agents in the scene are associated with their nearest lane segments to construct a lane sequence, which is then extracted into a summary vector by the time sequence module. After fusing the summary vector with the residual of the agent's initial context feature representation, it is concatenated with the initial context feature representation of the lane segment and input into the Transformer for a second interaction to obtain the context feature representation of the scene.

6. The spatiotemporal interactive trajectory prediction method based on trajectory-lane consistency according to claim 1, characterized in that, The spatiotemporal semantic consistency module uses a loss function during training to keep the predicted future temporal state aligned with the high-probability lane segment.

7. A spatiotemporal interactive trajectory prediction device based on trajectory-lane consistency, characterized in that, include: The feature module is used to perform initial feature encoding and spatiotemporal interaction modeling on the historical trajectory of the agent and the scene map in the driving scenario to obtain the context features of the scene. The context features of the scene include lane segment features and agent features. The encoding module is used to input the context features of the scene into the spatiotemporal semantic consistency module to obtain the refined temporal query features and the filtered scene context output by the spatiotemporal semantic consistency module. The refined temporal query features represent the future temporal state that incorporates lane semantics, and the filtered scene context represents the local interactive environment composed of high-probability lane segments. The decoding module is used to input the context features of the scene into the hierarchical multimodal decoder. In the first stage, the hierarchical multimodal decoder updates the pattern query and generates candidate trajectories through the interaction between the pattern query and the scene context. In the second stage, the updated pattern query is fused with the refined temporal query features and the filtered scene context to decode several refined trajectories and their corresponding probability distributions. The number of trajectories is the same as the number of pattern queries, and each pattern query represents a driving intention.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the spatiotemporal interactive trajectory prediction method based on trajectory lane consistency as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the spatiotemporal interactive trajectory prediction method based on trajectory lane consistency as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the spatiotemporal interactive trajectory prediction method based on trajectory lane consistency as described in any one of claims 1 to 6.

Images