Trajectory prediction method and system based on hybrid expert model, medium and product

Abstract

The invention discloses a trajectory prediction method and system based on a hybrid expert model, a medium and a product, and relates to the field of data processing systems specially suitable for the prediction purpose, and the method comprises the steps: obtaining an observation trajectory of any length; a preset encoder is utilized to encode the observation track, and track feature representation is obtained; inputting the track feature representation into a gating network, and calculating an optimal observation length; according to the optimal observation length, activating a corresponding target expert module in a preset hybrid expert model, and performing optimization processing on the observation trajectory to obtain an optimized trajectory; the optimization processing comprises the following steps: comparing the current length of the observation track with the optimal observation length; if the observation length is greater than or equal to the optimal observation length, executing observation pruning; if the observation length is smaller than the optimal observation length, reverse prediction is executed; and inputting the optimized trajectory into the trajectory prediction backbone network to obtain a future trajectory prediction result. By implementing the method, the trajectory prediction accuracy under the observation condition of any length can be improved.

Description

Technical Field

[0001] This application relates to the field of data processing systems specifically designed for prediction purposes, and more particularly to a trajectory prediction method, system, medium, and product based on a hybrid expert model. Background Technology

[0002] Trajectory prediction technology is used to predict the future movement paths of traffic participants based on their historical movement trajectories, and it forms the basis for path planning and obstacle avoidance decisions in autonomous driving systems. Deep learning-based trajectory prediction methods, which learn motion patterns through architectures such as recurrent neural networks, convolutional neural networks, or Transformers, have become the mainstream technology in this field.

[0003] In related technologies, deep learning-based trajectory prediction methods are generally trained using publicly available datasets, where trajectories have a fixed observation length. During training, the model receives a fixed-length sequence of historical trajectories, extracts spatiotemporal features through an encoder, and then generates future trajectory predictions through a decoder. Some methods introduce attention mechanisms to capture temporal dependencies or use graph neural networks to model the interactions of multiple traffic participants. This training method based on fixed-length inputs can achieve good prediction accuracy on standard datasets.

[0004] However, due to factors such as occlusion, sensor field of view limitations, sudden target appearance, or tracking interruption, the length of the historical trajectory acquired by the system varies. Too short an observation sequence results in insufficient temporal information for the model, leading to significant prediction bias; too long an observation sequence exceeds the length distribution during training, causing the position encoding and temporal modeling mechanisms to fail, and atypical motion patterns within it can interfere with model judgment, reducing prediction accuracy. Summary of the Invention

[0005] This application provides a trajectory prediction method, system, medium, and product based on a hybrid expert model, which can improve the accuracy of trajectory prediction under observation conditions of arbitrary length.

[0006] Firstly, this application provides a trajectory prediction method based on a hybrid expert model, applied to a trajectory prediction system. The method includes: acquiring an observation trajectory of arbitrary length; encoding the observation trajectory using a pre-set encoder to obtain a trajectory feature representation; inputting the trajectory feature representation into a gating network to calculate the optimal observation length adapted to the observation trajectory; activating the corresponding target expert module in the pre-set hybrid expert model based on the optimal observation length; optimizing the observation trajectory based on the target expert module to obtain an optimized trajectory; the optimization process includes: comparing the current length of the observation trajectory with the optimal observation length; if the current length is greater than or equal to the optimal observation length, performing observation pruning to remove interference information; if the current length is less than the optimal observation length, performing reverse prediction to supplement missing historical trajectories; and inputting the optimized trajectory into the trajectory prediction backbone network to obtain future trajectory prediction results.

[0007] In the above embodiments, the trajectory prediction system adopts the above technical solution, uses an encoder and a gating network to dynamically calculate the optimal observation length for each input trajectory of arbitrary length. Based on the comparison result of the optimal length and the current trajectory length, a specific expert module is activated to perform differential processing, which improves the quality of the data input to the final prediction network. This solves the problem of performance degradation when processing trajectory observations that are inconsistent with the distribution of training data, and enhances the accuracy of trajectory prediction.

[0008] In conjunction with some embodiments of the first aspect, in some embodiments, the step of encoding the observed trajectory using a preset encoder to obtain a trajectory feature representation specifically includes: inputting the coordinate sequence of the observed trajectory into a preset encoder to map it into a hidden layer state sequence; calculating the relative time interval between each time step in the observed trajectory and mapping the relative time interval to a rotation position encoding parameter; performing a rotation transformation on the hidden layer state sequence based on the rotation position encoding parameter to obtain an intermediate feature sequence carrying relative temporal information; and performing a pooling operation on the intermediate feature sequence to obtain a trajectory feature representation.

[0009] In the above embodiments, the trajectory prediction system adopts the above technical solution to dynamically model the relative positional relationship between trajectory points by learning the rotation parameters, so that the position encoding can flexibly handle variable-length sequences, determine the dynamic characteristics and inherent temporal patterns of the trajectory, and improve the performance of the model under variable inputs.

[0010] In conjunction with some embodiments of the first aspect, in some embodiments, the step of performing reverse prediction to supplement the missing historical trajectory if the current length is less than the optimal observation length specifically includes: calculating the absolute value of the length difference between the optimal observation length and the current length to obtain the number of time steps to be supplemented; inputting the trajectory feature representation into the reverse prediction network to generate a number of historical coordinate points equal to the number of time steps; and splicing the historical coordinate points in chronological order before the start of the observed trajectory to obtain the completed trajectory sequence, which is used as the optimized trajectory output.

[0011] In the above embodiments, the trajectory prediction system, by adopting the above technical solution, when it is determined that the current observation length is less than the optimal length, generates historical trajectory points that conform to the motion law based on the reverse prediction network, and splices them to the starting position of the original observation sequence. This makes up for the information loss caused by sensor occlusion, late appearance of the target, etc., and improves the prediction accuracy of short observation sequences.

[0012] In conjunction with some embodiments of the first aspect, in some embodiments, before the step of obtaining the observation trajectory of arbitrary length, the method further includes: obtaining the complete training sample trajectory in the training set, and truncating the complete training sample trajectory to obtain the truncated observation trajectory and the corresponding truncated real historical trajectory; inputting the truncated observation trajectory into the backpropagation network to be trained to generate the predicted historical trajectory; calculating the reconstruction loss value between the predicted historical trajectory and the truncated real historical trajectory; concatenating the predicted historical trajectory and the truncated observation trajectory and inputting them into the trajectory prediction backbone network to obtain the predicted future trajectory; calculating the prediction loss value between the predicted future trajectory and the real future trajectory; and performing gradient update on the network parameters of the backpropagation network based on the reconstruction loss value and the prediction loss value.

[0013] In the above embodiments, the trajectory prediction system adopts the above technical solution to input the spliced ​​complete trajectory into the backbone network to calculate the prediction loss value, thereby using the final prediction task performance as a direct guide for the training of the reverse prediction network, ensuring that the reverse prediction network learns high-quality historical information and ensuring the accuracy of supplementing historical information.

[0014] In conjunction with some embodiments of the first aspect, in some embodiments, after performing gradient updates on the network parameters of the inverse prediction network based on the reconstruction loss value and the prediction loss value, the method further includes: recording the activation counts of each expert module in the hybrid expert model in the training batch to obtain an activation count list; calculating the mean and standard deviation of each element in the activation count list to obtain distribution parameters; calculating the load imbalance coefficient based on the ratio of the standard deviation to the mean; when the load imbalance coefficient exceeds a preset balance threshold, applying a penalty gradient to the weight parameters of the gating network, and adjusting the weight parameters of the gating network based on the penalty gradient to balance the activation probabilities of each expert module.

[0015] In the above embodiments, the trajectory prediction system, by adopting the above technical solution, monitors and quantifies the activation frequency of each expert during training. When it detects that some experts are overused while others are idle, it will adjust the parameters of the gating network by applying a penalty gradient to ensure that each expert can be fully trained and optimized, thus guaranteeing the diversity and generalization ability of the entire hybrid expert system.

[0016] In conjunction with some embodiments of the first aspect, in some embodiments, after activating the corresponding target expert module in the preset hybrid expert model according to the optimal observation length, the method further includes: calculating the length difference between the current length and the optimal observation length to obtain a length deviation; when the observation trajectory is determined to be located in the classification boundary region based on the length deviation, extracting the activation probability distribution of each expert module output by the gating network, identifying candidate expert modules whose activation probability is greater than a preset probability threshold, and obtaining a candidate expert set; driving each candidate expert module in the candidate expert set to process the observation trajectory to obtain multiple candidate optimized trajectories; performing weighted fusion on the time step coordinates of the multiple time-aligned candidate optimized trajectories according to the activation probability corresponding to each candidate expert module as a weight coefficient to obtain a fused optimized trajectory; and replacing the optimized trajectory with the fused optimized trajectory and inputting it into the trajectory prediction backbone network.

[0017] In the above embodiments, the trajectory prediction system, by employing the aforementioned technical solution, identifies all highly probable candidate experts when the characteristics of the trajectory make it difficult to explicitly classify it into a specific optimal length. Through parallel processing and weighted fusion of results based on activation probabilities, it avoids drastic changes in prediction results due to small input perturbations, thereby improving the model's prediction performance when handling fuzzy and uncertain trajectories.

[0018] In conjunction with some embodiments of the first aspect, in some embodiments, after performing weighted fusion of the corresponding time step coordinates of multiple candidate optimized trajectories according to the activation probabilities corresponding to each candidate expert module as weight coefficients to obtain a fused optimized trajectory, the method further includes: calculating the displacement vector between adjacent time steps in the fused optimized trajectory to obtain a displacement sequence; performing second-order difference on the displacement sequence to obtain an acceleration feature sequence characterizing the trajectory smoothness; detecting abnormal time steps in the acceleration feature sequence whose magnitude exceeds a preset physical limit threshold to generate a set of motion mutations; performing smoothing correction on the coordinate points of the corresponding set of motion mutations in the fused optimized trajectory using a local polynomial interpolation algorithm to obtain a smooth fused trajectory that conforms to kinematic constraints, and using the smooth fused trajectory as the optimized trajectory input to the trajectory prediction backbone network.

[0019] In the above embodiments, the trajectory prediction system employs the above technical solution and uses a local polynomial interpolation algorithm for smooth correction, ensuring that the optimized trajectory finally input to the backbone prediction network is kinematically coherent and smooth, thereby improving the physical authenticity and credibility of the final prediction result.

[0020] In a second aspect, embodiments of this application provide a trajectory prediction system, the trajectory prediction system comprising: one or more processors and a memory; the memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors calling the computer instructions to cause the trajectory prediction system to perform the method described in the first aspect and any possible implementation thereof.

[0021] Thirdly, embodiments of this application provide a computer program product containing instructions that, when the computer program product is run on a trajectory prediction system, cause the trajectory prediction system to perform the method described in the first aspect and any possible implementation thereof.

[0022] Fourthly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on a trajectory prediction system, cause the trajectory prediction system to perform the method described in the first aspect and any possible implementation thereof.

[0023] Understandably, the trajectory prediction system provided in the second aspect, the computer program product provided in the third aspect, and the computer storage medium provided in the fourth aspect are all used to execute the methods provided in the embodiments of this application. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods, and will not be repeated here.

[0024] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0025] 1. By adopting a technical solution that involves acquiring an observation trajectory of arbitrary length, dynamically calculating its optimal observation length using an encoder and gating network, and then activating the corresponding observation pruning or reverse prediction module in the hybrid expert model for adaptive optimization based on the comparison between this length and the current length, this application can perform personalized input processing for each trajectory, providing the downstream prediction model with an input of better information quality and more suitable length, thereby improving the prediction accuracy and robustness of the trajectory prediction system in complex and dynamic scenarios.

[0026] 2. By adopting a technical solution that maps the relative time intervals between each time step in the trajectory to learnable rotational position encoding parameters, and performs rotational transformation on the hidden state sequence of the trajectory based on these parameters to inject relative temporal information, the encoder of this application can get rid of the dependence of traditional position encoding on fixed length or predefined patterns, realize flexible and adaptive temporal modeling of trajectory sequences of arbitrary length, and the encoder can learn data-driven positional relationship representation, better capture the dynamic characteristics of the trajectory, and improve the performance and robustness of the entire model in handling variable inputs.

[0027] 3. By adopting a technical solution that, during the training of the backpropagation network, not only is the reconstruction loss between the generated historical trajectory and the real historical trajectory calculated, but also the reconstruction loss is concatenated with the observed trajectory and input into the backbone prediction network to calculate and backpropagate the prediction loss of the final prediction task, this application constructs an end-to-end, task-oriented training closed loop. This achieves direct alignment between the learning objective of the backpropagation module and the utility function of the final trajectory prediction task, thereby improving the prediction accuracy of the model when processing short observation sequences. Attached Figure Description

[0028] Figure 1 This is a flowchart illustrating a trajectory prediction method based on a hybrid expert model in an embodiment of this application.

[0029] Figure 2 This is another flowchart illustrating the trajectory prediction method based on a hybrid expert model in this application embodiment;

[0030] Figure 3 This is a schematic diagram of the physical device structure of a trajectory prediction system in the embodiments of this application. Detailed Implementation

[0031] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification of this application, the singular expressions “a,” “an,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to any or all possible combinations including one or more of the listed items.

[0032] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0033] In the implementation scenarios of this application, such as in autonomous driving or robot navigation, trajectory prediction systems need to process data from sensors such as cameras and radar in real time to predict the future paths of surrounding traffic participants (such as pedestrians and vehicles). The observation data acquired by these sensors naturally has the characteristic of variable length, i.e., "arbitrary length observation", which may originate from the target just entering the sensor's field of view (short trajectory), the target reappearing after being briefly occluded (short trajectory), or the target being stably tracked for a long time (long trajectory).

[0034] The proposed "Hybrid Expert Model (MoE)" is an ensemble learning framework comprising multiple functionally specialized "expert" subnetworks and a "gated network." The gated network analyzes input observation trajectories of arbitrary length and selects one or more of the most suitable experts for processing. The expert module designed in this application has a dual function: "observation pruning" is used to handle excessively long trajectories that may contain atypical motion patterns such as sudden avoidance or hesitation, removing these interfering information by truncating them to the "optimal observation length"; while "backward prediction" is used to handle excessively short trajectories, supplementing temporal information by generating missing historical trajectory points to achieve the optimal observation length.

[0035] To better handle variable-length sequences, this application also designs a "Learnable Rotational Position Encoding (LRoPE)," which can adaptively learn the relationship between positions, outperforming traditional fixed-position encoding.

[0036] The goal of the entire process is to first process the input trajectory, regardless of its length, through this adaptive optimization framework to obtain an "optimized trajectory" with optimal length and clean information. Then, it is fed into a general "trajectory prediction backbone network" (such as a prediction model based on Transformer or RNN) for the final prediction of the future trajectory, thereby ensuring the accuracy and security of the prediction.

[0037] The following describes the process of the method provided in this implementation. Please refer to [link / reference]. Figure 1 This is a flowchart illustrating a trajectory prediction method based on a hybrid expert model in an embodiment of this application.

[0038] S101. Obtain observation trajectories of arbitrary length.

[0039] Among them, the observation trajectory of arbitrary length refers to the time series data collected by the trajectory prediction system through sensors (such as cameras and lidar) in the real physical world, which describes the motion state of an intelligent agent (such as a vehicle or pedestrian) over a period of time. The length of the sequence (i.e. the number of time steps it contains) is not fixed and is variable, and it is usually composed of a series of two-dimensional or three-dimensional coordinate points ordered by time.

[0040] Specifically, in applications such as autonomous driving or robot navigation, trajectory prediction systems continuously receive information about surrounding dynamic targets from the perception module. When a new target is detected, or the tracking ID of an existing target is confirmed, the system begins recording its continuous position coordinates, forming an observed trajectory. Because targets may enter or leave the sensor's sensing range at any time, be occluded by other objects causing tracking interruptions, or the sensor's own frame rate may change, the length of the historical trajectory that the system can acquire at the current moment when a prediction decision is needed is flexible and not fixed. For example, if a car has just appeared from around a corner, the system may only observe its trajectory for two time steps; while another car traveling on an open road ahead may have a complete observation spanning dozens of time steps.

[0041] That is, given an agent, its complete observation trajectory is defined as:

[0042] ,in These are the two-dimensional coordinates at time t. It is the length of the complete observation.

[0043] In practical applications, we obtain observations of trajectories of arbitrary length:

[0044] Its length is .

[0045] The objective of this invention is to design a frame that can accept elements of arbitrary length. And accurately predict its future trajectory. .

[0046] In some embodiments, this step can be implemented in several ways: Optionally, the trajectory prediction system can be configured with a data receiving module that subscribes in real time to structured data packets published by the upstream sensing system via a preset communication protocol (such as TCP / IP, ROS topic). Each data packet contains a unique identifier of the target, a timestamp, and the current (x, y, z) coordinates and state information such as velocity and acceleration. The system sorts the target information with the same identifier by timestamp, caches it, and combines it into an observed trajectory. Optionally, the trajectory prediction system can directly access a shared database or memory area. The sensing system continuously writes detected target trajectory fragments into this area, and the prediction system reads the latest trajectory fragment of the specified target as input based on the task trigger signal. It is understood that other methods can also be used to acquire data, such as replaying data from offline log files for testing; this is not limited here.

[0047] S102. Encode the observed trajectory using a preset encoder to obtain a trajectory feature representation.

[0048] The pre-set encoder refers to a deep learning model module with a predefined network structure that has been trained or is awaiting training. Its function is to convert the input time-series coordinate sequence into a fixed-dimensional high-dimensional vector that contains the spatiotemporal dynamic characteristics of the trajectory.

[0049] Specifically, after the trajectory prediction system acquires the observed trajectory of arbitrary length in step S101, it needs to transform it from the original coordinate space into a feature space that is easier for machine learning models to understand and process. The encoder undertakes this task. It processes each coordinate point in the trajectory individually or as a whole, capturing the motion patterns of the trajectory through its internal network layers (such as self-attention layers and recurrent units), such as speed, acceleration, turning trends, and more complex temporal dependencies. Finally, the encoder integrates and compresses this captured dynamic information into a single feature vector, i.e., the trajectory feature representation. This feature representation can be regarded as a high-level summary of "what kind of motion the entire historical trajectory has" and "how it might move next," and it will serve as the sole basis for subsequent gating network decisions.

[0050] That is, using a Transformer encoder right Encoding can generate its deep feature representation. This feature It will be sent to the access control network in the next step. .

[0051] In some embodiments, this step can be implemented in several ways: Optionally, a gated recurrent unit (GRU)-based encoder can be used. First, the coordinates of each time step are mapped to an initial embedding vector through a linear layer; then, the sequence of embedding vectors is sequentially fed into a multi-layer GRU network, and the hidden state of the last time step of the GRU is used as the trajectory feature representation. Optionally, a Transformer-based encoder can be used. First, an initial embedding is generated for each coordinate point of the trajectory sequence, and the Learnable Rotational Position Encoding (LRoPE) proposed in this application is added; then, the sequence is input into a multi-layer Transformer encoder module, which captures global temporal dependencies through a self-attention mechanism; finally, the sequence output by the Transformer is averaged or the output corresponding to the [CLS] label is taken to obtain the final trajectory feature representation. It is understood that other encoder architectures, such as LSTM or spatiotemporal graph convolutional networks (ST-GCN), can also be used, and are not limited here.

[0052] In some embodiments, it is necessary to ensure that the encoder is invariant to the translation and rotation of the input trajectory to improve the model's generalization ability. The same motion pattern (such as a left turn) may occur at different positions and orientations in the coordinate system, and the model needs to identify its inherent consistency. To address this, the trajectory prediction system performs coordinate system normalization before inputting the trajectory into the encoder. Specifically, the last point in the trajectory sequence is selected as the new origin, and the coordinates of all historical points are converted to relative coordinates relative to this origin. Simultaneously, the vector direction from the second-to-last point to the last point is calculated, and this direction is used as the new coordinate axis direction (e.g., the x-axis), and all relative coordinates are rotated. In this way, the "current" position of all input trajectories is unified to the origin, and the "current" orientation is unified to the positive x-axis direction.

[0053] S103. Input the trajectory feature representation into the gating network and calculate the optimal observation length that matches the observation trajectory.

[0054] The gating network is a lightweight neural network, typically a multilayer perceptron (MLP), which acts like a decision-maker or router. The optimal observation length refers to the ideal historical trajectory length that produces the most accurate prediction for a specific trajectory with current input; this length is dynamically variable and depends on the trajectory's motion characteristics.

[0055] Specifically, after generating the trajectory feature representation in step S102, this feature representation is fed into a gating network in the trajectory prediction system. The gating network analyzes the trajectory feature representation containing dynamic trajectory information and outputs a probability distribution or a discrete decision result. In this application, the output of the gating network directly corresponds to a length selection. For example, the gating network can output a B-dimensional vector, where B is the number of preset candidate optimal lengths (e.g., from 2 to the full observation length). The length of the vector represents the optimal observation length selected by the gating network for the current trajectory. This decision-making process is data-driven, meaning that the gating network learns through training to map different types of trajectory features (such as smooth straight travel, sharp turns, and slow creep) to different optimal observation lengths.

[0056] That is, the features in step S102 Sent to the access control network Output a sparse B-dimensional vector. Where B is the total number of experts. (Through) Operation (k=1) Only one element is 1, and its index corresponds to the optimal length chosen by the model for the current trajectory. This is represented as:

[0057] ;in These are learnable weights. It is noise used to increase exploration during training.

[0058] Optionally, the gated network is a multilayer perceptron containing several linear layers and nonlinear activation functions (such as ReLU). It receives trajectory feature representations. Through the weight matrix Perform a linear transformation, then apply the Softmax function to convert its output into a probability distribution, and finally determine the probability distribution by taking the index corresponding to the maximum probability (argmax) or using... Operation (when k=1) to determine unique Optionally, the gating network can be designed to be more complex, for example, by introducing a small attention module. This module performs attention calculations on the trajectory feature representation and a set of learnable length prototype vectors, where the length represented by the prototype with the highest attention weight is the length of the vector. Understandably, other methods can be used to make decisions, such as using a decision tree or a set of rules, which are not limited here.

[0059] In some embodiments, the training of the gating network lacks direct supervision signals, i.e., the optimal observation length is unknown. To address this, this application employs reinforcement learning or indirect supervision through end-to-end gradient propagation. Specifically, the decision-making (selection) of the gating network... This will affect subsequent trajectory optimization and the final prediction result. The final prediction loss... It will propagate backwards through the entire computation graph. Although from... arrive The path may contain errors due to pruning or splicing operations, but policy gradient methods (such as the REINFORCE algorithm) can be used to use the improvement or decline of the final prediction performance as a reward or penalty signal to guide the update of the gating network parameters. The gating network can determine the choice that minimizes the final prediction error and learn to assign appropriate optimal observation lengths to different trajectories.

[0060] S104. Based on the optimal observation length, activate the corresponding target expert module in the preset hybrid expert model.

[0061] The hybrid expert model (MoE) is a collection of multiple (B) expert modules. Each expert All are associated with a specific potential optimal length l. The target expert module refers to the one among all experts whose corresponding length l equals the optimal observation length. The expert. Activation refers to directing the computational task to that target expert module, which then performs the subsequent processing.

[0062] Specifically, the trajectory prediction system determines the optimal observation length through a gating network. Next comes the expert selection phase. The system maintains an expert pool, in which each expert... Both can handle trajectories that should ultimately be adjusted to length l. The system will... The value is used to precisely match the corresponding target expert module from this pool. For example, if the gating network determines that the optimal observation length for the current trajectory is 8 time steps ( =8), then the system will activate the expert. The activated expert module will receive the original observation trajectory and its feature representation, and will be responsible for performing the next optimization operation.

[0063] In some embodiments, this step can be implemented in several ways: optionally, direct index activation can be used. The output of the gated network is a one-hot vector g, in which only one element is 1, and its index is 1. The system maintains a list or dictionary of experts, which can be used directly. The target expert module is used as an index to retrieve its instance and call its forward method; optionally, weighted gating activation is employed. The output g of the gating network is a sparse vector. The system multiplies the input data by each element in g and then feeds it to each expert. Since g contains only one non-zero element (1), only the target expert receives valid input, while the inputs to other experts are zero. This is computationally equivalent to activating only the target expert. This approach is easier to implement in parallel computation and gradient propagation in some deep learning frameworks. Understandably, other methods can also be used to implement expert activation, such as through the dynamic construction of a conditional computation graph; this is not limited here.

[0064] In some embodiments, uneven expert load exists, meaning that during training, the gating network may over-rely on a few dominant experts, resulting in insufficient training for other experts and a decline in the model's generalization ability. To address this, this application introduces an expert load balancing loss. The specific implementation involves: in each training batch, calculating the frequency distribution of all data samples assigned to each expert within that batch. Then, quantifying the degree of load imbalance by calculating the square of the coefficient of variation (CV) of this frequency distribution. Loss function Defined as the square of the coefficient of variation, its mathematical expression is as follows:

[0065] ;

[0066] in, It is a gating network for the first The sparse selection vector output from each training sample. This represents the sum of the number of times each expert was selected in a batch, while CV (Concentration Values) measures the dispersion of the values ​​of each element in this summation vector. This training process penalizes parameter updates that lead to imbalanced expert selection, thereby incentivizing the gating network to explore and utilize all experts, ensuring that each expert is effectively trained, ultimately improving the model's robustness and prediction accuracy. That is, when expert selection in a batch is highly imbalanced, This produces a large value, which imposes a penalty gradient on the gating network's parameters during backpropagation, forcing the gating network to explore and utilize other undertrained experts in subsequent training. In this way, it ensures that all experts are effectively trained, improving the overall performance and robustness of the model.

[0067] S105. The observation trajectory is optimized based on the target expert module to obtain the optimized trajectory.

[0068] The optimization process involves adjusting the original observation trajectory based on the optimal observation length, making its length and content more conducive to prediction. This process includes two operations: observation pruning and reverse prediction. The optimized trajectory is the final trajectory sequence used as input to the backbone prediction network after the above processing.

[0069] Specifically, the activated target expert module Upon receiving the original observation trajectory (Its current length is) After that, the following logical judgments and operations will be executed: it will compare... and the optimal length it represents .

[0070] if ≥ This means that the current observation is too long and may contain interfering information. In this case, the expert module will perform an observation pruning operation, which simply truncates the observation from the end of the trajectory forward. At each point, discard earlier historical data and generate an optimized trajectory.

[0071] if < This means that the current observation information is insufficient. At this time, the expert module will perform a reverse prediction operation, that is, use a built-in sub-network (such as MLP) to generate a prediction based on the existing trajectory feature representation. - The missing historical trajectory points are pieced together and connected to the starting point of the original trajectory to form a trajectory of length [length missing]. The optimized trajectory.

[0072] Ultimately, this ensures the optimized trajectory of the output. The length of all are equal to .

[0073] In some embodiments, this step can be implemented in several ways: Optionally, for backward prediction, the expert module contains a multilayer perceptron (MLP). It receives trajectory feature representations Output a flattened vector that can be reshaped into ( - A 2x2 matrix represents the predicted historical coordinate points. Then perform the splicing operation. =Concat( , );in,

[0074] ;

[0075] Optionally, backward prediction can employ an autoregressive decoder architecture, such as a GRU or Transformer decoder, which uses trajectory features... Starting from the initial state, one historical point is generated at a time, and this generated point is used as the input for the next step until the required number of points are generated. This approach may generate historical trajectories with more temporal coherence. Understandably, other optimization methods can also be used, such as attention-weighted resampling instead of simple pruning for excessively long trajectories; this is not limited here.

[0076] In some embodiments, it is necessary to effectively train a subnetwork that performs inverse prediction. This ensures the generation of high-quality historical trajectories. Complete real-world trajectories are used directly. As a monitoring signal, it may not be optimal because It may also contain misleading information. To address this, this application proposes a dynamic expert distillation strategy. At each step of the training process, the complete observations are first... The input is fed into the ALS-MoE framework, which is in evaluation mode and whose parameters are frozen, resulting in a refined and improved trajectory version. Then, when training the inverse prediction module, the original [method / mechanism] is no longer used. Instead, use this high-quality "soft label". As a monitoring signal, by minimizing the currently generated optimized trajectory. and soft labels KL divergence loss on its overlapping portion ,Will The pure knowledge contained within is distilled to .

[0077] ;

[0078] ;

[0079] in, This represents the length of the optimized trajectory of any length, while This indicates the length of the complete observation after optimization. It is the smaller of these two length values. To transfer as much effective knowledge as possible to the optimized trajectory of arbitrary length... We are in soft labels and Loss is calculated on common segments to achieve efficient knowledge distillation.

[0080] This method can effectively filter noise and interference in the original supervision signal, ensuring the efficiency and accuracy of knowledge transfer.

[0081] S106. Optimize the trajectory input trajectory prediction backbone network to obtain the future trajectory prediction results.

[0082] The trajectory prediction backbone network refers to the deep learning model responsible for performing the core prediction task; it can be any advanced trajectory prediction model architecture. The future trajectory prediction result is the model's output, which is the sequence of path points most likely to be traversed by the agent in the next period of time (e.g., the next 3-5 seconds).

[0083] Specifically, after a series of adaptive optimizations in step S105, the trajectory prediction system obtains an optimized trajectory with the optimal length and pure information. This trajectory is now fed as input into the trajectory prediction backbone network. This backbone network functions similarly to standard prediction models in related technologies; it receives historical trajectories and outputs predictions of future trajectories. Since the framework of this application is plug-and-play, this backbone network can be any off-the-shelf, high-performance prediction model. The backbone network is already of high quality, allowing it to operate under more ideal input conditions and thus make more accurate predictions. Its output, predicting future trajectories, can be used in downstream modules, such as the planning and control systems of autonomous vehicles, to make safe driving decisions.

[0084] In some embodiments, this step can be implemented in several ways: Optionally, the trajectory prediction backbone network can be a Seq2Seq-based model, whose encoder part processes and optimizes the trajectory. The decoder part then autoregressively generates a sequence of future trajectory points; optionally, the backbone network can be a generative adversarial network (GAN) based model, whose generator is based on... Multiple candidate future trajectories are generated, and the discriminator is responsible for evaluating the authenticity of these trajectories, ultimately outputting a set of predictions that are both realistic and diverse. Understandably, other advanced prediction models can also be used as the backbone network, such as diffusion-based predictors; this is not a limitation here.

[0085] In some embodiments, this application describes how to perform end-to-end joint training of the proposed ALS-MoE framework with an arbitrary trajectory prediction backbone network. To this end, this application treats the entire process as a unified large network and defines a total training loss function. The total loss consists of a weighted sum of four parts: = +α +β +γ* .in, It is the original loss function of the downstream prediction backbone network itself (such as ADE / FDE). It is the experts who balance the losses; It is a loss due to knowledge distillation; This is the regularization loss of LRoPE; α, β, and γ are the balancing hyperparameters. During training, these are calculated... The gradients of all parameters of the entire network (including the ALS-MoE module and the backbone network) are updated in an integrated manner using the backpropagation algorithm.

[0086] The following provides a more detailed description of the process of the method provided in this implementation. Please refer to [link / reference]. Figure 2 This is another flowchart illustrating the trajectory prediction method based on a hybrid expert model in this application.

[0087] S201. Obtain the complete training sample trajectory in the training set, and truncate the complete training sample trajectory to obtain the truncated observation trajectory and the corresponding truncated real historical trajectory.

[0088] Here, a complete training sample trajectory refers to standard trajectory data with a fixed and sufficiently long observation length that has been preprocessed in the dataset used for model training, denoted as Truncating is a data augmentation technique that removes data randomly. The initial segment is used to simulate short observations encountered in the real world. The truncated observation trajectory is the latter half of the trajectory that is preserved, denoted as... The truncated true historical trajectory is the removed first half of the trajectory, which will serve as the supervised ground truth for the reverse prediction task.

[0089] Specifically, during the model training phase, in order for the reverse prediction module to learn how to supplement historical information, the trajectory prediction system needs to construct corresponding training samples. This step serves precisely this purpose. The system extracts a complete sample trajectory from the training set. Its length is Then, the system randomly selects a cutoff point and... It is divided into two parts. The portion from the cutoff point to the end of the trajectory is used as the arbitrary length of the simulated observation, i.e., the truncated observation trajectory. The portion of the trajectory that was cut off, from the starting point to the cutoff point, constitutes the truncated true historical trajectory. For example, given a complete trajectory of length 20, if the first 8 points are randomly truncated, then the last 12 points become... The first 8 points became This pair ( , This constitutes a sample for training the inverse prediction module.

[0090] In some embodiments, this step can be implemented in several ways: optionally, a uniform random truncation is used. For each complete training sample, from [1, A length value is uniformly and randomly sampled within the range of -1]. Then remove the previous track 1 point; optionally, a non-uniform truncation distribution can be used. To allow the model to focus more on the recovery of extremely short trajectories, a non-uniform sampling strategy can be designed, resulting in a longer truncation length (i.e., the retained points). Shorter (shorter) observations are more likely to be sampled. Understandably, other data augmentation methods can also be used to simulate observations of arbitrary length, such as adding random noise to trajectory points or partially occluding them; this is not limited here.

[0091] S202. Input the truncated observation trajectory into the inverse prediction network to be trained to generate the predicted historical trajectory.

[0092] The backpropagation network to be trained is the module in the hybrid expert model responsible for performing information supplementation. Its parameters are randomly initialized in the early stages of training. The predicted historical trajectory is an estimate of the missing historical portion generated by the network based on the features of the truncated observation trajectory.

[0093] Specifically, after preparing the training samples (truncating the observation trajectory) After that, the trajectory prediction system begins training the inverse prediction network. First, Its feature representation is obtained through encoding by an encoder. Then, this feature Feed into the reverse prediction network .network Based on this feature, a vector sequence is output. This vector sequence is its prediction of the truncated portion of the historical trajectory, denoted as . The goal of this process is to make Learn to deduce the most likely representation of the first half of the trajectory based solely on the characteristics of the second half.

[0094] In some embodiments, this step can be implemented in several ways: optionally, a back prediction network. It is a simple multilayer perceptron (MLP). It receives... After several fully connected layers and activation functions, a fixed-size vector is directly output, which can be reshaped into ( The shape of 2) represents all predicted historical coordinate points; optional, It could be a more complex decoder structure, such as a... The GRU network serves as the initial hidden state. This GRU network generates historical coordinate points step-by-step in an autoregressive manner. After generating each point, the information from that point is used as input for the next generation, until a complete set of hidden states is generated. This approach is better able to capture the temporal dependencies within historical trajectories. It is understandable that other generative models, such as the decoder portion of a variational autoencoder (VAE), could also be used; however, this is not a limitation here.

[0095] In some embodiments, inferring the complete history from a short trajectory is highly uncertain, as multiple plausible histories may exist. Forcing the network to precisely match a unique true historical trajectory (a multimodal problem supervised by a single-modal loss) can lead to modeaveraging, where the network learns to generate a fuzzy, average trajectory of all possible histories. To mitigate this problem, the framework of Generative Adversarial Networks (GANs) can be introduced. In this case, the reverse prediction network... As a generator, a discriminator network is introduced. The discriminator's task is to distinguish the true complete history (i.e., splicing ) and the complete history of its generation (i.e. splicing Through adversarial training, the generator The system is incentivized to generate more realistic historical trajectories that are not only closer in coordinates but also more difficult for the discriminator to detect in their overall motion patterns, thus generating more accurate historical predictions.

[0096] S203. Calculate the reconstruction loss value between the predicted historical trajectory and the truncated true historical trajectory.

[0097] The reconstruction loss value is a scalar used to quantify the historical trajectory generated by the reverse prediction network. ) and the true truncated historical trajectory ( The difference between ).

[0098] Specifically, in step S202, the predicted historical trajectory is generated. Next, the trajectory prediction system needs a quantitative metric to evaluate the quality of the prediction. This metric is the reconstruction loss. The system will compare point by point. and The coordinates correspond to the time steps. A common calculation method is to calculate the average of the L1 or L2 distances (i.e., Manhattan distance or Euclidean distance) between them. The smaller the loss value, the closer the predicted historical trajectory is to the true historical trajectory in terms of geometric location. The reconstruction loss value will be used as part of the gradient backpropagation to guide the update of the backpropagation network parameters, making the generated trajectory increasingly closer to the reality.

[0099] In some embodiments, this step can be implemented in several ways: Optionally, Average Displacement Error (ADE) can be used, which calculates the average Euclidean distance between corresponding points of two trajectories across all time steps; alternatively, Huber loss (SmoothL1 Loss) can be used, which behaves as L2 loss when the error is small and as L1 loss when the error is large. Compared to pure L2 loss, Huber loss is less sensitive to outliers, making the training process more stable. It is understood that other loss functions can also be used, such as losses considering the dynamic characteristics of the trajectories, such as differences in velocity or acceleration; this is not limited here.

[0100] In some embodiments, simply minimizing the reconstruction loss in coordinates may not be sufficient to learn high-quality motion patterns. For example, the network might learn to generate a trajectory that is roughly correct in location but very unsmooth or unrealistic in dynamics (such as velocity and acceleration). To address this, additional regularization terms can be added to the reconstruction loss. For example, predicted historical trajectories can be computed. The velocity and acceleration between adjacent points within the sequence are considered, and penalties are imposed for excessively large changes in velocity or acceleration. Furthermore, a loss function based on Dynamic Time Warping (DTW) can be designed, which allows for non-linear distortions on the time axis when comparing two sequences, is more tolerant of slight time asynchrony, and focuses more on the similarity of the overall shape of the two trajectories. This helps to learn more fundamental motion patterns.

[0101] S204. After splicing the predicted historical trajectory and the truncated observation trajectory, input them into the trajectory prediction backbone network to obtain the predicted future trajectory.

[0102] Here, splicing refers to combining the predicted historical trajectories generated in step S202. Compared with the original truncated observation trajectory Merging them according to time sequence forms a complete line of length [length missing]. The pseudo-historical trajectory. Predicting the future trajectory is a prediction of the future made by the backbone network based on this pseudo-historical trajectory.

[0103] Specifically, the core idea of ​​this step is to evaluate the actual utility of the information supplemented by the reverse prediction module for the final prediction task. The trajectory prediction system will... Placed Previously, a complete historical trajectory was constructed. =Concat( , Then, this calculated historical trajectory is fed into the trajectory prediction backbone network. The backbone network encodes it and generates predictions for future trajectories. That is, when encountering short observations, the system can first use backward prediction to complete the information before proceeding with the full process of future prediction.

[0104] In some embodiments, this step can be implemented in several ways: optionally, the concatenation operation is a simple array or tensor join. If yes( ,2) tensor yes( The tensors of (2) are concatenated to obtain ( + The tensor of ,2), i.e. The length of the trajectory; optionally, smoothing can be applied at the splicing points. To avoid The last point and An unnatural abrupt transition between the first two points can be addressed by performing local interpolation or smoothing the transition between these two points and their neighboring points during the splicing process. It is understood that the specific structure of the backbone network can be diverse, as described in S106 above, and is not limited here.

[0105] In some embodiments, how are gradient flows from different modules handled during joint training? The backbone prediction network and the backpropagation network may have different learning rates or optimization strategies. Directly propagating gradients from the final prediction loss back to the backpropagation network can lead to training instability. To address this, gradient pruning techniques can be employed to limit the norm of gradients propagated back to the backpropagation network, preventing gradient explosion. Additionally, different weights can be set for gradients from the reconstruction loss and gradients from the prediction loss to balance the goals of "looking similar" and "being useful for prediction." In more refined implementations, an alternating training strategy can even be used: fixing the backbone network parameters, training the backpropagation network for a few steps; then fixing the backpropagation network parameters, training the backbone network for a few steps, and so on, to decouple and stabilize the training process.

[0106] S205. Calculate the predicted loss value between the predicted future trajectory and the actual future trajectory.

[0107] Here, the true future trajectory refers to the trajectory in the training dataset that corresponds to the trajectory of the complete training samples. The paired, actual future paths. The predicted loss value is a scalar used to quantify the difference between the predicted future trajectory generated by the backbone network in step S204 and the actual future trajectory.

[0108] Specifically, the trajectory prediction system compares the predicted future trajectory output by the backbone network in step S204 with the actual future trajectory corresponding to that sample in the dataset. Similar to the reconstruction loss, this comparison is usually performed by calculating the geometric distance between the corresponding time step coordinates of the two trajectory sequences. This prediction loss value directly reflects the final prediction performance of the entire system after supplementing historical information. A smaller prediction loss value means that the information generated by the backpropagation module is beneficial, helping the backbone network make more accurate predictions. This loss value is a key signal driving the backpropagation network to learn information.

[0109] S206. Perform gradient updates on the network parameters of the inverse prediction network based on the reconstruction loss value and the prediction loss value.

[0110] Gradient update is a core step in deep learning model training. It refers to adjusting parameters such as weights and biases in the network using optimization algorithms (such as Adam and SGD) based on the gradient calculated from the loss function, in order to obtain a smaller loss in the next calculation.

[0111] Specifically, the trajectory prediction system performs a weighted sum of the reconstruction loss value calculated in step S203 and the prediction loss value calculated in step S205 to obtain a total loss for the backpropagation network. Then, the system uses an automatic differentiation mechanism to calculate this total loss with respect to the backpropagation network. The partial derivatives of all learnable parameters are called gradients. These gradients indicate the direction in which the parameters should be adjusted to minimize the total loss as quickly as possible. Finally, the system uses an optimizer (e.g., Adam) to update the parameters based on the calculated gradients. The parameters are set. By repeatedly executing the process from S201 to S206 on the entire training set, the backpropagation network will gradually learn to generate historical trajectories that are both realistic and beneficial to downstream tasks.

[0112] In some embodiments, this step can be implemented in several ways: optionally, the total loss is = * + * ,in It is the reconstruction loss. It is about predicting losses. and These are the hyperparameters used to balance the two. Then, the Adam optimizer performs a one-step parameter update; optionally, a curriculum learning strategy can be employed. In the early stages of training, a relatively large [parameter value] is set. and smaller The goal is to first teach the network to generate a relatively simple task: creating historically realistic trajectories. As training progresses, the complexity is gradually reduced. and increase This guides the network to focus more on the more difficult and core task of improving the final prediction performance. Understandably, other optimization techniques, such as learning rate decay and weight regularization, can also be used, but these are not limited here.

[0113] In some embodiments, the trajectory prediction system introduces a load balancing mechanism to ensure that all experts in the hybrid expert model are effectively trained. Specifically, the trajectory prediction system records the activation counts of each expert module in the hybrid expert model in the training batch, obtaining an activation count list; calculates the mean and standard deviation of each element in the activation count list to obtain distribution parameters; calculates the load imbalance coefficient based on the ratio of the standard deviation to the mean; and applies a penalty gradient to the weight parameters of the gating network when the load imbalance coefficient exceeds a preset balancing threshold, and adjusts the weight parameters of the gating network based on the penalty gradient to balance the activation probabilities of each expert module.

[0114] The activation count list is a vector of length B (total number of experts), where the i-th element represents the number of times expert i is selected in a training batch. Distribution parameters include the mean and standard deviation (std) of this list. The load imbalance coefficient is typically referred to as the coefficient of variation (CV), calculated as CV = std / mean. The penalty gradient is a gradient signal calculated based on this coefficient, designed to promote a more uniform activation distribution.

[0115] Specifically, after each training batch, the trajectory prediction system summarizes the routing decisions of the gating network for all samples in that batch, calculating the total number of activations for each expert. Based on this list of activations, the system calculates its coefficient of variation (CV). In an ideal equilibrium state, all experts should be activated approximately equally, at which point both the standard deviation and CV are close to 0. If the CV value is large, it indicates severe load balancing. The system then calculates a load balancing loss proportional to the square of the CV. This loss term will be added to the total loss for that batch. During backpropagation, This will generate a gradient that will act specifically on the parameters of the gating network. Its effect is to "penalize" those parameter settings that lead to unbalanced selection, and to encourage the network to select more of the neglected experts in the future.

[0116] In some embodiments, this step can be implemented in several ways: Optionally, the load balancing loss can be directly defined as... =w* , where w is the hyperparameter weight. This loss updates the gating network using the standard backpropagation algorithm; alternatively, a more direct auxiliary task can be employed. Besides the main task, the batch average activation probability distribution output by the gating network can be required to approximate a uniform distribution as closely as possible. Load balancing can also be achieved by calculating the KL divergence or cross-entropy between these two distributions as an auxiliary loss. Understandably, other load balancing strategies can also be used, such as introducing randomness during routing (e.g., the temperature parameter in Gumbel-Softmax) and gradual annealing; these are not limited here.

[0117] In some embodiments, excessively large load balancing weights may impair the model's predictive performance because the gating network is forced to make choices that are not optimal but still meet the balancing requirements. Conversely, excessively small weights may fail to effectively address the load imbalance problem. To address this, an adaptive weight adjustment strategy can be employed. For example, a target CV value (e.g., 0.1) can be set. When the actual CV is significantly higher than this target, the weight α of the load balancing loss is increased; when the CV is close to or lower than the target, α is decreased, or even set to 0. In this way, the system only forces load balancing when necessary, allowing the model to focus on optimizing the primary prediction task when the load is already relatively balanced, thus achieving a better balance between ensuring sufficient training of experts and improving model performance.

[0118] S207. Obtain observation trajectories of arbitrary length.

[0119] Refer to step S101, which will not be repeated here.

[0120] S208. Input the coordinate sequence of the observed trajectory into the preset encoder to obtain the hidden layer state sequence.

[0121] The coordinate sequence is a set of all coordinate points in the observed trajectory arranged in chronological order. The hidden state sequence is a high-dimensional vector sequence generated by the encoder after processing the input at each time step; it captures the accumulated trajectory information up to the current time step.

[0122] Specifically, the trajectory prediction system first transforms the coordinates (e.g., two-dimensional (x, y)) of each point on the observed trajectory into a higher-dimensional vector through an input embedding layer (usually a linear layer). This vector sequence is then fed into the main structure of the encoder (such as a Transformer or RNN). The encoder processes this sequence and generates a corresponding hidden state vector for each input time step. For example, for a trajectory of length L, the output is a hidden state sequence of length L, where the t-th vector in the sequence... This represents the model's understanding of the first t points of the trajectory.

[0123] In some embodiments, this step can be implemented in several ways: Optionally, a multi-layer LSTM network can be used. The coordinate embedding sequence is input into the LSTM, and the output hidden states at each time step constitute the hidden state sequence; alternatively, a multi-layer Transformer encoder can be used. The coordinate embedding sequence is input into the Transformer, and its output sequence is the hidden state sequence. It is understood that other sequence modeling methods, such as temporal convolutional networks (TCNs), can also be used, and this is not limited here.

[0124] In some embodiments, it is necessary to process trajectory data with different sampling rates. Real-world sensors may collect data at different frequencies, resulting in uneven time intervals between trajectory points. If the model does not consider this factor, it will misinterpret the dynamics of the motion. To address this, in addition to coordinate information, the time difference Δt between the current point and the previous point can be appended to the input embedding layer. This allows the model to explicitly obtain the time interval information during encoding, thereby more accurately inferring velocity and acceleration and understanding the true dynamics of the trajectory. For example, the same 1-meter displacement occurring within 0.1 seconds and within 1 second represents completely different motion states.

[0125] S209. Calculate the relative time interval between each time step in the observation trajectory, and map the relative time interval to the rotation position encoding parameter.

[0126] Here, the relative time interval refers to the time difference between any two time steps m and n in the trajectory sequence. The rotation position encoding parameters are a set of parameters used to construct the rotation matrix. In this application, these parameters are dynamically generated based on the relative time interval and are learnable.

[0127] Specifically, to enable architectures like Transformer, which lack temporal concepts, to understand the order and distance of sequences, positional encoding is required. This application employs an advanced Learnable Rotational Positional Encoding (LRoPE). This step is the core preparatory work for LRoPE. The trajectory prediction system calculates the relative time interval (mn) of each position m in the sequence relative to all other positions n. Then, instead of using a fixed formula, the system inputs this relative interval value into a small learnable network or directly multiplies it by a set of learnable parameters θ̃ to generate the final parameters for the rotation operation. This learnable parameter θ̃ is adaptively learned by the model during training based on data characteristics, and it determines how the time interval affects the rotation angle of the positional encoding.

[0128] In some embodiments, this step can be implemented in several ways: Optionally, the rotation position encoding parameter θ can be directly obtained by multiplying the relative position (mn) by a learnable fundamental frequency vector θ̃, i.e.:

[0129] =(mn)* ;

[0130] Optionally, a more complex mapping function can be designed, such as a small MLP, where the input is the relative position (mn) and the output is the rotation parameter θ. Understandably, the implementation details of rotation encoding can vary, such as which dimensions of the feature the rotation operation applies to; these are not limited here.

[0131] In some embodiments, the stability of the learned rotation parameter θ̃ is ensured to prevent it from becoming too large or too small during training, which could lead to numerical instability or model non-convergence. To address this, this application introduces a regularization loss. The loss function constrains the range of θ̃, for example, by penalizing θ̃ values ​​that are less than 0 or greater than 1 using the ReLU function:

[0132] Adding this loss term to the total training loss ensures that the learned fundamental frequency parameter θ̃ remains within a reasonable range, thus guaranteeing the stability and effectiveness of the LRoPE module.

[0133] S210. Based on the rotation position coding parameters, perform rotation transformation on the hidden layer state sequence to obtain an intermediate feature sequence carrying relative temporal information.

[0134] The rotation transformation refers to performing a complex multiplication-like or higher-dimensional rotation operation on the hidden state vector using a rotation matrix constructed from rotation position encoding parameters. The intermediate feature sequence is a feature sequence that has been injected with relative position information after the rotation transformation.

[0135] Specifically, before the attention mechanism computes the dot product of the query vector q and the key vector k, this application applies a rotation transformation to them. For the query vector at position m... and the key vector at position n The system will use the rotation parameters generated in step S209 based on the relative position (mn) to construct a rotation matrix R(mn). Then, it will respectively... and Apply this rotation (or apply it to only one of them, the effect is similar). After rotation... and The dot product depends not only on their contents but also on their relative positions (mn). This operation is equivalent to incorporating positional information into the attention computation. After performing this operation on the hidden states at all positions, an intermediate feature sequence carrying rich relative temporal information is obtained. That is, for the query vector in the attention mechanism... and key vector Through a relative position and learnable parameters rotation matrix To inject location information:

[0136] ;in, This is obtained by the model adaptively learning based on the characteristics of the data during training. To ensure the stability of the learning process, we also introduced a regularization loss. ,make sure The value is within a reasonable range. This allows the location encoding to better adapt to trajectory data of different lengths and patterns. That is:

[0137] .

[0138] In some embodiments, this step can be implemented in several ways: optionally, each hidden layer state vector The D-dimensional features are paired up and considered as D / 2 complex numbers. The rotation operation is to multiply each complex number by... ,in The rotation angle is calculated based on position t. Optionally, a block diagonal rotation matrix can be constructed, where each 2x2 block is a standard rotation matrix [[cosθ, -sinθ], [sinθ, cosθ]], and this large matrix is ​​then multiplied by the hidden state vector. Understandably, the specific implementation of the rotation can be adjusted; the key idea is to inject position information through rotation while maintaining the norm, which is not limited here.

[0139] S211. Perform pooling operation on the intermediate feature sequence to obtain the trajectory feature representation.

[0140] Pooling is a dimensionality reduction technique used to aggregate the features of a sequence into a single, fixed-size vector. The trajectory feature representation is this aggregated vector, which represents the entire trajectory and is input into the subsequent gating network.

[0141] Specifically, after processing in step S210, the trajectory prediction system obtains a feature sequence, where the vector at each time step contains rich contextual and temporal information. However, the gating network requires a single vector as input to make decisions. Therefore, the trajectory prediction system needs to compress this feature sequence into a single vector. A simple method is average pooling, which calculates the element-wise average of all vectors in the sequence. Another method is max pooling, which takes the maximum value of all vectors in each dimension. Through pooling, the spatiotemporal dynamics of the entire trajectory are summarized in the final trajectory feature representation. middle.

[0142] In some embodiments, this step can be implemented in several ways: Optionally, mean pooling can be used to calculate the arithmetic mean of all vectors in the intermediate feature sequence; alternatively, a special learnable [CLS] (classification) label can be added at the beginning of the input sequence, and after Transformer encoding, the output vector corresponding to this [CLS] label can be directly used as the trajectory feature representation of the entire sequence. This method has proven effective in many Transformer-based classification tasks. It is understood that other pooling strategies can also be used, such as attention-based weighted pooling, which is not limited here.

[0143] In some embodiments, simple pooling operations (such as average pooling) may lose important local information in the sequence because it treats all time steps equally. However, certain parts of the trajectory (such as the instant a turn begins) may contain more information about intent than other parts (such as constant speed in a straight line). Attention pooling can be used to address this. Specifically, a learnable query vector is introduced. The query vector is then subjected to attention calculations with each vector in the intermediate feature sequence, resulting in an attention weight distribution. This weight distribution is then used to perform a weighted summation of the feature sequences, yielding the final trajectory feature representation. In this way, the model learns to selectively focus on the most informative parts of the trajectory, thereby generating a more representative trajectory feature representation.

[0144] S212. Input the trajectory feature representation into the gating network and calculate the optimal observation length that matches the observation trajectory.

[0145] Refer to step S103, which will not be repeated here.

[0146] S213. Based on the optimal observation length, activate the corresponding target expert module in the preset hybrid expert model.

[0147] Refer to step S104, which will not be repeated here.

[0148] S214. The observation trajectory is optimized based on the target expert module to obtain the optimized trajectory.

[0149] Refer to step S105, which will not be repeated here.

[0150] It should be noted that during the optimization process, when the trajectory prediction system determines that the current observation information is insufficient (i.e., the current length is less than the optimal observation length), it will perform reverse prediction to supplement the missing historical trajectory. That is, the trajectory prediction system will calculate the absolute value of the length difference between the optimal observation length and the current length to obtain the number of time steps that need to be supplemented; input the trajectory feature representation into the reverse prediction network to generate a number of historical coordinate points equal to the number of time steps; and concatenate the historical coordinate points in chronological order before the beginning of the observed trajectory to obtain the completed trajectory sequence, which is used as the output of the optimized trajectory.

[0151] Among them, the absolute value of the length difference =| - | represents the length of the historical trajectory that needs to be predicted in reverse.

[0152] Specifically, when the gating network determines the current length Less than the optimal length At that time, the activated target expert module It will initiate its internal reverse prediction process. First, it calculates the amount that needs to be generated. Each historical point. Then, it will be represented by the trajectory features generated by the main encoder. As input, it is fed into its proprietary reverse prediction network. This network, typically an MLP or a small decoder, outputs a shape of ( The tensor of 2), i.e., the predicted tensor. These are historical coordinate points. Finally, the system performs a stitching operation to combine these... The newly generated points are placed on the original observation trajectory in reverse chronological order (from most recent history to oldest history). At the very front, forming a length of The optimized trajectory, with more complete information, is then output.

[0153] In some embodiments, this step can be implemented in several ways: optionally, a back prediction network. It is a feedforward network that generates all of them in parallel at once. 1.5 historical points. This is computationally efficient, but may not guarantee the temporal consistency between generated points; alternatively, It is an autoregressive RNN (such as GRU), which uses... Initialize its hidden state, then loop. In each iteration, it generates a historical point and uses the information of that point (or its embedding) as input for the next iteration. This approach better models the dynamic dependencies within the historical trajectory, generating smoother and more realistic trajectories. Understandably, other generative models, such as flow-based models or diffusion models, can also be used for reverse prediction; this is not a limitation here.

[0154] In some embodiments, it is necessary to ensure that the spliced ​​trajectory is smooth at the connection point to avoid jumps that do not conform to physical laws. This refers to the latest historical point predicted in reverse. [-1] and the first point of the original observation The transition between [0] and [0] needs to be natural. This can be achieved by adding a connection point smoothness regularization term to the loss function. This regularization term can be defined as follows: [-1] and The displacement vector between [0] and The difference between the average displacement vectors of the first few points in the sequence. By minimizing this difference, the backpropagation network can be incentivized to generate historical trajectories that can smoothly connect with existing observations. Alternatively, in the post-processing stage, local interpolation methods (such as cubic spline interpolation) can be used to smooth out several points near the splice point to eliminate any sharp transitions.

[0155] S215. Optimize the trajectory input trajectory prediction backbone network to obtain the future trajectory prediction results.

[0156] Refer to step S106, which will not be repeated here.

[0157] In some embodiments, to improve the robustness of prediction in decision boundary regions, the trajectory prediction system employs a soft decision and fusion strategy. Specifically, the system calculates the length difference between the current length and the optimal observed length to obtain a length deviation. When the observed trajectory is determined to be located in a classification boundary region based on the length deviation, the system extracts the activation probability distribution of each expert module output by the gating network, identifies candidate expert modules with activation probabilities greater than a preset probability threshold, and obtains a candidate expert set. Each candidate expert module in the candidate expert set is then driven to process the observed trajectory, resulting in multiple candidate optimized trajectories. Using the activation probabilities corresponding to each candidate expert module as weighting coefficients, the time step coordinates of the time-aligned multiple candidate optimized trajectories are weighted and fused to obtain a fused optimized trajectory. This fused optimized trajectory replaces the original optimized trajectory and is input into the trajectory prediction backbone network.

[0158] Wherein, the length deviation is the current trajectory length. Optimal length for gating network selection The difference between them. The classification boundary region refers to those trajectories whose characteristics make it difficult for the gating network to make a clear single choice; that is, they lie between two optimal length choices. The output before operation represents the selection probability of each expert. The candidate expert set consists of experts whose activation probabilities exceed a preset threshold (e.g., 0.1). Weighted fusion is performed by taking the coordinates of multiple optimized trajectories at each time step and averaging them according to the activation probabilities of the corresponding experts.

[0159] Specifically, for some ambiguous trajectories, the output probability distribution of the gating network may be relatively flat, with no single expert holding an absolute advantage. In this case, the system no longer forces the selection of the highest probability (hard decision), but instead initiates a soft decision process. It collects all candidate experts with a certain probability (probability greater than the threshold) and allows these experts to perform their respective optimization processes on the input trajectory in parallel, resulting in a set of different candidate optimized trajectories. For example, expert 8 might output a trajectory pruned to 8 points, while expert 10 might output a trajectory predicted backward to 10 points. Then, the system calculates a weighted average of the coordinates of these trajectories at each time step based on the initial activation probability of each expert, generating the final fused optimized trajectory. This method integrates the judgments of multiple experts, and the results are generally more stable.

[0160] In some embodiments, this step can be implemented in several ways: Optionally, time alignment can be based on end-of-trajectory alignment. Since the trajectory lengths output by different experts vary, they need to be aligned before fusion. One approach is to align based on the last time step of all trajectories, padding the shorter trajectories with zeros or specific values ​​at the beginning; alternatively, fusion can be a weighted average of coordinates or a weighted average of the feature space. That is, each candidate optimized trajectory is first passed through the encoder of the backbone network to obtain its own features, then these features are weighted and fused, and finally the fused features are fed into the decoder of the backbone network. It is understood that the methods for determining boundary regions can also be diverse; for example, the entropy value of the probability distribution output by the gating network can be used. A higher entropy value indicates greater uncertainty and a higher probability of being located in a boundary region; this is not limited here.

[0161] In some embodiments, to ensure the physical authenticity of the optimized trajectory, the trajectory prediction system adds a smoothing correction step. Specifically, the trajectory prediction system calculates the displacement vectors between adjacent time steps in the fused optimized trajectory to obtain a displacement sequence; performs second-order difference on the displacement sequence to obtain an acceleration feature sequence characterizing the smoothness of the trajectory; detects abnormal time steps in the acceleration feature sequence whose magnitude exceeds a preset physical limit threshold, generating a set of motion mutations; and uses a local polynomial interpolation algorithm to perform smoothing correction on the coordinate points of the corresponding set of motion mutations in the fused optimized trajectory to obtain a smooth fused trajectory that conforms to kinematic constraints. The smooth fused trajectory is then input as the optimized trajectory into the trajectory prediction backbone network.

[0162] In this context, the displacement sequence is the coordinate difference vector between adjacent points in the trajectory. The acceleration feature sequence is obtained by further differencing the displacement sequence, approximating the acceleration of the trajectory. The physical limit threshold is a reasonable upper limit for acceleration set based on the type of agent (e.g., human, vehicle). The motion mutation set is the index of time steps with anomalous acceleration. Local polynomial interpolation (such as the Savitzky-Golay filter) is an algorithm that smooths data while preserving its characteristic shape well.

[0163] Specifically, while the fused trajectory from the previous step is stable, it may contain uneven "peaks" or "corners" at certain points, corresponding to instantaneous, physically inflexible, large accelerations. This step aims to eliminate these imperfections. The system calculates the point-by-point acceleration of the fused trajectory and compares it to a preset physical limit. All points exceeding the acceleration limit are identified as "outliers." Then, the system applies a local polynomial interpolation algorithm to each outlier and two points within a small window surrounding it (e.g., two points before and after). This algorithm fits the data within this window with a low-order polynomial and replaces the original outlier coordinates with the value of the fitted polynomial at the outlier's location. This process is equivalent to replacing the original sharp corners with a smooth curve segment, thus making the entire trajectory kinematically consistent.

[0164] In some embodiments, this step can be implemented in several ways: Optionally, in addition to acceleration, abrupt changes in velocity and angular velocity can be detected simultaneously. Points where the velocity or angular velocity exceeds a preset threshold are also added to the set of motion abrupt changes. Optionally, in addition to local polynomial interpolation, the smoothing correction algorithm can also use B-spline fitting. That is, a piecewise polynomial curve (B-spline) is used to fit the entire fusion optimization trajectory. This fitted curve naturally has high-order continuity (i.e., smoothness) and can be used as the final smoothed fusion trajectory. It is understood that other smoothing filtering techniques in signal processing, such as moving average filtering, can also be used, which are not limited here.

[0165] In some embodiments, a balance needs to be struck between smoothing noise and preserving the true intent. Over-smoothing may erase some important, subtle features in the trajectory that foreshadow behavioral changes, such as slight lateral movement of a vehicle before a lane change. To address this, an adaptive smoothing strategy can be employed. The parameters of the smoothing algorithm (such as polynomial order and window size) can not be fixed but dynamically adjusted based on the local characteristics of the trajectory. For example, in areas of gentle trajectory change (such as straight-line driving), a larger window and a lower-order polynomial can be used for strong smoothing; while in areas of rapid trajectory change (such as turning), a smaller window and a higher-order polynomial are used to more accurately preserve the shape of the turn. This adaptive smoothing can effectively filter out artifacts while maximizing the preservation of the trajectory's effective dynamic information.

[0166] In this embodiment, by employing an adaptive observation trajectory length selection and optimization framework based on a hybrid expert model, and combining a series of innovative technologies such as learnable rotational position encoding designed specifically for variable-length sequences, a task-oriented reverse prediction training strategy, and dynamic expert distillation, this application can perform personalized preprocessing on observation trajectories of arbitrary length in the real world, adjusting them to the optimal state regardless of information overload or insufficient information. This scheme effectively solves the problem of severely degraded prediction performance and poor robustness of existing trajectory prediction methods when facing real observations of variable length that do not conform to ideal datasets, due to model assumption mismatch and improper information processing. Thus, it achieves high-precision prediction of the future trajectories of any traffic participant in complex dynamic environments.

[0167] The trajectory prediction system in the embodiments of this invention is described below from the perspective of hardware processing. Please refer to [link / reference]. Figure 3 This is a schematic diagram of the physical device structure of a trajectory prediction system in an embodiment of this application.

[0168] It should be noted that, Figure 3 The structure of the trajectory prediction system shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0169] like Figure 3 As shown, the trajectory prediction system includes a CPU 301, which can perform various appropriate actions and processes according to a program stored in ROM 302 or a program loaded from storage section 308 into RAM 303, such as executing the methods described in the above embodiments. RAM 303 also stores various programs and data required for system operation. The CPU 301, ROM 302, and RAM 303 are interconnected via bus 304. I / O interface 305 is also connected to bus 304.

[0170] The following components are connected to I / O interface 305: input section 306 including audio input devices, push-button switches, etc.; output section 307 including liquid crystal display (LCD) and audio output devices, indicator lights, etc.; storage section 308 including hard disks, etc.; and communication section 309 including network interface cards such as LAN (Local Area Network) cards, modems, etc. Communication section 309 performs communication processing via a network such as the Internet. Drive 310 is also connected to I / O interface 305 as needed. Removable media 311, such as disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on drive 310 as needed so that computer programs read from them can be installed into storage section 308 as needed.

[0171] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing computer programs for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 309, and / or installed from removable medium 311. When the computer program is executed by CPU 301, it performs the various functions defined in the present invention.

[0172] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, program segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those shown in the drawings.

[0173] Specifically, the trajectory prediction system of this embodiment includes a processor and a memory. The memory stores a computer program. When the computer program is executed by the processor, it implements the trajectory prediction method based on a hybrid expert model provided in the above embodiment.

[0174] In another aspect, the present invention also provides a computer-readable storage medium, which may be included in the trajectory prediction system described in the above embodiments; or it may exist independently and not assembled into the trajectory prediction system. The storage medium carries one or more computer programs that, when executed by a processor of the trajectory prediction system, cause the trajectory prediction system to implement the trajectory prediction method based on a hybrid expert model provided in the above embodiments.

[0175] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0176] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as meaning "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".

Claims

1. A trajectory prediction method based on a hybrid expert model, characterized in that, The method, applied to a trajectory prediction system, includes: Obtain observation trajectories of arbitrary length; The observed trajectory is encoded using a preset encoder to obtain a trajectory feature representation; The trajectory feature representation is input into a gating network to calculate the optimal observation length that fits the observation trajectory. Based on the optimal observation length, activate the corresponding target expert module in the preset hybrid expert model; The observation trajectory is optimized based on the target expert module to obtain an optimized trajectory. The optimization process includes: comparing the current length of the observation trajectory with the optimal observation length; if the current length is greater than or equal to the optimal observation length, then performing observation pruning to remove interference information; if the current length is less than the optimal observation length, then performing reverse prediction to supplement the missing historical trajectory. The optimized trajectory is input into the trajectory prediction backbone network to obtain the future trajectory prediction result.

2. The method according to claim 1, characterized in that, The step of encoding the observed trajectory using a preset encoder to obtain a trajectory feature representation specifically includes: The coordinate sequence of the observed trajectory is input into a preset encoder to obtain the hidden layer state sequence; Calculate the relative time interval between each time step in the observed trajectory, and map the relative time interval to rotational position encoding parameters; Based on the rotation position encoding parameters, the hidden layer state sequence is rotated to obtain an intermediate feature sequence carrying relative temporal information; A pooling operation is performed on the intermediate feature sequence to obtain the trajectory feature representation.

3. The method according to claim 1, characterized in that, The step of performing reverse prediction to supplement missing historical trajectories if the current length is less than the optimal observation length specifically includes: Calculate the absolute value of the length difference between the optimal observation length and the current length to obtain the number of time steps that need to be added; The trajectory feature representation is input into the inverse prediction network to generate historical coordinate points in a number equal to the number of time steps. The historical coordinate points are stitched together in chronological order before the starting point of the observed trajectory to obtain the completed trajectory sequence, which is then used as the optimized trajectory output.

4. The method according to claim 1, characterized in that, Prior to the step of obtaining the observation trajectory of arbitrary length, the method further includes: Obtain the complete training sample trajectory in the training set, and truncate the complete training sample trajectory to obtain the truncated observation trajectory and the corresponding truncated real historical trajectory. The truncated observation trajectory is input into the reverse prediction network to be trained to generate a predicted historical trajectory; Calculate the reconstruction loss value between the predicted historical trajectory and the truncated true historical trajectory; The predicted historical trajectory and the truncated observation trajectory are concatenated and then input into the trajectory prediction backbone network to obtain the predicted future trajectory. Calculate the prediction loss value between the predicted future trajectory and the actual future trajectory; Gradient updates are performed on the network parameters of the inverse prediction network based on the reconstruction loss value and the prediction loss value.

5. The method according to claim 4, characterized in that, After the step of performing gradient updates on the network parameters of the inverse prediction network based on the reconstruction loss value and the prediction loss value, the method further includes: Record the number of activations of each expert module in the hybrid expert model in the training batch to obtain a list of activation counts; Calculate the mean and standard deviation of each element in the list of activation counts to obtain the distribution parameters; The load imbalance coefficient is calculated based on the ratio of the standard deviation to the mean. When the load imbalance coefficient exceeds a preset balancing threshold, a penalty gradient is applied to the weight parameters of the gating network, and the weight parameters of the gating network are adjusted based on the penalty gradient to balance the activation probability of each expert module.

6. The method according to claim 1, characterized in that, After the step of activating the corresponding target expert module in the preset hybrid expert model according to the optimal observation length, the method further includes: Calculate the length difference between the current length and the optimal observed length to obtain the length deviation. When the observed trajectory is determined to be located in the classification boundary region based on the length deviation, the activation probability distribution of each expert module output by the gating network is extracted, and candidate expert modules with activation probabilities greater than a preset probability threshold are identified to obtain a set of candidate experts. Each candidate expert module in the candidate expert set is driven to process the observed trajectory to obtain multiple candidate optimized trajectories; Using the activation probability of each candidate expert module as a weighting coefficient, the time step coordinates of the time-aligned multiple candidate optimization trajectories are weighted and fused to obtain the fused optimization trajectory. The fused optimized trajectory is used to replace the optimized trajectory and input into the trajectory prediction backbone network.

7. The method according to claim 6, characterized in that, After the step of performing weighted fusion of the corresponding time step coordinates of the multiple candidate optimization trajectories according to the activation probability corresponding to each candidate expert module as a weight coefficient to obtain the fused optimization trajectory, the method further includes: Calculate the displacement vector between adjacent time steps in the fused optimized trajectory to obtain the displacement sequence; The displacement sequence is subjected to second-order difference to obtain an acceleration feature sequence characterizing the trajectory smoothness; Detect abnormal time steps in the acceleration feature sequence whose magnitude exceeds a preset physical limit threshold, and generate a set of motion mutations; The coordinates of the corresponding motion mutation set in the fused optimized trajectory are smoothed by using a local polynomial interpolation algorithm to obtain a smooth fused trajectory that meets the kinematic constraints. The smooth fused trajectory is then used as the optimized trajectory and input into the trajectory prediction backbone network.

8. A trajectory prediction system, characterized in that, The trajectory prediction system includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the trajectory prediction system to perform the method as described in any one of claims 1-7.

9. A computer-readable storage medium comprising instructions, characterized in that, When the instructions are executed on the trajectory prediction system, the trajectory prediction system performs the method as described in any one of claims 1-7.

10. A computer program product, characterized in that, When the computer program product is run on the trajectory prediction system, the trajectory prediction system performs the method as described in any one of claims 1-7.

Images