Method, system, device and medium for predicting target trajectory in vehicle surrounding environment

Abstract

This invention proposes a method, system, device, and medium for predicting the trajectory of a target in the surrounding environment of a vehicle. By integrating time-frequency analysis to capture the dynamic characteristics of the trajectory, the accuracy of the prediction is improved. By using current vehicle information, scene information, and historical trajectory information, the global driving trend and local motion details of the vehicle are captured. Its encoder-decoder structure is designed to generate wavelet coefficients, and the decoder integrates a wavelet attention mechanism, utilizing specific scale patterns in historical trajectory data to enhance the model's ability to learn driving patterns at different scales and improve prediction performance.

Description

Technical Field

[0001] This invention belongs to the field of target detection technology, and in particular relates to a method, system, device and medium for predicting the trajectory of targets in the surrounding environment of a vehicle. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] With the widespread adoption of navigation tools and location-tracking technologies such as smartphones, vehicle movement patterns captured in the form of data points such as location and timestamps have become increasingly common. These patterns are now considered a rich data source for intelligent transportation networks, providing crucial insights for practical applications such as motion prediction, anomaly detection, trajectory clustering, trajectory similarity measurement, and travel time estimation. The increasing availability and widespread application of vehicle motion data has prompted the creation of models to effectively learn from and utilize this data.

[0004] To improve the performance of practical applications, it is necessary to develop a trajectory learning model to effectively extract rich semantic information from these trajectories, including vehicle motion behavior and environmental information. Furthermore, efficient extraction of this information is also essential for reducing computational burden and response time. However, achieving these goals faces many challenges.

[0005] First, extracting continuous motion patterns from irregular and discrete trajectory points is challenging. Vehicle motion itself is fluid in both spatial and temporal dimensions. However, this motion is captured through discrete data points. The relationships between these points may not fully capture the fluidity of vehicle motion. Traditional sequential analysis models such as RNNs and Transformers focus only on the connections between these discrete points, limiting their ability to accurately replicate the dynamics of vehicle motion patterns. Methods combining Neural ODEs and custom-designed kernels with Transformer models explicitly extract the continuity of vehicle motion from intermittently recorded data points. While these methods show higher performance, they also incur significant computational costs, potentially hindering model efficiency. Neural ODEs require analytical integral equations, demanding substantial computation even with modern hardware, while Transformers involve quadratic computational complexity. Therefore, there is currently no cost-effective solution for accurately modeling continuous motion from discrete trajectory data points.

[0006] Secondly, capturing the underlying patterns of vehicle trajectories is a challenge. As mentioned above, short-term trajectory prediction is essentially a time series prediction problem, achieved by modeling the complex nonlinear transition patterns of driving trajectories (with multiple interrelated dynamic attributes at each instant). Although time modeling has been used to capture the autoregressive properties of dynamic attributes, studying the underlying driving patterns in sufficient detail remains a challenging task. Generally, the main dynamic characteristics arise from the vehicle's maneuvers during straight-line and turning phases, as well as predetermined driving actions. Considering vehicle safety and driver comfort, the intensity of maneuvering control is limited to obtain finite significance over the time trajectory sequence, making it impossible for traditional models to capture such maneuvering patterns. A practical solution is to utilize frequency domain information to capture informational patterns from time-frequency features, supporting the trajectory prediction task. In the TSF (Trajectory-Short Flow Forecasting) field, frequency domain analysis is used to decompose time series to facilitate in-depth reasoning.

[0007] Considering the time-series nature of trajectory prediction tasks, frequency information is also involved in trajectory attributes. For example, during driving, position and orientation always change along the overall evolution from the starting point to the destination; therefore, the coordinates of the driving trajectory can be considered a partial description of the global driving trend. Driven by driving intentions (going straight, turning, U-turn, etc.), the time series of position, orientation, and velocity, as local details of vehicle motion, also change accordingly. However, current methods mainly focus on time-domain modeling, and time-frequency analysis remains a nascent field in trajectory prediction research, lacking a fine-grained decomposition of frequency details.

[0008] Therefore, how to perform time-frequency analysis to capture potential patterns in driving trajectories, thereby achieving trajectory prediction from a more refined perspective, is a problem that needs to be solved. Summary of the Invention

[0009] To overcome the shortcomings of the prior art, the present invention provides a method, system, device and medium for predicting the trajectory of a target in the vehicle's surrounding environment. It combines wavelet analysis and encoder-decoder to capture the overall driving pattern and fine-grained details of vehicle motion, thereby improving the accuracy of prediction.

[0010] To achieve the above objectives, the present invention adopts the following technical solution:

[0011] In a first aspect, the present invention provides a method for predicting the trajectory of a target in the environment surrounding a vehicle, comprising:

[0012] Acquire historical trajectory data, current vehicle information, and current vehicle scene information;

[0013] Using a target vehicle encoder and a map encoder, the vehicle information and the vehicle scene information at the current time are encoded respectively to obtain vehicle encoding features and scene encoding features;

[0014] The historical trajectory data is subjected to discrete wavelet transform to decompose wavelet coefficients at different time scales and frequencies.

[0015] The high-dimensional trajectory embedding features of the historical trajectory data are extracted using an input embedding network. The high-dimensional trajectory embedding features are enhanced and convolutionally processed using a wavelet attention module to generate context awareness of the wavelet coefficients. The vehicle coding features, the scene coding features, and the context awareness of the wavelet coefficients are respectively input into different decoders corresponding to different wavelet transform coefficients to obtain different sub-band time-frequency attributes of the trajectory sequence.

[0016] By using the discrete wavelet inverse transform to perform inverse transform on the time-frequency attributes of different sub-bands of the obtained trajectory sequence, the trajectory of the vehicle at the next moment can be predicted.

[0017] Secondly, the present invention provides a target trajectory prediction system for the surrounding environment of a vehicle, comprising:

[0018] The acquisition module is configured to acquire historical trajectory data, vehicle information at the current moment, and vehicle scene information at the current moment.

[0019] The encoding module is configured to: use a target vehicle encoder and a map encoder to encode the vehicle information and the vehicle scene information at the current time, respectively, to obtain vehicle encoding features and scene encoding features;

[0020] The wavelet transform module is configured to perform discrete wavelet transform on the historical trajectory data to decompose wavelet coefficients of different time scales and frequencies.

[0021] The embedding module is configured to: extract high-dimensional trajectory embedding features from the historical trajectory data using an input embedding network; enhance and convolve the high-dimensional trajectory embedding features using a wavelet attention module to generate context awareness of the wavelet coefficients; and input the vehicle coding features, the scene coding features, and the context awareness of the wavelet coefficients into different decoders corresponding to different wavelet transform coefficients to obtain different sub-band time-frequency attributes of the trajectory sequence.

[0022] The prediction module is configured to use discrete wavelet inverse transform to perform inverse transform on the time-frequency attributes of different sub-bands of the obtained trajectory sequence to predict the trajectory of the vehicle at the next moment.

[0023] Thirdly, the present invention provides an electronic device including a memory and a processor, and computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the method described in the first aspect.

[0024] Fourthly, the present invention provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in the first aspect.

[0025] The above one or more technical solutions have the following beneficial effects:

[0026] In this invention, dynamic features of the trajectory are captured by integrating time-frequency analysis, thereby improving the accuracy of prediction; the global driving trend and local motion details of the vehicle are captured by using current vehicle information, scene information, and historical trajectory information; its encoder-decoder structure is designed to generate wavelet coefficients, and the decoder integrates a wavelet attention mechanism to utilize specific scale patterns in historical trajectory data, thereby enhancing the model's ability to learn driving modes at different scales and improving prediction performance.

[0027] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0028] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0029] Figure 1 This is a schematic diagram of the overall WMTP framework in Embodiment 1 of the present invention;

[0030] Figure 2 This is a schematic diagram of the encoder structure in Embodiment 1 of the present invention;

[0031] Figure 3 This is a schematic diagram of the decoder structure in Embodiment 1 of the present invention;

[0032] Figure 4 This is a schematic diagram of the wavelet attention mechanism processing in Embodiment 1 of the present invention;

[0033] Figure 5 (a)-(h) are descriptive analysis diagrams of motion prediction of the Argoverse dataset in Embodiment 1 of the present invention. Detailed Implementation

[0034] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0035] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0036] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0037] Example 1

[0038] This embodiment discloses a method for predicting the trajectory of a target in the surrounding environment of a vehicle, including:

[0039] Acquire historical trajectory data, current vehicle information, and current vehicle scene information;

[0040] Using a target vehicle encoder and a map encoder, the vehicle information and the vehicle scene information at the current time are encoded respectively to obtain vehicle encoding features and scene encoding features;

[0041] The historical trajectory data is subjected to discrete wavelet transform to decompose wavelet coefficients at different time scales and frequencies.

[0042] The high-dimensional trajectory embedding features of the historical trajectory data are extracted using an input embedding network. The high-dimensional trajectory embedding features are enhanced and convolutionally processed using a wavelet attention module to generate context awareness of the wavelet coefficients. The vehicle coding features, the scene coding features, and the context awareness of the wavelet coefficients are respectively input into different decoders corresponding to different wavelet transform coefficients to obtain different sub-band time-frequency attributes of the trajectory sequence.

[0043] By using the discrete wavelet inverse transform to perform inverse transform on the time-frequency attributes of different sub-bands of the obtained trajectory sequence, the trajectory of the vehicle at the next moment can be predicted.

[0044] This embodiment proposes a novel framework called Wavelet Mamba Trajectory Prediction (WMTP), which combines wavelet analysis with a Mamba backbone network to capture overall driving patterns and fine-grained vehicle motion details. This analysis method uses Discrete Wavelet Transform (DWT) to decompose trajectory data into wavelet coefficients representing different time scales and frequencies; subsequently, these coefficients can be used to reconstruct the trajectory through the Inverse Wavelet Transform (IDWT) process. In terms of trajectory prediction, the main objective is to extract the most effective wavelet coefficients from the input data, enabling the IDWT to accurately predict subsequent trajectory points.

[0045] This embodiment introduces Discrete Wavelet Transform (DWT) into time-frequency analysis. The filter bank derived from wavelet transform can effectively capture the overall motion trend and intricate motion details of the driving path. Specifically, the wavelet coefficients of each terminal node in the filter bank are called wavelet transform components (WTCs), and these components are arranged in ascending order of frequency. For example, in a three-level DWT, WTC0 corresponds to the coefficients of the highest-level approximation, while the set {WTC}... 4-i} i∈(0,3)∩i∈Z Represents the detail coefficients at level i. Through wavelet multi-resolution analysis, WTC0 can describe the overall trend of gradual changes in the time series, while {WTC 4-i} i∈(0,3)∩i∈Z This represents fine-grained dynamics characterized by rapid fluctuations.

[0046] Given that wavelet analysis encompasses both broader and more detailed dynamics, the time-frequency properties of trajectories revealed by wavelets can be used to uncover the intrinsic properties of driving dynamics. Based on this insight, this embodiment introduces the Wavelet-Mamba Trajectory Predictor framework, or WMTP, which aims to distinguish subtle differences between global driving patterns and local motions from a time-frequency perspective, thereby improving the ability to identify driving behavior and the accuracy of the prediction model.

[0047] State-space models (SSMs) are a set of models that use unobserved states to perform continuous autoregression on data. A prominent example is the S4 model, whose autoregressive process is defined as follows:

[0048]

[0049] y t =Ch t +Dx t (2)

[0050] Where, x t It is the input signal, h t Representing the potential state, y t This is the output signal. Parameters C and D are matrix coefficients. and It is derived from the zero-order hold (ZOH) and Euler's method, which discretizes continuous systems. Although the S4 model provides an efficient framework for sequence modeling, efforts are still underway to extend its applicability to longer and more complex sequences.

[0051] Building upon the S4 model, the S5 model integrates advanced features such as parallel scanning and MIMO-SSM, enhancing the SSM's ability to model sequences. Currently, a data-driven SSM layer called Mamba has been introduced. This layer parameterizes matrices Δ, B, and C based on sequence data, thereby improving sequence capture and modeling for extended sequences. The Mamba module is designed to scale linearly with sequence length, providing efficient memory usage and fast inference.

[0052] This embodiment develops a motion prediction model based on SSM and explores its potential in time series prediction tasks. For trajectory prediction tasks, the Wavelet-Mamba Trajectory Predictor (WMTP) method focuses on predicting wavelet coefficients, which enables the execution of the Discrete Wavelet Inverse Transform (IDWT) algorithm, which reconstructs the original trajectory sequence and predicts the location of subsequent time intervals.

[0053] To achieve this, this embodiment constructs a motion prediction model with an encoder-decoder structure, such as... Figure 1 As shown, the motion prediction model consists of a trajectory embedding layer, a pair of encoders, and a set of decoders aligned with wavelet elements, followed by a series of Inverse Discrete Wavelet Transform (IDWT) steps. The encoders extract features from surrounding vehicles and the map, and then concatenate the outputs. The trajectory embedding layer embeds historical trajectories. The outputs of both the encoders and the embedding layer serve as inputs to the decoders, which predict the output wavelet components. Subsequently, the IDWT synthesizes a series of trajectories from the wavelet elements, encompassing both historical and upcoming motions. The mean squared error of the reconstructed wavelet components is used as the loss function to optimize the motion prediction model parameters. Unlike traditional trajectory prediction techniques, the proposed WMTP framework in this embodiment aims not only to predict the trajectory at the next time step but also to reconstruct past trajectory sequences. This dual functionality allows for a comprehensive understanding of the multi-scale characteristics of motion patterns, thereby improving prediction accuracy.

[0054] In the motion prediction model proposed in this embodiment, historical trajectory data Initially, the input is converted into a high-dimensional vector through the embedding network, denoted as I in Equation (3). Simultaneously, the encoder extracts Agent and Map features and concatenates them, denoted as G in Equation (4). The concatenated result G, along with the high-dimensional vector of the historical trajectory, is passed to the decoder, denoted as H in Equation (5). This latent representation encapsulates the temporal dynamics of the trajectory, which aids in subsequent prediction steps and enables the learning of various driving behaviors.

[0055]

[0056] G=Cat(Encoder(A),Encoder(M′)) (4)

[0057] H = Gather(H′, G) (5)

[0058] Here, A and M′ represent vehicle state information and scene information, respectively. Cat(·) represents the concatenation operation, Gather(·) represents input information collection, and H′ represents the output of the input embedding network. Let N represent the historical trajectory point, M represent the selected M trajectory points, G represent the concatenated output of the two encoders, and H represent the input of the decoder.

[0059] The motion prediction model architecture proposed in this embodiment combines multiple decoders to facilitate in-depth feature extraction across various driving behavior scales. Within the context of L-level wavelet decomposition, the specific task of each decoder is to generate different sub-band time-frequency attributes of the trajectory sequence, denoted as Q. i , representing the wavelet transform coefficient WTC of the i-th wavelet. i Specifically:

[0060] Q i =Decoder i (H), i = 0, 1, ..., L (6)

[0061] Among them, Q i Let L represent the i-th wavelet coefficient, and L represent the wavelet decomposition level.

[0062] To enhance the model's learning capabilities, a novel wavelet attention mechanism is integrated into the decoder. This mechanism, along with historical trajectory embeddings, generates scale-specific features, which are then utilized by a Mamba-inspired module to grasp temporal dependencies.

[0063] Finally, the expected trajectory points are derived through the Inverse Discrete Wavelet Transform (IDWT) process. The basic characteristic of this process is that it incorporates stacked wavelet transform coefficients (WTC) from various decoders, as shown in the formula:

[0064]

[0065] in, This represents M historical trajectory points, from the NMth to the (N-1th)th point, for a total of M points.

[0066] To update the model parameters, a wavelet component MSE loss method is introduced, which measures the difference between the predicted and actual wavelet component values ​​to improve MRA capability and achieve model convergence.

[0067] The following is a detailed description of each part involved in this embodiment:

[0068] Input trajectory embedding: Each point in the trajectory is represented by a distinct vector in a three-dimensional (3D) space defined by its position and velocity. To effectively capture the latent features of the trajectory for further analysis in subsequent layers, these vectors must be transformed into a high-dimensional feature space. Therefore, this embodiment develops an input embedding network that employs linear mappings to enhance the representation of the trajectory sequence, as shown below:

[0069]

[0070] Where, I∈R M×D Represents the input trajectory sequence A high-dimensional representation of W. i1 ∈R d ×(D / / 2) and W i2 ∈R (D / / 2)×D The wavelet decomposition is used to linearly project the low-dimensional attribute vectors of trajectory points into a high-dimensional feature space, where d represents the level of the wavelet decomposition. The ReLU activation function, denoted by σ(·), is used to enhance the model's ability to capture nonlinear relationships. M represents the count of historical trajectory points, D represents the dimension of the feature space, and N refers to the Nth point to be predicted.

[0071] Input embedding networks can cleverly capture the intrinsic features of trajectory points, including the dynamic properties of individual points. Nevertheless, high-level semantic feature extraction is crucial for accurate trajectory prediction across a series of trajectories. To address this issue, this embodiment utilizes a Mamba-inspired encoder to extract features, resulting in dynamic, task-specific, and feature-rich scene embeddings. This approach enables sophisticated analysis of the interactions occurring within a sequence of trajectories.

[0072] like Figure 2 As shown, the encoder has two encoders: a target vehicle encoder and a mapping encoder. Both the target vehicle encoder and the mapping encoder are composed of multiple Mamba blocks, and their outputs are concatenated as input to the downstream decoder. The input to the Agent Encoder is the vehicle's state, such as speed, direction, and position. The input to the map encoder is map or environmental features, such as lane lines. The target vehicle encoder's task is to encode information related to the target vehicle's motion, such as speed and position. Simultaneously, the map encoder is responsible for encoding scene information. The resulting hidden states generated by the encoders are treated as high-level abstract semantic features, and their embedding method is as follows:

[0073] J,h e =Mamba(F,h0) (9)

[0074] in, Initialized to zero, representing the initial hidden state. The input to the Mamba module is F∈RM×D It means that h e This indicates the hidden state of the output, F represents the vehicle's state information or scene information, and J represents the output of the Mamba block.

[0075] The output of the Mamba module is divided into two parts: J∈R M×D and The variable S1 corresponds to the number of levels in the loop structure.

[0076] In this embodiment, the encoder employs Mamba blocks to extract high-level trajectory embeddings rich in temporal details, providing a comprehensive description of the input trajectory sequence. A key prerequisite for trajectory prediction in this embodiment is predicting the wavelet coefficients required for IDWT execution. Over-reliance on traditional wavelet feature engineering, such as simply decomposing the sequence and partially inputting it into the model, fails to fully utilize the multi-resolution capabilities of wavelet analysis.

[0077] Figure 3 This is a diagram of the decoder structure in this embodiment. The wavelet transform coefficients (WTC) serve as the time-frequency representation of the driving mode, providing a series of dynamic features at different resolutions. To cleverly dissect the trajectory embedding and conduct in-depth analysis of the overall trend and details of driving behavior, a wavelet attention mechanism is integrated into each decoder to extract hierarchical features from the trajectory data. The components generated by the decoder illustrate the overall driving trend and local motion details. The encoder's concatenated output is used for hidden state initialization, and the historical trajectory embedding, after passing through the wavelet attention module, is used as the input to the Mamba block. LN is a LayerNorm layer, and FC is a fully connected layer.

[0078] The motion prediction model in this embodiment includes multiple decoders, enabling each decoder to grasp the time-frequency characteristics of trajectory attributes at different scales. The purpose of the decoders is to obtain wavelet components. Based on a preset wavelet decomposition level, the number of decoders and the level are the same, generating wavelet components of the same level. For example, if the preset decomposition level is 2, then the two decoders need to output wavelet components of two different frequency bands. Each decoder includes several cascaded Mamba blocks. Each Mamba block is followed by a LayerNorm layer and a fully connected layer in sequence. The output of the fully connected layer after each Mamba block is used as the output of the decoder.

[0079] like Figure 4 As shown, the wavelet attention mechanism initially involves an enhancement phase where the trajectory embeddings derived from the input embedding network are modulated by attention coefficients to generate enhanced, scale-specific features, i.e., refined trajectory embeddings. These attention coefficients (scores) iThe vectors (i = 0, 1, ..., M-1) are derived from trajectory embeddings and used to evaluate the importance of past trajectory positions. A second stage then involves convolutional operations to refine these enhanced embeddings into contextual embeddings. These are then used to generate wavelet components and synchronized to match the time scale of their respective wavelet components.

[0080] Specifically, the Watt module's architecture comprises two phases: an enhancement phase and a convolution phase. The trajectory embeddings derived from the encoder are enhanced using attention scores to produce refined, scale-oriented features, fundamentally enriching the trajectory embeddings. Subsequently, the Watt module applies convolutional operations to these enhanced embeddings to generate WTC's context-aware embeddings. Mathematically, the enhancement process of the Watt module can be represented as follows:

[0081] I p =σ(I·W p1 )·W p2 (10)

[0082] E=γ(W s2 ·σ(W s1 ·I p (11)

[0083] I h =I+Diag(E·I) (12)

[0084] Among them, W p1 ∈R D×(D / / 2) and W p2 ∈R (D / / 2)×1 Used to embed the trajectory into a linear projection onto the aggregate feature I p ∈R M×1 Middle. W s1 ∈R (M / / 2)×M W s2 ∈R M×(M / / 2) Used to evaluate the importance of features within trajectory embeddings. The ReLU activation function is denoted by σ, while the Sigmoid activation function is denoted by γ. The feature E = R of Sigmoid activation is... M×1 It serves as an attention score, measuring the relevance of past trajectory points to upcoming trajectory points. Fine-grained trajectory embedding I h =R M×D It is achieved by combining the original embedding I with the attention-weighted embedding E=R M×1 This is derived through integration. The Diag(·) operator constructs a diagonal matrix from the input elements.

[0085] Once the accurate trajectory embeddings are obtained, subsequent tasks convert them into WTC context-relevant embeddings. To achieve this, the WATt module applies one-dimensional convolution operations to align the enhanced trajectory embeddings, resulting in highly relevant context embeddings C∈R.h×D These embeddings and the corresponding WTCs obtained by performing discrete wavelet transform on the historical trajectories maintain the same time dimension h. This process can be mathematically represented as follows:

[0086] C=σ(Conv1d(I h (13)

[0087] For the time dimension of C, due to downsampling, the length of the L-level WTC derived from the Discrete Wavelet Transform (DWT) is typically reduced to about half the length of the original sequence. Furthermore, DWT usually involves shifting the wavelet function to convolve the time series. When the filter approaches the boundaries of a finite signal, the convolution operation requires values ​​exceeding the signal limit, which can be addressed using signal spreading techniques. Therefore, the choice of wavelet and the spreading method both affect the precise length of the WTC. Choosing symmetric spreading is a common choice to ensure signal continuity at the boundaries. Using this method, the WTC length of the L-level DWT in a convolution-based algorithm can be derived using the following equation:

[0088]

[0089] b L =b L-1 (16)

[0090] Where L is the wavelet analysis level, l is half the length of the wavelet filter selected in the DWT program, and M is the number of historical trajectory points. i} i∈[0,L)∩i∈Z WTC represents the time length of high frequencies. L-i b i Indicates the duration of high-frequency events. (b) L WTC0 represents the duration of low-frequency events. (Operator) Give the largest integer less than or equal to the input.

[0091] After obtaining the context embeddings from the WATt module, the decoder's Mamba module further refines these embeddings to generate wavelet embeddings; then, a linear transformation is performed on these to obtain WTCs:

[0092] H w =Mamba(G,C) (17)

[0093] Q = FC(LN(H) w (18)

[0094] Among them, H w This indicates the output of the first Mamba block.

[0095] The Mamba blocks are preprocessed using the encoder's long-term memory, providing them with prior knowledge of their historical trajectory sequences and enhancing their ability to utilize features at specific scales. The resulting output feature H... w ∈R H×D The wavelet embedding is performed, and then the desired wavelet coefficients Q∈R are derived through LayerNorm operation LN(·) and linear projection FC(·). h×d .

[0096] In this embodiment, the Discrete Wavelet Inverse Transform (IDWT) module performs an inverse transform on the output of each decoder, i.e., the WTCs of each attribute, thereby reconstructing the historical trajectory sequence and predicting the trajectory attributes at the next moment. The preset reconstruction filter transformation coefficients... Convert to a trajectory attribute series of size d×(M+1).

[0097] Specifically, for each attribute, the filter sequentially merges low-frequency and high-frequency coefficient pairs until the time-domain sequence of the attribute is reconstructed. Mathematically, the time series derivation for the j-th attribute is as follows:

[0098]

[0099] All matrix indices start from zero. At the end of the time series, to explain signal spread, Trim(·) is used to eliminate redundant segments, retaining only the initial M+1 elements along the time axis. Notably, the first M elements correspond to the reconstructed historical trajectory sequence, while the last element represents the expected vehicle state for the next time interval.

[0100] To enable various decoders to grasp the time-frequency characteristics of trajectory attributes at different scales, this embodiment constructs a wavelet loss function. This function summarizes the mean square error of the predicted WTC and guides the network parameter updates, as shown below:

[0101]

[0102] Where L represents the depth of wavelet analysis, h L-k WTC represents the wavelet transform coefficient of the kth wavelet. k The length of g, where g=6, specifies the attribute count considered in this embodiment.

[0103] In formula (21), This corresponds to the i-th element of the k-th WTC predicted by the (k+1)-th decoder for the j-th attribute, and This represents the actual value used for comparison.

[0104] Trajectory prediction of targets in the surrounding environment of autonomous vehicles is crucial. Learning from trajectories is significant for extracting valuable semantic data to improve the accuracy of subsequent tasks. However, vehicle motion is continuous in both space and time, making efficient data extraction from scattered trajectory data points challenging. Furthermore, identifying subtle patterns in vehicle trajectories is another difficulty; these patterns are not obvious in the raw data and are difficult to discern from complex scene details associated with specific regions and road features. To address these issues, this embodiment proposes a novel framework called Wavelet-Mamba Trajectory Prediction (WMTP), which integrates wavelet analysis through state-space modeling to capture the overall trend of driving patterns and the details of vehicle motion. The method employs Discrete Wavelet Transform (DWT) to decompose trajectory data into wavelet coefficients representing different time scales and frequencies, and then uses these coefficients to reconstruct the trajectory through an Inverse Wavelet Transform (IDWT) process. By designing an encoder-decoder neural architecture, latent temporal features are learned from the input trajectory sequence and projected into the wavelet domain, thereby generating wavelet coefficient estimates for historical and future trajectories using scale-oriented decoders. By using the inverse wavelet transform (IDWT) module, these coefficients are reconstructed to improve the accuracy of the prediction.

[0105] The method proposed in this embodiment is illustrated by experiments below:

[0106] A. Experimental Setup

[0107] Datasets: The performance of the method was evaluated using the Argoverse 1 and Argoverse 2 motion prediction datasets. Argoverse 1 consists of 205,942 training sequences, 39,472 validation sequences, and 78,143 test sequences, each captured at a frequency of 10 Hz. The goal is to predict the trajectory for the next 3 seconds using 2 seconds of past data. Argoverse 2 consists of 200,000 training, 25,000 validation, and 25,000 test sequences, also sampled at 10 Hz. Its difference lies in that it uses a 5-second historical window to predict the motion for the next 6 seconds. Both datasets provide high-resolution maps.

[0108] Measurement methods: Primarily using standard metrics popular in the field of multimodal trajectory prediction, including minimum average displacement error (minADE). k Minimum final displacement error (minFDE) k ) and bit error rate (MR) k These metrics evaluate the accuracy of the best predicted trajectory for a single target vehicle among K possible hypotheses compared to the actual trajectory. minADE k The mean Euclidean distance between the predicted path and the ground truth is measured, while minFDE...k Focus only on endpoint differences. MR k minFDE k The proportion of sequences exceeding the 2-meter threshold.

[0109] Parameter settings: The model is configured with a latent vector dimension D of 128, 8 layers for both the encoder and decoder, 2 wavelet analysis levels, and 2 decoder modules. The multi-mode decoder is set to output mode k=6, which is standard practice. The Adam optimizer is used with a batch size of 16 and a learning rate starting at 1e-3, gradually decreasing to 1e-5 after 150 iterations.

[0110] like Figure 5 As shown, the model algorithm has the ability to predict the true, reasonable and accurate trajectories of all target vehicles in the scenario on the Argoverse 1 and 2 datasets.

[0111] To further investigate the performance contribution of the method proposed in this embodiment, particularly the impact of different wavelet analysis depths and wavelet attention modules on trajectory prediction, ablation experiments were conducted. The configuration overview is as follows:

[0112] A1: Level 1 WMTP-We explored the impact of more in-depth wavelet analysis on prediction accuracy, keeping other parameters consistent with the basic MTP model.

[0113] A2: Level 3 WMTP-Extended Analysis. This configuration increases the wavelet depth to 3 while preserving other hyperparameters.

[0114] B1: WMTP without WATT - This variant omits the wavelet attention module WATT to evaluate its contribution to trajectory prediction and its value in time-frequency analysis. It uses autoregressive inference to generate trajectories.

[0115] B2: Level 2 WMTP without WATT - This setting reflects B1 but employs two levels of wavelet analysis.

[0116] B3: Level 3 WMTP, without WATT - similar to B2, but with Level 3 wavelet analysis. This configuration further examines the role of the WATT module.

[0117] Table 3: Experimental results of segmentation ablation studies using the Argoverse 1 motion prediction dataset.

[0118] Models WAtt minADE_6 minFDE_6 MR_6 MDE WMTP w / 0.78 1.16 12.2 0.3727 A1 w / 0.80 1.17 12.3 0.3855 A2 w / 0.85 1.27 12.3 0.5271 B1 w / o 0.88 1.25 12.4 0.5090 B2 w / o 0.85 1.21 12.3 0.4642 B3 w / o 0.90 1.28 12.5 0.5186

[0119] Table 3 lists the experimental results of the ablation study and draws the following conclusions:

[0120] 1. Experimental results regarding wavelet analysis depth demonstrate that the proposed method maintains comparable performance across different analysis levels. Specifically, Level 2 wavelet analysis yields the strongest trajectory prediction, while Level 1 and Level 3 analyses lead to a decline in prediction. WMTP outperforms the A1 framework across all metrics, likely due to the high-frequency wavelet transform coefficients (WTC) capturing more detailed dynamics, thus facilitating fine-grained trajectory modeling in the decoder. However, the absence of a wavelet attention (WAtt) module in B2 is also an improvement over B1, suggesting that higher wavelet levels may be beneficial without WATt.

[0121] Compared to A1, the superior performance of WMTP can likely be attributed to the greater number of wavelet trajectories (WTCs) and richer trajectory dynamics details. In contrast, B2 and B3, lacking WATt compensation, perform poorly, suggesting that excessive wavelet depth can be detrimental without WATt compensation. This is primarily due to the shorter WTC lengths, as shown in Equation (14). The minimum WTC length for A2 and B3 is only 2, which hinders the decoder's ability to recognize trajectory evolution.

[0122] 2. Experimental results consistently demonstrate that the WATt module improves WTC prediction accuracy by capturing scale-oriented features and enriching the contextual representation of cross-scale trajectory sequences. The WATt module reduces prediction errors for all metrics at the equivalent wavelet level.

[0123] The performance degradation observed in level 3 wavelet analysis in A2 is related to the compromised temporal modeling of WTC. The WAtt module enhances the correlation between historical trajectory data and future motion at specific scales, enabling the WMTP framework to predict trajectory patterns more accurately. Therefore, it outperforms B3 among all proposed position component indices.

[0124] 3. As shown in Table 3, although the wavelet-based framework proposed in this embodiment has a simple encoder-decoder structure, it excels in trajectory prediction even without the WATt module. This is mainly due to the framework's time-frequency analysis, which cleverly captures global trends and local nuances across various driving modes. Simultaneously, the WMTP framework utilizes deep feature extraction to identify multi-resolution vehicle motion features. The MDE metrics in Table 3 demonstrate that the time-frequency analysis-based model in this embodiment can predict driving trajectories with high accuracy, highlighting its robustness and practicality. These findings further validate the effectiveness of time-frequency analysis in trajectory prediction tasks.

[0125] This invention introduces a novel time-frequency analysis method for predicting driving trajectories, providing a new perspective for improving trajectory pattern modeling. The newly developed framework, called WMTP, emphasizes the importance of time-frequency analysis in trajectory prediction and overcomes the limitations of traditional techniques in capturing a wide range of subtle trajectory features. This framework utilizes discrete wavelet transform to establish a general time-frequency model designed to refine wavelet coefficients, which is crucial for reconstructing past trajectories and predicting future locations. An encoder-decoder neural network architecture generates wavelet coefficients from historical trajectory data, which are then used for trajectory prediction via inverse discrete wavelet transform (IDWT). Furthermore, the framework introduces a wavelet attention mechanism into the neural network to identify and emphasize features at different scales, thereby improving the model's learning efficiency.

[0126] Example 2

[0127] The purpose of this embodiment is to provide a vehicle surrounding environment target trajectory prediction system, including:

[0128] The acquisition module is configured to acquire historical trajectory data, vehicle information at the current moment, and vehicle scene information at the current moment.

[0129] The encoding module is configured to: use a target vehicle encoder and a map encoder to encode the vehicle information and the vehicle scene information at the current time, respectively, to obtain vehicle encoding features and scene encoding features;

[0130] The wavelet transform module is configured to perform discrete wavelet transform on the historical trajectory data to decompose wavelet coefficients of different time scales and frequencies.

[0131] The embedding module is configured to: extract high-dimensional trajectory embedding features from the historical trajectory data using an input embedding network; enhance and convolve the high-dimensional trajectory embedding features using a wavelet attention module to generate context awareness of the wavelet coefficients; and input the vehicle coding features, the scene coding features, and the context awareness of the wavelet coefficients into different decoders corresponding to different wavelet transform coefficients to obtain different sub-band time-frequency attributes of the trajectory sequence.

[0132] The prediction module is configured to use discrete wavelet inverse transform to perform inverse transform on the time-frequency attributes of different sub-bands of the obtained trajectory sequence to predict the trajectory of the vehicle at the next moment.

[0133] In further embodiments, the following is also provided:

[0134] An electronic device includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor. When executed by the processor, the computer instructions perform the method described in Embodiment 1. For brevity, further details are omitted here.

[0135] It should be understood that in this embodiment, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0136] Memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of memory may also include non-volatile random access memory. For example, memory may also store information about the device type.

[0137] A computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in Embodiment 1.

[0138] The method in Embodiment 1 can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor. The software modules can reside in readily available storage media in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not provided here.

[0139] A computer program product includes a computer program that, when executed by a processor, implements the method described in Embodiment 1.

[0140] The present invention also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions included in program modules, which execute in a device on a target real or virtual processor to perform the processes / methods described above. Typically, program modules include routines, programs, libraries, objects, classes, components, data structures, etc., that perform specific tasks or implement specific abstract data types. In various embodiments, the functionality of program modules can be combined or divided among program modules as needed. The machine-executable instructions for the program modules can execute within a local or distributed device. In a distributed device, the program modules can reside in both local and remote storage media.

[0141] The computer program code used to implement the methods of the present invention may be written in one or more programming languages. This computer program code may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the computer or other programmable data processing device, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a computer, partially on a computer, as a stand-alone software package, partially on a computer and partially on a remote computer, or entirely on a remote computer or server.

[0142] In the context of this invention, computer program code or related data may be carried by any suitable carrier to enable a device, apparatus, or processor to perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, and the like. Examples of signals may include electrical, optical, radio, sound, or other forms of propagation signals, such as carrier waves, infrared signals, etc.

[0143] Those skilled in the art will recognize that the units and algorithm steps described in conjunction with the embodiments herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0144] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for predicting the trajectory of a target in the surrounding environment of a vehicle, characterized in that, include: Acquire historical trajectory data, current vehicle information, and current vehicle scene information; Using a target vehicle encoder and a map encoder, the vehicle information and the vehicle scene information at the current time are encoded respectively to obtain vehicle encoding features and scene encoding features; The historical trajectory data is subjected to discrete wavelet transform to decompose wavelet coefficients at different time scales and frequencies. A high-dimensional trajectory embedding feature is extracted from the historical trajectory data using an input embedding network. This high-dimensional trajectory embedding feature is then enhanced and convolved using a wavelet attention module to generate context-aware wavelet coefficients. The vehicle encoding feature, the scene encoding feature, and the context-aware wavelet coefficients are input into different decoders corresponding to different wavelet transform coefficients to obtain different sub-band time-frequency attributes of the trajectory sequence. Each decoder includes multiple cascaded Mamba blocks, each followed by a LayerNorm layer and a fully connected layer to process the output features of the Mamba block. The output of the fully connected layer corresponding to each Mamba block is used as the different sub-band time-frequency attributes of the trajectory sequence output by the corresponding encoder. The time-frequency attributes of different sub-bands of the obtained trajectory sequence are inversely transformed using discrete wavelet inverse transform to predict the trajectory of the vehicle at the next moment. Specifically, a high-dimensional trajectory embedding feature is extracted from the historical trajectory data using an input embedding network, and the high-dimensional trajectory embedding feature is enhanced and convolved using a wavelet attention module to generate the context-aware wavelet coefficients. ， ， ， ， in, and These represent linear projection matrices, respectively. , Used to evaluate the importance of features within trajectory embeddings; Represents the ReLU activation function; This represents the count of historical trajectory points; Represents the dimension of the feature space; High-dimensional trajectory embedding feature representation of historical trajectory sequences; This represents a one-dimensional convolution; E is the attention-weighted embedding. Indicates context embedding; For fine trajectory embedding.

2. The method for predicting the trajectory of a target in the surrounding environment of a vehicle as described in claim 1, characterized in that, The high-dimensional vector extracted from the historical trajectory data is specifically as follows: in, Indicates the input historical trajectory sequence High-dimensional trajectory embedding feature representation; matrix and Used to linearly project the low-dimensional attribute vectors of trajectory points into a high-dimensional feature space; Represents the ReLU activation function; This represents the count of historical trajectory points. The dimension of the feature space is represented by N, where N represents the Nth trajectory point.

3. The method for predicting the trajectory of a target in the surrounding environment of a vehicle as described in claim 1, characterized in that, The target vehicle encoder and the map encoder respectively use multiple sequentially connected Mamba blocks to encode the vehicle information and the vehicle scene information at the current time, so as to obtain vehicle coding features and scene coding features.

4. The method for predicting the trajectory of a target in the surrounding environment of a vehicle as described in claim 1, characterized in that, The determination of the context-aware time dimension of the wavelet transform coefficients is specifically as follows: ， ， ， in, It's at the level of wavelet analysis. It is half the length of the wavelet filter selected in the wavelet transform. It is the number of historical trajectory points. Represents the time length of high frequencies This indicates the largest integer less than or equal to the input.

5. The method for predicting the trajectory of a target in the surrounding environment of a vehicle as described in claim 1, characterized in that, For the different sub-band time-frequency attributes of each trajectory sequence, low-frequency and high-frequency coefficient pairs are merged sequentially until the time-domain sequence of the attributes is reconstructed.

6. A vehicle surrounding environment target trajectory prediction system, characterized in that, include: The acquisition module is configured to acquire historical trajectory data, vehicle information at the current moment, and vehicle scene information at the current moment. The encoding module is configured to: use a target vehicle encoder and a map encoder to encode the vehicle information and the vehicle scene information at the current time, respectively, to obtain vehicle encoding features and scene encoding features; The wavelet transform module is configured to perform discrete wavelet transform on the historical trajectory data to decompose wavelet coefficients of different time scales and frequencies. The embedding module is configured to: extract high-dimensional trajectory embedding features from the historical trajectory data using an input embedding network; enhance and convolve the high-dimensional trajectory embedding features using a wavelet attention module to generate context-aware wavelet coefficients; input the vehicle encoding features, the scene encoding features, and the context-aware wavelet coefficients into different decoders corresponding to different wavelet transform coefficients to obtain different sub-band time-frequency attributes of the trajectory sequence; each decoder includes multiple cascaded Mamba blocks, each Mamba block is followed by a LayerNorm layer and a fully connected layer to process the output features of the Mamba block; and the output of the fully connected layer corresponding to each Mamba block is used as the different sub-band time-frequency attributes of the trajectory sequence output by the corresponding encoder. Specifically, a high-dimensional trajectory embedding feature is extracted from the historical trajectory data using an input embedding network, and the high-dimensional trajectory embedding feature is enhanced and convolved using a wavelet attention module to generate the context-aware wavelet coefficients. ， ， ， ， in, and These represent linear projection matrices, respectively. , Used to evaluate the importance of features within trajectory embeddings; Represents the ReLU activation function; This represents the count of historical trajectory points; Represents the dimension of the feature space; High-dimensional trajectory embedding feature representation of historical trajectory sequences; This represents a one-dimensional convolution; E is the attention-weighted embedding. Indicates context embedding; For fine trajectory embedding; The prediction module is configured to use discrete wavelet inverse transform to perform inverse transform on the time-frequency attributes of different sub-bands of the obtained trajectory sequence to predict the trajectory of the vehicle at the next moment.

7. An electronic device, characterized in that, It includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, perform the method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, Used to store computer instructions, which, when executed by a processor, perform the method described in any one of claims 1-5.

Images