Trajectory prediction method and apparatus, and vehicle

Abstract

A trajectory prediction method and apparatus, and a vehicle, relating to the technical field of vehicle trajectory prediction. The method comprises: acquiring a first position sequence feature and a lane segment sequence feature of a vehicle to be predicted (step 301); the first position sequence feature being used for representing a position feature of the vehicle to be predicted at a plurality of historical moments, and the lane segment sequence feature being used for representing a positional relationship between the vehicle to be predicted and a surrounding lane segment; on the basis of the first position sequence feature and the lane segment sequence feature, performing an attention mechanism operation to obtain a first lane segment fused feature (step 302); and on the basis of the first lane segment fused feature, obtaining a predicted trajectory of the vehicle to be predicted (step 303).

Description

Trajectory prediction methods, devices and vehicles

[0001] This application claims priority to Chinese Patent Application No. 202510095366.X, filed on January 21, 2025, entitled “Trajectory Prediction Method, Apparatus and Vehicle”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of automotive technology, and more particularly to the field of automotive trajectory prediction technology, specifically to a trajectory prediction method, device, and vehicle. Background Technology

[0003] Autonomous vehicles, as leaders in the transformation and upgrading of the vehicle industry, have received considerable attention and recognition globally. The core of this technology lies in enabling vehicles to make accurate and timely decisions in various complex traffic environments, thereby ensuring driving safety and efficiency. To achieve this goal, the advance prediction of the intentions or trajectories of surrounding vehicles (such as other vehicles and pedestrians) is particularly important.

[0004] Among related technologies, the proposed Multi-PPTP uses a lightweight Mobile Net network as a feature encoder for the raster image, but it cannot make good use of the sparsity of traffic scene information, resulting in inaccurate trajectory prediction.

[0005] In related technologies, a vehicle trajectory feature extraction method is proposed to provide standardized feature data for vehicle trajectory classification or prediction. However, the introduction of sampling increases the uncertainty of the encoding process, leading to inaccurate trajectory prediction. Summary of the Invention

[0006] This application provides a trajectory prediction method, apparatus, and vehicle to at least address the technical problem of low feasibility in trajectory prediction in related technologies. The technical solution of this application is as follows:

[0007] According to a first aspect provided in this application, a trajectory prediction method is provided, the method comprising: acquiring a first position sequence feature and a lane segment sequence feature of a vehicle to be predicted; the first position sequence feature is used to characterize the position features of the vehicle to be predicted at multiple historical moments; the lane segment sequence feature is used to characterize the positional relationship between the vehicle to be predicted and surrounding lane segments; performing an attention mechanism operation based on the first position sequence feature and the lane segment sequence feature to obtain a first lane segment fusion feature; and obtaining a predicted trajectory of the vehicle to be predicted based on the first lane segment fusion feature.

[0008] Based on the above technical solution, the embodiments of this application integrate multi-dimensional information (first position sequence features and lane segment sequence features), and through attention mechanism operation, enable the model to automatically learn and emphasize the position sequence features and lane segment sequence features that have an important impact on the prediction results, thereby improving the accuracy and robustness of the predicted trajectory.

[0009] In some embodiments, the predicted trajectory of the vehicle to be predicted is obtained based on the first lane segment fusion features, including: acquiring vehicle interaction features and second lane segment fusion features corresponding to the surrounding vehicles of the vehicle to be predicted; the vehicle interaction features are used to characterize the positional relationship between the vehicle to be predicted and the surrounding vehicles; attention mechanism operation is performed based on the first lane segment fusion features, the vehicle interaction features and the second lane segment fusion features to obtain target fusion features; and the predicted trajectory of the vehicle to be predicted is generated based on the target fusion features.

[0010] Based on the above technical solution, this application embodiment integrates the second lane segment fusion feature and vehicle interaction feature corresponding to the surrounding vehicles on the basis of the above lane segment sequence features. This can accurately represent the relative position and dynamic relationship between the vehicle to be predicted and the surrounding vehicles. Through attention mechanism operation, the model can better understand the vehicle behavior pattern in complex traffic scenarios, thereby generating a more reliable predicted trajectory.

[0011] In some embodiments, obtaining a first location sequence feature of a vehicle to be predicted includes: obtaining historical location information of the vehicle to be predicted; processing the historical location information based on a first multilayer perceptron (MLP) to obtain a second location sequence feature of the vehicle to be predicted; fusing the second location sequence feature with a time embedding parameter to obtain a third location sequence feature; and performing an attention mechanism operation on the third location sequence feature to obtain the first location sequence feature.

[0012] Based on the above technical solution, this embodiment uses a first multilayer perceptron (MLP) to process historical location information, which can extract deep-level features of vehicle motion (second position sequence features). These second position sequence features help to more accurately predict the future position of the vehicle. Simultaneously, the introduction of a time embedding parameter allows the model to consider the influence of time factors, further improving the accuracy of the prediction.

[0013] In some embodiments, obtaining lane segment sequence features of the vehicle to be predicted includes: obtaining the location information of the surrounding lane segments of the vehicle to be predicted; and processing the location information of the surrounding lane segments based on a second multilayer perceptron (MLP) to obtain lane segment sequence features.

[0014] Based on the above technical solution, this embodiment of the application processes the position information of the surrounding lane segments using a second multilayer perceptron (MLP), which can efficiently extract the key features of the lane segments. As a feedforward neural network, the second multilayer perceptron (MLP) has a strong nonlinear fitting capability and can learn the complex relationship between lane segment position information and vehicle motion, thereby enabling the model to more accurately understand the structure and constraints of the lane segments in subsequent position prediction or trajectory planning.

[0015] In some embodiments, attention mechanism operations are performed based on first location sequence features and lane segment sequence features to obtain first lane segment fusion features, including: mapping the first location sequence features to a first query vector based on a third multilayer perceptron (MLP); mapping the first concatenated features to a first key vector and a first value vector based on a fourth multilayer perceptron (MLP); the first concatenated features are features obtained by concatenating the first location sequence features and lane segment sequence features; and performing attention mechanism operations based on the first query vector, the first key vector, and the first value vector to obtain the first lane segment fusion features.

[0016] Based on the above technical solution, this application embodiment utilizes a third-layer perceptron (MLP) and a fourth-layer perceptron (MLP) to flexibly map the first position sequence features and lane segment sequence features to query vectors, key vectors, and value vectors in a high-dimensional space. This mapping method preserves important information from the original features while reducing redundancy and noise. Furthermore, the attention mechanism dynamically adjusts the weights of different features during the fusion process based on the similarity between the query vector and the key vector, enabling the trajectory prediction model to more accurately understand the relationship between the first position sequence features and the lane segment sequence features, thereby generating the first lane segment fused feature.

[0017] In some embodiments, obtaining vehicle interaction features includes: obtaining the location information of surrounding vehicles of the vehicle to be predicted; and processing the location information of surrounding vehicles based on a fifth multilayer perceptron (MLP) to obtain vehicle interaction features.

[0018] Based on the above technical solution, this application embodiment can establish the connection relationship between the vehicle to be predicted and surrounding vehicles at the current moment, and extract edge features using embedded representation and relative coordinates to construct a global interaction code. This method can accurately capture the dynamic interaction relationship between various vehicles, providing effective technical support for intelligent processing in fields such as intelligent transportation and autonomous driving.

[0019] In some embodiments, attention mechanism operations are performed based on the first lane segment fusion feature, vehicle interaction feature, and second lane segment fusion feature to obtain the target fusion feature, including: mapping the first lane interaction feature to a second query vector based on a sixth multilayer perceptron MLP; mapping the second concatenated feature to a second key vector and a second value vector based on a seventh multilayer perceptron MLP; the second concatenated feature is the feature obtained by concatenating the first lane segment fusion feature, vehicle interaction feature, and second lane segment fusion feature; and performing attention mechanism operations based on the second query vector, second key vector, and second value vector to obtain the target fusion feature.

[0020] According to a second aspect of this application, a training method for a trajectory prediction model is provided. The method includes: acquiring first position sequence features and lane segment sequence features of a vehicle at a first sample time; the first position sequence features represent the vehicle's position at multiple historical times prior to the first sample time; the lane segment sequence features represent the positional relationship between the vehicle and surrounding lane segments at multiple historical times; acquiring the ground truth label and true trajectory of the target lane segment corresponding to the vehicle at a second sample time; processing the first position sequence features and lane segment sequence features based on the trajectory prediction model to obtain the predicted lane segment label and predicted trajectory corresponding to the second sample time; the trajectory prediction model is used to perform attention mechanism operations based on the first position sequence features and lane segment features to obtain a first lane segment fusion feature; obtaining the predicted trajectory based on the first lane segment fusion feature and determining the predicted lane segment label corresponding to the predicted trajectory; calculating a first loss based on the ground truth label and predicted lane segment label; calculating a second loss based on the true trajectory and predicted trajectory; and adjusting the trajectory prediction model according to the first loss and the second loss.

[0021] Based on the above technical solution, this application embodiment comprehensively considers the vehicle's historical location information and its positional relationship with surrounding lane segments. These two features work together in trajectory prediction, providing a more comprehensive reflection of the vehicle's driving status and possible routes. Simultaneously, through attention mechanism computation, the model can dynamically adjust the weights of different features in the prediction process, thereby more accurately capturing key information about the vehicle's movement and improving prediction accuracy.

[0022] In some embodiments, obtaining the first position sequence features and lane segment sequence features of the vehicle at the first sample time includes: obtaining historical position information of the vehicle at multiple historical times and surrounding lane segment position information of the vehicle to be predicted at multiple historical times; and processing the historical position information and surrounding lane segment position information based on the trajectory prediction model to obtain the first position sequence features and lane segment sequence features, respectively.

[0023] In some embodiments, the method further includes: acquiring vehicle interaction features of the vehicle and second lane segment fusion features corresponding to surrounding vehicles; the vehicle interaction features are used to characterize the positional relationship between the vehicle and surrounding vehicles at a first sample time; processing the first position sequence features and lane segment sequence features based on a trajectory prediction model to obtain a predicted lane segment label and a predicted trajectory at the second sample time, including: processing the first lane segment fusion features, vehicle interaction features, and second lane segment fusion features based on a trajectory prediction model to obtain a predicted lane segment label and a predicted trajectory at the second sample time; the trajectory prediction model is used to perform attention mechanism operations based on the first position sequence features and lane segment features to obtain a first lane segment fusion feature; performing attention mechanism operations based on the first lane segment fusion feature, vehicle interaction features, and second lane segment fusion features to obtain a target fusion feature; generating a predicted trajectory of the vehicle based on the target fusion feature, and determining the predicted lane segment label corresponding to the predicted trajectory.

[0024] According to a third aspect provided in this application, a vehicle is provided, including a trajectory prediction device.

[0025] According to a fourth aspect provided in this application, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute instructions to implement the method of the first aspect described above and any possible implementation thereof.

[0026] According to a fifth aspect provided in this application, a computer-readable storage medium is provided that, when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to perform the methods described in the first aspect and any possible implementation thereof.

[0027] According to the sixth aspect provided in this application, a computer program product is provided, the computer program product including computer instructions, which, when executed on an electronic device, cause the electronic device to perform the method described in the first aspect and any possible implementation thereof.

[0028] Therefore, the above-mentioned technical features of this application have the following beneficial effects:

[0029] (1) The embodiments of this application integrate multi-dimensional information (first position sequence features and lane segment sequence features), and through attention mechanism operation, the model can automatically learn and emphasize the position sequence features and lane segment sequence features that have an important impact on the prediction results, thereby improving the accuracy and robustness of the prediction.

[0030] (2) Based on the above lane segment sequence features, this application embodiment integrates the second lane segment fusion features and vehicle interaction features corresponding to the surrounding vehicles, so as to accurately characterize the relative position and dynamic relationship between the vehicle to be predicted and the surrounding vehicles. Through attention mechanism operation, the model can better understand the vehicle behavior patterns in complex traffic scenarios, thereby generating more reliable prediction trajectories.

[0031] (3) In this embodiment, a first multilayer perceptron (MLP) is used to process historical location information, which can extract deep-level features of vehicle motion (second position sequence features). These second position sequence features help to predict the future position of the vehicle more accurately. At the same time, the introduction of a time embedding parameter enables the model to consider the influence of time factors, further improving the accuracy of prediction.

[0032] (4) In this embodiment of the application, the position information of the surrounding lane segments is processed by the second multilayer perceptron (MLP), which can efficiently extract the key features of the lane segments. As a feedforward neural network, the second multilayer perceptron (MLP) has a strong nonlinear fitting capability and can learn the complex relationship between the position information of the lane segments and the vehicle motion, so that the model can more accurately understand the structure and constraints of the lane segments in subsequent position prediction or trajectory planning.

[0033] (5) In this embodiment, the first position sequence features and lane segment sequence features can be flexibly mapped to query vectors, key vectors, and value vectors in a high-dimensional space using a third-layer perceptron (MLP) and a fourth-layer perceptron (MLP). This mapping method preserves important information from the original features while reducing redundancy and noise. Furthermore, the attention mechanism dynamically adjusts the weights of different features during the fusion process based on the similarity between the query vector and the key vector, enabling the trajectory prediction model to more accurately understand the relationship between the first position sequence features and the lane segment sequence features, thereby generating the first lane segment fused features.

[0034] (6) This embodiment of the application establishes the connection relationship between the vehicle to be predicted and surrounding vehicles at the current time, and extracts edge features using embedded representation and relative coordinates to construct a global interaction code. This method can accurately capture the dynamic interaction relationship between various vehicles, providing effective technical support for intelligent processing in fields such as intelligent transportation and autonomous driving.

[0035] (7) This embodiment of the application comprehensively considers the vehicle's historical location information and its positional relationship with surrounding lane segments. These two features work together to predict the trajectory, which can more comprehensively reflect the vehicle's driving status and possible driving path. At the same time, through attention mechanism operation, the model can dynamically adjust the weights of different features in the prediction process, thereby more accurately capturing key information about the vehicle's driving and improving the accuracy of the prediction.

[0036] It should be noted that the technical effects of any of the implementation methods in aspects two through six can be found in the technical effects of the corresponding implementation methods in aspect one, and will not be repeated here.

[0037] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0038] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application, and do not constitute an undue limitation of this application.

[0039] Figure 1 is a schematic diagram of the structure of a vehicle according to an exemplary embodiment;

[0040] Figure 2 is a block diagram illustrating a trajectory prediction device according to an exemplary embodiment;

[0041] Figure 3 is a flowchart illustrating a trajectory prediction method according to an exemplary embodiment;

[0042] Figure 4 is a scene diagram illustrating a trajectory prediction method according to an exemplary embodiment;

[0043] Figure 5 is a scenario diagram illustrating another trajectory prediction method according to an exemplary embodiment;

[0044] Figure 6 is a scenario diagram illustrating another trajectory prediction method according to an exemplary embodiment;

[0045] Figure 7 is a scenario diagram illustrating another trajectory prediction method according to an exemplary embodiment;

[0046] Figure 8 is a scenario diagram illustrating another trajectory prediction method according to an exemplary embodiment;

[0047] Figure 9 is a block diagram illustrating another trajectory prediction device according to an exemplary embodiment;

[0048] Figure 10 is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation

[0049] To enable those skilled in the art to better understand the technical solutions of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0050] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0051] In the embodiments of this application, the words "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the words "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a specific manner.

[0052] First, the relevant technologies involved in this application will be explained to facilitate understanding by those skilled in the art.

[0053] In recent years, the widespread application of deep neural networks in trajectory prediction has indeed brought significant progress to fields such as autonomous driving and intelligent transportation. Among these, target-driven trajectory prediction methods, by decomposing the problem into two sub-tasks—predicting potential targets and estimating motion based on contextual features—have achieved more accurate predictions of future vehicle trajectories. However, while pursuing low prediction errors, these methods often neglect the physical feasibility of the predicted trajectory, a problem worthy of in-depth exploration.

[0054] To address the aforementioned issues, a more complex trajectory representation method can be employed to more comprehensively describe the vehicle's spatial occupancy, thereby generating a predicted trajectory and overcoming the problem of low feasibility. This method can more accurately reflect the actual space occupied by the vehicle, thus improving the physical feasibility of the prediction results.

[0055] In summary, while deep learning-based trajectory prediction methods have achieved significant results in reducing prediction errors, physical feasibility remains a key factor limiting their practical application. In the future, researchers need to incorporate more physical constraints and more complex trajectory representation methods into prediction models to ensure that the generated predicted trajectories are both accurate and feasible. This will provide a more reliable guarantee for the practical application of intelligent transportation systems such as autonomous driving.

[0056] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0057] The trajectory prediction method provided in this application can be applied to vehicles. Figure 1 shows a schematic diagram of a vehicle structure provided in this application embodiment. The vehicle 100 may include a chassis 110, a body 120, and wheels 130. It is understood that the vehicle 100 can be a gasoline-powered vehicle, an electric vehicle, a hybrid vehicle, a natural gas vehicle, a methanol vehicle, a solar-powered vehicle, etc.

[0058] For example, vehicle 100 can be a passenger car such as a sedan, sport utility vehicle (SUV), or multi-purpose vehicle (MPV), or a bus, truck, or semi-trailer. This application does not impose specific limitations in this regard.

[0059] It is understood that the above-mentioned components are merely examples of some components of vehicle 100, and are not a limitation on the specific structure of vehicle 100.

[0060] Optionally, for vehicle control, vehicle 100 may also include a trajectory prediction system 140. The trajectory prediction system 140 can predict the motion trajectory of vehicles around vehicle 100.

[0061] As shown in Figure 2, this application embodiment provides a block diagram of a trajectory prediction system 140. The trajectory prediction system 140 includes an acquisition module 210 and a trajectory prediction module 220.

[0062] The acquisition module 210 is used to acquire the first position sequence features and lane segment sequence features of the vehicle to be predicted; and sends the first position sequence features and lane segment sequence features to the trajectory prediction module 220.

[0063] In some embodiments, the first position sequence feature is used to characterize the position features of the vehicle to be predicted at multiple historical moments; the lane segment sequence feature is used to characterize the positional relationship between the vehicle to be predicted and the surrounding lane segments.

[0064] The trajectory prediction module 220 is used to perform attention mechanism operations based on the first position sequence features and lane segment sequence features to obtain the first lane segment fusion features, and based on the first lane segment fusion features, to obtain the predicted trajectory of the vehicle to be predicted.

[0065] It should be noted that the trajectory prediction system described in the embodiments of this application is for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and does not constitute a limitation on the technical solutions provided in the embodiments of this application. Those skilled in the art will understand that with the evolution of electronic devices and the emergence of other electronic devices, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems. The methods in the following embodiments can all be implemented in the trajectory prediction system with the above-described hardware structure.

[0066] The methods described in the following embodiments can all be implemented in a trajectory prediction system with the above-described hardware structure.

[0067] The trajectory prediction method provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0068] The trajectory prediction method of this application embodiment can be applied to predict the motion trajectory of a vehicle to be predicted. As shown in Figure 3, the trajectory prediction method may include steps 301-303. Step 301 can also be referred to as the "acquiring the first position sequence features and lane segment sequence features of the vehicle to be predicted" process, and steps 302-303 can be referred to as the "generating the predicted trajectory of the vehicle to be predicted" process. Steps 301-303 are described in detail below.

[0069] Step 301: Obtain the first position sequence features and lane segment sequence features of the vehicle to be predicted.

[0070] In the embodiments related to this application, the first position sequence feature is used to characterize the position features of the vehicle to be predicted at multiple historical moments; the lane segment sequence feature is used to characterize the positional relationship between the vehicle to be predicted and the surrounding lane segments.

[0071] In some embodiments, the trajectory prediction system can acquire historical location information of the vehicle to be predicted, process the historical location information based on a first multilayer perceptron (MLP) to obtain a second location sequence feature of the vehicle to be predicted, and then fuse the second location sequence feature with a time embedding parameter to obtain a third location sequence feature. An attention mechanism operation is then performed on the third location sequence feature to obtain a first location sequence feature.

[0072] For example, as shown in Figure 4, the trajectory prediction system can obtain the historical location information (H) of the vehicle to be predicted. i，tEach historical location information includes, but is not limited to, a timestamp and the coordinates of the vehicle to be predicted, where t represents different historical timestamps (t = 0, t = -1, t = -2) and i represents the i-th vehicle to be predicted.

[0073] Furthermore, the trajectory prediction system can examine the historical location information H of the vehicle to be predicted. i，t Check if the corresponding timestamp is valid. Location information with invalid timestamps (such as missing, incorrect format, etc.) or that is significantly different from other historical location information (such as excessive time jump) is considered invalid and its corresponding coordinate value is set to zero to avoid its impact in subsequent analysis.

[0074] For valid location data, calculate the coordinate increment ΔH between adjacent location information (e.g., with a step size of 1). i，t That is, Δx = x(t+1) - x(t) and Δy = y(t+1) - y(t) (or Δz in three-dimensional space), which represent the instantaneous direction and instantaneous speed information of the vehicle to be predicted.

[0075] In addition, instantaneous speed information can be combined with timestamp information to calculate the distance the vehicle to be predicted moves between adjacent positions, and then divided by the time interval to obtain instantaneous speed information. The calculated instantaneous direction and instantaneous speed information are then output for subsequent applications such as path planning, obstacle avoidance, and navigation.

[0076] Therefore, this application embodiment effectively improves the accuracy of the instantaneous direction and speed information of the vehicle to be predicted through timestamp validity checks and invalid position data processing mechanisms. Simultaneously, by using coordinate increments with a step size of 1 to represent the motion state of the vehicle to be predicted, reliable data support is provided for the motion analysis and control of the vehicle.

[0077] Optionally, as shown in Figure 5, in this embodiment of the application, a temporal encoder can be constructed using a Transformer model, and two residual modules can be constructed to calculate self-attention features and fully connected features respectively.

[0078] Furthermore, the trajectory prediction system can project the coordinate increment ΔHi,t onto a feature space of dimension d using a first multilayer perceptron (MLP) as the second position sequence feature e. i,t .

[0079] Furthermore, in this embodiment, a set of learnable parameters can be initialized as a temporal embedding, the length of which is equal to the time step of the historical location information of the vehicle to be predicted, and the initialized temporal embedding is concatenated with the second location feature sequence. The concatenation operation can be performed along the feature dimension, that is, the temporal embedding is treated as an additional dimension of the features of the vehicle to be predicted.

[0080] The concatenated features are then subjected to dimensional transformation to meet the requirements of subsequent trajectory prediction model processing. After initialization of temporal embedding, feature concatenation, and dimensional transformation, a third position sequence feature containing temporal information is obtained. This dimensional transformation may include adjusting the shape of the features, performing linear transformations, etc.

[0081] It should be understood that the dimension of this time embedding can be set according to the specific task requirements, and the parameters of this time embedding will be continuously optimized during the subsequent model training process in order to better capture temporal information.

[0082] Furthermore, through layer normalization and the first multilayer perceptron, the third position sequence features are mapped into query vector, key vector, and value vector features, as shown in the following formula 1:

[0083] {Q t ,K t V t}={φ Q (e i,t ),φ K (e i,t ),φ V (e i,t )} Formula 1

[0084] Where, φ Q (e i,t ) represents the query vector Query feature mapped to the third position sequence feature, φ K (e i,t ) represents the key vector Key feature mapped to the third position sequence feature, φ V (e i,t ) represents the value vector Value feature mapped to the third position sequence feature.

[0085] The first position sequence features are calculated using a self-attention mechanism, as shown in Formula 2 below:

[0086] in, This represents the first position sequence feature, where i represents the i-th vehicle to be predicted, and t1 and t2 represent different historical times. Let T represent the transpose of the feature matrix of the query vector Query for the vehicle to be predicted at time t1, where T represents the transpose of the matrix. The key vector Key feature matrix represents the vehicle to be predicted at time t2.

[0087] Preferably, as shown in Figure 5, the trajectory prediction system can employ multi-head attention for stable training, that is, dividing Q (query vector features), K (key vector features), and V (value vector features) into g groups, thus increasing the batch size to a factor of g, with each group of features having a dimension of g. After completing the attention calculation, the final attention features are obtained by concatenating the multi-head features.

[0088] Furthermore, by sequentially transforming the attention features through layer normalization and multilayer perceptron, fully connected features are obtained, as shown in Formula 3 below:

[0089] in, Indicates fully connected features. This represents the first position sequence feature.

[0090] Furthermore, residual connections are introduced to derive new vehicle features to be predicted, as shown in Equation 4 below:

[0091] in, Indicates the first position sequence feature, e represents the positional sequence features obtained by the self-attention mechanism. i,t Indicates the second position sequence feature, This is a fully connected feature.

[0092] To reduce overfitting, random depth and layer scaling are introduced into the residual connections. Random depth means that residual connections during model training have a certain probability of being discarded, and this probability increases with the number of layers, as shown in Equation 5 below:

[0093] In the formula, p l Let p be the abandonment probability of the l-th layer, L be the total number of layers, and p L This represents the abandonment probability of the last layer. Layer scaling involves independently scaling the residual features of each layer along the feature dimension, enriching the differences between features from different channels, as shown in Formula 6 below: e′=w ls ·e Formula 6

[0094] In the formula, e and e′ are the original residual features and the updated residual features, respectively, and w ls For learnable weights.

[0095] As shown in Figure 4, the first position sequence feature of the vehicle to be predicted is output by a temporal encoder. The temporal encoder can encode the historical state or behavior data of the vehicle to be predicted and extract features with temporal characteristics, which represent the state or behavior pattern of the vehicle to be predicted at the current moment.

[0096] In some embodiments, the trajectory prediction system can acquire the location information of the surrounding lane segments of the vehicle to be predicted, and process the location information of the surrounding lane segments based on the second multilayer perceptron (MLP) to obtain lane segment sequence features.

[0097] For example, the trajectory prediction system can determine the lane segment (i.e. the surrounding lane segment) at a preset distance in the direction of movement of the vehicle to be predicted based on the historical location information of the vehicle to be predicted, and process the coordinates of the selected surrounding lane segment according to the second multilayer perceptron (MLP) to obtain the location information of the surrounding lane segment.

[0098] Furthermore, the location information of the surrounding lane segments (i.e., the coordinates of the midpoints of the line segments of the surrounding lane segments) is transformed into the local coordinate system of the vehicle to be predicted, so that the position of the midpoint coordinates of the line segments relative to the vehicle to be predicted can be clearly expressed.

[0099] The midpoint coordinates of the transformed line segments are processed using a second multilayer perceptron. The second multilayer perceptron extracts deep features from the midpoint coordinates of the line segments through a nonlinear transformation of the input data. These deep features are then used as edge features (i.e., lane segment sequence features) in the graph structure.

[0100] Preferably, in this embodiment of the application, the lane segment sequence features processed by the second multilayer perceptron (MLP) can be combined with the first position sequence features of the vehicle to be predicted to construct a spatiotemporal graph structure between the lane segment and the vehicle to be predicted. This spatiotemporal graph structure not only reflects the spatial relationship between the lane segment and the vehicle to be predicted, but also incorporates the temporal state information of the vehicle to be predicted, thereby achieving a comprehensive description of the environment surrounding the vehicle to be predicted.

[0101] The embodiments of this application efficiently construct the relative positional relationship (spatiotemporal graph structure) between lane segments and vehicles to be predicted using the above method, providing strong support for subsequent tasks such as path planning and trajectory prediction.

[0102] Step 302: Perform attention mechanism operation based on the first position sequence features and lane segment sequence features to obtain the first lane segment fusion features.

[0103] In some embodiments, the trajectory prediction system can map the first position sequence features to a first query vector based on a third multilayer perceptron (MLP), and map the first concatenated features to a first key vector and a first value vector based on a fourth multilayer perceptron (MLP). Then, it can perform attention mechanism operations based on the first query vector, the first key vector, and the first value vector to obtain the first lane segment fusion features.

[0104] Among them, the first splicing feature is the feature obtained by splicing the first position sequence feature and the lane segment sequence feature;

[0105] For example, as shown in Figure 6, the trajectory prediction system can project the first location sequence features into a first query vector Q using a third multilayer perceptron (MLP). i,t The first key vector K is projected from the first location sequence features and the first concatenated lane segment sequence features obtained by the fourth multilayer perceptron (MLP). j,t and the first value vector V j,t The number of channels can be d.

[0106] As shown in Figure 4, the lane segment sequence features are fused into the first position sequence features using the heteroattention mechanism of the lane feature fusion module to obtain the residual features. That is, the following formula 7:

[0107] Furthermore, the trajectory prediction system can incorporate residual features After concatenation with the first location sequence features, normalization and processing by a fourth multilayer perceptron (MLP) are performed to obtain the updated location sequence features e′ of the vehicle to be predicted. i That is, the following formula 8:

[0108] Where η1 and η2 represent normalization functions, and φ represents the fourth multilayer perceptron (MLP) network structure of the fully connected layer.

[0109] The time encoder further integrates the directed lane segment features with the vehicle A to be predicted. i The node features are fused, and the vehicle sequence features to be predicted at t=0 are used as the basis for prediction. i,t |t=0 is the vehicle to be predicted embedded in b i .

[0110] Furthermore, a temporal encoder is used to analyze the lane segment sequence features and the updated position sequence features e′ of the vehicle to be predicted. iA fusion process is performed. Based on time-series encoding, the vehicle sequence feature e_(i,t)|t=0 at a specific time t=0 is selected. This feature includes both the dynamic information of the vehicle itself and the static and dynamic features of the influence of lane segment sequence features. This feature is set as the embedding representation b of the vehicle to be predicted. i The embedding vector b i It can comprehensively and compactly reflect the state of the vehicle to be predicted at the initial moment and its interaction with the surrounding environment.

[0111] It should be understood that time-series encoders can capture the dynamic changes of time-series data and effectively integrate information in the time dimension through their internal mechanisms (such as recurrent neural networks (RNN), long short-term memory networks (LSTM), or gated recurrent units (GRU), thereby generating a new feature set that integrates spatiotemporal characteristics.

[0112] Step 303: Based on the fusion features of the first lane segment, obtain the predicted trajectory of the vehicle to be predicted.

[0113] In some embodiments, the trajectory prediction system can acquire vehicle interaction features and second lane segment fusion features corresponding to surrounding vehicles of the vehicle to be predicted, and then perform attention mechanism operations based on the first lane segment fusion features, vehicle interaction features and second lane segment fusion features to obtain target fusion features, and generate the predicted trajectory of the vehicle to be predicted based on the target fusion features.

[0114] Among them, vehicle interaction features are used to characterize the positional relationship between the vehicle to be predicted and surrounding vehicles.

[0115] For example, as shown in Figure 4, the trajectory prediction module of the trajectory prediction system can input the target fusion features into multiple sets of multilayer perceptrons for decoding, obtain multiple predicted trajectories and the confidence level corresponding to each predicted trajectory, and determine the predicted trajectory with the highest confidence level among the multiple predicted trajectories as the predicted trajectory of the vehicle to be predicted.

[0116] In some embodiments, the target fusion features mentioned above can be obtained in the following way: the trajectory prediction system can map the first lane interaction features to the second query vector based on the sixth multilayer perceptron MLP, and map the second concatenation features to the second key vector and the second value vector based on the seventh multilayer perceptron MLP, and then perform attention mechanism operation based on the second query vector, the second key vector and the second value vector to obtain the target fusion features.

[0117] The second splicing feature is the feature obtained by splicing the first lane segment fusion feature, the vehicle interaction feature, and the second lane segment fusion feature.

[0118] For example, as shown in Figure 4, the lane target prediction module of the trajectory prediction system can project the first lane interaction features into a second query vector Q using a sixth multilayer perceptron (MLP). i,t The second concatenated feature, obtained by combining the first lane segment fusion feature, vehicle interaction feature, and second lane segment fusion feature through a seventh multilayer perceptron (MLP), is projected as the second key vector K. l,t Second value vector V l,t Where l represents the l-th surrounding vehicle (l = 1, 2, ...), and the number of lanes can be d.

[0119] The self-attention mechanism is used to fuse the features of the second lane segment into the features of the first position sequence to obtain the residual features. That is, the following formula 9:

[0120] Furthermore, the trajectory prediction system can incorporate residual features After concatenation with the first position sequence features, normalization and processing by a seventh multilayer perceptron (MLP) are performed to obtain the target fusion feature b′. i That is, the following formula 10:

[0121] Where η1 and η2 represent normalization functions, and φ represents the seventh multilayer perceptron (MLP) network structure of the fully connected layer.

[0122] In some embodiments, the vehicle interaction features mentioned above can be obtained in the following way: the trajectory prediction system can obtain the location information of surrounding vehicles of the vehicle to be predicted, and process the location information of surrounding vehicles based on the fifth multilayer perceptron (MLP) to obtain the vehicle interaction features.

[0123] For example, as shown in Figure 4, the interaction fusion module of the trajectory prediction system can obtain the position information of each vehicle around the vehicle to be predicted. This position information includes, but is not limited to, the vehicle's historical behavior, attributes, and current state. Then, it calculates the relative coordinates of the position information of the surrounding vehicles with respect to the vehicle to be predicted. These relative coordinates can reflect the spatial positional relationship between the vehicle to be predicted and the surrounding vehicles. The fifth multilayer perceptron (MLP) is used to process these relative coordinates to obtain the vehicle interaction features between the vehicle to be predicted and the surrounding vehicles. These vehicle interaction features not only include the spatial distance information between the vehicle to be predicted and the surrounding vehicles, but can also capture more complex interaction relationships, such as speed differences and movement directions, through the nonlinear mapping capability of the fifth multilayer perceptron.

[0124] Optionally, the trajectory prediction system can also construct a global interaction code using the vehicle interaction features, the location information of the vehicle to be predicted, and surrounding vehicles obtained above. That is, the location information of the vehicle to be predicted, surrounding vehicles, and vehicle interaction features are integrated into a graph structure, where the vehicle to be predicted is a node, and the connections between them are edges. Using graph neural networks (GNNs) or other graph processing algorithms, deeper information in the global interaction code can be further extracted to support subsequent decision-making, prediction, or optimization tasks.

[0125] It should be understood that the embodiments of this application establish the connection relationship between the vehicle to be predicted and surrounding vehicles at the current moment, and extract edge features using embedded representation and relative coordinates to construct a global interaction code. This method can accurately capture the dynamic interaction relationship between various vehicles, providing effective technical support for intelligent processing in fields such as intelligent transportation and autonomous driving.

[0126] The trajectory prediction method provided in this application comprehensively considers the vehicle position features (i.e., first position sequence features) at multiple historical moments. The method can capture the dynamic driving mode of the vehicle, thereby more accurately predicting its future trajectory. Furthermore, the introduction of lane segment sequence features, i.e. the positional relationship between the vehicle and the surrounding lane segments, provides additional contextual information, which helps to more accurately understand the vehicle's driving environment and further improve the accuracy of the prediction.

[0127] Meanwhile, the trajectory prediction method provided in this application relies on the vehicle's dynamic driving mode and its relative position to the lane segment, rather than a fixed path or pattern. This allows the model to adapt to different traffic scenarios and road structures, and the introduction of the attention mechanism enables the model to dynamically focus on important input features, thereby making more reasonable predictions under different circumstances.

[0128] As shown in Figure 7, this application embodiment provides a training method for a trajectory prediction model, which specifically includes the following steps 701-706.

[0129] Step 701: Obtain the first position sequence features and lane segment sequence features of the vehicle at the first sample time.

[0130] In the embodiments related to this application, the first position sequence feature represents the position of the vehicle at multiple historical moments prior to the first sample moment; the lane segment sequence feature represents the positional relationship between the vehicle and the surrounding lane segments at multiple historical moments.

[0131] In some embodiments, historical location information of the vehicle at multiple historical moments and surrounding lane segment location information of the vehicle to be predicted at multiple historical moments are obtained, and the historical location information and surrounding lane segment location information are processed based on the trajectory prediction model to obtain the first location sequence feature and the lane segment sequence feature, respectively.

[0132] It should be noted that the process of acquiring the trajectory prediction model and processing historical location information and surrounding lane segment location information is the same as the trajectory prediction method described above, and will not be repeated here.

[0133] Step 702: Obtain the ground truth label and true trajectory of the target lane segment corresponding to the vehicle at the second sample time.

[0134] In this embodiment, the given trajectory prediction dataset can be divided into a training set and a validation set in a 4:1 ratio. The training set comprises 80% of the total data and is used to train the trajectory prediction model; the validation set comprises 20% of the total data and is used to evaluate the model's prediction performance. The true trajectories corresponding to the second sample time of the vehicles included in the dataset are set as the ground truth labels for the final prediction output of the trajectory prediction model. It should be understood that these ground truth labels can serve as supervisory information to guide and optimize the model's prediction task training process.

[0135] For example, the lane data in the trajectory prediction dataset is preprocessed. This preprocessing step filters based on the vehicle's current direction of travel, that is, along the vehicle's direction of travel, it searches for surrounding lane segments within a certain distance range, which can be regarded as future candidate target vectors.

[0136] Within a defined distance range, surrounding lane segments that match the vehicle's direction of travel are selected to form a set of future candidate target vectors. Then, within this set of candidate target vectors, the distance between the vehicle's trajectory point at each moment and each surrounding lane segment is calculated. The closest surrounding lane segment is selected as the target lane segment label for that moment. These target lane segment labels constitute the ground truth label set for target lane segments, which is used to supervise the training of the auxiliary task detection head.

[0137] Step 703: Process the first position sequence features and lane segment sequence features based on the trajectory prediction model to obtain the predicted lane segment label and predicted trajectory corresponding to the second sample time.

[0138] In the embodiments related to this application, the trajectory prediction model is used to perform attention mechanism operations based on the first location sequence features and lane segment features to obtain the first lane segment fusion features; based on the first lane segment fusion features, a predicted trajectory is obtained, and the predicted lane segment label corresponding to the predicted trajectory is determined.

[0139] It should be noted that the process of determining the fusion features of the first lane segment and obtaining the predicted trajectory based on the fusion features of the first lane segment in the trajectory prediction model described above is the same as the process of the trajectory prediction method, and will not be repeated here. It should be understood that after obtaining the predicted trajectory, the predicted trajectory traveled by the vehicle will correspond to the corresponding lane segment, and the trajectory prediction model can use the corresponding lane segment as the predicted lane segment label.

[0140] Preferably, as shown in Figure 8, the trajectory prediction model can map the first location sequence features to a third query vector based on the Multilayer Perceptron (MLP), and map the lane segment features to a third key vector and a third value vector based on the MLP, i.e., the following formula 11: {Q i,t}={(φ Q (e i,t )} {K j,t V j,t}={(φ K (e j,t ),φ V (e j,t )} Formula 11

[0141] In the formula, φ Q (e i,t ) represents the first position sequence feature mapped to the third query vector Query feature, φ K (e j,t ) represents the third key vector Key feature mapped to the lane segment features, φ V (e j,t ) represents the third value vector Value feature mapped to the lane segment feature.

[0142] The third key vector, third value vector, and third query vector are processed using a multi-head cross-attention mechanism to determine the lane segment with the highest classification score in the lane segment set as the predicted lane segment. In other words, the multi-head cross-attention mechanism is used to calculate the multi-head attention coefficient of the vehicle to surrounding lane segments at each historical time point, as shown in Formula 12 below:

[0143] Where, α i,j,t The cross-attention coefficients are represented by `cross_attn`, which represents the single-head cross-attention calculation function. K represents the transpose of the first position sequence features of the i-th vehicle at time t. j,t and V j,t These represent the lane segment sequence characteristics at time t.

[0144] Furthermore, for each head, its corresponding cross-attention coefficients are summed, and the summation result is then normalized using the Softmax function. It should be understood that this step is used to calculate the classification score μ_(j,t_f) for each lane segment at the second sample time, based on the attention weights of each head.

[0145] Using the target lane segment label at each time step, the target lane segment classification score corresponding to the target lane segment label is selected from the calculated classification scores. This step ensures the consistency between the classification score used for subsequent loss calculation and the actual target lane segment.

[0146] In some embodiments, the trajectory prediction model can acquire vehicle interaction features of the vehicle and second lane segment fusion features corresponding to surrounding vehicles.

[0147] Among them, vehicle interaction features are used to characterize the positional relationship between the vehicle and surrounding vehicles at the first sample time.

[0148] It should be noted that the process of obtaining vehicle interaction features and second lane segment fusion features described above is the same as the process of obtaining vehicle interaction features and second lane segment fusion features in the trajectory prediction method, and will not be repeated here.

[0149] In some embodiments, the first lane segment fusion features, vehicle interaction features, and second lane segment fusion features are processed based on the trajectory prediction model to obtain the predicted lane segment label and predicted trajectory corresponding to the second sample time.

[0150] The trajectory prediction model is used to perform attention mechanism operations based on the first position sequence features and lane segment features to obtain the first lane segment fusion feature; based on the first lane segment fusion feature, vehicle interaction features and second lane segment fusion feature, hetero-attention mechanism operations are performed to obtain the target fusion feature; based on the target fusion feature, the predicted trajectory of the vehicle is generated and the predicted lane segment label corresponding to the predicted trajectory is determined.

[0151] It should be noted that the process of determining the first lane segment fusion feature and the target fusion feature in the trajectory prediction model, and obtaining the predicted trajectory based on the target fusion feature, is the same as the trajectory prediction method, and will not be elaborated here. It should be understood that after obtaining the predicted trajectory, the predicted trajectory traveled by the vehicle will correspond to a specific lane segment, and the trajectory prediction model can use the corresponding lane segment as the predicted lane segment label.

[0152] Step 704: Calculate the first loss based on the true value label of the target lane segment and the predicted lane segment label.

[0153] In some embodiments, based on the classification score of the selected target lane segment, the Binary Cross Entropy (BCE_loss) is used as the loss function to calculate the training loss for the auxiliary task. This loss function measures the error between the classification score predicted by the model and the actual lane target point, thereby guiding the model training process.

[0154] For example, the ground truth label of the target lane segment is matched with the predicted lane segment label, for example, based on the time or space dimension, to ensure that the two are compared at the same time point or spatial location, and the difference between the ground truth label of the target lane segment and the predicted lane segment label after matching is calculated. The difference includes, but is not limited to, lane segment position deviation, direction deviation, width deviation, etc.

[0155] Furthermore, the first loss function (loss) is calculated based on the above differences. goal This refers to the cross-entropy target classification loss (denoted as loss_goal). This loss function aims to quantify the degree of inconsistency between the predicted lane segment label and the ground truth label of the target lane segment. It can typically employ mathematical distance metrics (such as Euclidean distance, Manhattan distance, etc.) or probability metrics (such as cross-entropy loss, KL divergence, etc.).

[0156] It should be understood that the calculation result of the first loss function reflects the accuracy of the model's prediction; that is, the smaller the difference between the lane segment predicted by the model and the actual lane segment, the better the model's prediction performance.

[0157] Step 705: Calculate the second loss based on the actual trajectory and the predicted trajectory.

[0158] For example, the true trajectory and the predicted trajectory are synchronized in time or aligned in space to ensure that they are compared at the same point in time or in the same spatial location. For example, an appropriate metric is used to calculate the difference between the true trajectory and the predicted trajectory. This difference includes, but is not limited to, differences in the geometry of the trajectory (such as path length, curvature, etc.), differences in dynamic characteristics (such as changes in velocity, acceleration, etc.), and differences in time synchronization (such as time delay, time interval, etc.).

[0159] The second loss function is calculated based on the above differences. This second loss function aims to quantify the degree of inconsistency between the predicted trajectory and the actual trajectory, and is usually calculated using mathematical distance metrics (such as Euclidean distance, Manhattan distance, Fraser distance, etc.), dynamic characteristic metrics (such as velocity error, acceleration error, etc.), or time synchronization metrics (such as time delay loss, etc.).

[0160] Another example involves calculating the SmoothL1 regression loss `loss_reg` for both the predicted and true trajectories, and then calculating a second loss function `loss_cls` (cross-entropy classification loss) for the predicted trajectory. This cross-entropy classification loss aims to improve the confidence level corresponding to this modality, making it more accurately reflect the true situation.

[0161] Step 706: Adjust the trajectory prediction model based on the first loss and the second loss.

[0162] For example, the calculated first and second losses are fused to form a joint loss function. This can be achieved, for instance, through simple weighted summation, product, or other complex nonlinear combinations. The joint loss function aims to comprehensively reflect the model's performance in both lane segment prediction and trajectory prediction.

[0163] Furthermore, using gradient descent or other optimization algorithms, the parameters of the trajectory prediction model are adjusted based on the gradient information of the joint loss function. During the adjustment process, the model continuously attempts to reduce the joint loss, thereby gradually improving the accuracy of lane segment prediction and trajectory prediction.

[0164] It should be understood that during the model training phase, the adjusted model can be iteratively trained on datasets containing a wide variety of scenarios, while continuously monitoring changes in the joint loss. Through continuous parameter tuning and optimization, the model gradually converges to the optimal solution.

[0165] In the model validation phase, the trained model's performance is evaluated using an independent validation dataset. By comparing the model's predictions on the validation set with those on the actual data, the model's generalization ability and prediction accuracy are verified. Furthermore, based on the validation results, the model is further optimized and adjusted.

[0166] Another example is that the final training loss function loss_total is the same as the first loss function loss. goal The loss_total is a linear combination of the SmoothL1 regression loss_reg and the second loss function loss_cls. Specifically, the loss_total can be calculated using Equation 13: loss_total=(1-α)·[β·loss_cls+(1-β)·loss_reg]+α·loss_goal Equation 13

[0167] In this embodiment, the linear coefficients α and β are set to 0.2 and 0.5, respectively, to balance the influence of different loss terms and achieve the effect of lane target prediction assisting trajectory prediction. During training, this embodiment uses backpropagation technology in neural network model training to iteratively update the model parameters. By continuously adjusting the model parameters, the calculated result of the loss function loss_total continuously decreases. When the calculated result of the loss function is less than a certain threshold, the model is considered to have been trained to its optimal mode, at which point the model has high prediction accuracy and reliability.

[0168] The foregoing mainly describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the above functions, the image acquisition device or electronic device includes corresponding hardware structures and / or software modules for performing each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0169] This application embodiment can, according to the above method, exemplarily divide an image acquisition device or electronic device into functional modules. For example, the image acquisition device or electronic device may include functional modules corresponding to each functional division, or two or more functions may be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division; in actual implementation, there may be other division methods.

[0170] Figure 9 is a block diagram illustrating another trajectory prediction device according to an exemplary embodiment. This trajectory prediction device can be applied to trajectory prediction methods or training methods. Referring to Figure 9, the trajectory prediction device includes a processing unit 901 and an acquisition unit 902. The acquisition unit 902 is used to acquire a first position sequence feature and a lane segment sequence feature of the vehicle to be predicted; the first position sequence feature is used to characterize the position features of the vehicle to be predicted at multiple historical moments; the lane segment sequence feature is used to characterize the positional relationship between the vehicle to be predicted and surrounding lane segments; the processing unit 901 is used to perform attention mechanism operations based on the first position sequence feature and the lane segment sequence feature to obtain a first lane segment fusion feature; the processing unit 901 is also used to obtain the predicted trajectory of the vehicle to be predicted based on the first lane segment fusion feature.

[0171] In some embodiments, the processing unit 901 is specifically used to acquire vehicle interaction features and second lane segment fusion features corresponding to surrounding vehicles of the vehicle to be predicted; the vehicle interaction features are used to characterize the positional relationship between the vehicle to be predicted and surrounding vehicles; attention mechanism operation is performed based on the first lane segment fusion features, vehicle interaction features and second lane segment fusion features to obtain target fusion features; and the predicted trajectory of the vehicle to be predicted is generated based on the target fusion features.

[0172] In some embodiments, the processing unit 901 is specifically used to acquire historical location information of the vehicle to be predicted; process the historical location information based on a first multilayer perceptron (MLP) to obtain a second location sequence feature of the vehicle to be predicted; fuse the second location sequence feature with a time embedding parameter to obtain a third location sequence feature; and perform attention mechanism operation on the third location sequence feature to obtain a first location sequence feature.

[0173] In some embodiments, the processing unit 901 is specifically used to acquire the location information of the surrounding lane segments of the vehicle to be predicted; and to process the location information of the surrounding lane segments based on the second multilayer perceptron (MLP) to obtain lane segment sequence features.

[0174] In some embodiments, the processing unit 901 is specifically used to map the first position sequence features to a first query vector based on a third multilayer perceptron (MLP); to map the first concatenated features to a first key vector and a first value vector based on a fourth multilayer perceptron (MLP); the first concatenated features are features obtained by concatenating the first position sequence features and the lane segment sequence features; and to perform attention mechanism operations based on the first query vector, the first key vector, and the first value vector to obtain the first lane segment fusion features.

[0175] In some embodiments, the processing unit 901 is specifically used to acquire the location information of surrounding vehicles of the vehicle to be predicted; and to process the location information of surrounding vehicles based on the fifth multilayer perceptron (MLP) to obtain vehicle interaction features.

[0176] In some embodiments, the processing unit 901 is specifically used to map the first lane interaction feature to a second query vector based on the sixth multilayer perceptron MLP; to map the second concatenated feature to a second key vector and a second value vector based on the seventh multilayer perceptron MLP; the second concatenated feature is a feature obtained by concatenating the first lane segment fusion feature, the vehicle interaction feature and the second lane segment fusion feature; and to perform attention mechanism operation based on the second query vector, the second key vector and the second value vector to obtain the target fusion feature.

[0177] The acquisition unit 902 is further configured to acquire the first position sequence features and lane segment sequence features of the vehicle at the first sample time; the first position sequence features represent the vehicle's position at multiple historical times prior to the first sample time; the lane segment sequence features represent the positional relationship between the vehicle and surrounding lane segments at multiple historical times; the acquisition unit 902 is further configured to acquire the ground truth label and true trajectory of the target lane segment corresponding to the vehicle at the second sample time; the processing unit 901 is further configured to process the first position sequence features and lane segment sequence features based on the trajectory prediction model to obtain the predicted lane segment label and predicted trajectory corresponding to the second sample time; the trajectory prediction model is used to perform attention mechanism operations based on the first position sequence features and lane segment features to obtain the first lane segment fusion features; based on the first lane segment fusion features, the predicted trajectory is obtained, and the predicted lane segment label corresponding to the predicted trajectory is determined; the first loss is calculated based on the ground truth label of the target lane segment and the predicted lane segment label; the second loss is calculated based on the true trajectory and the predicted trajectory; and the trajectory prediction model is adjusted according to the first loss and the second loss.

[0178] In some embodiments, the processing unit 901 is specifically used to acquire historical location information of the vehicle at multiple historical moments and surrounding lane segment location information of the vehicle to be predicted at multiple historical moments; and to process the historical location information and surrounding lane segment location information based on the trajectory prediction model to obtain first location sequence features and lane segment sequence features, respectively.

[0179] In some embodiments, the processing unit 901 is further configured to acquire vehicle interaction features of the vehicle and second lane segment fusion features corresponding to surrounding vehicles of the vehicle; the vehicle interaction features are used to characterize the positional relationship between the vehicle and surrounding vehicles at a first sample time; and to process the first position sequence features and lane segment sequence features based on the trajectory prediction model to obtain the predicted lane segment label and predicted trajectory corresponding to the second sample time, including: processing the first lane segment fusion features, vehicle interaction features and second lane segment fusion features based on the trajectory prediction model to obtain the predicted lane segment label and predicted trajectory corresponding to the second sample time; the trajectory prediction model is used to perform attention mechanism operations based on the first position sequence features and lane segment features to obtain the first lane segment fusion features; to perform attention mechanism operations based on the first lane segment fusion features, vehicle interaction features and second lane segment fusion features to obtain the target fusion features; and to generate the predicted trajectory of the vehicle based on the target fusion features and determine the predicted lane segment label corresponding to the predicted trajectory.

[0180] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0181] Figure 10 is a block diagram of an electronic device according to an exemplary embodiment. As shown in Figure 10, the electronic device includes, but is not limited to, a processor 1001 and a memory 1002.

[0182] The memory 1002 described above is used to store the executable instructions of the processor 1001. It is understood that the processor 1001 is configured to execute instructions to implement the image acquisition method in the above embodiments.

[0183] It should be noted that those skilled in the art will understand that the electronic device structure shown in FIG10 does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than those shown in FIG10, or combine certain components, or have different component arrangements.

[0184] The processor 1001 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 1002, and by calling data stored in the memory 1002, it performs various functions and processes data, thereby providing overall monitoring of the electronic device. The processor 1001 may include one or more processing units. Optionally, the processor 1001 may integrate an application processor and a modem processor. The application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 1001.

[0185] The memory 1002 can be used to store software programs and various data. The memory 1002 may primarily include a program storage area and a data storage area. The program storage area may store the operating system, application programs required by at least one functional module (such as a determination unit, processing unit, etc.), etc. Furthermore, the memory 1002 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0186] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 1002 including instructions, which can be executed by a processor 1001 of an electronic device to implement the methods in the above embodiments.

[0187] In actual implementation, the functions of the acquisition unit 301, display unit 302, and control unit 302 in Figure 10 can all be implemented by the processor 1001 in Figure 10 calling the computer program stored in the memory 1002. The specific execution process can be found in the description of the method section in the above embodiment, and will not be repeated here.

[0188] Optionally, the computer-readable storage medium may be a non-transitory computer-readable storage medium, such as a read-only memory (ROM), random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device.

[0189] In an exemplary embodiment, this application also provides a computer program product including one or more instructions, which can be executed by the processor 1001 of an electronic device to perform the methods described above.

[0190] It should be noted that when one or more instructions in the computer-readable storage medium or computer program product are executed by the processor of an electronic device, they implement the various processes of the above method embodiments and achieve the same technical effect as the above method. To avoid repetition, they will not be described again here.

[0191] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0192] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0193] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the classified units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0194] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0195] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially, or the parts that contribute to the prior art, or part or all of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0196] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A trajectory prediction method, wherein, The method includes: Obtain the first position sequence feature and lane segment sequence feature of the vehicle to be predicted; the first position sequence feature is used to characterize the position feature of the vehicle to be predicted at multiple historical moments; the lane segment sequence feature is used to characterize the positional relationship between the vehicle to be predicted and the surrounding lane segments; An attention mechanism is performed based on the first location sequence features and the lane segment sequence features to obtain the first lane segment fusion feature; Based on the fusion features of the first lane segment, the predicted trajectory of the vehicle to be predicted is obtained.

2. The method of claim 1, wherein, The step of obtaining the predicted trajectory of the vehicle to be predicted based on the first lane segment fusion features includes: The vehicle interaction features and the second lane segment fusion features corresponding to the surrounding vehicles of the vehicle to be predicted are obtained; the vehicle interaction features are used to characterize the positional relationship between the vehicle to be predicted and the surrounding vehicles. An attention mechanism is performed based on the first lane segment fusion feature, the vehicle interaction feature, and the second lane segment fusion feature to obtain the target fusion feature; Based on the target fusion features, the predicted trajectory of the vehicle to be predicted is generated.

3. The method of claim 1 or 2, wherein, The process of obtaining the first location sequence features of the vehicle to be predicted includes: Obtain the historical location information of the vehicle to be predicted; The historical location information is processed based on the first multilayer perceptron (MLP) to obtain the second location sequence features of the vehicle to be predicted. The second position sequence feature and the time embedding parameter are fused to obtain the third position sequence feature; An attention mechanism operation is performed on the third position sequence features to obtain the first position sequence features.

4. The method according to any one of claims 1 to 3, wherein, Obtain lane segment sequence features of the vehicle to be predicted, including: Obtain the location information of the surrounding lane segments of the vehicle to be predicted; The location information of the surrounding lane segments is processed based on the second multilayer perceptron (MLP) to obtain the lane segment sequence features.

5. The method according to any one of claims 1 to 4, wherein, The first lane segment fusion feature is obtained by performing attention mechanism operations based on the first location sequence features and the lane segment sequence features, including: The first position sequence features are mapped to the first query vector based on the third multilayer perceptron (MLP). The first concatenated feature is mapped to a first key vector and a first value vector based on the fourth multilayer perceptron (MLP); the first concatenated feature is the feature obtained by concatenating the first position sequence feature and the lane segment sequence feature; The attention mechanism is used to perform the first lane segment fusion feature based on the first query vector, the first key vector, and the first value vector.

6. The method of claim 2, wherein, The acquisition of vehicle interaction features includes: Obtain the location information of surrounding vehicles of the vehicle to be predicted; The surrounding vehicle location information is processed using a fifth-layer perceptron (MLP) to obtain the vehicle interaction features.

7. The method of claim 2 or 6, wherein, The process of performing attention mechanism calculations based on the first lane segment fusion feature, the vehicle interaction feature, and the second lane segment fusion feature to obtain the target fusion feature includes: The first lane interaction feature is mapped to the second query vector based on the sixth multilayer perceptron (MLP). The second concatenated feature is mapped to a second key vector and a second value vector based on the seventh multilayer perceptron (MLP); the second concatenated feature is the feature obtained by concatenating the first lane segment fusion feature, the vehicle interaction feature and the second lane segment fusion feature. The attention mechanism is performed based on the second query vector, the second key vector, and the second value vector to obtain the target fusion feature.

8. A method of training a trajectory prediction model, wherein, The method includes: The vehicle's first position sequence feature and lane segment sequence feature are obtained at a first sample time. The first position sequence feature represents the vehicle's position at multiple historical times prior to the first sample time. The lane segment sequence feature represents the positional relationship between the vehicle and surrounding lane segments at the multiple historical times. Obtain the ground truth label and true trajectory of the target lane segment corresponding to the vehicle at the second sample time; The trajectory prediction model processes the first location sequence features and the lane segment sequence features to obtain the predicted lane segment label and predicted trajectory corresponding to the second sample time. The trajectory prediction model is used to perform attention mechanism operation based on the first location sequence features and the lane segment features to obtain the first lane segment fusion feature. Based on the first lane segment fusion feature, the predicted trajectory is obtained, and the predicted lane segment label corresponding to the predicted trajectory is determined. The first loss is calculated based on the true label of the target lane segment and the predicted lane segment label; Calculate the second loss based on the actual trajectory and the predicted trajectory; The trajectory prediction model is adjusted based on the first loss and the second loss.

9. The method of claim 8, wherein, Obtain the vehicle's first position sequence features and lane segment sequence features at the first sample time, including: Obtain the historical location information of the vehicle at multiple historical times and the surrounding lane segment location information of the vehicle to be predicted at the multiple historical times; Based on the trajectory prediction model, the historical location information and the surrounding lane segment location information are processed to obtain the first location sequence feature and the lane segment sequence feature, respectively.

10. The method of claim 8 or 9, wherein, The method further includes: The vehicle interaction features of the vehicle and the second lane segment fusion features corresponding to the surrounding vehicles of the vehicle are obtained; the vehicle interaction features are used to characterize the positional relationship between the vehicle and the surrounding vehicles at the first sample time. Based on the trajectory prediction model, the first location sequence features and the lane segment sequence features are processed to obtain the predicted lane segment label and predicted trajectory corresponding to the second sample time, including: The trajectory prediction model processes the first lane segment fusion feature, the vehicle interaction feature, and the second lane segment fusion feature to obtain the predicted lane segment label and predicted trajectory corresponding to the second sample time. The trajectory prediction model is used to perform attention mechanism operation based on the first position sequence feature and the lane segment feature to obtain the first lane segment fusion feature. The attention mechanism operation is also performed based on the first lane segment fusion feature, the vehicle interaction feature, and the second lane segment fusion feature to obtain the target fusion feature. Based on the target fusion feature, the predicted trajectory of the vehicle is generated, and the predicted lane segment label corresponding to the predicted trajectory is determined.

11. A trajectory prediction apparatus, wherein, The device includes a processing unit and an acquisition unit.

12. A vehicle, wherein, The vehicle includes the device as described in claim 11.

13. An electronic device, comprising: include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the method as described in any one of claims 1 to 7 or to implement the method as described in any one of claims 8 to 10.

14. A computer readable storage medium, wherein, When the computer-executable instructions stored in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is capable of performing the method as described in any one of claims 1 to 7 or implementing the method as described in any one of claims 8 to 10.

15. A computer program product comprising instructions, wherein, When the instructions are executed by a computer, the computer performs the method as described in any one of claims 1 to 7 or implements the method as described in any one of claims 8 to 10.

Images