Obstacle trajectory prediction methods, devices and storage media

By transforming obstacle historical trajectory features and high-precision map features into a local coordinate system and encoding them using a scene encoder, the problems of computational redundancy and low accuracy in existing methods are solved, achieving more efficient and accurate obstacle trajectory prediction.

CN118585792BActive Publication Date: 2026-11-17CHINA AUTOMOTIVE INNOVATION CORP
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202410720764.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-06-05
Publication Date
2026-11-17
Estimated Expiration
2044-06-05

AI Technical Summary

Technical Problem

Existing deep learning-based obstacle trajectory prediction methods fail to fully utilize the temporal and spatial variation characteristics of obstacles in the environment, resulting in computational redundancy and low trajectory prediction accuracy.

Method used

The historical trajectory features of obstacles and high-precision map features are transformed into local coordinate system and encoded using a scene encoder. Obstacle trajectory prediction is then performed using a scene encoder and trajectory decoder model, reducing the need for understanding geographic coordinates in the world coordinate system and focusing on the relative relationship and relative position between the trajectory and the map.

Benefits of technology

It improves the efficiency and accuracy of obstacle trajectory prediction, reduces the amount of computation, makes full use of the temporal and spatial information in the scene, and provides a more comprehensive contextual representation for the trajectory decoder.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118585792B_ABST
    Figure CN118585792B_ABST
Patent Text Reader

Abstract

This application discloses an obstacle trajectory prediction method, apparatus, and storage medium. The method includes: acquiring obstacle historical trajectory features and high-precision map features of a target vehicle; acquiring the position information of the starting point of the centerline of the lane where the target vehicle is located in the world coordinate system in a local map; establishing a local coordinate system based on the position information, and transforming the obstacle historical trajectory features and the high-precision map features into the local coordinate system to obtain local obstacle historical trajectory features and local high-precision map features; using a scene encoder to perform scene encoding on the local obstacle historical trajectory features and the local high-precision map features to obtain scene features; and inputting the scene features into a trajectory decoder for decoding processing to obtain the obstacle prediction trajectory of the target vehicle. This application can reduce the amount of computation and improve the efficiency and accuracy of obstacle trajectory prediction.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application belongs to the field of autonomous driving, and in particular relates to an obstacle trajectory prediction method, device and storage medium. Background Technology

[0002] Trajectory prediction has received significant attention in autonomous systems such as self-driving cars. It aims to predict the trajectory of an obstacle within the next few seconds based on its past trajectories and the surrounding environment, thus providing a basis for the vehicle's path planning. Obstacles can be understood as traffic participants such as vehicles and pedestrians around the vehicle. Due to the complexity of autonomous driving scenarios, environmental information is often fused when performing obstacle trajectory prediction to improve accuracy.

[0003] Traditional obstacle trajectory prediction methods can be categorized into three types: physical methods based on vehicle kinematic or dynamic models, machine learning methods based on application data-driven models, and deep learning methods that extract features from historical inputs and then predict future outputs through regression. Compared to physical and machine learning methods, deep learning-based methods are currently the mainstream research approach. However, existing deep learning-based methods fail to fully utilize the temporal and spatial variations of obstacles in the environment, leading to redundant feature extraction during trajectory prediction and consequently, lower trajectory accuracy.

[0004] Therefore, improving the efficiency and accuracy of obstacle trajectory prediction is a technical problem that urgently needs to be solved. Summary of the Invention

[0005] This application provides an obstacle trajectory prediction method, apparatus, and storage medium, which can reduce computational load and improve the efficiency and accuracy of obstacle trajectory prediction.

[0006] On one hand, embodiments of this application provide an obstacle trajectory prediction method, the method comprising:

[0007] Obtain the obstacle history trajectory features and high-precision map features of the target vehicle; the obstacle history trajectory features represent the information of the obstacle history trajectory in the world coordinate system; the high-precision map features represent the information of the high-precision map in the world coordinate system;

[0008] Obtain the position information of the starting point of the centerline of the lane where the target vehicle is located in the local map in the world coordinate system;

[0009] Based on the location information, a local coordinate system is established, and the obstacle historical trajectory features and the high-precision map features are transformed into the local coordinate system to obtain local obstacle historical trajectory features and local high-precision map features;

[0010] The scene features are obtained by using a scene encoder to encode the historical trajectory features of the local obstacles and the local high-precision map features.

[0011] The scene features are input into the trajectory decoder for decoding to obtain the obstacle prediction trajectory of the target vehicle. The obstacle prediction trajectory is the trajectory of the obstacle corresponding to the target time period after the current time.

[0012] In one exemplary embodiment, acquiring the obstacle history trajectory features and high-precision map features of the target vehicle includes:

[0013] The coordinate information of the obstacle is collected at a preset time sampling interval within a preset time period;

[0014] Based on the coordinate information of adjacent sampling times, the historical trajectory characteristics of the obstacle are determined;

[0015] Map information of the high-precision map is collected within a preset spatial range at a preset spatial sampling interval;

[0016] The high-precision map features are determined based on map information from adjacent sampling locations.

[0017] In one exemplary embodiment, the step of establishing a local coordinate system based on the location information and transforming the obstacle historical trajectory features and the high-precision map features into the local coordinate system to obtain local obstacle historical trajectory features and local high-precision map features includes:

[0018] A local coordinate system is established with the starting point of the lane centerline of the local map where the target vehicle is currently located as the origin, the lane direction as the first axis, and the direction perpendicular to the lane direction as the second axis; the local map is a map in the high-precision map that expands outward from the current location of the target vehicle within a preset spatial range.

[0019] Analyze the historical trajectory features of the obstacle and the high-precision map features, and determine the first dimension and the second dimension representing the coordinate position in the historical trajectory features of the obstacle and the high-precision map features, respectively;

[0020] Based on the location information, the coordinate position corresponding to the first dimension in the obstacle historical trajectory feature is normalized to the local coordinate system to obtain the local obstacle historical trajectory feature;

[0021] Based on the location information, the coordinate position corresponding to the second dimension in the high-precision map feature is normalized to the local coordinate system to obtain the local high-precision map feature.

[0022] In one exemplary embodiment, the scene encoder includes a first modal encoder, a second modal encoder, and a cascaded encoder. The scene encoder is used to encode the local obstacle historical trajectory features and the local high-precision map features to obtain scene features, including:

[0023] The first modal encoder is used to perform obstacle encoding processing on the local obstacle historical trajectory features to obtain obstacle trajectory features;

[0024] The local high-precision map features are processed by the second modal encoder to obtain map features;

[0025] The obstacle trajectory features and the map features are fused using the serial encoder to obtain scene features.

[0026] In one exemplary embodiment, the step of fusing the obstacle trajectory features and the map features using the serial encoder to obtain scene features includes:

[0027] The map features are used as the key matrix and value matrix of the cascade encoder, and the obstacle trajectory features are used as the initial query matrix of the cascade encoder. The key matrix, the value matrix and the initial query matrix are input into the cascade encoder for fusion encoding to obtain the scene features.

[0028] In one exemplary embodiment, the method further includes:

[0029] The reference trajectory is used as the initial query for the trajectory decoder; the reference trajectory is the obstacle prediction trajectory generated by the preset reference trajectory model.

[0030] Based on the initial query, the scene features are decoded using the trajectory decoder to obtain the obstacle prediction trajectory of the target vehicle.

[0031] In one exemplary embodiment, the method further includes:

[0032] The target time period following the current moment is divided into a first target time period and a second target time period;

[0033] The scene features are decoded using a local trajectory decoder to obtain a first predicted trajectory; the first predicted trajectory is the trajectory of the obstacle corresponding to the first target time period.

[0034] The first modal encoder is used to perform obstacle encoding processing on the first predicted trajectory to obtain intermediate trajectory features;

[0035] The intermediate trajectory features and the map features are fused using the serial encoder to obtain intermediate scene features;

[0036] The intermediate scene features are decoded using the local trajectory decoder to obtain a second predicted trajectory; the second predicted trajectory is the trajectory of the obstacle corresponding to the second target time period.

[0037] The first predicted trajectory and the second predicted trajectory are spliced ​​together to obtain the obstacle prediction trajectory of the target vehicle.

[0038] On the other hand, this application also provides an obstacle trajectory prediction device, the device comprising:

[0039] The feature acquisition module is used to acquire the obstacle historical trajectory features and high-precision map features of the target vehicle; the obstacle historical trajectory features represent the information of the obstacle historical trajectory in the world coordinate system; the high-precision map features represent the information of the high-precision map in the world coordinate system.

[0040] The starting point acquisition module is used to acquire the position information of the starting point of the centerline of the lane where the target vehicle is located in the local map in the world coordinate system;

[0041] The coordinate transformation module establishes a local coordinate system based on the location information and transforms the obstacle historical trajectory features and the high-precision map features into the local coordinate system to obtain local obstacle historical trajectory features and local high-precision map features.

[0042] The scene encoding module is used to encode the local obstacle historical trajectory features and the local high-precision map features using the scene encoder to obtain scene features;

[0043] The trajectory decoding module is used to input the scene features into the trajectory decoder for decoding processing to obtain the obstacle prediction trajectory of the target vehicle. The obstacle prediction trajectory is the trajectory of the obstacle corresponding to the target time period after the current time.

[0044] On the other hand, this application also provides an electronic device, the device including a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the obstacle trajectory prediction method as described above.

[0045] On the other hand, this application also provides a computer storage medium storing at least one instruction or at least one program, which is loaded and executed by a processor to implement the obstacle trajectory prediction method as described above.

[0046] On the other hand, this application also provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the obstacle trajectory prediction method as described above.

[0047] The obstacle trajectory prediction method provided in this application has the following technical advantages:

[0048] In this embodiment, the obstacle historical trajectory features and high-precision map features of the target vehicle in the world coordinate system are transformed into a local coordinate system centered on the centerline starting point of the lane where the target vehicle is located in the local map, resulting in local obstacle historical trajectory features and local high-precision map features. Then, a scene encoder is used to perform scene encoding on these features to obtain scene features. These scene features are then input into a trajectory decoder for decoding to obtain the trajectory of obstacles corresponding to the target time period after the current moment of the target vehicle. This coordinate system transformation eliminates the need for the scene encoder-trajectory decoder model to understand all geographical coordinates in the world coordinate system, reducing the computational load. Furthermore, transforming all features to a local coordinate system centered on the centerline starting point of the lane establishes a correlation between the trajectory and the map, allowing feature encoding to focus more on the relative relationship and position between the trajectory and the map, thus effectively improving the scientific rigor and accuracy of the encoding. Moreover, by fusing obstacle historical trajectory features and map features through the scene encoder, the temporal and spatial information within the scene can be fully utilized, providing a more comprehensive contextual representation for the trajectory decoder, thereby improving the efficiency and accuracy of trajectory prediction. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is an application environment diagram of an obstacle trajectory prediction method provided in the embodiments of this application.

[0051] Figure 2 This is a flowchart illustrating an obstacle trajectory prediction method provided in an embodiment of this application.

[0052] Figure 3 This is a schematic diagram of establishing a local coordinate system provided in an embodiment of this application.

[0053] Figure 4 This is a flowchart illustrating the scene coding process provided in the embodiments of this application.

[0054] Figure 5 This is a schematic flowchart of the segmented prediction process provided in the embodiments of this application.

[0055] Figure 6 This is a schematic diagram of the modules of the trajectory prediction model provided in the embodiments of this application.

[0056] Figure 7 This is a schematic diagram of the obstacle trajectory prediction device provided in the embodiments of this application.

[0057] Figure 8 This is a hardware structure block diagram of a server for an obstacle trajectory prediction method provided in an embodiment of this application. Detailed Implementation

[0058] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0059] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0060] Figure 1 This is an application environment diagram of an obstacle trajectory prediction method provided in the embodiments of this application.

[0061] In autonomous driving scenarios, accurate prediction of the trajectories of surrounding vehicles can assist autonomous vehicles in making reasonable, real-time decisions. For example... Figure 1 As shown, target vehicle 10 is an autonomous vehicle. During its operation, it is assumed that there are multiple dynamic traffic participants such as vehicles or pedestrians around it. These vehicles or pedestrians can be considered as obstacles to target vehicle 10, for example... Figure 1 Obstacles 11, 12, and 13, etc. Figure 1 In the diagram, dashed lines represent lane markings. By collecting historical trajectories and map information of obstacles, the future trajectory of obstacles over a given time period can be predicted, thus assisting autonomous vehicles in making decisions. For example, target vehicle 10 can collect the historical trajectory of obstacles for 5 seconds and then predict the future trajectory of obstacles within 6 seconds. Target vehicle 10 can then combine the predicted future trajectory of obstacles to perform actions such as obstacle avoidance and path planning.

[0062] In existing technologies, obstacle trajectory prediction methods can be categorized into three types: physical methods based on vehicle kinematic or dynamic models, machine learning methods based on application data-driven models, and deep learning methods that extract features from historical state inputs and then predict future state outputs through regression. Compared to physical and machine learning-based methods, deep learning-based methods are currently the mainstream research approach.

[0063] However, autonomous driving scenarios are complex, requiring the fusion of dynamic moving obstacles (such as pedestrians and vehicles) and static environmental information (such as lanes and traffic lights). The fusion of environmental information provides rich encoding information for deep learning-based trajectory prediction. Early work involved rendering high-resolution maps and vehicle / pedestrian trajectories into a single image, then using convolutional neural networks (CNNs) for learning. In practice, this method requires rendering each frame, consuming significant CPU resources and time. Therefore, CNN-based methods have limited feature extraction capabilities and struggle to model long-distance interaction information. Furthermore, to address the issues of limited receptive fields and high computational costs, current research has shifted towards vector-based encoding schemes. Through pooling, graph convolution, and attention mechanisms, vector-based methods can effectively aggregate sparse information in traffic scenarios. However, existing methods cannot fully utilize the temporal and spatial variations of obstacles in the environment, leading to redundant feature extraction during trajectory prediction and consequently, lower trajectory prediction accuracy.

[0064] Therefore, this application proposes an obstacle trajectory prediction method, which combines historical trajectories and high-precision maps and uniformly transforms them into a local coordinate system. Then, a scene-based coding method is used to enable all obstacles to share scene features, thereby improving the efficiency and accuracy of obstacle trajectory prediction.

[0065] Figure 2 This is a flowchart illustrating an obstacle trajectory prediction method provided in an embodiment of this application.

[0066] S201: Obtain the historical obstacle trajectory features and high-precision map features of the target vehicle.

[0067] Obstacle historical trajectory features represent information about the historical trajectory of obstacles in the world coordinate system; high-precision map features represent information about high-precision maps in the world coordinate system.

[0068] High-definition maps, also known as high-definition maps (HD maps), are typically used for autonomous driving assistance. High-definition maps have centimeter-level precision and contain rich road elements and traffic-related dynamic elements, such as detailed lane lines, road signs, traffic signs, traffic lights, lane curvature, and slope.

[0069] Preferably, the historical trajectory features of obstacles and the features of high-definition maps can be represented in vector form. Most elements in high-definition maps are spline curves, such as lanes, closed shapes such as intersections and pedestrian crossings, and points such as traffic lights. The trajectory of an obstacle is in the form of a directed spline curve relative to time. Therefore, the information of historical trajectories and the information of high-definition maps can both be approximated in vector form.

[0070] It should be noted that the data obtained in this step is in an unnormalized world coordinate system, meaning that the coordinates are unique for each geographical location, and no two geographical locations have the same coordinates. However, enabling a model to understand massive amounts of data in the world coordinate system requires significant computational resources and costs. Furthermore, trajectory prediction places greater emphasis on relative relationships, such as the relative positions between obstacle trajectories, between obstacle trajectories and the map, and between various elements within the map. Therefore, this embodiment transforms the historical obstacle trajectory features and high-precision map features from the world coordinate system to a relative coordinate system.

[0071] S203: Obtain the position information of the starting point of the centerline of the lane where the target vehicle is located in the local map in the world coordinate system;

[0072] A local map is an area centered on the target vehicle; it can be understood as a high-precision map of the local area corresponding to the detectable location of the target vehicle. In this embodiment, the origin is the centerline of the lane where the target vehicle is located. This location information can be represented as position coordinates in the world coordinate system. In one example, these position coordinates can be denoted as... In addition, the location information also includes the orientation angle of the centerline of the lane where the target vehicle is located in the world coordinate system. In one example, the orientation angle can be denoted as θ.

[0073] S205: Based on the location information, establish a local coordinate system, and transform the obstacle historical trajectory features and the high-precision map features into the local coordinate system to obtain local obstacle historical trajectory features and local high-precision map features;

[0074] Based on the position information of the starting point of the lane centerline obtained in step S203 above, a local coordinate system is established. This local coordinate system differs from the world coordinate system and can be considered as being obtained from the world coordinate system through operations such as rotation and translation. Compared to the world coordinate system, the local coordinate system places greater emphasis on the relative relationship between the map and the obstacle trajectory. The obstacle historical trajectory features and the high-precision map features are transformed into the local coordinate system to obtain local obstacle historical trajectory features and local high-precision map features. The local obstacle historical trajectory features and local high-precision map features can satisfy the rotation invariance required for trajectory prediction, and a general mode can be established. When performing joint prediction, with the starting point of different lanes as the origin, the feature representations of the local obstacle historical trajectory features and local high-precision map features of different target vehicles can remain basically consistent, thereby reducing redundant calculations of map information.

[0075] Specifically, the conversion formula for transforming the obstacle historical trajectory features and the high-precision map features to the local coordinate system is as follows:

[0076]

[0077] Among them, (x local y local (x) represents the position coordinates in the local coordinate system. world y world () represents the position coordinates in the local coordinate system. Here are the coordinates of the starting point of the lane's centerline obtained in step S203 above, and θ is the angle of the lane's orientation in the world coordinate system.

[0078] S207: Use a scene encoder to perform scene encoding on the local obstacle historical trajectory features and the local high-precision map features to obtain scene features;

[0079] A scene encoder can be viewed as a feature extractor used to represent scene features for autonomous driving, including local obstacle historical trajectory features and local high-precision map features. The scene encoder performs multimodal fusion of these two features to obtain a feature representation of the entire autonomous driving scene.

[0080] In addition, the local obstacle historical trajectory features and local high-precision map features obtained by sampling in time and space can be fused to obtain scene features that can establish the temporal and spatial relationships of obstacles, making full use of the temporal and spatial change features of obstacles in the environment.

[0081] S209: Input the scene features into the trajectory decoder for decoding processing to obtain the obstacle prediction trajectory of the target vehicle. The obstacle prediction trajectory is the trajectory of the obstacle corresponding to the target time period after the current time.

[0082] A trajectory decoder can be viewed as decoding the scene features obtained by the scene encoder according to their correspondences to obtain the trajectory for a future time period. For example, if the scene encoder inputs local obstacle historical trajectory features and local high-precision map features within 5 seconds, then the trajectory decoder can output the future trajectory of the obstacle within 6 seconds. It should be understood that the values ​​here are merely examples, and this application does not limit the duration of the input and output trajectories.

[0083] In one embodiment, the trajectory decoder can be a stack of one or more standard transformer cross-attention modules. The input is an initial query, which is then cross-processed with scene features obtained from the scene encoder to output a predicted trajectory.

[0084] In addition, during the encoding process, since the scene encoder retains the features of all obstacles, the trajectory decoder can directly output the predicted trajectories of all obstacles under this local high-precision map, thereby achieving joint prediction.

[0085] It's important to note that the scene encoder and trajectory decoder can be viewed as two parts of a trajectory prediction model, namely the scene encoder-trajectory decoder model. This trajectory prediction model can be trained using public datasets, such as the motion prediction dataset in Argoversev2. The loss function during training can use ERP (Edit Distance with Real Penalty) distance to represent the similarity between the predicted trajectory and the ground truth trajectory. ERP distance is an edit distance capable of handling local time shifts, taking into account temporal offsets and distortions when calculating the matching degree of time-series data.

[0086] In this embodiment, the obstacle historical trajectory features and high-precision map features of the target vehicle in the world coordinate system are transformed into a local coordinate system centered on the centerline starting point of the lane where the target vehicle is located in the local map, resulting in local obstacle historical trajectory features and local high-precision map features. Then, a scene encoder is used to perform scene encoding on these features to obtain scene features. These scene features are then input into a trajectory decoder for decoding to obtain the trajectory of obstacles corresponding to the target time period after the current moment of the target vehicle. This coordinate system transformation eliminates the need for the scene encoder-trajectory decoder model to understand all geographical coordinates in the world coordinate system, reducing the computational load. Furthermore, transforming all features to a local coordinate system centered on the centerline starting point of the lane establishes a correlation between the trajectory and the map, allowing feature encoding to focus more on the relative relationship and position between the trajectory and the map, thus effectively improving the scientific rigor and accuracy of the encoding. Moreover, by fusing obstacle historical trajectory features and map features through the scene encoder, the temporal and spatial information within the scene can be fully utilized, providing a more comprehensive contextual representation for the trajectory decoder, thereby improving the efficiency and accuracy of trajectory prediction.

[0087] In one embodiment, acquiring the obstacle historical trajectory features and high-precision map features of the target vehicle may include the following steps: collecting the coordinate information of the obstacle at a preset time sampling interval within a preset time period; determining the obstacle historical trajectory features based on the coordinate information of adjacent sampling times; collecting the map information of the high-precision map at a preset spatial sampling interval within a preset spatial range; and determining the high-precision map features based on the map information of adjacent sampling locations.

[0088] In the world coordinate system, an object can be represented as a point. A point can be represented by a multi-dimensional vector containing features such as its coordinate position (x, y, z), whether there is a traffic light, element type (lane or vehicle historical trajectory), whether it is at an intersection, and turning type (left turn, right turn, or straight ahead). The z-dimensional coordinate position can be omitted in practical applications to reduce computational load. Features of a point can be obtained through sampling. After determining the sampling start point and direction, uniform sampling can be performed on the high-precision map and the obstacle historical trajectory. Then, the sampled points are sequentially concatenated to obtain the high-precision map features and obstacle historical trajectory features, which are represented as vectors.

[0089] Specifically, for high-precision maps, sampling is performed at a preset spatial sampling interval, meaning that the spatial distances of points sampled from the high-precision map are all equal. For obstacle historical trajectories, sampling is performed at a preset temporal sampling interval, meaning that the time intervals of points sampled from the obstacle historical trajectory are all equal. Then, adjacent points are sequentially connected to form vectors. In one example, assuming each point has a dimension of 10, a total of 100 vectors are formed. These 100 vectors contain the historical trajectory features of the target obstacle, the position features of the target vehicle, the historical trajectory features of other obstacles within the road besides the target obstacle, and the high-precision map features of the target vehicle, with a proportion of 1:1:48:50 respectively. In this embodiment, by uniform sampling, complex scenes can be uniformly represented in vector form. Unlike traditional image or raster representation methods, vector representation effectively reduces the amount of data, facilitating efficient feature extraction from obstacle historical trajectories and high-precision maps.

[0090] In one embodiment, establishing a local coordinate system based on the location information and converting the obstacle historical trajectory features and the high-precision map features to the local coordinate system to obtain local obstacle historical trajectory features and local high-precision map features may include the following steps: establishing a local coordinate system with the starting point of the lane centerline of the local map where the target vehicle is currently located as the origin, the lane direction as the first axis, and the direction perpendicular to the lane direction as the second axis; the local map is a map in the high-precision map that extends outward from the current location of the target vehicle within a preset spatial range; parsing the obstacle historical trajectory features and the high-precision map features to determine the first dimension and the second dimension representing the coordinate position in the obstacle historical trajectory features and the high-precision map features respectively; normalizing the coordinate position corresponding to the first dimension in the obstacle historical trajectory features to the local coordinate system based on the location information to obtain the local obstacle historical trajectory features; normalizing the coordinate position corresponding to the second dimension in the high-precision map features to the local coordinate system based on the location information to obtain the local high-precision map features.

[0091] Figure 3 This is a schematic diagram of establishing a local coordinate system provided in an embodiment of this application.

[0092] like Figure 3 As shown, we will still take target vehicle 10 as an example. Figure 3 The area shown is a local map region, a high-precision map portion extending outwards from the current location of the target vehicle 10 within a preset spatial range. This preset spatial range can be set based on the actual spatial range that the target vehicle can perceive. Figure 3 In the diagram, dashed lines represent lane lines. Taking the starting point of the lane where target vehicle 10 is located as the origin O, a local coordinate system is established on the lane plane, with the lane's forward direction as the x-axis, the direction perpendicular to the lane's forward direction as the y-axis, and the direction perpendicular to the xOy plane as the z-axis. Then, in this local coordinate system, the position coordinates can be denoted as (x... local y local , z local It should be noted that in practical applications, the z-axis dimension can be omitted to reduce the amount of computation.

[0093] Since obstacle historical trajectory features and high-precision map features are multi-dimensional vectors, while coordinate transformation only involves position coordinates, it is first necessary to determine the first dimension representing the coordinate position in the obstacle historical trajectory features and the second dimension representing the coordinate position in the high-precision map features.

[0094] Specifically, based on the location information, the coordinate positions corresponding to the first dimension of the obstacle historical trajectory features are normalized to the local coordinate system to obtain the local obstacle historical trajectory features; based on the location information, the coordinate positions corresponding to the second dimension of the high-precision map features are normalized to the local coordinate system to obtain the local high-precision map features. The specific calculation process is described in step S205, and will not be repeated here.

[0095] Figure 4 This is a flowchart illustrating the scene coding process provided in the embodiments of this application.

[0096] In one embodiment, the scene encoder includes a first modal encoder, a second modal encoder, and a cascaded encoder. The step of using the scene encoder to encode the local obstacle historical trajectory features and the local high-precision map features to obtain scene features may include the following steps:

[0097] S401: Use the first modal encoder to perform obstacle encoding processing on the local obstacle historical trajectory features to obtain obstacle trajectory features;

[0098] Before using the first modality encoder for obstacle encoding, a multilayer perceptron (MLP) is first used to upscale the local obstacle historical trajectory features. In one example, the upscaled local obstacle historical trajectory features can have a dimension of 64.

[0099] The first modal encoder encoding process also involves obstacle position encoding. By encoding the historical trajectory features of local obstacles, the first modal encoder can more comprehensively understand the trajectory's positional information. Specifically, the position encoding formula is as follows:

[0100]

[0101] in, Each local obstacle historical trajectory feature can be viewed as a token. Therefore, the input to the first modal encoder is a sequence of tokens, where t represents the actual position of one of the local obstacle historical trajectory features in the entire sequence. For example, t equals 1 for the first token. P t This represents the position vector of the token. d represents the dimension of the token, for example, 64.

[0102] Next, the position vector P obtained by position encoding is... t After being stitched together with the historical trajectory features of local obstacles, the stitched features are then processed using the first modal encoder.

[0103] The first modal encoder is used to encode the features of the historical trajectory modality. It can be viewed as a self-attention mechanism and can consist of N layers of neural networks. The input to the first modal encoder can include a series of local obstacle historical trajectory features (in vector form). These input historical trajectory feature vectors are then encoded and fed into the encoder layer. The output of the first modal encoder can include intermediate prediction vectors (i.e., intermediate prediction results) corresponding to the input local obstacle historical trajectory features, denoted as obstacle trajectory features.

[0104] S403: The local high-precision map features are processed by the second modal encoder to obtain map features;

[0105] Before using a second modality encoder for map encoding, a multilayer perceptron (MLP) is first used to upscale the local high-precision map features. In one example, the upscaled local high-precision map features can have a dimension of 64.

[0106] The second modality encoder (DMO) is used to encode features for the map modality. It can also be viewed as a self-attention mechanism. Similarly, the DMO can be composed of N layers of neural networks. The input to the DMO can include a series of local high-precision map features (in vector form), which are then encoded and fed into the encoder layer. The output of the DMO can include intermediate prediction vectors corresponding to the input local high-precision map features, denoted as map features.

[0107] It should be noted that, in order to simplify the model, the widths of the first modal encoder and the second modal encoder are set to equal values ​​in this embodiment to avoid adding an extra projection layer to the model. Furthermore, the first modal encoder and the second modal encoder share the same depth in this embodiment.

[0108] S405: Use the serial encoder to fuse the obstacle trajectory features and the map features to obtain scene features.

[0109] After the output features from different modal encoders, since the first and second modal encoders share the same depth, the features from different modalities can be directly fused through a cascade encoder to further extract features from the entire scene. These features can serve as an intermediate result. In this step, the cascade encoder first concatenates the obstacle trajectory features and map features to connect the features of all modalities along the time and spatial dimensions. Then, it extracts the concatenated vector features through an attention mechanism to obtain the scene features.

[0110] It should be noted that the first modal encoder, the second modal encoder, and the cascade encoder can be considered as parts of the scene encoder; therefore, the scene encoder can be viewed as composed of one or more attention encoders. Using the scene encoder in this embodiment, information from different modalities in an autonomous driving scenario can be fused to generate a representation of the overall scene, fully utilizing the spatiotemporal and semantic information of the scene. Furthermore, by transforming the coordinate system and using scene encoding, scene features can be shared among each obstacle, thereby enabling the simultaneous prediction of the future trajectories of all obstacles within the scene, achieving joint prediction.

[0111] In one embodiment, step S405 may include:

[0112] The map features are used as the key and value matrices of the cascade encoder, and the obstacle trajectory features are used as the initial query matrix of the cascade encoder. The key matrix, the value matrix, and the initial query matrix are input into the cascade encoder for fusion encoding to obtain the scene features. In this embodiment, the cascade encoder can also be a cross-attention encoder, where the map encoding result is used as the key and value matrix of the cross-attention encoder, and the obstacle encoding result is used as the initial query matrix. These are input together into the cascade encoder for encoding to obtain the scene features.

[0113] In one embodiment, the obstacle trajectory prediction method provided in this application may further include: using a reference trajectory as the initial query of the trajectory decoder; the reference trajectory is an obstacle prediction trajectory generated by a preset reference trajectory model; and combining the initial query with the trajectory decoder to decode the scene features to obtain the obstacle prediction trajectory of the target vehicle.

[0114] In this embodiment, the trajectory decoder is a transformer decoder, including one or more cross-attention modules. In one implementation, the predicted trajectory can be obtained directly by decoding the scene features generated by the scene encoder. In another implementation, a reference trajectory generated by a reference trajectory model can be used as the initial query. In one example, the reference trajectory model can be a Target-drive Trajectory prediction (TNT) model. By inputting the reference trajectory as the initial query, the future trajectory states of surrounding obstacles can be explicitly input, thereby achieving interactive functionality.

[0115] To increase the prediction duration, this application proposes a segmented prediction scheme based on the above method. By dividing the prediction duration into multiple stages, each stage re-encodes the prediction trajectory using the output of the previous stage, and then continues to perform prediction, thereby obtaining a more accurate prediction trajectory.

[0116] Figure 5 This is a schematic flowchart of the segmented prediction process provided in the embodiments of this application.

[0117] S501: Divide the target time period after the current time into a first target time period and a second target time period;

[0118] In one implementation, the target time period after the current moment can be divided into a first target time period and a second target time period. In one example, assuming the target time period after the current moment is 6 seconds, it can be divided into two 3-second periods, the first 3 seconds being designated as the first target time period and the second 3 seconds as the second target time period. In another implementation, the target time period can also be divided according to other proportions.

[0119] S503: The scene features are decoded using a local trajectory decoder to obtain a first predicted trajectory; the first predicted trajectory is the trajectory of the obstacle corresponding to the first target time period;

[0120] The local trajectory decoder structure is similar to the trajectory decoder structure described in the previous steps, except that the output dimension is different. In the example above, the first predicted trajectory is the future trajectory within the first 3 seconds.

[0121] S505: Use the first modal encoder to perform obstacle encoding processing on the first predicted trajectory to obtain intermediate trajectory features;

[0122] The first modal encoder is a historical trajectory encoder. The first predicted trajectory is used as a new historical trajectory for obstacle encoding, which can improve the accuracy of trajectory prediction and prevent the lack of reference in the later stages of the predicted target segment, which would cause the predicted trajectory to deviate significantly from the true value. The intermediate trajectory features can provide a reference for trajectory prediction in the later time period.

[0123] S507: The intermediate trajectory features and the map features are fused using the serial encoder to obtain intermediate scene features;

[0124] Similar to step S405, after obtaining the intermediate trajectory features, the intermediate trajectory features and map features are fused using a serial encoder to obtain intermediate scene features.

[0125] S509: The intermediate scene features are decoded using the local trajectory decoder to obtain a second predicted trajectory; the second predicted trajectory is the trajectory of the obstacle corresponding to the second target time period;

[0126] Similar to step S503, the intermediate scene features are decoded using a local trajectory decoder to obtain the second predicted trajectory. In one example, the second predicted trajectory is the future trajectory within the second 3 seconds.

[0127] S511: The first predicted trajectory and the second predicted trajectory are spliced ​​together to obtain the obstacle prediction trajectory of the target vehicle.

[0128] By dividing the target time period to be predicted into a first time period and a second time period, and utilizing the trajectory encoding information and map information from the previous time period, the predicted trajectory for the next time period can be continuously predicted. While maintaining prediction accuracy, the prediction duration can be increased. By sharing map information, the computational load can be reduced by avoiding repeated calculations of map features in each prediction. Furthermore, segmented prediction can also improve prediction accuracy to some extent.

[0129] Figure 6 This is a schematic diagram of the modules of the trajectory prediction model provided in the embodiments of this application.

[0130] Figure 6The trajectory prediction model is a transformer model, comprising a scene encoder 610 and a trajectory decoder 620. The scene encoder outputs scene features to the trajectory decoder 620, which can use a reference trajectory 621 as an initial query to output the predicted trajectory. The scene encoder's input includes historical obstacle trajectories 611 and a high-precision map 612. The historical obstacle trajectory 611 can be considered a specific example of the historical obstacle trajectory features in step S201, and the high-precision map 612 can be considered a specific example of the high-precision map features in step S201. Features of the historical obstacle trajectory 611 are extracted by a self-attention module 613, and features of the high-precision map 612 are extracted by a self-attention module 614. These features are then concatenated and further extracted using an attention module 615 to obtain scene features representing the overall autonomous driving scene. By using this layered attention-based feature extraction method, and fusing information from different modalities multiple times, the spatiotemporal and semantic information of the scene can be fully utilized, providing a comprehensive contextual representation for the decoder's output predicted trajectory.

[0131] As can be seen from the technical solutions provided in the embodiments of this application above, this application transforms the obstacle historical trajectory features and high-precision map features of the target vehicle in the world coordinate system to a local coordinate system established with the starting point of the centerline of the lane where the target vehicle is located in the local map as the center, thus obtaining local obstacle historical trajectory features and local high-precision map features. Then, a scene encoder is used to perform scene encoding on the local obstacle historical trajectory features and local high-precision map features to obtain scene features. The scene features are input into the trajectory decoder for decoding processing to obtain the trajectory of the obstacle corresponding to the target time period after the current time of the target vehicle. Through the coordinate system transformation, the scene encoder-trajectory decoder model does not need to understand all the geographical coordinates in the world coordinate system, reducing the computational load of the scene encoder-trajectory decoder model. In addition, transforming all features to a local coordinate system centered on the starting point of the lane centerline can establish a correlation between the trajectory and the map, making the feature encoding pay more attention to the relative relationship and relative position between the trajectory and the map, thereby effectively improving the scientificity and accuracy of the encoding. Furthermore, by fusing historical obstacle trajectory features and map features through a scene encoder, the temporal and spatial information of the scene can be fully utilized, providing a more comprehensive contextual representation for the trajectory decoder, thereby improving the efficiency and accuracy of trajectory prediction. During scene encoding, the scene encoder employs hierarchical encoding, performing attention encoding on the features of each modality, concatenating the encoded features, and then using the attention module again for feature extraction. This results in richer and more accurate scene features output by the scene encoder. By transforming the coordinate system and using scene encoding, scene features can be shared among all obstacles, enabling the simultaneous prediction of the future trajectories of all obstacles within the scene, achieving joint prediction. Additionally, by performing multiple segmented predictions, utilizing the predicted trajectory information and map information from the previous stage to continuously predict the predicted trajectory for the next stage, a basis can be provided for the prediction of the next stage, improving the accuracy of trajectory prediction in the next stage.

[0132] Figure 7 This is a schematic diagram of the obstacle trajectory prediction device provided in the embodiments of this application.

[0133] like Figure 7 As shown, the device 700 includes:

[0134] The feature acquisition module 701 is used to acquire the obstacle historical trajectory features and high-precision map features of the target vehicle; the obstacle historical trajectory features represent the information of the obstacle historical trajectory in the world coordinate system; the high-precision map features represent the information of the high-precision map in the world coordinate system.

[0135] The starting point acquisition module 702 is used to acquire the position information of the starting point of the centerline of the lane where the target vehicle is located in the local map in the world coordinate system.

[0136] The coordinate transformation module 703 establishes a local coordinate system based on the location information, and transforms the obstacle historical trajectory features and the high-precision map features into the local coordinate system to obtain local obstacle historical trajectory features and local high-precision map features;

[0137] The scene encoding module 704 is used to perform scene encoding on the local obstacle historical trajectory features and the local high-precision map features using the scene encoder to obtain scene features;

[0138] The trajectory decoding module 705 is used to input the scene features into the trajectory decoder for decoding processing to obtain the obstacle prediction trajectory of the target vehicle. The obstacle prediction trajectory is the trajectory of the obstacle corresponding to the target time period after the current time.

[0139] In some embodiments, the feature acquisition module may include:

[0140] The time sampling submodule is used to collect the coordinate information of the obstacle at a preset time sampling interval within a preset time period;

[0141] The obstacle historical trajectory feature determination submodule is used to determine the obstacle historical trajectory features based on the coordinate information of adjacent sampling times;

[0142] The spatial sampling submodule is used to collect map information of the high-precision map within a preset spatial range at a preset spatial sampling interval;

[0143] The high-precision map feature determination module is used to determine the high-precision map features based on map information from adjacent sampling locations.

[0144] In some embodiments, the coordinate transformation module includes:

[0145] The local coordinate system establishment submodule is used to establish a local coordinate system with the starting point of the lane centerline of the local map where the target vehicle is currently located as the origin, the lane direction as the first axis, and the direction perpendicular to the lane direction as the second axis; the local map is a map in the high-precision map that expands a preset spatial range in all directions with the current location of the target vehicle as the center.

[0146] The dimension determination module is used to analyze the obstacle historical trajectory features and the high-precision map features, and to determine the first dimension and the second dimension representing the coordinate position in the obstacle historical trajectory features and the high-precision map features, respectively.

[0147] The trajectory normalization submodule is used to normalize the coordinate position corresponding to the first dimension in the obstacle historical trajectory feature to the local coordinate system according to the location information, so as to obtain the local obstacle historical trajectory feature.

[0148] The map normalization submodule is used to normalize the coordinate position corresponding to the second dimension in the high-precision map feature to the local coordinate system based on the location information, so as to obtain the local high-precision map feature.

[0149] In some embodiments, the scene encoding module includes:

[0150] The first encoding submodule is used to perform obstacle encoding processing on the local obstacle historical trajectory features using the first modal encoder to obtain obstacle trajectory features;

[0151] The second encoding submodule is used to perform map encoding processing on the local high-precision map features using the second modal encoder to obtain map features;

[0152] The serial encoding submodule is used to fuse the obstacle trajectory features and the map features using the serial encoder to obtain scene features.

[0153] In some embodiments, the concatenation coding submodule includes:

[0154] The fusion encoding unit is used to use the map features as the key matrix and value matrix of the cascade encoder, use the obstacle trajectory features as the initial query matrix of the cascade encoder, and input the key matrix, the value matrix and the initial query matrix into the cascade encoder for fusion encoding to obtain the scene features.

[0155] In some embodiments, the device 700 further includes:

[0156] The initial query determination submodule is used to use the reference trajectory as the initial query of the trajectory decoder; the reference trajectory is the obstacle prediction trajectory generated by the preset reference trajectory model.

[0157] The decoding submodule is used to combine the initial query and use the trajectory decoder to decode the scene features to obtain the obstacle prediction trajectory of the target vehicle.

[0158] In some embodiments, the device 700 further includes:

[0159] The time period division submodule is used to divide the target time period after the current time into a first target time period and a second target time period;

[0160] The first predicted trajectory determination submodule is used to decode the scene features using a local trajectory decoder to obtain a first predicted trajectory; the first predicted trajectory is the trajectory of the obstacle corresponding to the first target time period.

[0161] The intermediate trajectory feature determination submodule is used to perform obstacle encoding processing on the first predicted trajectory using the first modal encoder to obtain intermediate trajectory features;

[0162] The intermediate scene feature determination submodule is used to fuse the intermediate trajectory features and the map features using the serial encoder to obtain intermediate scene features;

[0163] The second predicted trajectory determination submodule is used to decode the intermediate scene features using the local trajectory decoder to obtain the second predicted trajectory; the second predicted trajectory is the trajectory of the obstacle corresponding to the second target time period;

[0164] The trajectory stitching module is used to stitch together the first predicted trajectory and the second predicted trajectory to obtain the obstacle prediction trajectory of the target vehicle.

[0165] The apparatus and method embodiments described herein are based on the same inventive concept.

[0166] This application provides an electronic device including a processor and a memory. The memory stores at least one instruction or at least one program, which is loaded and executed by the processor to implement the method provided in the above method embodiments.

[0167] Embodiments of this application also provide a computer storage medium, which can be disposed in a terminal to store at least one instruction or at least one program related to implementing a method as provided in the above method embodiments, wherein the at least one instruction or at least one program is loaded and executed by the processor to implement the method provided in the above method embodiments.

[0168] Embodiments of this application also provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method provided in the above-described method embodiments.

[0169] Optionally, in this embodiment, the storage medium may be located at at least one of the multiple network servers in a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0170] The memory described in this application embodiment can be used to store software programs and modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for the functions, etc.; the data storage area may store data created according to the use of the device, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory may also include a memory controller to provide the processor with access to the memory.

[0171] The methods provided in this application can be executed on mobile terminals, computer terminals, servers, or similar computing devices. Taking running on a server as an example... Figure 8 This is a hardware structure block diagram of a server for an obstacle trajectory prediction method provided in an embodiment of this application. For example... Figure 8As shown, server 800 can be considered a specific example of an electronic device. Server 800 can vary significantly due to differences in configuration or performance, and may include one or more Central Processing Units (CPUs) 810 (CPUs 810 may include, but are not limited to, microprocessors (MCUs) or programmable logic devices (FPGAs), a memory 830 for storing data, and one or more storage media 820 (e.g., one or more mass storage devices) for storing application programs 823 or data 822. The memory 830 and storage media 820 can be temporary or persistent storage. The program stored in the storage media 820 may include one or more modules, each module including a series of instruction operations on the server. Furthermore, the CPU 810 may be configured to communicate with the storage media 820 and execute the series of instruction operations in the storage media 820 on the server 800. Server 800 may also include one or more power supplies 860, one or more wired or wireless network interfaces 850, one or more input / output interfaces 840, and / or one or more operating systems 821, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0172] The input / output interface 840 can be used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of server 800. In one example, the input / output interface 840 includes a network interface controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the input / output interface 840 may be a radio frequency (RF) module for wireless communication with the Internet.

[0173] Those skilled in the art will understand that Figure 8 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, server 800 may also include... Figure 8 The more or fewer components shown, or having the same Figure 8 The different configurations shown.

[0174] As can be seen from the embodiments of the obstacle trajectory prediction method, apparatus, and storage medium provided in this application, this application transforms the historical obstacle trajectory features and high-precision map features of the target vehicle in the world coordinate system to a local coordinate system established with the starting point of the centerline of the lane where the target vehicle is located in the local map as the center, thus obtaining local obstacle historical trajectory features and local high-precision map features. Then, a scene encoder is used to perform scene encoding on the local obstacle historical trajectory features and local high-precision map features to obtain scene features. The scene features are input into the trajectory decoder for decoding processing to obtain the trajectory of the obstacle corresponding to the target time period after the current time of the target vehicle. Through the coordinate system transformation, the scene encoder-trajectory decoder model does not need to understand all the geographical coordinates in the world coordinate system, reducing the computational load of the scene encoder-trajectory decoder model. In addition, transforming all features to a local coordinate system centered on the starting point of the lane centerline can establish a correlation between the trajectory and the map, making the feature encoding pay more attention to the relative relationship and relative position between the trajectory and the map, thereby effectively improving the scientificity and accuracy of the encoding. Furthermore, by fusing historical obstacle trajectory features and map features through a scene encoder, the temporal and spatial information of the scene can be fully utilized, providing a more comprehensive contextual representation for the trajectory decoder, thereby improving the efficiency and accuracy of trajectory prediction. During scene encoding, the scene encoder employs hierarchical encoding, performing attention encoding on the features of each modality, concatenating the encoded features, and then using the attention module again for feature extraction. This results in richer and more accurate scene features output by the scene encoder. By transforming the coordinate system and using scene encoding, scene features can be shared among all obstacles, enabling the simultaneous prediction of the future trajectories of all obstacles within the scene, achieving joint prediction. Additionally, by performing multiple segmented predictions, utilizing the predicted trajectory information and map information from the previous stage to continuously predict the predicted trajectory for the next stage, a basis can be provided for the prediction of the next stage, improving the accuracy of trajectory prediction in the next stage.

[0175] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0176] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0177] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer storage medium, such as a read-only memory, a disk, or an optical disk.

[0178] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method of obstacle trajectory prediction, the method comprising: receiving a plurality of images of a scene; determining a plurality of obstacle trajectories based on the plurality of images; and outputting the plurality of obstacle trajectories. include: Obtain the target vehicle's historical obstacle trajectory features and high-precision map features; Obtain the position information of the starting point of the centerline of the lane where the target vehicle is located in the local map in the world coordinate system; Based on the location information, a local coordinate system is established, and the obstacle historical trajectory features and the high-precision map features are transformed into the local coordinate system to obtain local obstacle historical trajectory features and local high-precision map features; The local coordinate system is established with the starting point of the lane centerline of the local map where the target vehicle is currently located as the origin, the lane direction as the first axis, and the direction perpendicular to the lane direction as the second axis. The scene features are obtained by using the scene encoder in the trajectory prediction model to encode the historical trajectory features of the local obstacles and the local high-precision map features. The scene features are input into the trajectory decoder in the trajectory prediction model for decoding processing to obtain the obstacle prediction trajectory of the target vehicle; the obstacle prediction trajectory is the trajectory of the obstacle corresponding to the target time period after the current time; during the training process of the trajectory prediction model, the loss function represents the similarity between the predicted trajectory and the ground truth trajectory.

2. The method of claim 1, wherein, The acquisition of the target vehicle's historical obstacle trajectory features and high-precision map features includes: The coordinate information of the obstacle is collected at a preset time sampling interval within a preset time period; Based on the coordinate information of adjacent sampling times, the historical trajectory characteristics of the obstacle are determined; Map information of the high-precision map is collected within a preset spatial range at a preset spatial sampling interval; The high-precision map features are determined based on map information from adjacent sampling locations.

3. The method of claim 1, wherein, The step of establishing a local coordinate system based on the location information and transforming the obstacle historical trajectory features and the high-precision map features into the local coordinate system to obtain local obstacle historical trajectory features and local high-precision map features includes: Analyze the historical trajectory features of the obstacle and the high-precision map features, and determine the first dimension and the second dimension representing the coordinate position in the historical trajectory features of the obstacle and the high-precision map features, respectively; Based on the location information, the coordinate position corresponding to the first dimension in the obstacle historical trajectory feature is normalized to the local coordinate system to obtain the local obstacle historical trajectory feature; Based on the location information, the coordinate position corresponding to the second dimension in the high-precision map feature is normalized to the local coordinate system to obtain the local high-precision map feature.

4. The method of claim 1, wherein, The scene encoder includes a first modal encoder, a second modal encoder, and a cascaded encoder. The scene encoder is used to encode the local obstacle historical trajectory features and the local high-precision map features to obtain scene features, including: The first modal encoder is used to perform obstacle encoding processing on the local obstacle historical trajectory features to obtain obstacle trajectory features; The local high-precision map features are processed by the second modal encoder to obtain map features; The obstacle trajectory features and the map features are fused using the serial encoder to obtain scene features.

5. The method of claim 4, wherein, The process of fusing the obstacle trajectory features and the map features using the serial encoder to obtain scene features includes: The map features are used as the key matrix and value matrix of the cascade encoder, and the obstacle trajectory features are used as the initial query matrix of the cascade encoder. The key matrix, the value matrix and the initial query matrix are input into the cascade encoder for fusion encoding to obtain the scene features.

6. The method of claim 1, wherein, After obtaining the scene features, the method further includes: The reference trajectory is used as the initial query for the trajectory decoder; the reference trajectory is the obstacle prediction trajectory generated by the preset reference trajectory model. Based on the initial query, the scene features are decoded using the trajectory decoder to obtain the obstacle prediction trajectory of the target vehicle.

7. The method according to claim 4, characterized in that, After obtaining the scene features, the method further includes: The target time period following the current moment is divided into a first target time period and a second target time period; The scene features are decoded using a local trajectory decoder to obtain a first predicted trajectory; the first predicted trajectory is the trajectory of the obstacle corresponding to the first target time period. The first modal encoder is used to perform obstacle encoding processing on the first predicted trajectory to obtain intermediate trajectory features; The intermediate trajectory features and the map features are fused using the serial encoder to obtain intermediate scene features; The intermediate scene features are decoded using the local trajectory decoder to obtain a second predicted trajectory; the second predicted trajectory is the trajectory of the obstacle corresponding to the second target time period. The first predicted trajectory and the second predicted trajectory are spliced ​​together to obtain the obstacle prediction trajectory of the target vehicle.

8. An obstacle trajectory prediction device, characterized in that, include: The feature acquisition module is used to acquire the obstacle history trajectory features and high-precision map features of the target vehicle; The starting point acquisition module is used to acquire the position information of the starting point of the centerline of the lane where the target vehicle is located in the local map in the world coordinate system; The coordinate transformation module is used to establish a local coordinate system based on the location information, and to transform the obstacle historical trajectory features and the high-precision map features into the local coordinate system to obtain local obstacle historical trajectory features and local high-precision map features; The local coordinate system is established with the starting point of the lane centerline of the local map where the target vehicle is currently located as the origin, the lane direction as the first axis, and the direction perpendicular to the lane direction as the second axis. The scene encoding module is used to encode the local obstacle historical trajectory features and the local high-precision map features using the scene encoder in the trajectory prediction model to obtain scene features; The trajectory decoding module is used to input the scene features into the trajectory decoder in the trajectory prediction model for decoding processing to obtain the obstacle prediction trajectory of the target vehicle; the obstacle prediction trajectory is the trajectory of the obstacle corresponding to the target time period after the current time; during the training process of the trajectory prediction model, the loss function represents the similarity between the predicted trajectory and the ground truth trajectory.

9. An electronic device, characterized in that, The electronic device includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, the at least one instruction or at least one program being loaded and executed by the processor to implement the obstacle trajectory prediction method as described in any one of claims 1-7.

10. A computer storage medium, characterized in that, The computer storage medium stores at least one instruction or at least one program, which is loaded and executed by a processor to implement the obstacle trajectory prediction method as described in any one of claims 1-7.

Citation Information

Patent Citations

  • Map construction method and device, electronic device and computer readable storage medium

    CN113865580A

  • Trajectory prediction method and apparatus, and computer device and storage medium

    WO2022222095A1