Vehicle trajectory prediction method, device and equipment, vehicle and storage medium

Abstract

The application discloses a vehicle trajectory prediction method, device, equipment, vehicle and storage medium, and belongs to the technical field of vehicles, wherein the method comprises the following steps: acquiring a plurality of first trajectory points of a second vehicle in a historical time period, the second vehicle being located within a preset range of the first vehicle, the historical time period comprising a plurality of historical time points, the plurality of historical time points corresponding to the plurality of first trajectory points in a one-to-one manner; determining trajectory offset information of the second vehicle in the historical time period according to the plurality of first trajectory points, the trajectory offset information being used for indicating M trajectory offset points or being used for indicating that there is no trajectory offset point, M being a positive integer; and predicting a trajectory of the second vehicle in a first time period according to the trajectory offset information and the plurality of first trajectory points to obtain a target trajectory. The application can improve the prediction accuracy of the trajectory of the adjacent vehicle.

Description

Technical Field

[0001] This application belongs to the field of vehicle technology, and particularly relates to a vehicle trajectory prediction method, device, equipment, vehicle, and storage medium. Background Technology

[0002] In autonomous driving systems, accurately predicting the movement trajectories of surrounding traffic participants is a necessary prerequisite for various assistance functions. As a result, vehicle trajectory prediction has become one of the key technologies to ensure traffic safety and efficiency.

[0003] Whether in urban, elevated, or highway scenarios, the lateral movement of adjacent vehicles cutting into the lane can easily cause adverse reactions in your vehicle. If the prediction of these lateral movements is delayed, it may lead to insufficient deceleration or even a collision. Currently, existing trajectory prediction methods are insufficient in predicting the lateral movements of adjacent vehicles, resulting in low accuracy in predicting their trajectories at a future timeframe. Summary of the Invention

[0004] This application provides a vehicle trajectory prediction method, apparatus, device, vehicle, and storage medium, which can improve the accuracy of predicting the trajectories of nearby vehicles.

[0005] In a first aspect, embodiments of this application provide a vehicle trajectory prediction method, applied to a first vehicle, the method comprising:

[0006] The second vehicle is located within a preset range of the first vehicle, and the historical time period includes multiple historical moments, each of which corresponds one-to-one with a number of the first trajectory points.

[0007] Based on multiple first trajectory points, determine the trajectory offset information of the second vehicle within the historical time period. The trajectory offset information is used to indicate M trajectory offset points, or to indicate that there are no trajectory offset points, where M is a positive integer.

[0008] Based on the trajectory offset information and multiple first trajectory points, the trajectory of the second vehicle within a first time period is predicted to obtain the target trajectory.

[0009] In one embodiment of this application, the step of predicting the trajectory of the second vehicle within a first time period based on the trajectory offset information and multiple first trajectory points to obtain a target trajectory includes:

[0010] If the trajectory offset information indicates M trajectory offset points, for each trajectory offset point, at least one first trajectory point corresponding to the trajectory offset point is obtained, and the historical time corresponding to at least one first trajectory point is greater than or equal to the historical time corresponding to the trajectory offset point.

[0011] Based on at least one first trajectory point corresponding to each trajectory offset point, and multiple first trajectory points, the trajectory of the second vehicle within a first time period is predicted to obtain the target trajectory.

[0012] In one embodiment of this application, the step of predicting the trajectory of the second vehicle within a first time period based on at least one first trajectory point corresponding to each trajectory offset point, and multiple first trajectory points, to obtain a target trajectory, includes:

[0013] Based on at least one first trajectory point corresponding to each trajectory offset point, a position mask matrix corresponding to each trajectory offset point is constructed. The position mask matrix corresponding to each trajectory offset point is used to characterize the association relationship between the trajectory offset point and the corresponding at least one first trajectory point.

[0014] For each position mask matrix corresponding to the trajectory offset point, the position mask matrix is ​​added to the data feature matrix to obtain the sub-encoding matrix corresponding to the trajectory offset point. The data feature matrix is ​​obtained by extracting features from multiple first trajectory points.

[0015] The sub-encoding matrices corresponding to the trajectory offset points are concatenated sequentially to obtain the target encoding matrix;

[0016] The target encoding matrix is ​​decoded using a trajectory prediction model to obtain the target trajectory. The trajectory prediction model is based on a self-attention mechanism.

[0017] In one embodiment of this application, the step of adding the position mask matrix and the data feature matrix to obtain the sub-encoding matrix corresponding to the trajectory offset point includes:

[0018] The position mask matrix is ​​transformed to obtain the target mask matrix corresponding to the trajectory offset point. The dimension of the target mask matrix corresponding to the trajectory offset point is the same as the dimension of the data feature matrix.

[0019] The data feature matrix is ​​added to the target mask matrix corresponding to the trajectory offset point to obtain the sub-encoding matrix corresponding to the trajectory offset point.

[0020] In one embodiment of this application, constructing a position mask matrix corresponding to each trajectory offset point based on at least one first trajectory point corresponding to each trajectory offset point includes:

[0021] For each trajectory offset point, the elements of the target row in the base matrix are set to preset values ​​to obtain the position mask matrix corresponding to the trajectory offset point. The base matrix is ​​a zero matrix of U rows and U columns, where U is the total number of the multiple first trajectory points. The target row is the row corresponding to other trajectory points in the base matrix. The other trajectory points are the trajectory points other than the trajectory offset point and at least one first trajectory point corresponding to the trajectory offset point among the multiple first trajectory points.

[0022] In one embodiment of this application, the step of predicting the trajectory of the second vehicle within a first time period based on the trajectory offset information and multiple first trajectory points to obtain a target trajectory includes:

[0023] If the trajectory offset information indicates that there is no trajectory offset point, then feature extraction is performed on multiple first trajectory points to obtain a data feature matrix of multiple first trajectory points;

[0024] The target encoding matrix is ​​obtained by calculating based on the data feature matrix.

[0025] The target encoding matrix is ​​decoded using a trajectory prediction model to obtain the target trajectory. The trajectory prediction model is based on a self-attention mechanism.

[0026] In one embodiment of this application, obtaining multiple first trajectory points of the second vehicle within a historical time period includes:

[0027] Multiple initial trajectory points of the second vehicle within a historical time period are obtained. The multiple historical times correspond one-to-one with the multiple initial trajectory points. Each initial trajectory point is a trajectory point of the second vehicle in a first coordinate system. The first coordinate system is determined based on the pose of the first vehicle at the historical time corresponding to each initial trajectory point.

[0028] Multiple initial trajectory points are mapped to a second coordinate system to obtain multiple first trajectory points. The second coordinate system is determined based on the pose of the first vehicle at a target historical time among the multiple historical times. The target historical time is later than other historical times among the multiple historical times other than the target historical time.

[0029] Secondly, embodiments of this application provide a vehicle trajectory prediction device, applied to a first vehicle, the device comprising:

[0030] The acquisition module is used to acquire multiple first trajectory points of the second vehicle within a historical time period. The second vehicle is located within a preset range of the first vehicle. The historical time period includes multiple historical moments, and each of the multiple historical moments corresponds one-to-one with a multiple of the first trajectory points.

[0031] The determining module is used to determine the trajectory offset information of the second vehicle within the historical time period based on multiple first trajectory points. The trajectory offset information is used to indicate M trajectory offset points, or to indicate that there are no trajectory offset points, where M is a positive integer.

[0032] The prediction module is used to predict the trajectory of the second vehicle within a first time period based on the trajectory offset information and multiple first trajectory points, so as to obtain the target trajectory.

[0033] Thirdly, embodiments of this application provide an electronic device, including: a processor and a memory storing computer program instructions;

[0034] When the processor executes the computer program instructions, it implements the vehicle trajectory prediction method as described in the first aspect.

[0035] Fourthly, embodiments of this application provide a vehicle that includes the electronic equipment described in the third aspect.

[0036] Fifthly, embodiments of this application provide a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the vehicle trajectory prediction method as described in the first aspect.

[0037] The vehicle trajectory prediction method, apparatus, device, vehicle, and storage medium provided in this application embodiment acquire multiple first trajectory points of a second vehicle within a historical time period. Based on these first trajectory points, trajectory offset information of the second vehicle within the historical time period is determined. Subsequently, based on the trajectory offset information and the multiple first trajectory points, the trajectory of the second vehicle within the first time period is predicted to obtain a target trajectory. The trajectory offset information indicates either the presence of M trajectory offset points or the absence of any trajectory offset points. These steps, based on the trajectory offset information of the second vehicle within the historical time period, allow for different prediction methods depending on whether trajectory offset points exist (i.e., whether M trajectory offset points exist or no trajectory offset points exist), thereby improving the accuracy of predicting the trajectory of neighboring vehicles (i.e., the second vehicle). Attached Figure Description

[0038] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a flowchart illustrating the vehicle trajectory prediction method provided in an embodiment of this application;

[0040] Figure 2 This is a schematic diagram of constructing a location mask matrix provided in an embodiment of this application;

[0041] Figure 3 This is another schematic flowchart of the vehicle trajectory prediction method provided in the embodiments of this application;

[0042] Figure 4 This is a schematic diagram of the vehicle trajectory prediction device provided in the embodiments of this application;

[0043] Figure 5 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0044] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0045] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0046] In all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. Additionally, when embodiments of this application require access to sensitive personal information, separate permission or consent from the user is obtained through pop-ups or redirects to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments obtained.

[0047] To address the problems of the prior art, embodiments of this application provide a vehicle trajectory prediction method, apparatus, device, vehicle, and storage medium. The vehicle trajectory prediction method provided in this application embodiment will be described first below.

[0048] Figure 1 This is a flowchart illustrating a vehicle trajectory prediction method provided in an embodiment of this application. Figure 1 As shown, the vehicle trajectory prediction method provided in this application embodiment is applied to a first vehicle, specifically to the electronic equipment of the first vehicle. The vehicle trajectory prediction method provided in this application embodiment includes the following steps 101-103, wherein:

[0049] Step 101: Obtain multiple first trajectory points of the second vehicle within a historical time period. The second vehicle is located within a preset range of the first vehicle. The historical time period includes multiple historical moments, and each of the multiple historical moments corresponds one-to-one with a multiple of the first trajectory points.

[0050] The second vehicle is a vehicle located within a preset range of the first vehicle. There can be one or more second vehicles, and their number is not limited. When there are multiple second vehicles, trajectory prediction can be performed for each individual second vehicle. The preset range can be determined by combining data such as the acquisition range of the image acquisition device set on the first vehicle, the detection range of the ranging device set on the first vehicle, or the safe driving distance of the vehicle, or it can be set by a relevant object. For example, the preset range can be a range less than 10 meters away from the first vehicle, or a range less than 15 meters away from the first vehicle, or other ranges. Furthermore, the second vehicle can be determined from the vehicles located within the preset range of the first vehicle in various ways. For example, all vehicles located within the preset range of the first vehicle can be directly considered as second vehicles. Alternatively, vehicles located within the preset range of the first vehicle and determined in response to a selection operation by a relevant object can be considered as second vehicles. The selection operation can be a touch operation, voice input operation, or button operation, etc. Alternatively, vehicles located within the preset range of the first vehicle and exhibiting lane departure signs can also be considered as second vehicles.

[0051] The historical time period can be a time interval of a certain duration preceding the current moment. The current moment refers to the starting moment for predicting the trajectory of the second vehicle. The historical time period includes multiple historical moments, and these multiple historical moments correspond one-to-one with multiple first trajectory points. That is, at each historical moment, a first trajectory point can be collected. Through the first trajectory point, the position of the second vehicle relative to the first vehicle at that historical moment can be described.

[0052] Step 102: Based on multiple first trajectory points, determine the trajectory offset information of the second vehicle within the historical time period. The trajectory offset information is used to indicate M trajectory offset points, or to indicate that there are no trajectory offset points, where M is a positive integer.

[0053] Based on multiple first trajectory points, the trajectory offset information of the second vehicle within a historical time period can be determined. This trajectory offset information indicates whether the second vehicle has M trajectory offset points within the historical time period, or whether no trajectory offset points exist. A trajectory offset point refers to a location where the vehicle's trajectory undergoes a significant change. For example, multiple first trajectory points can be compared, and the first trajectory point with the largest deviation from its adjacent first trajectory points can be identified as the trajectory offset point, thus obtaining the trajectory offset information of the second vehicle within the historical time period. Alternatively, a clustering algorithm can be used to process the multiple first trajectory points to determine the trajectory offset information of the second vehicle within the historical time period.

[0054] Alternatively, based on cubic spline interpolation, least squares method, Kalman filter algorithm, or other data fitting methods, multiple first trajectory points are fitted to obtain the trajectory curve of the second vehicle within the historical time period. By differentiating the trajectory curve, it can be determined whether there are trajectory offset points, thus obtaining the trajectory offset information of the second vehicle within the historical time period. Furthermore, assuming the first trajectory point is (x... t ,y t ,ω t Then, the coordinate value x of the first trajectory point in the first direction can be extracted from the first trajectory point. t And the coordinates y of the first trajectory point in the second direction. t The first trajectory point (x) after processing is obtained. t ,y t ). At the processed first trajectory point (x) t ,y t After fitting, the trajectory offset information of the second vehicle within the historical time period is determined by referring to the aforementioned steps. This reduces the waste of computational resources and improves the computational efficiency in determining the trajectory offset information. Here, t refers to the historical time corresponding to the first trajectory point, and x... t This refers to the coordinates of the first trajectory point at historical time t in the first direction. The first direction can be perpendicular to the length direction of the first vehicle at historical time t. t ω is the coordinate value of the first trajectory point at historical time t in the second direction. The second direction can be parallel to the length direction of the first vehicle at historical time t. t It is the relative angle between the second vehicle and the first vehicle at historical time t.

[0055] Step 103: Based on the trajectory offset information and multiple first trajectory points, predict the trajectory of the second vehicle within the first time period to obtain the target trajectory.

[0056] The first time period refers to the time period following the current moment, and it is later than the historical time period. The target trajectory can be any trajectory point of the second vehicle at any moment within the first time period, multiple trajectory points of the second vehicle within the first time period, or a segment of the second vehicle's driving trajectory within the first time period.

[0057] Based on trajectory offset information and multiple first trajectory points, the trajectory of the second vehicle within a first time period can be predicted to obtain the target trajectory. If the second vehicle has trajectory offset points within a historical time period, it can be considered that the second vehicle's driving trajectory has deviated significantly within that historical time period, potentially affecting the normal driving of the first vehicle. Therefore, for the second vehicle with trajectory offset points within a historical time period, its action characteristics within that historical time period can be emphasized. For example, if the trajectory offset information indicates M trajectory offset points, a position mask matrix can be constructed based on the historical times corresponding to each of the M trajectory offset points to emphasize the first trajectory point following each trajectory offset point. Subsequently, the constructed position mask matrix and multiple first trajectory points can be processed using models based on self-attention mechanisms, long short-term memory networks, or graph neural network models to predict the trajectory of the second vehicle within the first time period and obtain the target trajectory.

[0058] If the second vehicle does not have any trajectory deviation points within the historical time period, it can be assumed that the second vehicle is unlikely to affect the normal driving of the first vehicle. Therefore, a simpler trajectory prediction method can be used to determine the target trajectory. For example, a deep learning model (such as a model based on self-attention, a long short-term memory network, or a graph neural network model) can be used to process multiple trajectory points to predict the target trajectory of the two vehicles within the first time period. This improves trajectory prediction efficiency, increases the response time of the first vehicle, and allows it more time to control its speed, reducing the risk of a collision.

[0059] In this embodiment, multiple first trajectory points of the second vehicle within a historical time period are acquired. Based on these first trajectory points, trajectory offset information of the second vehicle within the historical time period is determined. Subsequently, based on the trajectory offset information and the multiple first trajectory points, the trajectory of the second vehicle within the first time period is predicted to obtain the target trajectory. The trajectory offset information indicates either the presence or absence of a trajectory offset point. These steps, based on the trajectory offset information of the second vehicle within the historical time period, allow for different prediction methods depending on whether the second vehicle has trajectory offset points within the historical time period (i.e., whether there are M trajectory offset points or no trajectory offset points), thereby improving the accuracy of predicting the trajectory of neighboring vehicles (i.e., the second vehicle).

[0060] In one embodiment of this application, obtaining multiple first trajectory points of the second vehicle within a historical time period includes:

[0061] Multiple initial trajectory points of the second vehicle within a historical time period are obtained. The multiple historical times correspond one-to-one with the multiple initial trajectory points. Each initial trajectory point is a trajectory point of the second vehicle in a first coordinate system. The first coordinate system is determined based on the pose of the first vehicle at the historical time corresponding to each initial trajectory point.

[0062] Multiple initial trajectory points are mapped to a second coordinate system to obtain multiple first trajectory points. The second coordinate system is determined based on the pose of the first vehicle at a target historical time among the multiple historical times. The target historical time is later than other historical times among the multiple historical times other than the target historical time.

[0063] Multiple initial trajectory points of the second vehicle within a historical time period can be acquired using ranging devices (such as millimeter-wave radar, lidar, etc.), image acquisition devices, or other equipment installed on the first vehicle. Alternatively, if the first and second vehicles are in communication relationship, the first vehicle can acquire multiple initial trajectory points sent by the second vehicle. Or, multiple initial trajectory points can be acquired using a global positioning system, high-precision maps, or intelligent transportation systems. Each initial trajectory point corresponds one-to-one with multiple historical moments; that is, one initial trajectory point can be acquired for each historical moment. The initial trajectory points are used to describe the position of the second vehicle relative to the first vehicle. Each initial trajectory point is a trajectory point of the second vehicle in a first coordinate system. The first coordinate system is determined based on the pose of the first vehicle at each historical moment; that is, based on data such as the geometric center, heading, and position of the first vehicle at each historical moment, a first coordinate system can be established for each historical moment.

[0064] Understandably, when the first vehicle moves, multiple initial trajectory points may be located in different coordinate systems. Therefore, to facilitate subsequent trajectory prediction, coordinate system transformation can be performed on these multiple initial trajectory points, converting them to the same coordinate system (i.e., the second coordinate system). Specifically, the second coordinate system is determined based on the pose of the first vehicle at a target historical moment from among multiple historical moments. That is, based on data such as the geometric center, heading, and position of the first vehicle at the target historical moment, the second coordinate system can be established. The target historical moment can be later than any other historical moment among the multiple historical moments. For example, if the multiple historical moments include 0 seconds, 0.1 seconds, and 0.2 seconds, then 0.2 seconds can be used as the target historical moment. Alternatively, the target historical moment can be determined from among the multiple historical moments based on the actual situation.

[0065] In the process of mapping multiple initial trajectory points to the second coordinate system, for example, multiple initial trajectory points can be mapped to the second coordinate system based on coordinate transformation relationships, thereby obtaining multiple first trajectory points. Taking the example that the target historical time is later than other historical times among multiple historical times besides the target historical time, the coordinate transformation relationship satisfies the following equations (1) and (2):

[0066]

[0067] Among them, t end It is the target historical moment. At the target historical moment t end The coordinates of the initial trajectory point in the first direction of the first coordinate system. At the target historical moment t end The coordinates of the initial trajectory point in the second direction of the first coordinate system. At the target historical moment t end The relative angle between the second vehicle and the first vehicle in the first coordinate system. At the target historical moment t end The coordinates of the first trajectory point in the first direction of the second coordinate system. At the target historical moment t end The coordinates of the first trajectory point in the second direction of the second coordinate system. The target historical moment t end The relative angle between the second vehicle and the first vehicle in the second coordinate system. It is the coordinate value of the first trajectory point in the first direction of the second coordinate system at historical time t-1. It is the coordinate value of the first trajectory point in the second direction of the second coordinate system at historical time t-1. It is the relative angle between the second vehicle and the first vehicle in the second coordinate system at historical time t-1, and dω is ω t With ω t-1 The difference, ω t Let ω be the relative angle between the second vehicle and the first vehicle in the first coordinate system at historical time t. t-1 It is the relative angle between the second vehicle and the first vehicle in the first coordinate system at historical time t-1. t It is the coordinate value of the initial trajectory point at historical time t in the first direction of the first coordinate system, y t dx is the coordinate value of the initial trajectory point at historical time t in the second direction of the first coordinate system, where x is the coordinate value of the initial trajectory point. t With x t-1 The difference, x t-1 dy is the coordinate value of the initial trajectory point at historical time t-1 in the first direction of the first coordinate system, where dy is the yt With y t-1 The difference, y t-1 It is the coordinate value of the initial trajectory point in the second direction of the first coordinate system at the historical time t-1.

[0068] Through the above steps, multiple initial trajectory points can be transformed from different coordinate systems (i.e., the first coordinate system) to the same coordinate system (i.e., the second coordinate system), resulting in multiple first trajectory points. This eliminates the coordinate system differences that may exist between different trajectory points, allowing all trajectory points to be compared, analyzed, and calculated under a unified reference system, thus reducing data errors caused by coordinate system differences.

[0069] In one embodiment of this application, the step of predicting the trajectory of the second vehicle within a first time period based on the trajectory offset information and multiple first trajectory points to obtain a target trajectory includes:

[0070] If the trajectory offset information indicates M trajectory offset points, for each trajectory offset point, at least one first trajectory point corresponding to the trajectory offset point is obtained, and the historical time corresponding to at least one first trajectory point is greater than or equal to the historical time corresponding to the trajectory offset point.

[0071] Based on at least one first trajectory point corresponding to each trajectory offset point, and multiple first trajectory points, the trajectory of the second vehicle within a first time period is predicted to obtain the target trajectory.

[0072] Based on the trajectory offset information and multiple first trajectory points, the trajectory of the second vehicle within the first time period can be predicted to obtain the target trajectory. Specifically, when the trajectory offset information indicates M trajectory offset points, for each trajectory offset point, at least one corresponding first trajectory point is obtained, and the historical time corresponding to at least one first trajectory point is greater than the historical time corresponding to the trajectory offset point. For example, assuming the trajectory offset information indicates two trajectory offset points (i.e., trajectory offset point 1 and trajectory offset point 2), the historical time corresponding to trajectory offset point 1 is 0.2 seconds, and the historical time corresponding to trajectory offset point 2 is 0.6 seconds, then for trajectory offset point 1, multiple first trajectory points with a corresponding historical time greater than 0.2 seconds can be obtained, and for trajectory offset point 2, at least one first trajectory point with a corresponding historical time greater than 0.6 seconds can be obtained.

[0073] Because of the existence of trajectory offset points, it can be assumed that the second vehicle has a significant deviation, posing a greater safety risk. Therefore, multiple first trajectory points can be processed to emphasize at least one first trajectory point corresponding to each trajectory offset point when predicting the target trajectory of the second vehicle, thereby improving trajectory prediction accuracy. For example, a prompt word can be constructed based on at least one first trajectory point corresponding to each trajectory offset point, and the prompt word, along with multiple first trajectory points, can be input into a deep learning model (such as a model based on a self-attention mechanism) to obtain the target trajectory output by the deep learning model. Alternatively, a position mask matrix can be constructed based on at least one first trajectory point corresponding to each trajectory offset point. Simultaneously, data features of multiple first trajectory points can be extracted to obtain a data feature matrix of multiple first trajectory points. Each element in the position mask matrix corresponds to the validity of a certain position in the input sequence. Through the constructed position mask matrix, the deep learning model can distinguish between valid and invalid data. Therefore, after obtaining the position mask matrix and the data feature matrix, the target trajectory can be predicted by the deep learning model based on the position mask matrix and the data feature matrix.

[0074] The above steps, when the trajectory offset information indicates M trajectory offset points, emphasize the first trajectory point after the occurrence of the trajectory offset point. This allows for the focus on more important data during trajectory prediction, enabling rapid processing of the more important data and improving the accuracy and efficiency of vehicle trajectory prediction.

[0075] In one embodiment of this application, the step of predicting the trajectory of the second vehicle within a first time period based on at least one first trajectory point corresponding to each trajectory offset point, and multiple first trajectory points, to obtain a target trajectory, includes:

[0076] Based on at least one first trajectory point corresponding to each trajectory offset point, a position mask matrix corresponding to each trajectory offset point is constructed. The position mask matrix corresponding to each trajectory offset point is used to characterize the association relationship between the trajectory offset point and the corresponding at least one first trajectory point.

[0077] For each position mask matrix corresponding to the trajectory offset point, the position mask matrix is ​​added to the data feature matrix to obtain the sub-encoding matrix corresponding to the trajectory offset point. The data feature matrix is ​​obtained by extracting features from multiple first trajectory points.

[0078] The sub-encoding matrices corresponding to the trajectory offset points are concatenated sequentially to obtain the target encoding matrix;

[0079] The target encoding matrix is ​​decoded using a trajectory prediction model to obtain the target trajectory. The trajectory prediction model is based on a self-attention mechanism.

[0080] Given M trajectory offset points indicated by trajectory offset information, a position mask matrix can be constructed for each trajectory offset point based on at least one corresponding first trajectory point. The position mask matrix for each trajectory offset point represents the association between the trajectory offset point and its corresponding first trajectory point. That is, for each trajectory offset point, important data (i.e., first trajectory points whose corresponding historical time is greater than or equal to the historical time corresponding to the trajectory offset point) and invalid data (i.e., first trajectory points whose corresponding historical time is less than the historical time corresponding to the trajectory offset point) among multiple first trajectory points can be distinguished based on the trajectory offset point. Subsequently, for the position mask matrix corresponding to each trajectory offset point, the position mask matrix can be added to the data feature matrix to obtain the sub-encoding matrix corresponding to the trajectory offset point. This allows for the identification of important features (i.e., features corresponding to important data) and invalid features (i.e., features corresponding to invalid data) in the data feature matrix.

[0081] By using methods such as horizontal concatenation, vertical concatenation, stacking, or element-wise concatenation, the sub-encoding matrices corresponding to the trajectory offset points can be sequentially concatenated to obtain the target encoding matrix. Then, a trajectory prediction model can decode the target encoding matrix to obtain the target trajectory. This trajectory prediction model employs a self-attention mechanism, such as a Transformer-based model or Bidirectional Encoder Representations from Transformers (BERT). Taking a Transformer-based trajectory prediction model as an example, the target encoding matrix can be decoded using a pre-trained encoder within the trajectory prediction model to obtain the target trajectory.

[0082] Thus, when the trajectory offset information indicates M trajectory offset points, by emphasizing the first trajectory point after the occurrence of the trajectory offset point, and by using the trajectory prediction model to predict the trajectory based on the processed data, the accuracy of vehicle trajectory prediction can be improved.

[0083] In one embodiment of this application, the step of adding the position mask matrix and the data feature matrix to obtain the sub-encoding matrix corresponding to the trajectory offset point includes:

[0084] The position mask matrix is ​​transformed to obtain the target mask matrix corresponding to the trajectory offset point. The dimension of the target mask matrix corresponding to the trajectory offset point is the same as the dimension of the data feature matrix.

[0085] The data feature matrix is ​​added to the target mask matrix corresponding to the trajectory offset point to obtain the sub-encoding matrix corresponding to the trajectory offset point.

[0086] Typically, the dimension of the location mask matrix is ​​the same as the total number of the first trajectory points. For example, if the total number of the first trajectory points is 21, then the dimension of the location mask matrix can be [21, 21]. In this case, there is a possibility that the dimension of the location mask matrix is ​​different from the dimension of the data feature matrix, making it impossible to add the location mask matrix and the data feature matrix directly. Therefore, a dimension transformation can be performed on the location mask matrix to make the dimension of the resulting target mask matrix the same as the dimension of the data feature matrix. For example, suppose the dimension of the data feature matrix is ​​[dim, dim], where dim is set according to the actual situation, and the dimension of the location mask matrix is ​​[21, 21]. Then, the dimension of the location mask matrix can be transformed using various methods such as Multi-Layer Perceptron (MLP) networks, pooling operations, and convolution operations to obtain the target mask matrix corresponding to the trajectory offset point, and the dimension of the target mask matrix corresponding to the trajectory offset point is [dim, dim]. Then, the data feature matrix can be added to the target mask matrix corresponding to the trajectory offset point to obtain the sub-encoding matrix corresponding to the trajectory offset point. The above steps, by performing dimensional transformation on the position mask matrix, facilitate subsequent matrix operations.

[0087] In one embodiment of this application, constructing a position mask matrix corresponding to each trajectory offset point based on at least one first trajectory point corresponding to each trajectory offset point includes:

[0088] For each trajectory offset point, the elements of the target row in the base matrix are set to preset values ​​to obtain the position mask matrix corresponding to the trajectory offset point. The base matrix is ​​a zero matrix of U rows and U columns, where U is the total number of the multiple first trajectory points. The target row is the row corresponding to other trajectory points in the base matrix. The other trajectory points are the trajectory points other than the trajectory offset point and at least one first trajectory point corresponding to the trajectory offset point among the multiple first trajectory points.

[0089] Based on at least one first trajectory point corresponding to each trajectory offset point, a position mask matrix corresponding to each trajectory offset point can be constructed. Specifically, the base matrix is ​​a U-row, U-column zero matrix, meaning all elements in the base matrix have a value of 0. Here, U is a positive integer. Each first trajectory point has a corresponding row in the base matrix. Then, for each trajectory offset point, the target row in the base matrix can be determined, which is the row corresponding to other trajectory points in the base matrix. Other trajectory points refer to trajectory points other than the at least one first trajectory point corresponding to the trajectory offset point. Then, the elements of the target row in the base matrix can be set to preset values ​​to obtain the position mask matrix corresponding to the trajectory offset point. The preset values ​​are set according to the actual situation; for example, the preset value can be negative infinity, 1, or other values.

[0090] For example, continuing with the case of two trajectory offset points (trajectory offset point 1 and trajectory offset point 2), assuming the total duration of the historical time period is 2 seconds, and the interval between any two adjacent historical moments is 0.1 seconds, the total number of first trajectory points is 21, and the basic matrix is ​​a 21-row, 21-column zero matrix. Then, for trajectory offset point 1, we can determine at least one first trajectory point corresponding to trajectory offset point 1 (i.e., the first trajectory point corresponding to a historical moment greater than 0.2 seconds, totaling 19). Other trajectory points include the first trajectory point corresponding to 0 seconds and the first trajectory point corresponding to 0.1 seconds. Subsequently, we can set the elements of the rows corresponding to other trajectory points in the basic matrix to preset values, while keeping the values ​​of the elements of the rows corresponding to at least one first trajectory point corresponding to trajectory offset point 1 at 0, thus obtaining the position mask matrix corresponding to trajectory offset point 1. For trajectory offset point 2, at least one first trajectory point corresponding to trajectory offset point 2 can be determined (the first trajectory point corresponding to a historical time greater than 0.6 seconds, a total of 15). Other trajectory points include the first trajectory point corresponding to 0 seconds, 0.1 seconds, 0.2 seconds, 0.3 seconds, 0.4 seconds, and 0.5 seconds. Subsequently, the elements of the rows corresponding to other trajectory points in the basic matrix can be set to preset values, and the values ​​of the elements of the rows corresponding to at least one first trajectory point corresponding to trajectory offset point 2 can still be 0, thus obtaining the position mask matrix corresponding to trajectory offset point 2.

[0091] Furthermore, if the trajectory deviation point is determined by differentiating the trajectory curve of the second vehicle over a historical time period, the time corresponding to the trajectory deviation point (i.e., the time when the second vehicle deviated from its trajectory) may fall between two adjacent historical moments. For example, if the time corresponding to the trajectory deviation point is 0.25 seconds, it may fall between two adjacent historical moments (such as 0.2 seconds and 0.3 seconds). Therefore, the smaller of the two adjacent historical moments (i.e., 0.2 seconds) can be taken as the historical moment corresponding to the trajectory deviation point. Then, referring to the aforementioned steps, a position mask matrix can be constructed based on the historical moment corresponding to the trajectory deviation point.

[0092] The above steps can construct a position mask matrix corresponding to each trajectory offset point, so as to construct a target encoding matrix based on the position mask matrix corresponding to each trajectory offset point, and then predict the target trajectory based on the target encoding matrix.

[0093] Figure 2 This is a schematic diagram illustrating the construction of a target mask matrix provided in an embodiment of this application. For example... Figure 2 As shown, in one example, by fitting multiple first trajectory points, the trajectory curve of the second vehicle within a historical time period can be obtained. The historical time period includes six historical moments: 0 seconds, 0.1 seconds, 0.2 seconds, 0.3 seconds, 0.4 seconds, and 0.5 seconds. The total number of first trajectory points is six, and the fundamental matrix is ​​a 6x6 zero matrix. After differentiating the trajectory curve, the trajectory offset point can be determined. Assuming the historical moment corresponding to the trajectory offset point is 0.4 seconds, and the other trajectory points include the first trajectory points corresponding to 0 seconds, 0.1 seconds, 0.2 seconds, and 0.3 seconds, and the rows corresponding to these other trajectory points in the fundamental matrix are the first four rows, then the elements of the target row (i.e., the first four rows of the fundamental matrix) can be set to preset values ​​(while the elements of the last two rows of the fundamental matrix remain 0). This yields the position mask matrix. If the dimension of the data feature matrix is ​​[12, 12], after performing dimension transformation on the location mask matrix through the MLP network, a target mask matrix with dimension [12, 12] can be obtained.

[0094] In one embodiment of this application, the step of predicting the trajectory of the second vehicle within a first time period based on the trajectory offset information and multiple first trajectory points to obtain a target trajectory includes:

[0095] If the trajectory offset information indicates that there is no trajectory offset point, then feature extraction is performed on multiple first trajectory points to obtain a data feature matrix of multiple first trajectory points;

[0096] The target encoding matrix is ​​obtained by calculating based on the data feature matrix.

[0097] The target encoding matrix is ​​decoded using a trajectory prediction model to obtain the target trajectory. The trajectory prediction model is based on a self-attention mechanism.

[0098] If the trajectory deviation information indicates the absence of a trajectory deviation point, the risk of a collision is low, so a relatively simple trajectory prediction method can be used to predict the trajectory of the second vehicle within the first time period. Specifically, features can be extracted from multiple first trajectory points using methods such as MLP networks, Long Short-Term Memory networks, positional encoding, or convolutional neural networks, resulting in a data feature matrix for multiple first trajectory points. Further, the query matrix, key matrix, and value matrix are all made equal to the aforementioned extracted data feature matrices. Based on the query matrix, key matrix, and value matrix, attention weights are calculated sequentially, and a weighted average of the value matrix is ​​performed using these attention weights to obtain the target encoding matrix.

[0099] Then, the target trajectory is obtained by decoding the target encoding matrix using a trajectory prediction model. This trajectory prediction model employs a self-attention mechanism (such as a Transformer-based model or BERT). Taking a Transformer-based model as an example, the target encoding matrix can be decoded using a pre-trained encoder within the trajectory prediction model to obtain the target trajectory. These steps, for the second vehicle without significant trajectory deviation, simplify the trajectory prediction process and improve its efficiency.

[0100] Figure 3 This is another flowchart illustrating the vehicle trajectory prediction method provided in this application. For example... Figure 3 As shown below, the vehicle trajectory prediction method provided in the embodiments of this application will be illustrated by example.

[0101] Step 1: Obtain multiple initial trajectory points of the second vehicle within the historical time period.

[0102] The historical time period includes multiple historical moments. For the initial trajectory point (x... t ,y t ,ω t ), its coordinate values ​​in the first direction of the first coordinate system. Coordinate values ​​in the second direction of the first coordinate system The relative angle between the second vehicle and the first vehicle in the first coordinate system t1,t2,…t endThis refers to multiple historical moments. For example, the total duration of a historical time period can be 2 seconds, and the interval between multiple historical moments is 0.1 seconds, meaning the time interval between any two adjacent historical moments is 0.1 seconds. In this case, the total number of multiple initial trajectory points is 21.

[0103] Step 2: Perform coordinate transformation on multiple initial trajectory points to obtain multiple first trajectory points.

[0104] For each initial trajectory point, we can refer to equations (1) and (2) to determine the initial trajectory point (x). t ,y t ,ω t Mapping to t end In the coordinate system (i.e., the second coordinate system) at that time (i.e., the target's historical time), the first trajectory point can be obtained.

[0105] Step 3: Fit a trajectory curve based on multiple first trajectory points.

[0106] Using cubic spline interpolation, multiple first trajectory points are... A trajectory curve is fitted. Specifically, it can be extracted from multiple first trajectory points. Data, and cubic spline interpolation method is used to... The data is processed to obtain the trajectory curve. The trajectory curve satisfies...

[0107] Step 4: Differentiate the trajectory curve and calculate the number of inflection points.

[0108] That is, to differentiate the trajectory curve and calculate... The number of curve inflection points M (i.e., the number of trajectory offset points) is calculated. If M = 0, proceed to step 5; if M > 0, proceed to step 6.

[0109] Step 5: Use a self-attention encoder to encode the target encoding matrix.

[0110] For trajectory curves without trajectory offset points, a self-attention encoder from the Transformer architecture can be used. The data is encoded. Specifically, an MLP network is used to encode the data. The data is processed into a data feature matrix E with dimensions [dim, dim], and the query matrix Q, key matrix K, and value matrix V of the self-attention encoder are all equal to the data feature matrix E. After encoding, the target encoding matrix can be obtained.

[0111] Step 6: Initialize the loop parameter n and the encoding matrix R.

[0112] At this point, n = 0, and the encoding matrix R is an empty matrix.

[0113] Step 7: Determine if n is greater than M. If not, proceed to step 8; if yes, proceed to step 12.

[0114] Step 8: Based on the trajectory offset point and multiple first trajectory points, determine the data feature matrix of multiple first trajectory points and the position mask matrix corresponding to the trajectory offset point.

[0115] Using MLP networks The data is processed into a data feature matrix E of dimension [dim, dim], and the position mask matrix P is initialized. n (i.e., the fundamental matrix), at this time the position mask matrix P n It is a zero matrix with dimensions [21, 21]. Therefore, for the m-th trajectory offset point, we can determine the historical time corresponding to the trajectory offset point, and determine the number s of times less than the historical time corresponding to the trajectory offset point among multiple historical times, and then adjust the position mask matrix P. n The elements of the first s rows are set to negative infinity, thus obtaining the position mask matrix corresponding to the trajectory offset point. m is a positive integer less than or equal to M.

[0116] Step 9: Perform dimensional transformation on the position mask matrix to obtain the target mask matrix.

[0117] The position mask matrix P corresponding to the trajectory offset point is obtained through an MLP network. n Perform a dimensional transformation to obtain the target mask matrix corresponding to the trajectory offset points. Target mask matrix The dimension is [dim, dim].

[0118] Step 10: Calculate the sub-encoding matrix based on the target mask matrix and the data feature matrix.

[0119] target mask matrix Add the sub-encoding matrix R to the data feature matrix E and perform self-attention encoding to obtain the sub-encoding matrix R. n .

[0120] Step 11: Update the loop parameter n and concatenate the sub-encoding matrix to the encoding matrix to obtain the target encoding matrix.

[0121] Update the loop parameters so that n = n + 1, and change the sub-encoding matrix R. n Concatenate to the encoding matrix R, R = [R|R n ], obtain the target encoding matrix, and jump to step 7.

[0122] Step 12: Decode the target encoding matrix to obtain the target trajectory.

[0123] The target trajectory is obtained by decoding the target encoding matrix using the decoder in the Transformer architecture.

[0124] The vehicle trajectory prediction method provided in this application model performs additional attention mechanism modeling on a second vehicle with obvious cutting-in movements, thereby emphasizing the lateral movement characteristics of the second vehicle. During model encoding, the additional attention layer data and the original attention layer data are concatenated and output. After calculation by the deep learning model, the trajectory prediction accuracy of the second vehicle can be improved, and the response time of the first vehicle can be increased, giving the first vehicle more time to control its speed and reducing the risk of a collision.

[0125] Figure 4 This is a schematic diagram of the vehicle trajectory prediction device provided in an embodiment of this application. Figure 4 As shown, the vehicle trajectory prediction device 400 includes:

[0126] The acquisition module 401 is used to acquire multiple first trajectory points of the second vehicle within a historical time period. The second vehicle is located within a preset range of the first vehicle. The historical time period includes multiple historical moments, and each of the multiple historical moments corresponds one-to-one with a multiple of the first trajectory points.

[0127] The determining module 402 is used to determine the trajectory offset information of the second vehicle within the historical time period based on multiple first trajectory points. The trajectory offset information is used to indicate M trajectory offset points, or to indicate that there are no trajectory offset points, where M is a positive integer.

[0128] The prediction module 403 is used to predict the trajectory of the second vehicle within a first time period based on the trajectory offset information and multiple first trajectory points, so as to obtain the target trajectory.

[0129] In one embodiment of this application, the prediction module 403 includes:

[0130] The first acquisition submodule is used to acquire at least one first trajectory point corresponding to each trajectory offset point if the trajectory offset information indicates M trajectory offset points, wherein the historical time corresponding to at least one first trajectory point is greater than or equal to the historical time corresponding to the trajectory offset point.

[0131] The first prediction submodule is used to predict the trajectory of the second vehicle within a first time period based on at least one first trajectory point corresponding to each trajectory offset point and multiple first trajectory points, so as to obtain the target trajectory.

[0132] In one embodiment of this application, the first prediction submodule includes:

[0133] The construction unit is configured to construct a position mask matrix corresponding to each trajectory offset point based on at least one first trajectory point corresponding to each trajectory offset point. The position mask matrix corresponding to the trajectory offset point is used to characterize the association relationship between the trajectory offset point and the corresponding at least one first trajectory point.

[0134] The calculation unit is used to add the position mask matrix to the data feature matrix for each of the trajectory offset points to obtain the sub-encoding matrix corresponding to the trajectory offset point. The data feature matrix is ​​obtained by extracting features from multiple first trajectory points.

[0135] The splicing unit is used to sequentially splice the sub-encoding matrices corresponding to the trajectory offset points to obtain the target encoding matrix;

[0136] The decoding unit is used to decode the target encoding matrix through a trajectory prediction model to obtain the target trajectory. The trajectory prediction model adopts a self-attention mechanism-based model.

[0137] In one embodiment of this application, the computing unit includes:

[0138] The transformation subunit is used to perform dimensional transformation on the position mask matrix to obtain the target mask matrix corresponding to the trajectory offset point, wherein the dimension of the target mask matrix corresponding to the trajectory offset point is the same as the dimension of the data feature matrix.

[0139] The calculation subunit is used to add the data feature matrix to the target mask matrix corresponding to the trajectory offset point to obtain the sub-encoding matrix corresponding to the trajectory offset point.

[0140] In one embodiment of this application, the building unit includes:

[0141] A sub-unit is set up to set the elements of the target row in the base matrix to preset values ​​for each trajectory offset point to obtain the position mask matrix corresponding to the trajectory offset point. The base matrix is ​​a zero matrix of U rows and U columns, where U is the total number of the multiple first trajectory points. The target row is the row corresponding to other trajectory points in the base matrix. The other trajectory points are the trajectory points other than the trajectory offset point and at least one first trajectory point corresponding to the trajectory offset point among the multiple first trajectory points.

[0142] In one embodiment of this application, the prediction module 403 includes:

[0143] The extraction submodule is used to extract features from multiple first trajectory points if the trajectory offset information indicates that there is no trajectory offset point, thereby obtaining a data feature matrix of multiple first trajectory points.

[0144] The calculation submodule is used to calculate the target encoding matrix based on the data feature matrix;

[0145] The second prediction submodule is used to decode the target encoding matrix through a trajectory prediction model to obtain the target trajectory. The trajectory prediction model adopts a self-attention mechanism-based model.

[0146] In one embodiment of this application, the acquisition module 401 includes:

[0147] The second acquisition submodule is used to acquire multiple initial trajectory points of the second vehicle within a historical time period. The multiple historical times correspond one-to-one with the multiple initial trajectory points. Each initial trajectory point is a trajectory point of the second vehicle in a first coordinate system. The first coordinate system is determined according to the pose of the first vehicle at the historical time corresponding to each initial trajectory point.

[0148] The mapping submodule is used to map multiple initial trajectory points to a second coordinate system to obtain multiple first trajectory points. The second coordinate system is determined based on the pose of the first vehicle at a target historical time among multiple historical times. The target historical time is later than other historical times among multiple historical times other than the target historical time.

[0149] The vehicle trajectory prediction device 400 provided in this application embodiment can implement all the processes implemented in the aforementioned vehicle trajectory prediction method embodiment and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0150] Figure 5 This is a schematic diagram of the structure of the electronic device provided in an embodiment of this application. For example... Figure 5 As shown, the electronic device may include a processor 501 and a memory 502 storing computer program instructions.

[0151] Specifically, the processor 501 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0152] Memory 502 may include mass storage for data or instructions. For example, and not limitingly, memory 502 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 502 may include removable or non-removable (or fixed) media. Where appropriate, memory 502 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 502 is non-volatile solid-state memory.

[0153] Memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the methods according to the first or second aspect of this disclosure.

[0154] The processor 501 reads and executes computer program instructions stored in the memory 502 to implement any of the vehicle trajectory prediction methods in the above embodiments.

[0155] In one example, the in-vehicle equipment may also include a communication interface 503 and a bus 510. Wherein, as... Figure 5 As shown, the processor 501, memory 502, and communication interface 503 are connected through bus 510 and complete communication with each other.

[0156] The communication interface 503 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0157] Bus 510 includes hardware, software, or both, that couples components of a camera extrinsic calibration method or verification device together. For example, and not as a limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 510 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0158] In addition, this application provides a vehicle that includes the above-described electronic equipment.

[0159] Vehicles can be private cars, such as sedans, SUVs, MPVs, or pickup trucks. Vehicles can also be commercial vehicles, such as vans, buses, small trucks, or large semi-trailers. Vehicles can be either gasoline-powered or new energy vehicles. When a vehicle is a new energy vehicle, it can be a hybrid or a pure electric vehicle.

[0160] Furthermore, in conjunction with the vehicle trajectory prediction methods in the above embodiments, this application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the vehicle trajectory prediction methods in the above embodiments.

[0161] Alternatively, this application embodiment can provide a computer program product for implementation, wherein when the instructions in the computer program product are executed by an electronic device, the electronic device implements any of the vehicle trajectory prediction methods in the above embodiments.

[0162] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described as examples. However, the method process of this application is not limited to the specific steps described. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0163] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0164] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0165] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0166] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A vehicle trajectory prediction method, characterized in that, Applied to a first vehicle, the method includes: The second vehicle is located within a preset range of the first vehicle, and the historical time period includes multiple historical moments, each of which corresponds one-to-one with a number of the first trajectory points. Based on multiple first trajectory points, determine the trajectory offset information of the second vehicle within the historical time period. The trajectory offset information is used to indicate M trajectory offset points, or to indicate that there are no trajectory offset points, where M is a positive integer. Based on the trajectory offset information and multiple first trajectory points, the trajectory of the second vehicle within a first time period is predicted to obtain the target trajectory. The step of predicting the trajectory of the second vehicle within a first time period based on the trajectory offset information and multiple first trajectory points to obtain the target trajectory includes: If the trajectory offset information indicates M trajectory offset points, for each trajectory offset point, at least one first trajectory point corresponding to the trajectory offset point is obtained, and the historical time corresponding to at least one first trajectory point is greater than or equal to the historical time corresponding to the trajectory offset point; based on at least one first trajectory point corresponding to each trajectory offset point and multiple first trajectory points, the trajectory of the second vehicle in the first time period is predicted to obtain the target trajectory. If the trajectory offset information indicates that there is no trajectory offset point, then feature extraction is performed on multiple first trajectory points to obtain a data feature matrix of multiple first trajectory points; a target encoding matrix is ​​calculated based on the data feature matrix; the target encoding matrix is ​​decoded through a trajectory prediction model to obtain the target trajectory, wherein the trajectory prediction model adopts a model based on a self-attention mechanism.

2. The method according to claim 1, characterized in that, The step of predicting the trajectory of the second vehicle within a first time period based on at least one first trajectory point corresponding to each trajectory offset point, and multiple first trajectory points, to obtain the target trajectory, includes: Based on at least one first trajectory point corresponding to each trajectory offset point, a position mask matrix corresponding to each trajectory offset point is constructed. The position mask matrix corresponding to each trajectory offset point is used to characterize the association relationship between the trajectory offset point and the corresponding at least one first trajectory point. For each position mask matrix corresponding to the trajectory offset point, the position mask matrix is ​​added to the data feature matrix to obtain the sub-encoding matrix corresponding to the trajectory offset point. The data feature matrix is ​​obtained by extracting features from multiple first trajectory points. The sub-encoding matrices corresponding to the trajectory offset points are concatenated sequentially to obtain the target encoding matrix; The target encoding matrix is ​​decoded using a trajectory prediction model to obtain the target trajectory. The trajectory prediction model is based on a self-attention mechanism.

3. The method according to claim 2, characterized in that, The step of adding the position mask matrix to the data feature matrix to obtain the sub-encoding matrix corresponding to the trajectory offset point includes: The position mask matrix is ​​transformed to obtain the target mask matrix corresponding to the trajectory offset point. The dimension of the target mask matrix corresponding to the trajectory offset point is the same as the dimension of the data feature matrix. The data feature matrix is ​​added to the target mask matrix corresponding to the trajectory offset point to obtain the sub-encoding matrix corresponding to the trajectory offset point.

4. The method according to claim 2, characterized in that, The step of constructing a position mask matrix corresponding to each trajectory offset point based on at least one first trajectory point corresponding to each trajectory offset point includes: For each trajectory offset point, the elements of the target row in the base matrix are set to preset values ​​to obtain the position mask matrix corresponding to the trajectory offset point. The base matrix is ​​a zero matrix of U rows and U columns, where U is the total number of the multiple first trajectory points. The target row is the row corresponding to other trajectory points in the base matrix. The other trajectory points are the trajectory points other than the trajectory offset point and at least one first trajectory point corresponding to the trajectory offset point among the multiple first trajectory points.

5. The method according to claim 1, characterized in that, The acquisition of multiple first trajectory points of the second vehicle within a historical time period includes: Multiple initial trajectory points of the second vehicle within a historical time period are obtained. The multiple historical times correspond one-to-one with the multiple initial trajectory points. Each initial trajectory point is a trajectory point of the second vehicle in a first coordinate system. The first coordinate system is determined based on the pose of the first vehicle at the historical time corresponding to each initial trajectory point. Multiple initial trajectory points are mapped to a second coordinate system to obtain multiple first trajectory points. The second coordinate system is determined based on the pose of the first vehicle at a target historical time among the multiple historical times. The target historical time is later than other historical times among the multiple historical times other than the target historical time.

6. A vehicle trajectory prediction device, characterized in that, Applied to a first vehicle, the device includes: The acquisition module is used to acquire multiple first trajectory points of the second vehicle within a historical time period. The second vehicle is located within a preset range of the first vehicle. The historical time period includes multiple historical moments, and each of the multiple historical moments corresponds one-to-one with a multiple of the first trajectory points. The determining module is used to determine the trajectory offset information of the second vehicle within the historical time period based on multiple first trajectory points. The trajectory offset information is used to indicate M trajectory offset points, or to indicate that there are no trajectory offset points, where M is a positive integer. The prediction module is used to predict the trajectory of the second vehicle within a first time period based on the trajectory offset information and multiple first trajectory points, so as to obtain the target trajectory. The prediction module includes: The first acquisition submodule is used to acquire at least one first trajectory point corresponding to each trajectory offset point if the trajectory offset information indicates M trajectory offset points, wherein the historical time corresponding to at least one first trajectory point is greater than or equal to the historical time corresponding to the trajectory offset point. The first prediction submodule is used to predict the trajectory of the second vehicle in a first time period based on at least one first trajectory point corresponding to each trajectory offset point and multiple first trajectory points, so as to obtain the target trajectory. The extraction submodule is used to extract features from multiple first trajectory points if the trajectory offset information indicates that there is no trajectory offset point, thereby obtaining a data feature matrix of multiple first trajectory points. The calculation submodule is used to calculate the target encoding matrix based on the data feature matrix; The second prediction submodule is used to decode the target encoding matrix through a trajectory prediction model to obtain the target trajectory. The trajectory prediction model adopts a self-attention mechanism-based model.

7. An electronic device, characterized in that, include: Processor and memory storing computer program instructions; When the processor executes the computer program instructions, it implements the vehicle trajectory prediction method as described in any one of claims 1-5.

8. A vehicle, characterized in that, The vehicle includes the electronic equipment as described in claim 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the vehicle trajectory prediction method as described in any one of claims 1-5.

Images