Vehicle trajectory prediction method, device and equipment, vehicle and storage medium

Abstract

The invention discloses a vehicle track prediction method, device and equipment, a vehicle and a storage medium, and belongs to the technical field of vehicles, and the method comprises the steps: obtaining a plurality of first track points of a second vehicle in a historical time period, the second vehicle being located in a preset range of the first vehicle, the historical time period comprising a plurality of historical moments, the plurality of historical moments are in one-to-one correspondence with the plurality of first track points; according to the multiple first track points, track offset information of the second vehicle in the historical time period is determined, the track offset information is used for indicating M track offset points or indicating that no track offset point exists, and M is a positive integer; and according to the trajectory offset information and the plurality of first trajectory points, predicting the trajectory of the second vehicle in the first time period to obtain a target trajectory. According to the invention, the prediction accuracy of the adjacent vehicle trajectory can be improved.

Description

Technical Field

[0001] This application belongs to the technical field of vehicles, and particularly relates to a vehicle trajectory prediction method, device, equipment, vehicle, and storage medium. Background Art

[0002] In an autonomous driving system, accurately predicting the motion trajectories of surrounding traffic participants is a necessary prerequisite for various auxiliary functions. Thus, vehicle trajectory prediction has become one of the key technologies to ensure traffic safety and efficiency.

[0003] Whether in an urban scenario, an elevated scenario, or a highway scenario, the cross-cutting motion of neighboring vehicles traveling in the adjacent lanes of the host vehicle can easily lead to poor performance of the host vehicle. If the prediction of the neighboring vehicle with a cross-cutting motion lags, it may result in the host vehicle not decelerating in time or even causing a collision accident. Currently, the existing trajectory prediction methods have insufficient prediction of the cross-cutting motion of neighboring vehicles, resulting in a low prediction accuracy of the trajectories of neighboring vehicles in a future time period. Summary of the Invention

[0004] Embodiments of this application provide a vehicle trajectory prediction method, device, equipment, vehicle, and storage medium, which can improve the prediction accuracy of neighboring vehicle trajectories.

[0005] In a first aspect, embodiments of this application provide a vehicle trajectory prediction method, which is applied to a first vehicle. The method includes:

[0006] Obtain multiple first trajectory points of a second vehicle within a historical time period. The second vehicle is located within a preset range of the first vehicle. The historical time period includes multiple historical moments, and the multiple historical moments correspond to the multiple first trajectory points one by one;

[0007] Determine trajectory offset information of the second vehicle within the historical time period according to the multiple first trajectory points. The trajectory offset information is used to indicate M trajectory offset points, or is used to indicate the absence of trajectory offset points, where M is a positive integer;

[0008] Predict the trajectory of the second vehicle within a first time period according to the trajectory offset information and the multiple first trajectory points to obtain a target trajectory.

[0009] In an embodiment of this application, the predicting the trajectory of the second vehicle within a first time period according to the trajectory offset information and the multiple first trajectory points to obtain a target trajectory includes:

[0010] If the trajectory offset information indicates M trajectory offset points, for each of the trajectory offset points, obtain at least one first trajectory point corresponding to the trajectory offset point, and the historical time corresponding to at least one of the first trajectory points is greater than or equal to the historical time corresponding to the trajectory offset point;

[0011] According to at least one of the first trajectory points corresponding to each of the trajectory offset points and the multiple first trajectory points, predict the trajectory of the second vehicle within the first time period to obtain a target trajectory.

[0012] In an embodiment of the present application, the predicting the trajectory of the second vehicle within the first time period according to at least one of the first trajectory points corresponding to each of the trajectory offset points and the multiple first trajectory points to obtain a target trajectory includes:

[0013] According to at least one of the first trajectory points corresponding to each of the trajectory offset points, construct a position mask matrix corresponding to each of the trajectory offset points, and the position mask matrix corresponding to the trajectory offset point is used to characterize the association relationship between the trajectory offset point and the corresponding at least one of the first trajectory points;

[0014] For the position mask matrix corresponding to each of the trajectory offset points, add the position mask matrix to the data feature matrix to obtain a sub-encoding matrix corresponding to the trajectory offset point, and the data feature matrix is obtained by performing feature extraction on the multiple first trajectory points;

[0015] Successively splice the sub-encoding matrices corresponding to the trajectory offset points to obtain a target encoding matrix;

[0016] Through a trajectory prediction model, perform decoding processing on the target encoding matrix to obtain the target trajectory, and the trajectory prediction model adopts a model based on a self-attention mechanism.

[0017] In an embodiment of the present application, the adding the position mask matrix to the data feature matrix to obtain a sub-encoding matrix corresponding to the trajectory offset point includes:

[0018] Perform dimension conversion on the position mask matrix to obtain a target mask matrix corresponding to the trajectory offset point, and the dimension of the target mask matrix corresponding to the trajectory offset point is the same as the dimension of the data feature matrix;

[0019] Add the data feature matrix to the target mask matrix corresponding to the trajectory offset point to obtain a sub-encoding matrix corresponding to the trajectory offset point.

[0020] In one embodiment of the present application, constructing a position mask matrix corresponding to each of the trajectory offset points according to at least one of the first trajectory points corresponding to each of the trajectory offset points includes:

[0021] For each of the trajectory offset points, set the elements of the target row in the base matrix to a preset value to obtain the position mask matrix corresponding to the trajectory offset point. The base matrix is a zero matrix with U rows and U columns, where U is the total number of the first trajectory points. The target row is the row corresponding to other trajectory points in the base matrix, and the other trajectory points are the trajectory points among the first trajectory points except the trajectory offset point and at least one of the first trajectory points corresponding to the trajectory offset point.

[0022] In one embodiment of the present application, predicting the trajectory of the second vehicle in the first time period according to the trajectory offset information and a plurality of the first trajectory points to obtain a target trajectory includes:

[0023] If the trajectory offset information indicates that there is no trajectory offset point, perform feature extraction on a plurality of the first trajectory points to obtain a data feature matrix of the plurality of the first trajectory points;

[0024] Perform calculations according to the data feature matrix to obtain a target encoding matrix;

[0025] Perform decoding processing on the target encoding matrix through a trajectory prediction model to obtain the target trajectory. The trajectory prediction model uses a model based on the self-attention mechanism.

[0026] In one embodiment of the present application, obtaining a plurality of first trajectory points of the second vehicle in the historical time period includes:

[0027] Obtain a plurality of initial trajectory points of the second vehicle in the historical time period. A plurality of the historical moments correspond to the plurality of initial trajectory points one by one. Each of the initial trajectory points is a trajectory point of the second vehicle in the first coordinate system, and the first coordinate system is determined according to the pose of the first vehicle at the historical moment corresponding to each of the initial trajectory points;

[0028] Map the plurality of initial trajectory points to the second coordinate system to obtain the plurality of first trajectory points. The second coordinate system is determined according to the pose of the first vehicle at the target historical moment among the plurality of historical moments, and the target historical moment is later than other historical moments except the target historical moment among the plurality of historical moments.

[0029] In a second aspect, an embodiment of the present application provides a vehicle trajectory prediction device applied to a first vehicle. The device includes:

[0030] An acquisition module, configured to acquire a plurality of first trajectory points of a second vehicle within a historical time period, where the second vehicle is located within a preset range of the first vehicle, and the historical time period includes a plurality of historical moments, and the plurality of historical moments correspond to the plurality of first trajectory points one by one;

[0031] A determination module, configured to determine trajectory deviation information of the second vehicle within the historical time period according to the plurality of first trajectory points, where the trajectory deviation information is used to indicate M trajectory deviation points, or is used to indicate that there are no trajectory deviation points, and M is a positive integer;

[0032] A prediction module, configured to predict the trajectory of the second vehicle within a first time period according to the trajectory deviation information and the plurality of first trajectory points, to obtain a target trajectory.

[0033] In a third aspect, an embodiment of the present application provides an electronic device, including: a processor and a memory storing computer program instructions;

[0034] When the processor executes the computer program instructions, the vehicle trajectory prediction method described in the first aspect is implemented.

[0035] In a fourth aspect, an embodiment of the present application provides a vehicle, where the vehicle includes the electronic device described in the third aspect.

[0036] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the vehicle trajectory prediction method described in the first aspect is implemented.

[0037] The vehicle trajectory prediction method, device, equipment, vehicle and storage medium provided by the embodiments of the present application obtain a plurality of first trajectory points of a second vehicle within a historical time period, then determine the trajectory deviation information of the second vehicle within the historical time period according to the plurality of first trajectory points, and subsequently predict the trajectory of the second vehicle within a first time period according to the trajectory deviation information and the plurality of first trajectory points, to obtain a target trajectory, where the trajectory deviation information is used to indicate the existence of M trajectory deviation points, or is used to indicate that there are no trajectory deviation points. The above steps can adopt different prediction methods according to different situations of whether there are trajectory deviation points of the second vehicle within the historical time period (that is, there are M trajectory deviation points, or there are no trajectory deviation points) based on the trajectory deviation information of the second vehicle within the historical time period, and improve the prediction accuracy of the trajectory of the neighboring vehicle (that is, the second vehicle). Description of the Drawings

[0038] To more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the accompanying drawings required for the embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0039] Figure 1 is a flowchart of a vehicle trajectory prediction method provided by an embodiment of the present application;

[0040] Figure 2 is a schematic diagram of constructing a position mask matrix provided by an embodiment of the present application;

[0041] Figure 3 is another flowchart of a vehicle trajectory prediction method provided by an embodiment of the present application;

[0042] Figure 4 is a schematic diagram of the structure of a vehicle trajectory prediction device provided by an embodiment of the present application;

[0043] Figure 5 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Detailed Embodiments

[0044] The following will describe in detail the features and exemplary embodiments of various aspects of the present application. To make the purpose, technical solutions and advantages of the present application more clear and understandable, the present application will be further described in detail below in combination with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than limiting the present application. For those skilled in the art, the present application can be implemented without some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by showing examples of the present application.

[0045] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, elements defined by the statement "including..." do not exclude the existence of additional identical elements in the process, method, article or device including the said elements.

[0046] In each specific embodiment of the present application, when it comes to performing relevant processing based on data related to the user's identity or characteristics, such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first. Moreover, the collection, use, and processing of such data will comply with relevant laws, regulations, and standards. In addition, when the embodiments of the present application need to obtain the user's sensitive personal information, the user's separate permission or separate consent will be obtained through methods such as pop-up windows or redirecting to a confirmation page. After clearly obtaining the user's separate permission or separate consent, the necessary user-related data for the normal operation of the embodiments of the present disclosure will be obtained.

[0047] To solve the problems of the prior art, the embodiments of the present application provide a vehicle trajectory prediction method, apparatus, device, vehicle, and storage medium. First, the vehicle trajectory prediction method provided by the embodiments of the present application will be introduced below.

[0048] Figure 1 is a flowchart of the vehicle trajectory prediction method provided by the embodiments of the present application. As Figure 1 shown, the vehicle trajectory prediction method provided by the embodiments of the present application is applied to a first vehicle and can specifically be applied to the electronic device of the first vehicle. The vehicle trajectory prediction method provided by the embodiments of the present application includes the following steps 101 to 103, where:

[0049] Step 101, obtain a plurality of first trajectory points of a second vehicle within a historical time period, where the second vehicle is located within a preset range of the first vehicle, the historical time period includes a plurality of historical moments, and the plurality of historical moments correspond to the plurality of first trajectory points one by one.

[0050] The second vehicle is a vehicle within a preset range of the first vehicle. Herein, the number of second vehicles can be one or multiple, and there is no limit on the number. When the number of second vehicles is multiple, trajectory prediction can be performed separately for each second vehicle. The preset range can be determined by combining data such as the acquisition range of the image acquisition device set on the first vehicle, the detection range of the ranging device set on the first vehicle, or the safe driving distance of the vehicle, or it can also be set by a relevant object. For example, the preset range can be a range with a distance less than 10 meters from the first vehicle, or a range with a distance less than 15 meters from the first vehicle, or other ranges. In addition, the second vehicle can be determined from the vehicles within the preset range of the first vehicle in multiple ways. Exemplarily, all vehicles within the preset range of the first vehicle can be directly used as the second vehicle. Or, a vehicle within the preset range of the first vehicle and determined in response to the selection operation of a relevant object can be used as the second vehicle. Among them, the selection operation can be a touch operation, a voice input operation, or a button operation, etc. Or, a vehicle within the preset range of the first vehicle and showing signs of lane deviation can also be used as the second vehicle.

[0051] The historical time period can be a time period with a certain duration before the current moment. The current moment refers to the starting moment for predicting the trajectory of the second vehicle. The historical time period includes multiple historical moments, and the multiple historical moments correspond one-to-one with multiple first trajectory points. That is to say, at each historical moment, a first trajectory point can be collected. Through the first trajectory point, the position of the second vehicle relative to the first vehicle at the historical moment can be described.

[0052] Step 102: Determine the trajectory deviation information of the second vehicle within the historical time period according to the multiple first trajectory points, where the trajectory deviation information is used to indicate M trajectory deviation points, or is used to indicate the absence of trajectory deviation points, and M is a positive integer.

[0053] According to the multiple first trajectory points, the trajectory deviation information of the second vehicle within the historical time period can be determined. Among them, the trajectory deviation information is used to indicate that there are M trajectory deviation points of the second vehicle within the historical time period, or to indicate the absence of trajectory deviation points. A trajectory deviation point refers to a position where the driving trajectory of the vehicle changes significantly. Exemplarily, multiple first trajectory points can be compared, and the first trajectory points with a large deviation from adjacent first trajectory points among the multiple first trajectory points can be determined as trajectory deviation points to obtain the trajectory deviation information of the second vehicle within the historical time period. Or, a clustering algorithm can be used to process the multiple first trajectory points to determine the trajectory deviation information of the second vehicle within the historical time period.

[0054] Alternatively, based on the cubic spline interpolation method, the least squares method, the Kalman filtering algorithm, or other data fitting methods, fit multiple first trajectory points to obtain the trajectory curve of the second vehicle within the historical time period. By taking the derivative of the trajectory curve, it is possible to determine whether there are trajectory deviation points, and thus obtain the trajectory deviation information of the second vehicle within the historical time period. In addition, assuming that the first trajectory point is (x t , y t , ω t ), then the coordinate value x t of the first trajectory point in the first direction and the coordinate value y t of the first trajectory point in the second direction can be extracted from the first trajectory points to obtain the processed first trajectory point (x t , y t ). After fitting the processed first trajectory point (x t , y t ), refer to the foregoing steps to determine the trajectory deviation information of the second vehicle within the historical time period. In this way, waste of computing resources can be reduced, and the computing efficiency when determining the trajectory deviation information can be improved. Wherein, t refers to the historical moment corresponding to the first trajectory point, x t is the coordinate value of the first trajectory point in the first direction at the historical moment t, the first direction can be perpendicular to the length direction of the first vehicle at the historical moment t, y t is the coordinate value of the first trajectory point in the second direction at the historical moment t, the second direction can be parallel to the length direction of the first vehicle at the historical moment t, and ω t is the relative angle between the second vehicle and the first vehicle at the historical moment t.

[0055] Step 103, according to the trajectory deviation information and the multiple first trajectory points, predict the trajectory of the second vehicle within the first time period to obtain a target trajectory.

[0056] Wherein, the first time period refers to the time period after the current moment, and the first time period is later than the historical time period. The target trajectory can be the trajectory point of the second vehicle at any moment within the first time period, can be the trajectory points of the second vehicle at multiple moments within the first time period, or can also be a driving trajectory of the second vehicle within the first time period.

[0057] Based on the trajectory offset information and multiple first trajectory points, the trajectory of the second vehicle in the first time period can be predicted to obtain the target trajectory. If there are trajectory offset points of the second vehicle in the historical time period, it can be considered that the driving trajectory of the second vehicle has a large offset in the historical time period, and there is a possibility of affecting the normal driving of the first vehicle. Therefore, for the second vehicle with trajectory offset points in the historical time period, the action characteristics of the second vehicle in the historical time period can be emphasized. Exemplarily, if the trajectory offset information indicates M trajectory offset points, a position mask matrix can be constructed based on the historical moments corresponding to the M trajectory offset points respectively, so as to emphasize the first trajectory points after each trajectory offset point through the position mask matrix. Subsequently, models such as a model based on the self-attention mechanism, a long short-term memory network, or a graph neural network model can be used to process the constructed position mask matrix and multiple first trajectory points, thereby predicting the trajectory of the second vehicle in the first time period to obtain the target trajectory.

[0058] If there are no trajectory offset points of the second vehicle in the historical time period, it can be considered that the possibility of the second vehicle affecting the normal driving of the first vehicle is relatively low. Therefore, a relatively simple trajectory prediction method can be used to determine the target trajectory. Exemplarily, a deep learning model (such as a model based on the self-attention mechanism, a long short-term memory network, or a graph neural network model) can be used to process multiple trajectory points, thereby predicting the target trajectory of the second vehicle in the first time period. In this way, the trajectory prediction efficiency can be improved, the response time of the first vehicle can be increased, enabling the first vehicle to have a longer time for speed control and reducing the risk of collision accidents.

[0059] In this embodiment, by obtaining multiple first trajectory points of the second vehicle in the historical time period, then determining the trajectory offset information of the second vehicle in the historical time period according to the multiple first trajectory points, and subsequently predicting the trajectory of the second vehicle in the first time period based on the trajectory offset information and the multiple first trajectory points to obtain the target trajectory, where the trajectory offset information is used to indicate the existence of trajectory offset points or the non-existence of trajectory offset points. The above steps can adopt different prediction methods according to different situations of whether there are trajectory offset points of the second vehicle in the historical time period (i.e., there are M trajectory offset points or there are no trajectory offset points) based on the trajectory offset information of the second vehicle in the historical time period, improving the prediction accuracy of the trajectory of the neighboring vehicle (i.e., the second vehicle).

[0060] In an embodiment of the present application, the obtaining of multiple first trajectory points of the second vehicle in the historical time period includes:

[0061] Obtain multiple initial trajectory points of the second vehicle within a historical time period. The multiple historical moments correspond one-to-one with the multiple initial trajectory points. Each initial trajectory point is a trajectory point of the second vehicle in a first coordinate system, and the first coordinate system is determined according to the pose of the first vehicle at the historical moment corresponding to each initial trajectory point.

[0062] Map the multiple initial trajectory points into a second coordinate system to obtain multiple first trajectory points. The second coordinate system is determined according to the pose of the first vehicle at the target historical moment among the multiple historical moments, and the target historical moment is later than other historical moments among the multiple historical moments except the target historical moment.

[0063] Through ranging devices (such as millimeter-wave radar, lidar, etc.), image acquisition devices or other devices installed on the first vehicle, multiple initial trajectory points of the second vehicle within a historical time period can be obtained. Alternatively, when there is a communication relationship between the first vehicle and the second vehicle, the first vehicle can obtain multiple initial trajectory points sent by the second vehicle. Alternatively, through the global positioning system, high-precision map or intelligent transportation system, etc., the first vehicle can obtain multiple initial trajectory points. Among them, the multiple initial trajectory points correspond one-to-one with the multiple historical moments, that is, one initial trajectory point can be collected at each historical moment. The initial trajectory points are used to describe the position of the second vehicle relative to the first vehicle. Each initial trajectory point is a trajectory point of the second vehicle in the first coordinate system. The first coordinate system is determined according to the pose of the first vehicle at each historical moment, that is, based on data such as the geometric center of the first vehicle, the head orientation of the first vehicle, and the position of the first vehicle at each historical moment, the first coordinate system at each historical moment can be established.

[0064] It can be understood that when the first vehicle moves, the multiple initial trajectory points may be in different coordinate systems. Therefore, for the convenience of subsequent trajectory prediction, the multiple initial trajectory points can be subjected to coordinate system conversion processing to convert the multiple initial trajectory points to the same coordinate system (i.e., the second coordinate system). Specifically, the second coordinate system is determined according to the pose of the first vehicle at the target historical moment among the multiple historical moments, that is, based on data such as the geometric center of the first vehicle, the head orientation of the first vehicle, and the position of the first vehicle at the target historical moment, the second coordinate system can be established. The target historical moment can be later than other historical moments among the multiple historical moments except the target historical moment. For example, if the multiple historical moments include 0 seconds, 0.1 seconds, and 0.2 seconds, then 0.2 seconds can be used as the target historical moment. Alternatively, the target historical moment can also be determined from the multiple historical moments according to the actual situation.

[0065] In the process of mapping multiple initial trajectory points to the second coordinate system, exemplarily, multiple initial trajectory points can be mapped to the second coordinate system based on the coordinate transformation relationship, so as to obtain multiple first trajectory points. Taking the target historical moment being later than other historical moments among the multiple historical moments as an example, the coordinate transformation relationship satisfies the following formula (1) and formula (2):

[0066]

[0067] where t end is the target historical moment, is the coordinate value of the initial trajectory point in the first direction of the first coordinate system at the target historical moment t end ; is the coordinate value of the initial trajectory point in the second direction of the first coordinate system at the target historical moment t end ; is the relative angle between the second vehicle and the first vehicle in the first coordinate system at the target historical moment t end ; is the coordinate value of the first trajectory point in the first direction of the second coordinate system at the target historical moment t end ; is the coordinate value of the first trajectory point in the second direction of the second coordinate system at the target historical moment t end ; is the relative angle between the second vehicle and the first vehicle in the second coordinate system at the target historical moment t end ; is the coordinate value of the first trajectory point in the first direction of the second coordinate system at the historical moment t - 1, is the coordinate value of the first trajectory point in the second direction of the second coordinate system at the historical moment t - 1, is the relative angle between the second vehicle and the first vehicle in the second coordinate system at the historical moment t - 1, dω is the difference between ω t and ω t-1 , ω t is the relative angle between the second vehicle and the first vehicle in the first coordinate system at the historical moment t, ω t-1 is the relative angle between the second vehicle and the first vehicle in the first coordinate system at the historical moment t - 1. x t is the coordinate value of the initial trajectory point in the first direction of the first coordinate system at the historical moment t, y t is the coordinate value of the initial trajectory point in the second direction of the first coordinate system at the historical moment t, dx is the difference between x t and x t-1 , x t-1 is the coordinate value of the initial trajectory point in the first direction of the first coordinate system at the historical moment t - 1, dy is yt The difference from y t-1 y, t-1 is the coordinate value of the initial trajectory point at historical moment t - 1 in the second direction of the first coordinate system.

[0068] Through the above steps, multiple collected initial trajectory points can be converted from different coordinate systems (i.e., the first coordinate system) to the same coordinate system (i.e., the second coordinate system), obtaining multiple first trajectory points. In this way, the coordinate system differences that may exist among different trajectory points can be eliminated, enabling all trajectory points to be unified under a reference system for comparison, analysis, and calculation, and reducing data errors caused by coordinate system differences.

[0069] In an embodiment of the present application, predicting the trajectory of the second vehicle in the first time period based on the trajectory offset information and multiple first trajectory points to obtain a target trajectory includes:

[0070] If the trajectory offset information indicates M trajectory offset points, for each of the trajectory offset points, obtain at least one first trajectory point corresponding to the trajectory offset point, and the historical moment corresponding to at least one first trajectory point is greater than or equal to the historical moment corresponding to the trajectory offset point;

[0071] Predict the trajectory of the second vehicle in the first time period based on at least one first trajectory point corresponding to each of the trajectory offset points and multiple first trajectory points to obtain a target trajectory.

[0072] Based on the trajectory offset information and multiple first trajectory points, the trajectory of the second vehicle in the first time period can be predicted to obtain a target trajectory. Specifically, when the trajectory offset information indicates M trajectory offset points, at this time, for each trajectory offset point, obtain at least one first trajectory point corresponding to the trajectory offset point, and the historical moment corresponding to at least one first trajectory point is greater than the historical moment corresponding to the trajectory offset point. For example, assume that the trajectory offset information indicates the existence of 2 trajectory offset points (i.e., trajectory offset point 1 and trajectory offset point 2), the historical moment corresponding to trajectory offset point 1 is 0.2 seconds, and the historical moment corresponding to trajectory offset point 2 is 0.6 seconds. At this time, for trajectory offset point 1, multiple first trajectory points with corresponding historical moments greater than 0.2 seconds can be obtained, and for trajectory offset point 2, at least one first trajectory point with a corresponding historical moment greater than 0.6 seconds can be obtained.

[0073] Due to the existence of trajectory deviation points, it can be considered that there is a relatively obvious deviation of the second vehicle at this time, and the safety risk of the vehicle is relatively high. Therefore, multiple first trajectory points can be processed to emphasize at least one first trajectory point corresponding to each trajectory deviation point when predicting the target trajectory of the second vehicle, thereby improving the trajectory prediction accuracy. Exemplarily, a prompt word can be constructed according to at least one first trajectory point corresponding to each trajectory deviation point, and the prompt word and multiple first trajectory points are input into a deep learning model (such as a model based on the self-attention mechanism) together to obtain the target trajectory output by the deep learning model. Alternatively, a position mask matrix can be constructed according to at least one first trajectory point corresponding to each trajectory deviation point. At the same time, data features of multiple first trajectory points are extracted to obtain a data feature matrix of multiple first trajectory points. Each element in the position mask matrix corresponds to the validity of a certain position in the input sequence. By constructing the position mask matrix, the deep learning model can distinguish valid data and invalid data in the data. Therefore, after obtaining the position mask matrix and the data feature matrix, through the deep learning model, the target trajectory can be predicted according to the position mask matrix and the data feature matrix.

[0074] In the above steps, when the trajectory deviation information indicates M trajectory deviation points, by performing an emphasizing process on the first trajectory points after the appearance of the trajectory deviation points, important data can be taken into account during trajectory prediction, and then the important data can be processed quickly, thereby improving the accuracy and efficiency of vehicle trajectory prediction.

[0075] In an embodiment of the present application, predicting the trajectory of the second vehicle in the first time period according to at least one first trajectory point corresponding to each trajectory deviation point and multiple first trajectory points to obtain a target trajectory includes:

[0076] Constructing a position mask matrix corresponding to each trajectory deviation point according to at least one first trajectory point corresponding to each trajectory deviation point, where the position mask matrix corresponding to the trajectory deviation point is used to represent the association relationship between the trajectory deviation point and at least one first trajectory point corresponding thereto;

[0077] For the position mask matrix corresponding to each trajectory deviation point, adding the position mask matrix and the data feature matrix to obtain a sub-encoding matrix corresponding to the trajectory deviation point, where the data feature matrix is obtained by feature extraction of multiple first trajectory points;

[0078] Sequentially splicing the sub-encoding matrices corresponding to the trajectory deviation points to obtain a target encoding matrix;

[0079] Decode the target encoding matrix through a trajectory prediction model to obtain the target trajectory, where the trajectory prediction model adopts a model based on a self-attention mechanism.

[0080] When the trajectory offset information indicates M trajectory offset points, a position mask matrix corresponding to each trajectory offset point can be constructed according to at least one first trajectory point corresponding to each trajectory offset point. The position mask matrix corresponding to the trajectory offset point is used to represent the association relationship between the trajectory offset point and the corresponding at least one first trajectory point. That is, for each trajectory offset point, important data (i.e., the first trajectory points whose corresponding historical moments are greater than or equal to the historical moment corresponding to the trajectory offset point) and invalid data (i.e., the first trajectory points whose corresponding historical moments are less than the historical moment corresponding to the trajectory offset point) among the multiple first trajectory points can be distinguished based on the trajectory offset point. Subsequently, for the position mask matrix corresponding to each trajectory offset point, the position mask matrix can be added to the data feature matrix to obtain the sub-encoding matrix corresponding to the trajectory offset point. In this way, important features (i.e., features corresponding to important data) and invalid features (i.e., features corresponding to invalid data) in the data feature matrix can be identified.

[0081] Through methods such as horizontal splicing, vertical splicing, stacking, or element-wise splicing, the sub-encoding matrices corresponding to the trajectory offset points can be sequentially spliced to obtain the target encoding matrix. Then, through the trajectory prediction model, the target encoding matrix can be decoded to obtain the target trajectory. Among them, the trajectory prediction model adopts a model based on a self-attention mechanism, such as a model based on the Transformer architecture or Bidirectional Encoder Representations from Transformers (BERT). Taking the trajectory prediction model adopting a model based on the Transformer architecture as an example, the target encoding matrix can be decoded through the pre-trained encoder in the trajectory prediction model to obtain the target trajectory.

[0082] In this way, when the trajectory offset information indicates M trajectory offset points, by emphasizing the first trajectory points after the trajectory offset points appear and performing trajectory prediction by the trajectory prediction model based on the processed data, the trajectory prediction accuracy of the vehicle can be improved.

[0083] In an embodiment of the present application, the adding the position mask matrix to the data feature matrix to obtain the sub-encoding matrix corresponding to the trajectory offset point includes:

[0084] Perform dimension conversion on the position mask matrix to obtain the target mask matrix corresponding to the trajectory offset point, where the dimension of the target mask matrix corresponding to the trajectory offset point is the same as the dimension of the data feature matrix;

[0085] Add the data feature matrix to the target mask matrix corresponding to the trajectory offset point to obtain the sub-encoding matrix corresponding to the trajectory offset point.

[0086] Generally, the dimension of the position mask matrix is consistent with the total number of multiple first trajectory points. For example, if the total number of multiple first trajectory points is 21, then the dimension of the position mask matrix can be [21, 21]. At this time, there is a possibility that the dimension of the position mask matrix is different from the dimension of the data feature matrix, and the position mask matrix and the data feature matrix cannot be added. Therefore, the dimension of the position mask matrix can be converted so that the dimension of the obtained target mask matrix is the same as the dimension of the data feature matrix. Exemplarily, assume that the dimension of the data feature matrix is [dim, dim], where dim is set according to the actual situation, and the dimension of the position mask matrix is [21, 21]. Then, the dimension of the position mask matrix can be converted in various ways such as a Multi-Layer Perceptron (MLP) network, pooling operation, convolution operation, etc. to obtain the target mask matrix corresponding to the trajectory offset point, and the dimension of the target mask matrix corresponding to the trajectory offset point is [dim, dim]. Then, the data feature matrix can be added to the target mask matrix corresponding to the trajectory offset point to obtain the sub-encoding matrix corresponding to the trajectory offset point. Through the above steps, by converting the dimension of the position mask matrix, subsequent matrix operations can be facilitated.

[0087] In an embodiment of the present application, constructing the position mask matrix corresponding to each trajectory offset point according to at least one of the first trajectory points corresponding to each trajectory offset point includes:

[0088] For each trajectory offset point, set the elements of the target row in the base matrix to a preset value to obtain the position mask matrix corresponding to the trajectory offset point. The base matrix is a zero matrix with U rows and U columns, where U is the total number of multiple first trajectory points, and the target row is the row corresponding to other trajectory points in the base matrix. The other trajectory points are the trajectory points among the multiple first trajectory points except the trajectory offset point and at least one of the first trajectory points corresponding to the trajectory offset point.

[0089] Based on at least one first trajectory point corresponding to each trajectory offset point, a position mask matrix corresponding to each trajectory offset point can be constructed. Specifically, the base matrix is a zero matrix with U rows and U columns, that is, the values of all elements included in the base matrix are 0. Where U is a positive integer. Each first trajectory point has a corresponding row in the base matrix. Then, for each trajectory offset point, the target row in the base matrix can be determined, that is, the row corresponding to other trajectory points in the base matrix. Other trajectory points refer to the trajectory points other than at least one first trajectory point corresponding to the trajectory offset point among multiple trajectory points. Then, the elements of the target row in the base matrix can be set to a preset value, so as to obtain the position mask matrix corresponding to the trajectory offset point. Where the preset value is set according to the actual situation. For example, the preset value can be negative infinity, can be 1, or can also be other values.

[0090] Exemplarily, continuing with the example where there are 2 trajectory offset points (i.e., trajectory offset point 1 and trajectory offset point 2), assuming that the total duration included in the historical time period is 2 seconds, and the interval between every two adjacent historical moments is 0.1 second. At this time, the total number of multiple first trajectory points is 21, and the base matrix is a zero matrix with 21 rows and 21 columns. Then, for trajectory offset point 1, at least one first trajectory point corresponding to trajectory offset point 1 can be determined (that is, the first trajectory points corresponding to historical moments greater than 0.2 second, a total of 19), and other trajectory points include the first trajectory point corresponding to 0 second and the first trajectory point corresponding to 0.1 second. Subsequently, the elements of the rows corresponding to other trajectory points in the base matrix can be set to the preset value, and the values of the elements of the rows corresponding to at least one first trajectory point corresponding to trajectory offset point 1 remain 0, so as to obtain the position mask matrix corresponding to trajectory offset point 1. For trajectory offset point 2, at least one first trajectory point corresponding to trajectory offset point 2 can be determined (the first trajectory points corresponding to historical moments greater than 0.6 second, a total of 15), and other trajectory points include the first trajectory point corresponding to 0 second, the first trajectory point corresponding to 0.1 second, the first trajectory point corresponding to 0.2 second, the first trajectory point corresponding to 0.3 second, the first trajectory point corresponding to 0.4 second, and the first trajectory point corresponding to 0.5 second. Subsequently, the elements of the rows corresponding to other trajectory points in the base matrix can be set to the preset value, and the values of the elements of the rows corresponding to at least one first trajectory point corresponding to trajectory offset point 2 remain 0, so as to obtain the position mask matrix corresponding to trajectory offset point 2.

[0091] In addition, if the trajectory curve of the second vehicle in the historical time period is differentiated to determine the trajectory offset point, the moment corresponding to the trajectory offset point (i.e., the moment when the second vehicle has a trajectory offset) may be between two adjacent historical moments. For example, the moment corresponding to the trajectory offset point is 0.25 seconds, which is between two adjacent historical moments (such as 0.2 seconds and 0.3 seconds). Then, the smaller of the two adjacent historical moments (i.e., 0.2 seconds) can be used as the historical moment corresponding to the trajectory offset point. Subsequently, referring to the foregoing steps, a position mask matrix can be constructed based on the historical moment corresponding to the trajectory offset point.

[0092] Through the above steps, a position mask matrix corresponding to each trajectory offset point can be constructed, so as to construct a target encoding matrix based on the position mask matrix corresponding to each trajectory offset point, and then predict the target trajectory according to the target encoding matrix.

[0093] Figure 2 It is a schematic diagram of constructing a target mask matrix provided by an embodiment of the present application. As Figure 2 shown, in one example, after fitting multiple first trajectory points, the trajectory curve of the second vehicle in the historical time period can be obtained. Among them, the historical time period includes 6 historical moments, namely 0 second, 0.1 second, 0.2 second, 0.3 second, 0.4 second, and 0.5 second. The total number of multiple first trajectory points is 6, and the base matrix is a 6-row and 6-column zero matrix. After differentiating the trajectory curve, the trajectory offset point can be determined. Suppose the historical moment corresponding to the trajectory offset point is 0.4 seconds, that is, the other trajectory points include the first trajectory point corresponding to 0 second, the first trajectory point corresponding to 0.1 second, the first trajectory point corresponding to 0.2 second, and the first trajectory point corresponding to 0.3 second, and the rows corresponding to the other trajectory points in the base matrix are the first 4 rows of the base matrix. Then, the elements of the target rows (i.e., the first 4 rows of the base matrix) can be set to a preset value (at this time, the values of the elements in the last 2 rows of the base matrix are still 0). In this way, a position mask matrix can be obtained. If the dimension of the data feature matrix is [12, 12], after dimension conversion of the position mask matrix through the MLP network, a target mask matrix with a dimension of [12, 12] can be obtained.

[0094] In an embodiment of the present application, the predicting the trajectory of the second vehicle in the first time period according to the trajectory offset information and the multiple first trajectory points to obtain a target trajectory includes:

[0095] If the trajectory offset information indicates that there is no trajectory offset point, feature extraction is performed on the multiple first trajectory points to obtain a data feature matrix of the multiple first trajectory points;

[0096] Calculation is performed according to the data feature matrix to obtain a target encoding matrix;

[0097] Decode the target encoding matrix through a trajectory prediction model to obtain the target trajectory, where the trajectory prediction model uses a model based on the self-attention mechanism.

[0098] When the trajectory offset information indicates that there is no trajectory offset point, since the risk of a collision accident is relatively low at this time, a relatively simple trajectory prediction method can be used to predict the trajectory of the second vehicle within the first time period. Specifically, multiple first trajectory points can be feature-extracted through methods such as an MLP network, a long short-term memory network, a position encoding method, or a convolutional neural network, so as to obtain a data feature matrix of multiple first trajectory points. Further, make the query matrix, the key matrix, and the value matrix all equal to the aforementioned extracted data feature matrix, and based on the query matrix, the key matrix, and the value matrix, sequentially perform the steps of calculating the attention weights and weighted-averaging the value matrix using the attention weights, so as to obtain the target encoding matrix.

[0099] Then, decode the target encoding matrix through the trajectory prediction model to obtain the target trajectory. Among them, the trajectory prediction model uses a model based on the self-attention mechanism (such as a model based on the Transformer architecture or BERT). Taking the trajectory prediction model using a model based on the Transformer architecture as an example, the target encoding matrix can be decoded through a pre-trained encoder in the trajectory prediction model to obtain the target trajectory. The above steps can improve the trajectory prediction efficiency by simplifying the trajectory prediction process for the second vehicle without obvious trajectory offset.

[0100] Figure 3 It is another schematic flowchart of the vehicle trajectory prediction method provided by the embodiments of the present application. As Figure 3 shown, the vehicle trajectory prediction method provided by the embodiments of the present application will be illustrated by way of example below.

[0101] Step 1, obtain multiple initial trajectory points of the second vehicle within the historical time period.

[0102] Among them, the historical time period includes multiple historical moments. For the initial trajectory point (x t , y t , ω t ), its coordinate value in the first direction of the first coordinate system The coordinate value in the second direction of the first coordinate system The relative angle between the second vehicle and the first vehicle in the first coordinate system t1, t2, … t endThat is, multiple historical moments. For example, the total duration of the historical time period can be 2 seconds, and the value interval of multiple historical moments is 0.1 second, that is, the time interval between every two adjacent historical moments is 0.1 second. At this time, the total number of multiple initial trajectory points is 21.

[0103] Step 2: Perform coordinate transformation processing on multiple initial trajectory points to obtain multiple first trajectory points.

[0104] For each initial trajectory point, with reference to Equation (1) and Equation (2), the initial trajectory point (x t , y t , ω t ) can be mapped to the coordinate system (i.e., the second coordinate system) at time t end (i.e., the target historical moment). In this way, the first trajectory point

[0105] Step 3: Fit a trajectory curve based on multiple first trajectory points.

[0106] Using the cubic spline interpolation method, multiple first trajectory points are fitted into a trajectory curve. Specifically, data can be extracted from multiple first trajectory points , and the cubic spline interpolation method is used to process the data, so as to obtain a trajectory curve. The trajectory curve satisfies

[0107] Step 4: Differentiate the trajectory curve and calculate the number of curve inflection points.

[0108] That is, differentiate the trajectory curve and calculate the number of curve inflection points M (i.e., the number of trajectory deviation points) at. If M = 0, go to Step 5; if M > 0, go to Step 6.

[0109] Step 5: Use a self-attention encoder for encoding to obtain a target encoding matrix.

[0110] For a trajectory curve without trajectory deviation points, the self-attention encoder in the Transformer architecture can be used to encode the data. Specifically, an MLP network is used to process the data into a data feature matrix E with dimensions [dim, dim], and the query matrix Q, key matrix K, and value matrix V of the self-attention encoder are all equal to the data feature matrix E. After encoding, the target encoding matrix can be obtained.

[0111] Step 6: Initialize the loop parameter n and initialize the encoding matrix R.

[0112] At this time, n = 0, and the encoding matrix R is an empty matrix.

[0113] Step 7, determine whether n is greater than M, if not, go to step 8, if so, go to step 12.

[0114] Step 8: Based on the trajectory offset point and the plurality of first trajectory points, determine a data feature matrix of the plurality of first trajectory points and a position mask matrix corresponding to the trajectory offset point.

[0115] Using the MLP network The data is processed into a data feature matrix E of dimension [dim, dim] and the position mask matrix P is initialized n (ie basic matrix), at this time the position mask matrix P n is a zero matrix with dimension [21, 21]. Then, for the mth trajectory offset point, we can determine the historical moment corresponding to the trajectory offset point, and determine the number of moments s that are smaller than the historical moment corresponding to the trajectory offset point in multiple historical moments, and then convert the position mask matrix P n The elements of the first s rows of are set to negative infinity, thereby obtaining the position mask matrix corresponding to the trajectory offset point. m is a positive integer less than or equal to M.

[0116] Step 9: Perform dimension conversion on the position mask matrix to obtain the target mask matrix.

[0117] The position mask matrix P corresponding to the trajectory offset point is obtained through the MLP network n Perform dimension conversion to obtain the target mask matrix corresponding to the trajectory offset point Target mask matrix The dimension is [dim, dim].

[0118] Step 10: Calculate based on the target mask matrix and the data feature matrix to obtain a sub-coding matrix.

[0119] The target mask matrix Add it to the data feature matrix E and perform self-attention encoding to obtain the sub-encoding matrix R n .

[0120] Step 11: Update the loop parameter n and concatenate the sub-coding matrix to the coding matrix to obtain the target coding matrix.

[0121] Update the loop parameters so that n=n+1 and transform the sub-coding matrix R n Spliced to the encoding matrix R, R = [R | R n ], get the target encoding matrix, and jump to step 7.

[0122] Step 12: Decode the target encoding matrix to obtain the target trajectory.

[0123] The target encoding matrix is decoded by the decoder in the Transformer architecture to obtain the target trajectory.

[0124] The vehicle trajectory prediction method provided by the embodiments of this application performs additional attention mechanism modeling on the second vehicle with obvious cut-in actions to emphasize the lateral movement characteristics of the second vehicle. During model encoding, the data of the additional attention layer and the data of the original attention layer are concatenated and output. After calculation by the deep learning model, the trajectory prediction accuracy of the second vehicle can be improved, the response time of the first vehicle can be increased, so that the first vehicle has more sufficient time for speed control, and the risk of vehicle collision can be reduced.

[0125] Figure 4 It is a schematic structural diagram of the vehicle trajectory prediction device provided by the embodiments of this application. As Figure 4 shown, the vehicle trajectory prediction device 400 includes:

[0126] An acquisition module 401, configured to acquire a plurality of first trajectory points of a second vehicle within a historical time period, where the second vehicle is within a preset range of the first vehicle, the historical time period includes a plurality of historical moments, and the plurality of historical moments correspond to the plurality of first trajectory points one by one;

[0127] A determination module 402, configured to determine trajectory offset information of the second vehicle within the historical time period according to the plurality of first trajectory points, where the trajectory offset information is used to indicate M trajectory offset points, or is used to indicate that there are no trajectory offset points, and M is a positive integer;

[0128] A prediction module 403, configured to predict the trajectory of the second vehicle within a first time period according to the trajectory offset information and the plurality of first trajectory points to obtain a target trajectory.

[0129] In an embodiment of this application, the prediction module 403 includes:

[0130] A first acquisition sub-module, configured to, if the trajectory offset information indicates M trajectory offset points, for each of the trajectory offset points, acquire at least one first trajectory point corresponding to the trajectory offset point, and the historical moment corresponding to at least one of the first trajectory points is greater than or equal to the historical moment corresponding to the trajectory offset point;

[0131] A first prediction sub-module, configured to predict the trajectory of the second vehicle within a first time period according to at least one first trajectory point corresponding to each of the trajectory offset points and the plurality of first trajectory points to obtain a target trajectory.

[0132] In an embodiment of this application, the first prediction sub-module includes:

[0133] A construction unit, configured to construct a position mask matrix corresponding to each of the trajectory offset points according to at least one of the first trajectory points corresponding to each of the trajectory offset points, where the position mask matrix corresponding to the trajectory offset point is used to characterize the association relationship between the trajectory offset point and the corresponding at least one of the first trajectory points;

[0134] A calculation unit, configured to add the position mask matrix to the data feature matrix for the position mask matrix corresponding to each of the trajectory offset points to obtain a sub-encoding matrix corresponding to the trajectory offset point, where the data feature matrix is obtained by performing feature extraction on a plurality of the first trajectory points;

[0135] A splicing unit, configured to sequentially splice the sub-encoding matrices corresponding to the trajectory offset points to obtain a target encoding matrix;

[0136] A decoding unit, configured to perform decoding processing on the target encoding matrix through a trajectory prediction model to obtain the target trajectory, where the trajectory prediction model uses a model based on a self-attention mechanism.

[0137] In an embodiment of the present application, the calculation unit includes:

[0138] A conversion sub-unit, configured to perform dimension conversion on the position mask matrix to obtain a target mask matrix corresponding to the trajectory offset point, where the dimension of the target mask matrix corresponding to the trajectory offset point is the same as the dimension of the data feature matrix;

[0139] A calculation sub-unit, configured to add the data feature matrix to the target mask matrix corresponding to the trajectory offset point to obtain a sub-encoding matrix corresponding to the trajectory offset point.

[0140] In an embodiment of the present application, the construction unit includes:

[0141] A setting sub-unit, configured to set the elements of the target row in the base matrix to a preset value for each of the trajectory offset points to obtain a position mask matrix corresponding to the trajectory offset point, where the base matrix is a zero matrix with U rows and U columns, U is the total number of the plurality of the first trajectory points, the target row is the row corresponding to other trajectory points in the base matrix, and the other trajectory points are the trajectory points other than the trajectory offset point and the corresponding at least one of the first trajectory points among the plurality of the first trajectory points.

[0142] In an embodiment of the present application, the prediction module 403 includes:

[0143] An extraction sub-module, configured to perform feature extraction on a plurality of the first trajectory points to obtain a data feature matrix of the plurality of the first trajectory points if the trajectory offset information indicates that there is no trajectory offset point;

[0144] A calculation sub-module, configured to perform calculations based on the data feature matrix to obtain a target coding matrix;

[0145] A second prediction sub-module, configured to perform decoding processing on the target coding matrix through a trajectory prediction model to obtain the target trajectory, where the trajectory prediction model adopts a model based on a self-attention mechanism.

[0146] In an embodiment of the present application, the acquisition module 401 includes:

[0147] A second acquisition sub-module, configured to acquire a plurality of initial trajectory points of the second vehicle within a historical time period, where the plurality of historical moments correspond one-to-one to the plurality of initial trajectory points, and each initial trajectory point is a trajectory point of the second vehicle in a first coordinate system, and the first coordinate system is determined according to the pose of the first vehicle at the historical moment corresponding to each initial trajectory point;

[0148] A mapping sub-module, configured to map the plurality of initial trajectory points to a second coordinate system to obtain the plurality of first trajectory points, where the second coordinate system is determined according to the pose of the first vehicle at a target historical moment among the plurality of historical moments, and the target historical moment is later than other historical moments except the target historical moment among the plurality of historical moments.

[0149] The vehicle trajectory prediction device 400 provided by the embodiments of the present application can implement each process implemented by the foregoing vehicle trajectory prediction method embodiments and achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0150] Figure 5 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. As Figure 5 shown, the electronic device may include a processor 501 and a memory 502 storing computer program instructions.

[0151] Specifically, the foregoing processor 501 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be an integrated circuit configured to implement one or more embodiments of the present application.

[0152] The memory 502 may include a mass storage for data or instructions. By way of example and not limitation, the memory 502 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 502 may include removable or non-removable (or fixed) media. Where appropriate, the memory 502 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, the memory 502 is a non-volatile solid state memory.

[0153] The memory may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage media device, an optical storage media device, a flash memory device, an electrical, optical, or other physical / tangible memory storage device. Thus, in general, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the methods according to the first aspect or the second aspect of the present disclosure.

[0154] The processor 501 reads and executes the computer program instructions stored in the memory 502 to implement any one of the vehicle trajectory prediction methods in the above embodiments.

[0155] In one example, the in-vehicle device may further include a communication interface 503 and a bus 510. Among them, as Figure 5 shown, the processor 501, the memory 502, and the communication interface 503 are connected via the bus 510 and complete communication with each other.

[0156] The communication interface 503 is mainly used to implement communication between the various modules, devices, units, and / or devices in the embodiments of the present application.

[0157] The bus 510 includes hardware, software, or both, and couples the components of the camera extrinsic calibration method or verification device to each other. By way of example and not limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable bus or a combination of two or more of these. Where appropriate, the bus 510 may include one or more buses. Although the embodiments of the present application describe and illustrate specific buses, the present application contemplates any suitable bus or interconnect.

[0158] In addition, an embodiment of the present application provides a vehicle, which includes the above-mentioned electronic device.

[0159] The vehicle may be a private car, such as a sedan, an SUV, an MPV, or a pickup truck, etc. The vehicle may also be an operating vehicle, such as a minivan, a bus, a small truck, or a large trailer, etc. The vehicle may be a fuel vehicle or a new energy vehicle. When the vehicle is a new energy vehicle, it may be a hybrid vehicle or a pure electric vehicle.

[0160] In addition, in combination with the vehicle trajectory prediction method in the above embodiments, an embodiment of the present application may be implemented by providing a computer storage medium. Computer program instructions are stored on the computer storage medium; when the computer program instructions are executed by a processor, any one of the vehicle trajectory prediction methods in the above embodiments is implemented.

[0161] In addition, an embodiment of the present application may be provided to be implemented by a computer program product. When the instructions in the computer program product are executed by an electronic device, the electronic device implements any one of the vehicle trajectory prediction methods in the above embodiments.

[0162] It should be clear that the present application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described as examples. However, the method process of the present application is not limited to the described specific steps, and those skilled in the art can make various changes, modifications, and additions, or change the order between steps after understanding the spirit of the present application.

[0163] The functional blocks shown in the above-described structural block diagrams can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a functional card, and so on. When implemented in software, the elements of the present application are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via a data signal carried in a carrier wave. A "machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical discs, hard disks, fiber optic media, radio frequency (RF) links, and so on. The code segment can be downloaded via a computer network such as the Internet, an intranet, and so on.

[0164] It should also be noted that the exemplary embodiments mentioned in the present application describe some methods or systems based on a series of steps or devices. However, the present application is not limited to the order of the above steps, that is, the steps can be executed in the order mentioned in the embodiments, or different from the order in the embodiments, or several steps can be executed simultaneously.

[0165] Aspects of the present disclosure have been described above with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block in the flowcharts and / or block diagrams, and the combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the functions / actions specified in one or more blocks of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It should also be understood that each block in the block diagram and / or flowchart, and the combinations of blocks in the block diagram and / or flowchart, can also be implemented by dedicated hardware that performs the specified functions or actions, or by a combination of dedicated hardware and computer instructions.

[0166] As described above, this is only the specific implementation manner of the present application. Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems, modules, and units described above can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein. It should be understood that the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered within the protection scope of the present application.

Claims

1. A vehicle trajectory prediction method, characterized in that, Applied to a first vehicle, the method includes: Obtaining a plurality of first trajectory points of a second vehicle within a historical time period, the second vehicle being located within a preset range of the first vehicle, the historical time period including a plurality of historical moments, and the plurality of historical moments corresponding one-to-one to the plurality of first trajectory points; Determining trajectory deviation information of the second vehicle within the historical time period according to the plurality of first trajectory points, the trajectory deviation information being used to indicate M trajectory deviation points, or being used to indicate that there are no trajectory deviation points, where M is a positive integer; Predicting the trajectory of the second vehicle within a first time period according to the trajectory deviation information and the plurality of first trajectory points to obtain a target trajectory.

2. The method according to claim 1, wherein The predicting the trajectory of the second vehicle within a first time period according to the trajectory deviation information and the plurality of first trajectory points to obtain a target trajectory includes: If the trajectory deviation information indicates M trajectory deviation points, for each of the trajectory deviation points, obtaining at least one first trajectory point corresponding to the trajectory deviation point, and the historical moment corresponding to at least one of the first trajectory points being greater than or equal to the historical moment corresponding to the trajectory deviation point; Predicting the trajectory of the second vehicle within a first time period according to at least one first trajectory point corresponding to each of the trajectory deviation points and the plurality of first trajectory points to obtain a target trajectory.

3. The method according to claim 2, wherein The predicting the trajectory of the second vehicle within a first time period according to at least one first trajectory point corresponding to each of the trajectory deviation points and the plurality of first trajectory points to obtain a target trajectory includes: Constructing a position mask matrix corresponding to each of the trajectory deviation points according to at least one first trajectory point corresponding to each of the trajectory deviation points, the position mask matrix corresponding to the trajectory deviation point being used to characterize the association relationship between the trajectory deviation point and the corresponding at least one first trajectory point; For the position mask matrix corresponding to each of the trajectory deviation points, adding the position mask matrix to a data feature matrix to obtain a sub-encoding matrix corresponding to the trajectory deviation point, the data feature matrix being obtained by performing feature extraction on the plurality of first trajectory points; Sequentially splicing the sub-encoding matrices corresponding to the trajectory deviation points to obtain a target encoding matrix; Performing decoding processing on the target encoding matrix through a trajectory prediction model, the trajectory prediction model being a model based on a self-attention mechanism.

4. The method according to claim 3, characterized in that, The adding the position mask matrix to the data feature matrix to obtain a sub-encoding matrix corresponding to the trajectory deviation point includes: Performing dimension conversion on the position mask matrix to obtain a target mask matrix corresponding to the trajectory deviation point, the dimension of the target mask matrix corresponding to the trajectory deviation point being the same as the dimension of the data feature matrix; Adding the data feature matrix to the target mask matrix corresponding to the trajectory deviation point to obtain a sub-encoding matrix corresponding to the trajectory deviation point.

5. The method according to claim 3, wherein The constructing a position mask matrix corresponding to each of the trajectory deviation points according to at least one first trajectory point corresponding to each of the trajectory deviation points includes: For each of the trajectory offset points, set the elements of the target row in the base matrix to a preset value to obtain a position mask matrix corresponding to the trajectory offset point. The base matrix is a zero matrix with U rows and U columns, where U is the total number of the first trajectory points. The target row is the row in the base matrix corresponding to other trajectory points, and the other trajectory points are the trajectory points among the first trajectory points except the trajectory offset point and at least one of the first trajectory points corresponding to the trajectory offset point.

6. The method according to claim 1, characterized in that Predicting the trajectory of the second vehicle within a first time period based on the trajectory offset information and the first trajectory points, to obtain a target trajectory, includes: If the trajectory offset information indicates that there is no trajectory offset point, perform feature extraction on the first trajectory points to obtain a data feature matrix of the first trajectory points; Perform calculations based on the data feature matrix to obtain a target encoding matrix; Decode the target encoding matrix through a trajectory prediction model to obtain the target trajectory. The trajectory prediction model uses a model based on the self-attention mechanism.

7. The method according to claim 1, characterized in that, Obtaining a plurality of first trajectory points of the second vehicle within a historical time period includes: Obtain a plurality of initial trajectory points of the second vehicle within the historical time period. A plurality of the historical moments correspond one-to-one with the plurality of initial trajectory points. Each initial trajectory point is a trajectory point of the second vehicle in a first coordinate system, and the first coordinate system is determined according to the pose of the first vehicle at the historical moment corresponding to each initial trajectory point; Map the plurality of initial trajectory points to a second coordinate system to obtain the plurality of first trajectory points. The second coordinate system is determined according to the pose of the first vehicle at a target historical moment among the plurality of historical moments, and the target historical moment is later than other historical moments except the target historical moment among the plurality of historical moments.

8. A vehicle trajectory prediction device, characterized in that, Applied to a first vehicle, the device includes: An acquisition module, configured to acquire a plurality of first trajectory points of a second vehicle within a historical time period. The second vehicle is located within a preset range of the first vehicle. The historical time period includes a plurality of historical moments, and the plurality of historical moments correspond one-to-one with the plurality of first trajectory points; A determination module, configured to determine trajectory offset information of the second vehicle within the historical time period according to the plurality of first trajectory points. The trajectory offset information is used to indicate M trajectory offset points, or is used to indicate that there is no trajectory offset point, where M is a positive integer; A prediction module, configured to predict the trajectory of the second vehicle within a first time period according to the trajectory offset information and the plurality of first trajectory points to obtain a target trajectory.

9. An electronic device, characterized in that, Including: A processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the vehicle trajectory prediction method according to any one of claims 1-7 is implemented.

10. A vehicle, characterized in that, The vehicle includes the electronic device according to claim 9.

11. A computer-readable storage medium, characterized in that, Computer program instructions are stored on a computer-readable storage medium. When the computer program instructions are executed by a processor, the vehicle trajectory prediction method according to any one of claims 1-7 is implemented.

Images