Trajectory prediction model scheduling method for automatic driving of vehicle

Abstract

The invention belongs to the technical field of automatic driving, and particularly relates to a vehicle automatic driving trajectory prediction model scheduling method, which comprises the following steps of: 1, data preprocessing: filling original historical trajectory data of a vehicle into grids according to a certain rule, and processing each grid into a graph; each graph represents the current state of the vehicle at the current time, and one node in the graph represents one vehicle; step 2, constructing a hierarchical LSTM (Long Short Term Memory) encoder; step 3, constructing an encoder-decoder model based on a space-time diagram; 4, an attention mechanism is introduced for prediction, the attention mechanism is introduced on the basis of the LSTM vehicle track prediction model based on the space-time diagram, the structure is reasonable, the precision is high in the use process, and the motion track of the vehicle can be accurately predicted; the adaptability is high, the method can adapt to the change of vehicle motion and the change of environment, and the prediction result can be quickly updated; the comprehensiveness is good, and various factors influencing the vehicle motion trail can be considered.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a trajectory prediction model scheduling method for vehicle autonomous driving. Background Art

[0002] In the autonomous driving system, the vehicle trajectory prediction results determine the path planning and self-control of the vehicle. With the development of autonomous driving systems, more and more trajectory prediction algorithms are deployed in autonomous driving systems, including kinematics-based rule prediction algorithms and deep learning-based rule prediction algorithms. A large number of algorithms are running at the same time, and the prediction results are uneven, which seriously affects the operating efficiency of the trajectory prediction algorithm and the real-time and accuracy of the selection algorithm and prediction results. In the limited computing power of the vehicle, different algorithms need to be calculated, and the algorithm with the most accurate prediction results is selected, which seriously increases the computing load and computing resource loss of the vehicle. Different trajectory prediction algorithms have different running times, different prediction trajectory accuracy, and different quantization accuracy. This leads to different prediction trajectory results in different scenarios with different trajectory prediction algorithms and different quantization accuracy. Due to the speed of inference time of different algorithms, computing resources are seriously occupied, affecting the operating efficiency and real-time performance of trajectory prediction.

[0003] In the prior art, the application number is CN202310500722.2, and the application name is a trajectory prediction model scheduling method and system for vehicle autonomous driving. In the technical solution recorded, in the offline training stage, the historical data set required for trajectory prediction is obtained and classified, and each trajectory prediction model is trained using historical data sets of different clusters of different road scenes, and the pre-trained trajectory prediction model that meets the preset accuracy threshold is saved; in the online scheduling stage, the current road scene and the current vehicle data are obtained, and the historical data set closest to the cluster of the same road scene in the offline training stage is matched, and the corresponding several pre-trained trajectory prediction models are used as the preliminary trajectory prediction models, among which the one with the highest total score is used as the final trajectory prediction model for autonomous driving trajectory prediction.

[0004] This application does not take into account the interference of weather factors on driving trajectory prediction, which affects the real-time performance of trajectory prediction. Based on the above problems, we propose a new trajectory prediction model scheduling method for vehicle autonomous driving. Summary of the invention

[0005] The purpose of this section is to summarize some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0006] In view of the problems existing in the prior art, the present invention is proposed.

[0007] Therefore, the purpose of the present invention is to provide a trajectory prediction model scheduling method for vehicle autonomous driving, which can achieve high precision and accurate prediction of the vehicle's motion trajectory during use; strong adaptability, able to adapt to changes in vehicle motion and environmental changes, and can quickly update the prediction results; good comprehensiveness, able to consider a variety of factors that affect the vehicle's motion trajectory.

[0008] To solve the above technical problems, according to one aspect of the present invention, the present invention provides the following technical solutions:

[0009] A trajectory prediction model scheduling method for vehicle automatic driving comprises the following steps:

[0010] Step 1: Data preprocessing: fill the original historical trajectory data of the vehicle into the grid according to certain rules, and then process each grid into a graph; each graph represents the current state of the vehicle at the current time, and a node in the graph represents a vehicle. Each graph has three attributes: the first is the node coordinate attribute, the second is the time edge attribute, and the third is the space edge attribute. By preprocessing the historical trajectory of the vehicle in this way, the spatiotemporal characteristics of the interaction between vehicles can be effectively captured;

[0011] Step 2: Construct a hierarchical LSTM encoder. Based on the data format of the trajectory data preprocessed into a spatiotemporal graph, design the corresponding hierarchical encoder according to the data format of the model input to extract the spatiotemporal interaction features of the vehicle historical trajectory data;

[0012] Step 3: Construct an encoder-decoder model based on the spatiotemporal graph. The encoder analyzes the input sequence and the decoder generates the output sequence. The encoder extracts the spatiotemporal interaction features of the vehicle trajectory data. The decoder decodes the features after the three-layer encoder is cascaded to obtain the predicted vehicle trajectory.

[0013] Step 4: Introduce the attention mechanism for prediction. On the basis of the LSTM vehicle trajectory prediction model based on the space-time graph, the attention mechanism is introduced; the attention mechanism is introduced on the time edge and space edge attributes of the space-time graph respectively. By introducing the space-time attention component, the accuracy of vehicle trajectory prediction is improved.

[0014] As a preferred solution of the trajectory prediction model scheduling method for vehicle automatic driving described in the present invention, the specific process of step one is to use the collected vehicle trajectory data and the collected historical trajectory data to perform data preprocessing operations, including data cleaning, integration, reduction, transformation, and sampling.

[0015] As a preferred solution of the trajectory prediction model scheduling method for vehicle automatic driving described in the present invention, wherein: the step one also includes the extraction of data hidden information. By inputting a considerable scale of data sets into the neural network model, the neural network can learn many times more information than the original data, providing a strong data support for vehicle trajectory prediction.

[0016] As a preferred solution of the trajectory prediction model scheduling method for vehicle autonomous driving described in the present invention, wherein: in the step 2, the data input to the model is in the spatiotemporal graph data format, so the designed hierarchical encoder is the corresponding spatial edge LSTM encoder, node coordinate LSTM encoder, and time edge LSTM encoder; the three-layer encoder extracts the spatial features, trajectory coordinate hidden features, and time series features of the data respectively; through the hierarchical encoder, the richer spatiotemporal interaction features of the vehicle trajectory data can be effectively extracted.

[0017] As a preferred solution of the trajectory prediction model scheduling method for vehicle autonomous driving described in the present invention, wherein: in the step 2, each layer of LSTM includes a Dropout layer and a fully connected layer, and the vehicle trajectory prediction result is obtained through an activation function.

[0018] As a preferred solution of the trajectory prediction model scheduling method for vehicle automatic driving described in the present invention, wherein: the step three mainly includes three key elements, a node set V, a spatial edge set Es, and a time edge set Er; the graph (V, Es) is formed by expanding G along with the time edge set Er, and the key elements of the graph G are replaced by corresponding LSTM components, namely, the time edge LSTM, the space edge LSTM, and the node LSTM. The feature vector of the time edge is the vector difference of the position coordinates of the same node at two time steps, the feature vector of the space edge is the vector difference of the position coordinates of two nodes at the same time step, and the feature vector of the node LSTM is the position coordinate vector of the node.

[0019] As a preferred solution of the trajectory prediction model scheduling method for vehicle automatic driving described in the present invention, in the step three, the key vehicle trajectory data are collected using the US-101 and I-80 data sets in the NGSIM data set.

[0020] As a preferred solution of the trajectory prediction model scheduling method for vehicle autonomous driving described in the present invention, wherein: in the step four, the influence of surrounding vehicles on the movement of the target vehicle is obtained by calculating the attention weights of the hidden layer of the historical trajectory of the surrounding vehicles, different weights are assigned to neighboring vehicles, and global feature information about neighboring vehicles is added, so that the output at each moment can take into account the historical trajectory points that are most relevant to it, thereby further improving the accuracy of vehicle trajectory prediction.

[0021] Compared with the prior art, the beneficial effects of the present invention are as follows: by preprocessing the historical trajectory of the vehicle through this method, the spatiotemporal characteristics of the interaction between vehicles can be effectively captured. During use, the vehicle's motion trajectory can be accurately predicted with high precision; it has strong adaptability, can adapt to changes in vehicle motion and environmental changes, and can quickly update the prediction results; it has good comprehensiveness, can consider a variety of factors that affect the vehicle's motion trajectory, and introduces an attention mechanism on the time edge and space edge attributes of the spatiotemporal graph respectively. By introducing the spatiotemporal attention component, the accuracy of vehicle trajectory prediction is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the present invention will be described in detail below in combination with the accompanying drawings and detailed embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0023] Figure 1 It is a schematic diagram of the step flow structure of the present invention. DETAILED DESCRIPTION

[0024] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0025] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0026] Secondly, the present invention is described in detail with reference to schematic diagrams. When describing the embodiments of the present invention in detail, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.

[0027] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0028] The present invention provides the following technical solutions: a trajectory prediction model scheduling method for vehicle automatic driving, which can accurately predict the motion trajectory of the vehicle with high accuracy during use; has strong adaptability, can adapt to changes in vehicle motion and environmental changes, and can quickly update the prediction results; has good comprehensiveness, and can consider multiple factors that affect the vehicle motion trajectory;

[0029] Example 1

[0030] A trajectory prediction model scheduling method for vehicle automatic driving comprises the following steps:

[0031] Step 1: Data preprocessing, fill the original historical trajectory data of the vehicle into the grid according to certain rules, and then process each grid into a graph; each graph represents the current state of the vehicle at the current time, and a node in the graph represents a vehicle. Each graph has three attributes: the first is the node coordinate attribute, the second is the time edge attribute, and the third is the space edge attribute; by preprocessing the historical trajectory of the vehicle in this way, the spatiotemporal characteristics of the interaction between vehicles can be effectively captured; the specific process is to use the collected vehicle trajectory data and the collected historical trajectory data to perform data preprocessing operations, including data cleaning, integration, reduction, transformation, and sampling; it also includes the extraction of hidden information in the data. By inputting a considerable data set into the neural network model, the neural network can learn many times more information than the original data, providing a strong data support for vehicle trajectory prediction;

[0032] Step 2: Construct a hierarchical LSTM encoder. Based on the data format of the trajectory data preprocessed into a spatiotemporal graph, design the corresponding hierarchical encoder according to the data format of the model input to extract the spatiotemporal interaction features of the vehicle's historical trajectory data. The model input data is in the spatiotemporal graph data format, so the designed hierarchical encoder is the corresponding spatial edge LSTM encoder, node coordinate LSTM encoder, and time edge LSTM encoder. The three-layer encoder extracts the spatial features of the data, the trajectory coordinate hidden features, and the temporal features respectively. The hierarchical encoder can effectively extract richer spatiotemporal interaction features of the vehicle trajectory data. Each layer of LSTM contains a Dropout layer and a fully connected layer, and the vehicle trajectory prediction result is obtained through the activation function.

[0033] Step 3: Construct an encoder-decoder model based on the spatiotemporal graph. The encoder analyzes the input sequence and the decoder generates the output sequence. The encoder extracts the spatiotemporal interaction features of the vehicle trajectory data. The decoder decodes the features after the three-layer encoder is cascaded to obtain the predicted vehicle trajectory, which mainly contains three key elements: node set V, spatial edge set Es, and temporal edge set Er. The graph (V, Es) is formed by G expanding with the temporal edge set Er. The key elements of the graph G are replaced by LSTM components, namely temporal edge LSTM, spatial edge LSTM, and node LSTM. The feature vector of the temporal edge is the vector difference of the position coordinates of the same node at two time steps, the feature vector of the spatial edge is the vector difference of the position coordinates of two nodes at the same time step, and the feature vector of the node LSTM is the position coordinate vector of the node. The key vehicle trajectory data are collected using the US-101 and I-80 datasets in the NGSIM dataset.

[0034] Step 4: Introduce the attention mechanism for prediction. On the basis of the LSTM vehicle trajectory prediction model based on the space-time graph, the attention mechanism is introduced; the attention mechanism is introduced on the time edge and space edge attributes of the space-time graph respectively. The introduction of the space-time attention component improves the accuracy of vehicle trajectory prediction. The influence of surrounding vehicles on the movement of the target vehicle is obtained by calculating the attention weights of the hidden layer of the historical trajectory of surrounding vehicles, and different weights are assigned to neighboring vehicles. The global feature information about neighboring vehicles is increased, so that the output at each moment can take into account the historical trajectory points that are most relevant to it, thereby further improving the accuracy of vehicle trajectory prediction.

[0035] Working principle: This method can effectively capture the spatiotemporal characteristics of vehicle interactions by preprocessing the historical trajectories of vehicles. During use, it can accurately predict the vehicle's motion trajectory with high precision. It has strong adaptability and can adapt to changes in vehicle motion and environmental changes, and can quickly update the prediction results. It is comprehensive and can consider a variety of factors that affect the vehicle's motion trajectory. It introduces attention mechanisms on the time edge and space edge attributes of the spatiotemporal graph. The introduction of the spatiotemporal attention component improves the accuracy of vehicle trajectory prediction.

[0036] Although the present invention has been described above with reference to the embodiments, various modifications may be made thereto and parts thereof may be replaced by equivalents without departing from the scope of the present invention. In particular, as long as there is no structural conflict, the various features in the embodiments disclosed in the present invention may be used in combination with each other in any manner, and the fact that these combinations are not exhaustively described in this specification is only for the sake of omitting space and saving resources. Therefore, the present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A trajectory prediction model scheduling method for vehicle automatic driving, characterized by: The steps include: Step 1: Data preprocessing: fill the original historical trajectory data of the vehicle into the grid according to certain rules, and then process each grid into a graph; each graph represents the current state of the vehicle at the current time, and a node in the graph represents a vehicle. Each graph has three attributes: the first is the node coordinate attribute, the second is the time edge attribute, and the third is the space edge attribute. By preprocessing the historical trajectory of the vehicle in this way, the spatiotemporal characteristics of the interaction between vehicles can be effectively captured; Step 2: Construct a hierarchical LSTM encoder. Based on the data format of the trajectory data preprocessed into a spatiotemporal graph, design the corresponding hierarchical encoder according to the data format of the model input to extract the spatiotemporal interaction features of the vehicle historical trajectory data; Step 3: Construct an encoder-decoder model based on the spatiotemporal graph. The encoder analyzes the input sequence and the decoder generates the output sequence. The encoder extracts the spatiotemporal interaction features of the vehicle trajectory data. The decoder decodes the features after the three-layer encoder is cascaded to obtain the predicted vehicle trajectory. Step 4: Introduce the attention mechanism for prediction. On the basis of the LSTM vehicle trajectory prediction model based on the space-time graph, the attention mechanism is introduced; the attention mechanism is introduced on the time edge and space edge attributes of the space-time graph respectively. By introducing the space-time attention component, the accuracy of vehicle trajectory prediction is improved.

2. The trajectory prediction model scheduling method for vehicle automatic driving according to claim 1, characterized in that: The specific process of step one is to use the collected vehicle trajectory data and the collected historical trajectory data to perform data preprocessing operations, including data cleaning, integration, reduction, transformation, and sampling.

3. The trajectory prediction model scheduling method for vehicle automatic driving according to claim 1 is characterized in that: The step one also includes the extraction of hidden information in the data. By inputting a considerable-scale data set into the neural network model, the neural network can learn many times more information than the original data, providing a strong data support for vehicle trajectory prediction.

4. The trajectory prediction model scheduling method for vehicle automatic driving according to claim 1, characterized in that: In the step 2, the data input to the model is in the format of spatiotemporal graph data, so the designed hierarchical encoders are the corresponding spatial edge LSTM encoder, node coordinate LSTM encoder, and time edge LSTM encoder; the three-layer encoders extract the spatial features, trajectory coordinate hidden features, and time series features of the data respectively; The hierarchical encoder can effectively extract richer spatiotemporal interaction features of vehicle trajectory data.

5. The trajectory prediction model scheduling method for vehicle automatic driving according to claim 1, characterized in that: In the step 2, each LSTM layer includes a Dropout layer and a fully connected layer, and the vehicle trajectory prediction result is obtained through the activation function.

6. The trajectory prediction model scheduling method for vehicle automatic driving according to claim 1, characterized in that: The step three mainly includes three key elements, namely, a node set V, a spatial edge set Es, and a time edge set Er; the graph (V, Es) is formed by expanding G along with the time edge set Er, and the key elements of the graph G are replaced by LSTM components respectively, namely, time edge LSTM, spatial edge LSTM, and node LSTM. The feature vector of the time edge is the vector difference of the position coordinates of the same node at two time steps, the feature vector of the spatial edge is the vector difference of the position coordinates of two nodes at the same time step, and the feature vector of the node LSTM is the position coordinate vector of the node.

7. The trajectory prediction model scheduling method for vehicle automatic driving according to claim 1 is characterized in that: In step 3, the key vehicle trajectory data are collected using the US-101 and I-80 datasets in the NGSIM dataset.

8. The trajectory prediction model scheduling method for vehicle automatic driving according to claim 1, characterized in that: In the step 4, the influence of surrounding vehicles on the movement of the target vehicle is obtained by calculating the attention weights of the hidden layer of the historical trajectory of the surrounding vehicles, assigning different weights to neighboring vehicles, and adding global feature information about neighboring vehicles, so that the output at each moment can take into account the historical trajectory points that are most relevant to it, thereby further improving the accuracy of vehicle trajectory prediction.

Images