Track prediction method based on digital twin platform

Abstract

The invention discloses a trajectory prediction method based on a digital twin platform. The method comprises the steps of collecting information such as the position, the speed, the driving state and the surrounding environment state of a vehicle in real time; after initial processing is carried out on the information, the vehicle transmits the processed information to a track prediction platform located at the cloud end by means of a built-in communication module of the vehicle; secondly, extracting data features of vehicles and roadside units through a multi-channel mechanism, and sending the extracted features to a Transform module to predict a future trajectory; and finally, the platform feeds back the prediction result to the vehicle, so that a driver can timely adjust the current track to avoid traffic accidents. Compared with other schemes, the method has the advantages that ADE and FDE indexes are the lowest, the model efficiency is the best, and in addition, the average reasoning delay is 11.83 ms within the trajectory prediction time step length of 5 seconds.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent vehicles, and specifically relates to a trajectory prediction method based on a digital twin platform. Background Art

[0002] Smart cars equipped with technologies such as V2X (Vehicle to Everything), cloud computing, and smart sensors provide users with high-quality services and convenient travel experiences. Trajectory prediction has shown great value in traffic safety because it can perceive the vehicle's surrounding environment. Currently, mainstream trajectory prediction solutions can be divided into two categories: trajectory prediction based on a single vehicle and trajectory prediction based on vehicle-road collaboration.

[0003] Single-vehicle-based trajectory prediction schemes rely on the vehicle's own collected historical trajectory and driving status information to predict the vehicle's trajectory at the next moment. However, this approach has some obvious limitations. First, since the data comes from a single vehicle, it leads to the neglect of the interactions between traffic participants. Second, it may not be able to accurately capture the driving styles of different drivers, which may vary due to personal habits, emotional states, and other factors. Finally, due to the limited field of view and capabilities of the single-vehicle perception system, it may not be able to fully obtain the status information of surrounding vehicles, which may lead to inaccurate prediction results in complex traffic scenarios.

[0004] To overcome these shortcomings, researchers have proposed trajectory prediction schemes based on vehicle-road collaboration. This type of scheme can more comprehensively perceive the vehicle's surrounding environment by integrating resources such as vehicle networking technology, roadside infrastructure and cloud computing. Specifically, it uses various information provided by the target vehicle, surrounding vehicles and roadside units to achieve a more accurate prediction of the vehicle's future trajectory. Trajectory prediction schemes based on vehicle-road collaboration are further divided into vehicle-to-vehicle schemes and vehicle-to-road collaborative schemes. The vehicle-to-vehicle scheme predicts the next action of the target vehicle and surrounding vehicles by analyzing their historical trajectories and motion states. The advantage of this method is that it can overcome the problem of single-vehicle trajectory prediction schemes lacking information about the status of surrounding vehicles. However, its limitation is that the information source is limited to the vehicle itself, and the surrounding road features, such as road shape and traffic signs, may be ignored, which may affect the vehicle's driving trajectory. The vehicle-to-road collaborative scheme effectively utilizes road information and assists trajectory prediction through data collected by roadside units. This scheme can provide more comprehensive traffic environment information, including traffic flow, signal light status, etc., thereby improving the accuracy of prediction. However, it also has its limitations. First, the surrounding vehicle status information used by this solution may come from estimation rather than direct perception, which may introduce additional uncertainty. Second, trajectory prediction solutions based on vehicle-road collaboration often rely on the vehicle's own computing power, but on-board computing resources are usually limited, and communication bandwidth may also become a bottleneck, limiting the application of complex algorithms and thus affecting the accuracy of trajectory prediction.

[0005] Although the trajectory prediction scheme based on vehicle-road collaboration has obvious advantages in improving prediction accuracy, it still faces challenges in computing resources, communication bandwidth, and information accuracy. Future research needs to further optimize the algorithm, improve computing efficiency, and explore more efficient information fusion and communication strategies to fully tap the potential of the vehicle-road collaboration trajectory prediction scheme and provide users with a safer and more reliable travel experience. Summary of the invention

[0006] In order to overcome the shortcomings of the prior art, the present invention provides a trajectory prediction method based on a digital twin platform, which collects information such as the vehicle's position, speed, driving status, and surrounding environment status in real time; after the initial processing of the information, the vehicle relies on its own built-in communication module to transmit the processed information to the trajectory prediction platform located in the cloud; then, the data features of the vehicle and roadside unit are extracted through a multi-channel mechanism, and the extracted features are sent to the Transformer module to predict the future trajectory; finally, the platform feeds back the prediction results to the vehicle so that the driver can adjust the current trajectory in time to avoid traffic accidents. Compared with other solutions, the ADE and FDE indicators of the present invention are the lowest, and the model efficiency is the best. In addition, within the trajectory prediction time step of 5 seconds, the average inference delay is 11.83ms.

[0007] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0008] Step 1: vectorize the collected vehicle information and map information;

[0009] For a vehicle, the dynamic state at time t is represented by x, y, v x ,v y ,a x ,a y ,φ, represented by a high-dimensional vector of 7 features; (x, y) describes the spatial position of the target vehicle, (v x ,v y ,a x ,a y ) represents the tendency of the vehicle to move in the x and y directions, where v x 、a x Respectively represent the vehicle's velocity and acceleration in the lateral direction, v y , a y They represent the velocity and acceleration in the longitudinal direction, respectively, and φ represents the direction of the vehicle;

[0010] Use D n , D s , D w , D e , D n-e , D s-e , D n-w , D s-w Indicates the straight-line distance between vehicles in eight directions: north, south, west, east, northeast, southeast, northwest and southwest;

[0011] Use V n 、V s 、V w 、V e 、V n-e 、V s-e 、V n-w 、V s-w Respectively represent the motion state differences between vehicles in eight directions: north, south, west, east, northeast, southeast, northwest and southwest;

[0012] The auxiliary factor a is used to represent the acceleration difference between vehicles in the scene;

[0013] use represents the angle difference between vehicles in the selected scene;

[0014] For map information, the collected map information is converted into a ψ and coordinates (P x ,P y), which is used to distinguish different road features such as straight traffic, roundabouts and intersections;

[0015] Step 2: Call the channel attention mechanism to extract the influence weights of different features from vehicles and roadside units RSU. The original vehicle and map feature vectors first pass through the fully connected layer, which further transforms and maps the feature vectors. The mapping relationship is shown in the following formula:

[0016] h=Wx+b

[0017] Where h represents the output vector of the vehicle and the map, x represents the original input feature vector of the vehicle and the map, W is the weight matrix, and b is the bias vector;

[0018] After the fully connected layer, an activation function is used to perform a nonlinear transformation on the previously transformed vector;

[0019] The Sigmoid function is used to normalize the result vector of the activation function. The Sigmoid function maps the original feature weights to a probability distribution that indicates the contribution of each channel to the prediction result.

[0020] Finally, the output weight of the Sigmoid function is multiplied by the original feature vectors of the vehicle and the roadside unit RSU to obtain the output vector;

[0021] Step 3: The output vector from the multi-channel attention mechanism is first passed through position encoding; to obtain position encoding, the unique encoding vectors from different positions are merged with the output vector from step 2 into a new vector; secondly, the new vector is used as the input of the transformer encoder, which consists of an attention mechanism, a feedforward neural network, and a residual layer; in the attention mechanism, the new vector is decomposed into query Q, key K, and value V; then, the similarity between the Query vector and the Key vector is calculated to obtain the attention weight that reflects the importance of each Query-Key correspondence; the attention weight is assigned to the corresponding Value vector, and the feature representation after the corresponding attention mechanism is obtained by summing, and the calculation formula is as follows:

[0022]

[0023] Next, this process is repeated H times; finally, they are concatenated through the Concatenate layer to obtain the global self-attention feature map, which is calculated as follows:

[0024] Multihead(Q,K,V)=Concat(head1,···,head h )W o

[0025] head i=Attention(QW i Q ,KW i K ,VW i V ),

[0026] i=1,2,...,H

[0027] This weight matrix merges all outputs into the final output vector to capture a richer feature representation;

[0028] After executing the attention mechanism, the transformer encoder introduces a set of feedforward neural networks with residual layers to compress the entire input sequence into a hidden state tensor HS; HS is used as the initial hidden state as the input of the transformer decoder; the transformer decoder gradually generates the target sequence through the attention mechanism; in the transformer decoder, the initial input X obtained at time t and HS is used t To predict the X of the next vehicle with step length t+1 t+1 ; On this basis, the output X at time t+1 is t+1 and hidden state HS+1 as the initial input and initial hidden state of Transformer Decoder, predicting X at time t+2 t+2 ; This prediction process will be repeated until X is completed at prediction step t+n t+n The prediction of the trajectory is then fed back to the vehicle for route adjustment.

[0029] Preferably, the activation function is a rectified linear unit (ReLU).

[0030] The beneficial effects of the present invention are as follows:

[0031] The present invention provides a trajectory prediction solution based on a digital twin platform. By combining the digital twin platform with trajectory prediction, the vehicle and environment information in the scene is collected in real time, the problem of limited vehicle field of view and insufficient vehicle computing resources is solved, and the accuracy of trajectory prediction is further improved. Specifically, through various sensors, the present invention can collect information such as the position, speed, driving status, and surrounding environment status of the vehicle in real time. After the initial processing of the information, the vehicle relies on its own built-in communication module to transmit the processed information to the trajectory prediction platform located in the cloud. Then, the present invention designs a multi-channel mechanism to extract data features of vehicles and roadside units, and sends the extracted features to the Transformer module to predict future trajectories. Finally, the platform feeds back the prediction results to the vehicle so that the driver can adjust the current trajectory in time to avoid traffic accidents. Compared with other solutions, the ADE and FDE indicators of the present invention are the lowest, and the model efficiency is the best. In addition, within the trajectory prediction time step of 5 seconds, the average inference delay is 11.83ms. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a flow chart for implementing the trajectory prediction solution based on the digital twin platform in the present invention;

[0033] Figure 2 The following are comparison diagrams of the average displacement error and the final displacement error in different scenarios of the present invention: (a) average displacement error of a three-lane straight road, (b) average displacement error of a four-lane straight road, (c) average displacement error of a nine-lane intersection, (d) final displacement error of a three-lane straight road, (e) final displacement error of a four-lane straight road, and (f) final displacement error of a nine-lane intersection.

[0034] Figure 3 Comparison diagram of the predicted trajectory and the actual trajectory in different scenarios in the present invention, (a) three-lane straight road, (b) nine-lane ring road, (a) four-lane ring road. DETAILED DESCRIPTION

[0035] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0036] The purpose of the present invention is to provide a trajectory prediction solution based on a digital twin platform to solve the problems of limited vehicle field of view and insufficient vehicle computing resources, and to further improve the accuracy of trajectory prediction.

[0037] according to Figure 1 As shown, the present invention provides a trajectory prediction solution based on a digital twin platform, comprising the following steps:

[0038] Step 1: First, vectorize the collected vehicle information and map information. For a vehicle, the dynamic state at time t can be expressed as x, y, vx ,v y ,a x ,a y ,φ, is represented by a high-dimensional vector of 7 features. (x,y) describes the spatial position of the target vehicle, (v x ,v y ,a x ,a y ) represents the tendency of the vehicle to move in two directions, and φ represents the direction of the vehicle. In addition, the characteristics of the vehicle also include differences in position, motion state, and supplementary factors. The position of the vehicle is divided into eight directions: north, south, west, east, northeast, southeast, northwest, and southwest. In different directions, the present invention uses D n , D s , D w , D e , D n-e , D s-e , D n-w , D s-w represents the straight-line distance between vehicles. In addition to the position of the vehicle, it is also necessary to consider the difference in motion state between the target volume and nearby vehicles. The motion state difference is represented by the variable V n 、V s 、V w 、V e 、V n-e 、V s-e 、V n-w 、V s-w express.

[0039] In addition, auxiliary factors such as a represent the acceleration differences between vehicles in the scene.

[0040] For map information, the present invention converts the collected map information into a ψ and coordinates (P x ,P y ). The point set converted by the present invention is used to distinguish different road features of straight traffic, roundabouts and intersections. The feature bodies in the traffic scene will help the model capture the influence weights of different factors in the next stage.

[0041] Step 2: Call the channel attention mechanism to extract the influence weights of different features from vehicles and roadside units RSU. The original vehicle and map feature vectors first pass through the fully connected layer, which further transforms and maps the feature vectors. The mapping relationship is shown in the following formula:

[0042] h=Wx+b

[0043] Where h and x represent the output vector and original feature vector of the vehicle and map respectively, W is the weight matrix, and b is the bias vector. After the fully connected layer, activation functions such as rectified linear unit (ReLU) are used to perform nonlinear transformation on the previously transformed vector. It can learn more complex function equations. Then, the result vector of the activation function is normalized using the Sigmod function. The Sigmod function processes the obtained weight vector to ensure the rationality of the attention weight. The Sigmoid function maps the original feature weight to a probability distribution that indicates the degree of contribution of each channel to the prediction result. Finally, the output weight of the Sigmoid function is multiplied by the original feature vector of the vehicle and roadside unit RSU to obtain the output vector.

[0044] In step 3, the output vector from the multi-channel attention mechanism will first pass through the position encoding part. In order to obtain the position encoding, the unique encoding vectors from different positions are merged with them into a new vector. Secondly, the new vector is used as the input of the transformer encoder, which consists of an attention mechanism, a feedforward neural network, and a residual layer. In the attention mechanism, these vectors are decomposed into query Q, key K, and value V. Then, the similarity between the Query vector and the Key vector is calculated to obtain the attention weights reflecting the importance of each Query-Key correspondence. These weights are assigned to the corresponding Value vectors, and the feature representation after the corresponding attention mechanism is obtained by summing them. The calculation formula is as follows:

[0045]

[0046] Next, this process is repeated H times. Finally, they are concatenated through the Concatenate layer to obtain the global self-attention feature map. The calculation formula is as follows:

[0047] Multihead(Q,K,V)=Concat(head1,···,head h )W o

[0048] head i =Attention(QW i Q ,KW i K ,VW i V ),

[0049] i=1,2,...,H

[0050] This weight matrix merges all outputs into the final output vector to capture a richer feature representation.

[0051] After executing the attention mechanism, the Transformer Encoder introduces a set of feedforward neural networks with residual layers to compress the entire input sequence into a hidden state tensor HS. HS is used as the initial hidden state as the input of the Transformer Decoder. The Transformer Decoder gradually generates the target sequence through the attention mechanism. In the Transformer Decoder, the initial input X obtained at time t and HS is used. t To predict the X of the next vehicle with step length t+1 t+1 On this basis, the output X at time t+1 is t+1 and hidden state HS+1 as the initial input and initial hidden state of Transformer Decoder, predicting X at time t+2 t+2 This prediction process will be repeated until X is completed at prediction step t+n. t+n The trajectory prediction result is then fed back to the vehicle to adjust the driving route.

[0052] Example:

[0053] refer to Figure 2 As shown, Figure 2 The proposed trajectory prediction scheme is compared with LSTM, Social-LSTM, and CS-LSTM, and its accuracy is judged by the visualization results of ADE / FDE in three scenarios. The figure shows the comparison of prediction accuracy in different scenarios within the next 5 seconds. The ordinate in the figure represents the prediction accuracy of different schemes for ADE / FDE values, and the abscissa represents the prediction time step. According to the qualitative results, it can be seen that the scheme of the present invention has high accuracy and flexibility in various scenarios. As the compensation increases, the accuracy of trajectory prediction gradually decreases, but compared with other schemes, the scheme of the present invention still maintains a high accuracy in three different scenarios. The results show the robustness of the scheme. In addition, the scheme makes full use of information from different scales and sources to improve the performance and effectiveness of trajectory prediction.

[0054] As shown in reference tables 1, 2, and 3, Tables 1, 2, and 3 respectively show the comparison between the proposed schemes in three scenarios: straight road, roundabout, and intersection, and the current LSTM, Social-LSTM, and CS-LSTM. The present invention analyzes the superiority of schemes with different time steps in different scenarios. CS-LSTM performs well in the first 15 seconds of the straight lane and the first 15 seconds of the intersection. However, as the prediction step size gradually increases, the index of the scheme of the present invention at 5s is significantly smaller than that of the benchmark, which indicates that the scheme of the present invention has superior performance in long-term trajectory prediction in different scenarios.

[0055] Table 1 Prediction accuracy of different schemes (ADE / FDE) on straight track

[0056]

[0057] Table 2 Prediction accuracy of different schemes (ADE / FDE) under loop

[0058]

[0059] Table 3 Prediction accuracy of different schemes (ADE / FDE) at intersections

[0060]

[0061] refer to Figure 3 As shown, Figure 3 The trajectory prediction results based on the digital twin platform were shown in three typical scenarios: 3-lane straight, 4-lane roundabout, and 9-lane intersection. Based on the digital twin platform, it can predict the trajectory of all vehicles in an area. This means it can track vehicles and provide timely feedback to warn drivers to avoid traffic accidents. Figure 3 In the figure, the red and green lines represent the real trajectory and the predicted trajectory within 5 seconds, respectively, and the yellow line represents the prediction deviation. As the compensation amount increases, the prediction deviation gradually increases, but the prediction accuracy remains at a high level. In addition, the scheme of the present invention performs well in all scenarios, indicating that the scheme of the present invention is robust.

[0062] As shown in reference Table 4, Table 4 shows a comparison of the inference delay required for different time steps. The present invention defines the inference time as the time cost of predicting the future trajectory of the target vehicle in a scenario. The results are shown in Table 2, which shows the distribution of inference time under different step lengths. The present invention sets a prediction time step of 1-5 seconds in the experiment. The difference in inference time between different time steps is relatively small. This is because the smaller model size of the present invention leads to a shorter inference time.

[0063] Table 4 Comparison of inference delay required for different time steps (ms)

[0064]

[0065] Without loss of generality, this embodiment experiments on the trajectory prediction scheme based on the digital twin platform on the INTERACTION dataset. INTERACTION was developed by the MSC Lab at the University of California, Berkeley, which uses drones to collect data from different cities in China, the United States, and Germany. It is divided into three parts: training, validation, and testing. The training set, validation set, and test set contain 47,584, 11,794, and 22,644 cases, respectively. Diverse scenarios and driving data improve the robustness and accuracy of the model.

[0066] The embodiment uses average displacement error (ADE) and final displacement error (FDE) to evaluate the positioning accuracy. All experiments are performed on a PC equipped with an AMD5000 CPU and a GeForce RTX3060 GPU.

Claims

1. A trajectory prediction method based on a digital twin platform, characterized in that: The steps include: Step 1: vectorize the collected vehicle information and map information; For a vehicle, the dynamic state at time t is represented by x, y, v x ,v y ,a x ,a y ,φ, represented by a high-dimensional vector of 7 features; (x, y) describes the spatial position of the target vehicle, (v x ,v y ,a x ,a y ) represents the tendency of the vehicle to move in the x and y directions, where v x 、a x Respectively represent the vehicle's lateral velocity and acceleration, v y , a y They represent the velocity and acceleration in the longitudinal direction, respectively, and φ represents the direction of the vehicle; Use D n , D s , D w , D e , D n-e , D s-e , D n-w , D s-w Indicates the straight-line distance between vehicles in eight directions: north, south, west, east, northeast, southeast, northwest and southwest; Use V n 、V s 、V w 、V e 、V n-e 、V s-e 、V n-w 、V s-w Respectively represent the motion state differences between vehicles in eight directions: north, south, west, east, northeast, southeast, northwest and southwest; The auxiliary factor a is used to represent the acceleration difference between vehicles in the scene; use represents the angle difference between vehicles in the selected scene; For map information, the collected map information is converted into a ψ and coordinates (P x ,P y ), which is used to distinguish different road features such as straight traffic, roundabouts and intersections; Step 2: Call the channel attention mechanism to extract the influence weights of different features from vehicles and roadside units RSU. The original vehicle and map feature vectors first pass through the fully connected layer, which further transforms and maps the feature vectors. The mapping relationship is shown in the following formula: h=Wx+b Where h represents the output vector of the vehicle and the map, x represents the original input feature vector of the vehicle and the map, W is the weight matrix, and b is the bias vector; After the fully connected layer, an activation function is used to perform a nonlinear transformation on the previously transformed vector; The Sigmoid function is used to normalize the result vector of the activation function. The Sigmoid function maps the original feature weights to a probability distribution that indicates the contribution of each channel to the prediction result. Finally, the output weight of the Sigmoid function is multiplied by the original feature vectors of the vehicle and the roadside unit RSU to obtain the output vector; Step 3: The output vector from the multi-channel attention mechanism is first passed through position encoding; to obtain position encoding, the unique encoding vectors from different positions are merged with the output vector from step 2 into a new vector; secondly, the new vector is used as the input of the transformer encoder, which consists of an attention mechanism, a feedforward neural network, and a residual layer; in the attention mechanism, the new vector is decomposed into query Q, key K, and value V; then, the similarity between the Query vector and the Key vector is calculated to obtain the attention weight that reflects the importance of each Query-Key correspondence; the attention weight is assigned to the corresponding Value vector, and the feature representation after the corresponding attention mechanism is obtained by summing, and the calculation formula is as follows: Next, this process is repeated H times; finally, they are concatenated through the Concatenate layer to obtain the global self-attention feature map, which is calculated as follows: Multihead(Q,K,V)=Concat(head1,···,head h )W o head i =Attention(QW i Q ,KW i K ,VW i V ), i=1,2,...,H This weight matrix merges all outputs into the final output vector to capture a richer feature representation; After executing the attention mechanism, the transformer encoder introduces a set of feedforward neural networks with residual layers to compress the entire input sequence into a hidden state tensor HS; HS is used as the initial hidden state as the input of the transformer decoder; the transformer decoder gradually generates the target sequence through the attention mechanism; in the transformer decoder, the initial input X obtained at time t and HS is used t To predict the X of the next vehicle with step length t+1 t+1 ; On this basis, the output X at time t+1 is t+1 and hidden state HS+1 as the initial input and initial hidden state of Transformer Decoder, predicting X at time t+2 t+2 ; This prediction process will be repeated until X is completed at prediction step t+n t+n The prediction of the trajectory is then fed back to the vehicle for route adjustment.

2. According to a trajectory prediction method based on a digital twin platform according to claim 1, it is characterized in that: The activation function is the rectified linear unit ReLU.

3. A computer program, characterized in that The computer program enables a computer to execute the method according to any one of claims 1 to 2.

4. An electronic device, characterized in that: include: Processor and memory; The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device executes the method as claimed in any one of claims 1 to 2.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 2 is implemented.

6. A chip, characterized in that: include: A processor, configured to call and run a computer program from a memory, so that a device equipped with the chip executes a method as claimed in any one of claims 1 to 2.

7. A computer program product, characterized in that The computer program product comprises a computer storage medium storing a computer program, wherein the computer program comprises instructions executable by at least one processor, and when the instructions are executed by the at least one processor, the method according to any one of claims 1 to 2 is implemented.

Images