Vehicle trajectory prediction system, predicting method thereof and model training method thereof

Abstract

A model training method is provided for training a converter model for a vehicle trajectory prediction system. The method includes: generating a first key vector and a first value vector from a first attention layer based on map information, object information, time information, and a reference trajectory; generating a second training trajectory from a third attention layer using the converter model based on a first training query vector, the first key vector, and the first value vector; and calculating a first loss value based on the second training trajectory and the reference trajectory, optimizing the converter model using the first loss value, and adding the positive label to the first training trajectory, in response to the first training trajectory lacking a positive label and the displacement error between the second training trajectory and the reference trajectory falling within a preset range.

Description

Technical Field

[0001] This invention relates to a vehicle trajectory prediction system, its prediction method, and the prediction method thereof, and particularly to a vehicle trajectory prediction system, its prediction method, and the prediction method thereof that predicts vehicle trajectories through an Early-Match-Take-All operation. Prior Technology

[0002] The technological development of vehicle trajectory prediction systems is crucial to the safety of autonomous driving. Most existing vehicle trajectory prediction systems use a winner-take-all computational principle to generate multiple predicted trajectories in parallel, then compare these trajectories and select the optimal solution to optimize the prediction model.

[0003] The drawback of the "winner-takes-all" approach is that the multiple trajectories generated by the prediction system are unaware of each other's existence, so the prediction system cannot use the relationships between these trajectories to optimize the prediction data; furthermore, the method of generating multiple prediction trajectories first and then selecting the optimal solution will cause the system to consume a lot of computing resources.

[0004] How to improve the aforementioned shortcomings of the winner-takes-all calculation principle, and thus enhance the prediction quality of vehicle trajectory prediction technology, is an important issue that needs to be addressed by those skilled in the art. Summary of the Invention

[0005] This disclosure provides a vehicle trajectory prediction system. The system includes a data collection interface, a processor, and a database. The data collection interface collects map information, object information, and time information. The processor is coupled to the data collection interface and executes a converter model. The converter model includes an encoder and a decoder. The encoder generates a first key vector and a first value vector from a first attention layer based on the map information, object information, and time information. The decoder generates a first query vector from a second attention layer based on the query vector corresponding to the first predicted trajectory, and generates a second predicted trajectory from a third attention layer based on the first query vector, the first key vector, and the first value vector. The database is coupled to the processor and stores the first and second predicted trajectories. The first predicted trajectory is generated earlier than the second predicted trajectory.

[0006] This disclosure provides a prediction method for executing a converter model in a vehicle trajectory prediction system. The prediction method includes: collecting map information, object information, and time information; and executing the converter model. The converter model includes an encoder and a decoder. The prediction method further includes: generating a first key vector and a first value vector from a first attention layer using the encoder based on the map information, object information, and time information; and generating a first query vector from a second attention layer using the decoder based on a query vector corresponding to the first predicted trajectory, and generating a second predicted trajectory from a third attention layer based on the first query vector, the first key vector, and the first value vector. The first predicted trajectory is generated earlier than the second predicted trajectory.

[0007] This disclosure provides a model training method for training a converter model for a vehicle trajectory prediction system. The model training method includes: performing a first round of training based on map information, object information, time information, and a reference trajectory. The first round of training includes: generating a first key vector and a first value vector from a first attention layer using the converter model based on the map information, object information, and time information; generating a first training query vector from a second attention layer using the converter model based on the query vector corresponding to the first training trajectory; and generating a second training trajectory from a third attention layer based on the first training query vector, the first key vector, and the first value vector. The generation time of the first training trajectory is earlier than the generation time of the second training trajectory. The first round of training further includes: in response to the first training trajectory not having a positive label and the displacement error between the second training trajectory and the reference trajectory falling within a preset range, calculating a first loss value based on the second training trajectory and the reference trajectory; optimizing the converter model using the first loss value; and adding the positive label to the second training trajectory in response to the first training trajectory having a positive label; and adding a negative label to the second training trajectory in response to the first training trajectory having a positive label.

[0008] In summary, the vehicle trajectory prediction system and method disclosed in this document can predict the trajectory of a vehicle based on past trajectory data and information about the surrounding environment. Furthermore, the model training method in this document can generate multiple training trajectories for the vehicle and add a positive label to the earliest training trajectory that meets the matching criteria (i.e., the displacement error between the training trajectory and the reference trajectory falls within a preset range), thereby achieving an "Early Match-Take-All" operation. Through the technology disclosed in this document, the vehicle trajectory prediction system may be able to predict more accurate autonomous vehicle movement trajectories and improve the system's computational efficiency. Simple Explanation of the Diagram

[0009] Figure 1 is a schematic diagram of a vehicle trajectory prediction system according to an embodiment of this disclosure. Figure 2 is a schematic diagram of a database and converter model according to an embodiment of this disclosure. Figure 3 is a schematic diagram of a database and converter model according to an embodiment of this disclosure. Figure 4 is a flowchart of a model training method according to an embodiment of this disclosure. Figure 5 is a schematic diagram of a multi-round model training method according to an embodiment of this disclosure. Implementation

[0010] The embodiments of this disclosure will be described below with reference to the relevant drawings. In the drawings, the same reference numerals denote the same or similar elements or method flows.

[0011] Please refer to Figure 1, which is a block diagram of a vehicle trajectory prediction system 100 according to an embodiment of this disclosure. The vehicle trajectory prediction system 100 may include a database 110, a processor 120, and a data collection interface 130.

[0012] The vehicle trajectory prediction system 100 may be an in-vehicle artificial intelligence computing device installed in a single autonomous vehicle. However, this disclosure is not limited thereto; the vehicle trajectory prediction system 100 may also be a terminal server that performs trajectory prediction and data collection for one or more autonomous vehicles via wireless communication technology. The vehicle trajectory prediction system 100 may compute, train, and utilize a transformer model 200.

[0013] Database 110 may be used to store computer-executable instructions, training data (during the training phase of converter model 200), learnable parameters of converter model 200, input data to be processed by converter model 200, and / or output data generated by converter model 200. In some embodiments, database 110 may be implemented using random access memory modules, read-only memory modules, flash memory, hard disks, cache memory, static random access memory, dynamic random access memory, non-volatile memory, solid-state drives, optical storage media, or other equivalent storage components. In some embodiments, database 110 stores instructions executable by processor 120 for performing the detailed steps described in subsequent paragraphs.

[0014] Processor 120 is coupled to database 110. Processor 120 includes a graphics processing unit (GPU), application-specific integrated circuit (ASIC), tensor processing unit (TPU), digital signal processor (DSP), field-programmable gate array (FPGA), single-instruction multiple-data (SIMD) processor, and / or any equivalent processing circuitry, etc. Compared to a general-purpose central processing unit (CPU), processor 120 can accelerate various computational tasks related to neural network model estimation (e.g., training, prediction, preprocessing, etc.).

[0015] The data collection interface 130 is coupled to the processor 120. The data collection interface 130 can be any type of data transceiver or port used to receive various data related to the autonomous vehicle, such as the vehicle's coordinates in the GPS, image information analyzed by the image recognition system, map information of the area where the vehicle is located, etc. Users or other external devices can input relevant data for autonomous vehicle trajectory prediction to the converter model 200 through the data collection interface 130, or the converter model 200 can receive the output results generated by the input.

[0016] In embodiments of the present invention, the converter model 200 can receive various types of information collected by the data collection interface 130, and predict the trajectory of a target vehicle (autonomous vehicle) using the various types of data. The converter model 200 can also perform model training using the various types of data to optimize its own predictive capabilities.

[0017] Please refer to Figure 2, which is a schematic diagram of a database 110 and a converter model 200 according to an embodiment of this disclosure. The converter model 200 in Figure 2 illustrates the architectural details of the converter model 200 in Figure 1. The database 110 and the converter model 200 are coupled to each other, and the converter model 200 includes an encoder 220 and a decoder 240.

[0018] In some embodiments, the database 110 may be used for the historical trajectories HT[1]-HT[n] of the target vehicle. "n" represents the number of historical trajectories. The target vehicle may be a self-driving car predicted by the vehicle trajectory prediction system 100, and the historical trajectories may be prediction data generated by the vehicle trajectory prediction system 100 for the target vehicle before the current time. In other words, the historical trajectories HT[1]-HT[n] may be multiple trajectories predicted by the vehicle trajectory prediction system 100 in the past.

[0019] In some other embodiments, the database 110 may store only one historical trajectory, that is, the database 110 may only contain the historical trajectory HT[1], but not the historical trajectory HT[n]. In other embodiments, the database 110 may also not contain any historical trajectories. For example, when the vehicle trajectory prediction system 100 has just started operating, since the vehicle trajectory prediction system 100 has not yet made predictions for the target vehicle, the database 110 has not yet stored any historical trajectories.

[0020] The encoder 220 can generate scene encoding based on the map information M_DAT, object information A_DAT, and time information T_DAT collected by the data collection interface 130.

[0021] Encoder 220 can generate map tensor MD220 based on the scene encoding of map information M_DAT. Encoder 220 can analyze the scene encoding of object information A_DAT and time information T_DAT to generate object tensor ATD220.

[0022] Map information M_DAT can be used to describe the target vehicle and multiple movement paths of one or more objects (agents) around the target vehicle, such as lanes or sidewalks in a city, town, or specific scene. The map tensor MD220 has a tensor size of [M, D], where M is the number of channels referenced by the map information M_DAT (i.e., the number of lanes or sidewalks mentioned above), and D is a preset encoding dimension (e.g., 128). The map tensor MD220 has multiple map embedding values.

[0023] Object information A_DAT can be used to describe the target vehicle and one or more objects around the target vehicle. Time information T_DAT can be used to record multiple time points when the data collection interface 130 receives various information. The object tensor ATD220 has a tensor size of [A, T, D], where A is the number of the target vehicle and one or more objects around the target vehicle corresponding to the object information A_DAT, T is the number of time points recorded in the time information T_DAT, and D is a preset encoding dimension (e.g., 128). The object tensor ATD220 can contain multiple target vehicle embedding values ​​and multiple moving object embedding values.

[0024] The encoder 220 has an attention layer. The attention of the encoder 220 can generate a key vector K220 and a value vector V220 based on the plurality of map embedding values, the plurality of target vehicle embedding values, and the plurality of moving object embedding values.

[0025] Decoder 240 can generate key vectors and value vectors based on historical trajectories HT[1]-HT[n]. Furthermore, decoder 240 can generate query vector Q240 using the key vectors and value vectors generated based on historical trajectories HT[1]-HT[n], and the query vector Q240P corresponding to the latest generated trajectory in historical trajectories HT[1]-HT[n]. If the database 110 does not store historical trajectories, decoder 240 can receive an empty value from the database 110 and generate query vector Q240 based on the empty value.

[0026] Furthermore, the decoder 240 can receive the key vector K220 and the value vector V220 from the encoder 220. Based on the query vector Q240, the key vector K220, and the value vector V220, the decoder 240 can sequentially predict the trajectories T[1]-T[m] and the confidence scores CV[1]-CV[m]. The predicted trajectories T[1]-T[m] can correspond to the confidence scores CV[1]-CV[m] respectively. "m" can represent the total number of all predicted trajectories generated by the converter model 200, including the predicted trajectories generated at the current time.

[0027] Furthermore, the decoder 240 includes a memory attention layer 242, a factorized attention layer 244, and an output layer 246.

[0028] The memory attention layer 242 can receive historical trajectories HT[1]-HT[n] and query vector Q240P or the null value to generate query vector Q240.

[0029] The query vector Q240 can be transferred to the database 110 as the query vector Q240P for the next trajectory prediction.

[0030] The decomposed attention layer 244 can receive a query vector Q240, a key vector K220, and a value vector V220. The decomposed attention layer 244 can perform temporal attention, agent-map attention, and agent-agent attention.

[0031] Through the aforementioned time attention mechanism, the decomposed attention layer 244 can use the query vector Q240 to focus on the historical features of the target vehicle at multiple time points described in the key vector K220 and the value vector V220 (i.e., the historical trajectory encoding of the target vehicle).

[0032] Through the object-map attention mechanism, the decomposed attention layer 244 can use the query vector Q240 to focus on the correspondence between the target vehicle and the map information M_DAT described in the key vector K220 and the value vector V220. Specifically, the query vector of the object-map attention mechanism is the output data of the temporal attention mechanism, and the key vector and value vector of the object-map attention mechanism are the encodings of the map information M_DAT around the target vehicle.

[0033] Through the object-to-object attention mechanism, the decomposed attention layer 244 can use the query vector Q240 to focus on the correspondence between the target vehicle and other moving objects nearby, as described in the key vector K220 and value vector V220, at the current moment. Specifically, the query vector of the object-to-object attention mechanism is the output data of the object-to-map attention mechanism, while the key vector and value vector of the object-to-object attention mechanism are the encodings of other moving objects around the target vehicle.

[0034] It is worth mentioning that the key vectors and value vectors of the temporal attention mechanism, the object-map attention mechanism, and the object-object attention mechanism are all derived from a subset of the scene encoding of the encoder 220.

[0035] Furthermore, the decomposed attention layer 244 can generate multiple trajectory embedding values ​​TE[1]-TE[m] based on the query vector Q240, the key vector K220, and the value vector V220.

[0036] The output layer 246 can be a multilayer perceptron (MLP). The output layer 246 can sequentially receive the trajectory embedding values ​​TE[1]-TE[m] generated by the decomposed attention layer 244, and generate the predicted trajectory T[1]-T[m] and confidence score CV[1]-CV[m].

[0037] In summary, as can be seen from the embodiment in Figure 2, the converter model 200 of the vehicle trajectory prediction system 100 can generate the current prediction query vector Q240 from the query vector Q240P of the historical trajectory, thereby enhancing the continuity between the current predicted trajectory and the past trajectory.

[0038] Please refer to Figures 2 and 3 simultaneously. Figure 3 is a schematic diagram of a database 110 and a converter model 200 according to an embodiment of this disclosure. The converter model 200 in Figure 3 further reveals the model architecture details of the converter model 200 in Figure 2.

[0039] In the embodiment of Figure 3, encoder 220 includes an embedding module 222, a normalization module 224, a first attention layer 226, and an output layer 229.

[0040] The embedding module 222 can receive map information M_DAT, object information A_DAT and time information T_DAT, and generate the multiple map embedding values, the multiple target vehicle embedding values ​​and the multiple moving object embedding values.

[0041] The normalization module 224 can generate a query vector Q224, a key vector K224, and a value vector V224 based on the multiple map embedding values, the multiple target vehicle embedding values, and the multiple moving object embedding values.

[0042] The first attention layer 226 can receive query vector Q224, key vector K224, and value vector V224. The output data of the first attention layer 226, the multiple map embedding values, the multiple target vehicle embedding values, and the multiple moving object embedding values ​​can be combined and transmitted to the output layer 229 to output key vector K220 and value vector V220.

[0043] In this embodiment, the memory attention layer 242 of the decoder 240 includes an embedding module 2422, a normalization module 2424, and a second attention layer 2426. The decomposed attention layer 244 of the decoder 240 includes a normalization module 2442 and a third attention layer 2444.

[0044] The embedding module 2422 can receive historical trajectories HT[1]-HT[n] and generate embedding values ​​related to the historical trajectories.

[0045] The normalization module 2424 can generate a key vector K242, a value vector V242, and a query vector (not shown) based on the embedded values ​​related to the historical trajectory mentioned above.

[0046] The second attention layer 2426 can receive the key vector K242, the value vector V242, and the query vector Q240P.

[0047] The output data of the second attention layer 2426 and the embedded values ​​related to the historical trajectory mentioned above can be combined to serve as the output data of the memory attention layer 242. The output data of the memory attention layer 242 can be transmitted to the normalization module 2442, which can output a query vector Q240, a key vector, and a value vector (not shown).

[0048] The third attention layer 2444 can receive the key vector K220, the value vector V220, and the query vector Q240. The output data of the memory attention layer 242 and the output data of the third attention layer 2444 can be combined into the trajectory embedding value TE[1]-TE[m] and output to the output layer 246.

[0049] Please refer to Figures 2 and 4 simultaneously. Figure 4 is a flowchart of a model training method 400 according to an embodiment of this disclosure. In this embodiment, the model training method 400 is used for the training phase of the converter model 200. During the training phase, the vehicle trajectory prediction system 100 can train and optimize the converter model 200 through the model training method 400.

[0050] In step S410, the converter model 200 can obtain past training trajectories from the database 110. Specifically, the model training method 400 can be executed multiple times, generating multiple training trajectories accordingly. The "past training trajectories" mentioned in step S410 are one or more training trajectories generated by executing one or more model training methods 400 before the current time. If the model training method 400 has not been executed before the current time (i.e., no past training trajectories are stored in the database 110), the database 110 can provide null values.

[0051] In step S420, the encoder 220 of the converter model 200 can generate a training key vector and a training value vector from the first attention layer 226 based on map information M_DAT, object information A_DAT, and time information T_DAT. The training key vector may correspond to the key vector K220 of the second image, and the training value vector may correspond to the value vector V220 of the second image. It should be noted that the training key vector and training value vector in step S420 are used to train the converter model 200, while the key vector K220 and value vector V220 can be used simultaneously for both executing and training the converter model 200.

[0052] In step S430, the decoder 240 of the converter model 200 can generate a training query vector from the second attention layer 2426 based on past training trajectories. Specifically, the training query vector can be generated based on the query vectors corresponding to past training trajectories. The query vectors corresponding to past training trajectories can correspond to the query vector Q240P in Figure 2, and the training query vector can correspond to the query vector Q240P generated in Figure 2 via the second attention layer 2426 and the normalization module 2442. It should be noted that the training query vector in step S430 is used to train the converter model 200, while the query vector Q240 can be used to both execute and train the converter model 200.

[0053] In step S440, the decoder 240 of the converter model 200 can generate the current training trajectory and the confidence score corresponding to the current training trajectory by the third attention layer 2444 based on the training query vector, the training key vector and the training value vector.

[0054] In step S450, the converter model 200 can determine whether the past training trajectories have positive labels. Regardless of the number of past training trajectories, as long as any of them have been positively labeled, step S490 is executed; otherwise, if none of the past training trajectories have been positively labeled, step S460 is executed.

[0055] In step S460, the converter model 200 determines whether the displacement error between the current training trajectory and the reference trajectory falls within a preset range. Specifically, the reference trajectory can be an ideal trajectory preset by the system, and the preset range can be preset to 2 meters. For example, as long as the displacement error between the current training trajectory and the reference trajectory is within 2 meters, the converter model 200 can determine that the current training trajectory meets the matching conditions. It should be noted that the preset range in this disclosure is not limited to 2 meters, and those skilled in the art can adjust the value of the preset range according to actual applications. If the judgment result of step S460 is "yes", that is, the displacement error falls within the preset range, step S470 is executed; otherwise, if the judgment result of step S460 is "no", the "current training trajectory" in this step can be stored in the database 110, and step S410 is executed.

[0056] In step S470, the converter model 200 can establish a positive label for the current training trajectory, set a loss value based on the current training trajectory and the reference trajectory, and store the current training trajectory in the database 110.

[0057] In step S480, the processor 120 can optimize the converter model 200 using the loss value. After adding a positive label to the current training trajectory, the converter model 200 can also adjust the confidence score of the current training trajectory to be close to 1, indicating that the converter model 200 considers the current training trajectory as a calculation result with a high degree of matching.

[0058] After completing step S480, the vehicle trajectory prediction system 100 completes the earliest match take-all operation based on the current training trajectory.

[0059] Regarding the "earliest match takes all" operation, after the vehicle trajectory prediction system 100 identifies the earliest match (i.e., the training trajectory that first falls into the preset interval in sequence), the system will not continue to generate training trajectories in similar directions. Instead, it will generate trajectories in other directions. This allows for faster training of trajectory data with different directions, thereby improving the system's computational efficiency. For example, if the earliest match's trajectory direction is to make the target vehicle turn left, the subsequent training trajectories of the earliest match will be to make the target vehicle head in different directions, such as going straight or turning right.

[0060] In step S490, the converter model 200 can add a negative sample to the current training trajectory and make the confidence score of the current training trajectory approach 0.

[0061] If the vehicle trajectory prediction system 100 enters step S490 when executing model training method 400, it means that one of the previous training trajectories already has a positive label and completed the earliest match take-all operation long before the current training trajectory was generated. The current training trajectory in step S490 is just the "training trajectory of the earliest match".

[0062] After either step S480 or step S490 is completed, the vehicle trajectory prediction system 100 can repeat step S410 and continue to execute the model training method 400 to generate more training trajectories.

[0063] Please refer to Figures 2, 4, and 5 simultaneously. Figure 5 is a schematic diagram of a multi-round model training method 500 according to an embodiment of this disclosure. During the training phase of the converter model 200, the vehicle trajectory prediction system 100 can generate multiple loss values ​​through the multi-round model training method 500, and optimize the converter model 200 multiple times using the multiple loss values.

[0064] In step S510, the vehicle trajectory prediction system 100 can provide the converter model 200 with map information M_DAT, object information A_DAT, time information T_DAT, and the reference trajectory mentioned above. Furthermore, the database 110 can provide the converter model 200 with null values ​​Tini. The null value Tini is a null value that conforms to the data format of the predicted trajectory, and its confidence score is 0.

[0065] In step S520, the converter model 200 can perform the first round of training to generate multiple predicted trajectories T11, T12, and T13. The first round of training is the first time that the converter model 200 executes the model training method 400, and through the various steps contained in the model training method 400, it sequentially generates multiple predicted trajectories T11, T12, and T13.

[0066] The confidence score for predicted trajectory T11 is 0.2, the confidence score for predicted trajectory T12 is 0.4, and the confidence score for predicted trajectory T13 is 0.5. In this embodiment, predicted trajectory T12 is given a positive label. That is, the converter model 200 sets the loss value for the first round of training based on the predicted trajectory T12 and the reference trajectory.

[0067] In step S530, the database 110 can perform a reduced-order sorting of multiple predicted trajectories T11, T12, and T13. Specifically, the database 110 sorts the predicted trajectories according to their confidence scores, resulting in multiple descending-order trajectories. As shown in Figure 5, predicted trajectory T13 is rearranged to the first position in the reduced-order sequence and becomes the descending trajectory TA11; predicted trajectory T12 becomes the descending trajectory TA12; and predicted trajectory T11 is rearranged to the third position in the reduced-order sequence and becomes the descending trajectory TA13.

[0068] In step S540, the converter model 200 performs a second round of training, generating multiple predicted trajectories T21, T22, and T23. This second round of training involves the converter model 200 executing model training method 400 for the second time. The confidence score for predicted trajectory T21 is 0.7, the confidence score for predicted trajectory T22 is 0.3, and the confidence score for predicted trajectory T23 is 0.1. The converter model 200 adds a positive label to predicted trajectory T22 and sets the loss value for the second round of training based on predicted trajectory T22 and the reference trajectory.

[0069] It is worth mentioning that, corresponding to the architecture in Figure 2, when the converter model 200 generates the predicted trajectory T21 using model training method 400, its query vector Q240P can be the query vector Q240 used to generate the predicted trajectory T11. Furthermore, when the converter model 200 generates the predicted trajectory T22 using model training method 400, its query vector Q240P can be the query vector Q240 used to generate the predicted trajectory T12. When the converter model 200 generates the predicted trajectory T23 using model training method 400, its query vector Q240P can be the query vector Q240 used to generate the predicted trajectory T13.

[0070] After completing step S540, the vehicle trajectory prediction system 100 continues to perform multiple training rounds, making the multiple predicted trajectories generated in subsequent rounds more scene-compatible and having a more appropriate confidence score. Ideally, after performing multiple training rounds (e.g., five rounds), the vehicle trajectory prediction system 100 should be able to make the first predicted trajectory generated in the next round (e.g., the sixth round) meet the matching conditions.

[0071] In summary, the vehicle trajectory prediction system and method disclosed in this document can predict the trajectory of a vehicle based on past trajectory data and information surrounding the vehicle (e.g., other moving objects or a map). Furthermore, the model training method in this document can generate multiple training trajectories for the vehicle and add a positive label to the earliest matching training trajectory, thus achieving an "earliest match wins" operation. Through the technology disclosed in this document, the vehicle trajectory prediction system may be able to predict more accurate autonomous vehicle trajectories and improve the system's computational efficiency.

[0072] The above are merely preferred embodiments of this disclosure. Various modifications and equivalent changes can be made to this disclosure without departing from its scope or spirit. In summary, all modifications and equivalent changes to this disclosure made within the scope of the following claims are within the scope of this disclosure.

[0073] 100: Vehicle Trajectory Prediction System 110: Database 120: Processor 130: Data Collection Interface 200: Converter Model 220: Encoder 240: Decoder MD220: Map Tensor ATD220: Object Tensor 242: Memory Attention Layer 244: Decompositional Attention Layer 246,229,246: Output layer M_DAT: Map Information A_DAT: Object Information T_DAT: Time Information K220, K224, K242: Key Vectors V220, V224, V242: Value vectors HT[1]-HT[n]: Historical trajectory Q240, Q240P, Q224: Query vectors TE[1]-TE[m]: Trajectory embedding value T[1]-T[m],T11,T12,T13,T21,T22,T23: Predicted trajectory CV[1]-CV[m]: Confidence score 222,2422: Embedded Module 224,2424,2442: Normalization Module 226: First Attention Layer 2426: Second Attention Layer 2444: Third Attention Layer 400: Model Training Methods S410, S420, S430, S440, S450, S460, S470, S480, S490, S510, S520, S530, S540: Steps 500: Multi-round model training method Tini: null value TA11, TA12, TA13: Descending trajectories

Claims

1. A vehicle trajectory prediction system, comprising: a database; a data collection interface for collecting map information, object information, and time information; and a processor coupled to the database and the data collection interface, the processor executing a converter model, wherein in a training phase of the converter model, the database provides a reference trajectory to the converter model, and the processor further being configured to: execute a first round of training, wherein the first round of training comprises: generating a first key vector and a first value vector by a first attention layer based on the map information, the object information, and the time information through the converter model; generating a first training query vector by a second attention layer based on a query vector corresponding to a first training trajectory through the converter model, and generating a second training trajectory by a third attention layer based on the first training query vector, the first key vector, and the first value vector, wherein the generation time of the first training trajectory is earlier than the generation time of the second training trajectory, and the second training trajectory is oriented towards a first direction; In response to the first training trajectory not having a positive label and a displacement error between the second training trajectory and the reference trajectory falling within a preset range, a first loss value is calculated based on the second training trajectory and the reference trajectory, the converter model is optimized using the first loss value, and the positive label is added to the second training trajectory; and in response to the first training trajectory having the positive label, a negative label is added to the second training trajectory; in response to the second training trajectory not having the positive label or the negative label, multiple training trajectories generated after the second training trajectory are oriented towards the first direction; and in response to the second training trajectory having the positive label, these training trajectories generated after the second training trajectory are oriented towards a second direction different from the first direction.

2. The vehicle trajectory prediction system as described in claim 1, wherein the converter model includes an encoder for: generating a map tensor based on the map information, and generating an object tensor based on the object information and the time information, wherein the map tensor has multiple map embedding values, and the object tensor includes multiple target vehicle embedding values ​​and multiple moving object embedding values; generating the first key vector and the first value vector based on the map embedding values, the target vehicle embedding values, and the moving object embedding values, wherein the tensor size of the map tensor is [M, D], where M is the number of channels referenced by the map information, and D is a preset encoding dimension, and the tensor size of the object tensor is [A, T, D], where A is the number of target vehicles and multiple moving objects corresponding to the object information, and T is the number of time points in the time information.

3. The vehicle trajectory prediction system as described in claim 1, wherein the first round of training further comprises: in response to the second training trajectory having the positive label, approaching the confidence score of the second training trajectory to 1; and in response to the second training trajectory having the negative label, approaching the confidence score of the second training trajectory to 0.

4. The vehicle trajectory prediction system as described in claim 1, wherein the processor is further configured to: sort the first training trajectory and the second training trajectory in descending order based on the confidence scores of the first training trajectory and the second training trajectory to obtain a first descending trajectory and a second descending trajectory; and perform a second round of training, wherein the second round of training includes: generating a third training query vector by the second attention layer based on a query vector of the first descending trajectory and the first descending trajectory and the second descending trajectory through the converter model, and generating a third training trajectory by the third attention layer based on the third training query vector, the first key vector and the first value vector; and calculating a second loss value based on the third training trajectory and the reference trajectory in response to the displacement error between the third training trajectory and the reference trajectory falling into the preset interval, optimizing the converter model through the second loss value, and adding the positive label to the third training trajectory.

5. A model training method for training a converter model of a vehicle trajectory prediction system, the model training method comprising: performing a first round of training based on map information, object information, time information, and a reference trajectory, wherein the first round of training comprises: generating a first key vector and a first value vector by a first attention layer using the converter model based on the map information, the object information, and the time information; generating a first training query vector by a second attention layer using the converter model based on a query vector corresponding to a first training trajectory, and generating a second training trajectory by a third attention layer based on the first training query vector, the first key vector, and the first value vector, wherein the generation time of the first training trajectory is earlier than the generation time of the second training trajectory, and the second training trajectory is oriented in a first direction; responding to the first training trajectory not having a positive label and a displacement error between the second training trajectory and the reference trajectory falling within a preset range, calculating a first loss value based on the second training trajectory and the reference trajectory, optimizing the converter model using the first loss value, and adding the positive label to the second training trajectory; In response to the first training trajectory having the positive label, a negative label is added to the second training trajectory; in response to the second training trajectory not having the positive label or the negative label, multiple training trajectories generated after the second training trajectory are oriented toward the first direction; and in response to the second training trajectory having the positive label, the training trajectories generated after the second training trajectory are oriented toward a second direction different from the first direction.

6. The model training method as described in claim 5, wherein the first round of training further comprises: in response to the second training trajectory having the positive label, approaching the confidence score of the second training trajectory to 1; and in response to the second training trajectory having the negative label, approaching the confidence score of the second training trajectory to 0.

7. The model training method as described in claim 5 further comprises: sorting the first training trajectory and the second training trajectory in descending order based on the confidence scores of the first training trajectory and the second training trajectory to obtain a first descending trajectory and a second descending trajectory; and performing a second round of training, wherein the second round of training comprises: generating a third training query vector from the second attention layer based on a query vector of the first descending trajectory and the first descending trajectory and the second descending trajectory through the converter model, and generating a third training trajectory from the third attention layer based on the third training query vector, the first key vector, and the first value vector; in response to the displacement error between the third training trajectory and the reference trajectory falling into the preset interval, calculating a second loss value based on the third training trajectory and the reference trajectory, optimizing the converter model through the second loss value, and adding the positive label to the third training trajectory.

Images