Track prediction method and device fusing kinematics and environmental cognition

By integrating kinematics and environmental cognition, the features extracted and interactive learning of vehicle historical trajectories, surrounding vehicle trajectories and traffic environment information are combined with kinematic simulation models to make iterative prediction, which solves the traditional trajectory prediction algorithm in complex environments The accuracy and transparency issues achieve higher trajectory prediction accuracy and interpretability.

CN119037470AInactive Publication Date: 2024-11-29JIOTAO INTELLIGENT (BEIJING) TECHNOLOGY CO LTD
View PDF 0 Cites 6 Cited by

Patent Information

Application Number
CN202411107740.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-13
Publication Date
2024-11-29
Estimated Expiration
Not applicable · inactive patent

Smart Images

  • Figure CN119037470A_ABST
    Figure CN119037470A_ABST
Patent Text Reader

Abstract

The invention relates to a trajectory prediction method and device fusing kinematics and environmental cognition. The method comprises the following steps: based on a pre-constructed coding module, carrying out feature extraction and interactive learning on a first historical track, a second historical track and traffic environment information after preprocessing to obtain a target vehicle track feature, a surrounding vehicle advanced track feature related to the target vehicle and an environment advanced feature; inputting the target vehicle trajectory features, the surrounding vehicle advanced trajectory features and the environment advanced features into a pre-constructed prediction decoding module for decoding prediction processing to obtain a predicted acceleration corresponding to the current moment; and inputting the predicted acceleration and the current position and the current speed of the target vehicle into a kinematics simulation model for iterative motion simulation to obtain a trajectory prediction result of a future preset time step length. The acceleration and trajectory prediction and iteration can be carried out from the perspective of gradual simulation from the perspective of kinematics, so that the interpretability can be ensured, and the trajectory prediction accuracy can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of intelligent driving and artificial intelligence technology, and in particular to a trajectory prediction method and device integrating kinematics and environmental cognition. Background Art

[0002] Trajectory prediction plays a key role in many applications, such as path planning for autonomous vehicles, flight route planning for drones, and intelligent traffic management. Taking intelligent driving as an example, with the increasingly urgent demand for the implementation of intelligent driving and the safety requirements of autonomous driving, accurately predicting the future trajectory of the vehicle is crucial for downstream planning tasks, among which the vehicle trajectory prediction algorithm is the key to promoting the implementation of autonomous driving technology.

[0003] In the process of realizing the concept of the present disclosure, the inventors found that there are at least the following technical problems in the related art: Traditional trajectory prediction algorithms mainly rely on physical models and simple machine learning methods, but these methods have limited prediction accuracy in complex environments. Some algorithms introduce deep neural networks, but the prediction process of such algorithms is a black box algorithm, and the internal decision-making process is not transparent, which poses a major obstacle in the field of autonomous driving where safety and explainability need to be guaranteed. Therefore, the related art lacks a trajectory prediction solution that can both improve the explainability of the prediction process and guarantee the desired prediction performance. Summary of the invention

[0004] In order to solve the above technical problems or at least partially solve the above technical problems, an embodiment of the present disclosure provides a trajectory prediction method and device that integrates kinematics and environmental cognition.

[0005] In the first aspect, the embodiment of the present disclosure provides a trajectory prediction method integrating kinematics and environmental cognition. The above method includes: preprocessing the first historical trajectory of the target vehicle, the second historical trajectory of other vehicles and the traffic environment information in the vehicle driving scene; based on the pre-constructed encoding module, the pre-processed first historical trajectory, the second historical trajectory and the traffic environment information are feature extracted and interactively learned to obtain the target vehicle trajectory features, the high-level trajectory features of the surrounding vehicles related to the target vehicle and the high-level environmental features; the above target vehicle trajectory features, the above high-level trajectory features of the surrounding vehicles and the high-level environmental features are input into the pre-constructed prediction decoding module for decoding prediction processing to obtain the predicted acceleration corresponding to the current moment; the above predicted acceleration, the current position and the current speed of the above target vehicle are input into the kinematic simulation model for iterative motion simulation to obtain the trajectory prediction result of the preset time step in the future; wherein, in the iterative motion simulation process, the predicted speed and predicted position of a single time step are iteratively input into the above encoding module for re-encoding to obtain the latest trajectory features of the target vehicle; the above latest trajectory features continue to be used as the input of the above prediction decoding module and obtain the predicted acceleration at the next moment; iterate multiple rounds until the simulation of the preset time step is completed.

[0006] In some embodiments, the predicted acceleration is obtained by summing or weighted summing the following three accelerations: a first predicted acceleration generated by the self-driving action of the target vehicle, a second predicted acceleration generated by the surrounding vehicles affecting the target vehicle, and a third predicted acceleration generated by the traffic environment affecting the target vehicle.

[0007] In some embodiments, the prediction decoding module includes: a first decoding module, a second decoding module and a third decoding module. The target vehicle trajectory features, the surrounding vehicle high-level trajectory features and the environment high-level features are input into the pre-built prediction decoding module for decoding prediction processing to obtain the predicted acceleration corresponding to the current moment, including: inputting the target vehicle trajectory features and the target point features into the first decoding module, and outputting the first predicted acceleration generated by the vehicle's self-driving action; wherein the target point features are features corresponding to the predicted target point position obtained by predicting the target vehicle trajectory features and the environment high-level features; inputting the target vehicle trajectory features and the surrounding vehicle high-level trajectory features into the second decoding module, and outputting the second predicted acceleration generated by the surrounding vehicles affecting the target vehicle; inputting the target vehicle trajectory features and the environment high-level features into the third decoding module, and outputting the third predicted acceleration generated by the traffic environment affecting the target vehicle.

[0008] In some embodiments, the method further includes: inputting the target vehicle trajectory features and the high-level environmental features into a pre-built target point encoding module for target point prediction processing, and obtaining target point features corresponding to the predicted target point position.

[0009] In some embodiments, the above-mentioned encoding module includes: a target vehicle trajectory encoding module, a surrounding vehicle trajectory encoding module, a traffic environment encoding module, a first cross-learning module, and a second cross-learning module. Wherein, based on the pre-constructed encoding module, the pre-processed first historical trajectory, the second historical trajectory and the traffic environment information are subjected to feature extraction and interactive learning to obtain the target vehicle trajectory features, the surrounding vehicle high-level trajectory features and the environment high-level features related to the target vehicle, including: the pre-processed first historical trajectory and the predicted trajectory under the iteration step constitute a splicing vector, the above-mentioned splicing vector is input into the above-mentioned target vehicle trajectory encoding module for feature extraction processing, and the target vehicle trajectory features are output; wherein, in the initial state, the above-mentioned predicted trajectory is an empty vector; in the iterative motion simulation process, the predicted trajectory under the above-mentioned iteration step is the result of step-by-step accumulation of the predicted position and predicted speed of a single time step corresponding to the output of each iteration round; the pre-processed second historical trajectory is input into the above-mentioned surrounding vehicle trajectory encoding module, and the second of each other vehicle is subjected to the step-by-step accumulation of the predicted position and predicted speed of the surrounding vehicle trajectory encoding module. Feature extraction is performed on each trajectory point in the historical trajectory, and the global trajectory feature of each other vehicle is obtained based on self-attention learning between each trajectory point; then the global trajectory features between other vehicles are interactively learned to obtain the trajectory features of surrounding vehicles; the above-mentioned target vehicle trajectory features and the above-mentioned surrounding vehicle trajectory features are input into the above-mentioned first cross-learning module for cross-attention learning to obtain the high-level trajectory features of surrounding vehicles related to the target vehicle; the pre-processed traffic environment information is input into the above-mentioned traffic environment encoding module for feature extraction of local environment, internal interactive learning of each local environment, and global interactive learning between multiple local environments to obtain the global features of the environment; the above-mentioned target vehicle trajectory features and the above-mentioned global features of the environment are input into the above-mentioned second cross-learning module for cross-attention learning to obtain the high-level features of the environment related to the target vehicle.

[0010] In some embodiments, the first historical trajectory, the second historical trajectory and the traffic environment information are all located in the global coordinate system. The preprocessing of the first historical trajectory of the target vehicle, the second historical trajectory of other vehicles and the traffic environment information in the vehicle driving scene includes: performing coordinate conversion processing on the first historical trajectory, the second historical trajectory and the traffic environment information, and converting them to the representation results in the reference coordinate system corresponding to the observation angle of the target vehicle. The current position of the target vehicle is taken as the coordinate origin, the displacement direction of the current timestamp of the target vehicle compared to the previous timestamp is taken as the positive direction of the first direction, and the first direction and the second direction perpendicular to the first direction are constructed as the reference coordinate system.

[0011] In some embodiments, the above method also includes: performing multimodal processing in the encoding module, mapping the target vehicle trajectory features, the high-level trajectory features of the surrounding vehicles related to the target vehicle, and the high-level features of the environment to obtain multiple modal results; wherein, after decoding and predicting processing and iterative motion simulation are performed through a pre-built prediction decoding module, a plurality of trajectory prediction results corresponding to each modal result are obtained; for each trajectory prediction result, re-encoding is performed based on the above-mentioned encoding module to obtain the predicted trajectory features corresponding to each modality; the predicted trajectory features, the high-level trajectory features of the surrounding vehicles, and the high-level features of the environment corresponding to each modality are inputted into a pre-built probability decoding network, and the trajectory score of the current modality is outputted; the trajectory scores of all modalities are inputted into the soft maximization function to obtain the probability values ​​of multiple trajectory prediction results; and the trajectory prediction result with the largest probability value is outputted as the final predicted trajectory.

[0012] In some embodiments, the trajectory prediction result of the preset time step in the future is a short-term prediction trajectory at the second level; the predicted trajectory finally output is a short-term prediction trajectory at the second level. In the above vehicle driving scenario, with the above target vehicle as the center, vehicles within a preset size range around the above target vehicle are regarded as other vehicles, and traffic elements within a preset size range around the above target vehicle are regarded as traffic environment; the above traffic environment includes one of the following combinations: lanes, lanes and traffic lights, lanes and roadblocks; the above traffic environment information includes one of the following combinations: lane information, lane information and traffic light information, lane information and roadblock information.

[0013] In the second aspect, the embodiments of the present disclosure provide a method for constructing a trajectory prediction model that integrates kinematics and environmental cognition. The above method includes: based on a preprocessing module, preprocessing the first training history trajectory of the target vehicle in the vehicle driving scenario, the second training history trajectory of other vehicles, and the training traffic environment information; based on the encoding module to be trained, extracting features and interactively learning the preprocessed first training history trajectory, the second training history trajectory, and the training traffic environment information to obtain the target vehicle training trajectory features, the surrounding vehicle training high-level trajectory features and the environment training high-level features related to the target vehicle; inputting the target vehicle training trajectory features, the surrounding vehicle training high-level trajectory features and the environment training high-level features into the prediction decoding module to be trained for decoding and prediction processing to obtain the training prediction acceleration corresponding to the current moment; the above training prediction acceleration, the current position and current speed of the above target vehicle are decoded and predicted. The prediction results of the training trajectory of the preset time step in the future are obtained by inputting the prediction trajectory into the kinematic simulation model for iterative motion simulation; wherein, during the iterative motion simulation process, the predicted speed and predicted position of a single time step are iteratively input into the encoding module for re-encoding to obtain the latest training trajectory characteristics of the target vehicle; the latest training trajectory characteristics continue to be input into the prediction decoding module and the training prediction acceleration of the next moment is obtained; multiple rounds are iterated until the simulation of the preset time step is completed; the actual trajectory corresponding to the target vehicle at the preset time step in the future is used as the training label, and the encoding module and the prediction decoding module are synchronously trained; a trajectory prediction model is constructed based on the preprocessing module, the trained encoding module, the trained prediction decoding module and the kinematic simulation model.

[0014] In some embodiments, the method further includes: performing multimodal processing in the encoding module, mapping the target vehicle training trajectory features, the surrounding vehicle training advanced trajectory features related to the target vehicle, and the environment training advanced features to obtain multiple modal results; wherein, after decoding and predicting processing and iterative motion simulation are performed by the pre-built prediction decoding module, a plurality of training trajectory prediction results corresponding to each modal result are obtained; for each training trajectory prediction result, re-encoding is performed based on the encoding module to obtain the training prediction trajectory features corresponding to each modality; the training prediction trajectory features, the surrounding vehicle training advanced trajectory features, and the environment training advanced features corresponding to each modality are input into the probability decoding network to be trained, and the training trajectory score of the current modality is output; the training trajectory scores of all modalities are input into the soft maximization function to obtain the probability values ​​of the various training trajectory prediction results; the training trajectory prediction result with the largest probability value is output as the final training prediction trajectory. wherein, the loss function is determined by the gap between the training prediction trajectory and the true trajectory, and the encoding module, the prediction decoding module, and the probability decoding network are trained synchronously; the trajectory prediction model also includes a trained probability decoding network.

[0015] In the third aspect, the embodiments of the present disclosure provide a vehicle trajectory prediction method based on a trajectory prediction model. The above method includes: obtaining the first historical trajectory of the target vehicle, the second historical trajectory of other vehicles and traffic environment information in the vehicle driving scene; using the above first historical trajectory, the above second historical trajectory and traffic environment information as inputs of the trajectory prediction model, and outputting the predicted trajectory of the above target vehicle; wherein the above trajectory prediction model is a model constructed by the method provided in the above second aspect embodiment.

[0016] In a fourth aspect, an embodiment of the present disclosure provides a trajectory prediction device that integrates kinematics and environmental cognition. The above-mentioned device includes: a preprocessing module, a feature extraction module, an acceleration prediction module and a kinematic simulation module. The above-mentioned preprocessing module is used to preprocess the first historical trajectory of the target vehicle, the second historical trajectory of other vehicles and the traffic environment information in the vehicle driving scene. The above-mentioned feature extraction module is used to perform feature extraction and interactive learning on the preprocessed first historical trajectory, the second historical trajectory and the traffic environment information based on a pre-constructed encoding module, and obtain the target vehicle trajectory features, the surrounding vehicle high-level trajectory features and the environment high-level features related to the target vehicle. The above-mentioned acceleration prediction module is used to input the target vehicle trajectory features, the surrounding vehicle high-level trajectory features and the environment high-level features into a pre-constructed prediction decoding module for decoding and prediction processing to obtain the predicted acceleration corresponding to the current moment. The kinematic simulation module is used to input the predicted acceleration, the current position and the current speed of the target vehicle into the kinematic simulation model for iterative motion simulation to obtain the trajectory prediction result of the future preset time step; wherein, during the iterative motion simulation process, the predicted speed and predicted position of a single time step are iteratively input into the encoding module for re-encoding to obtain the latest trajectory characteristics of the target vehicle; the latest trajectory characteristics continue to serve as the input of the prediction decoding module and obtain the predicted acceleration at the next moment; multiple rounds are iterated until the simulation of the preset time step is completed.

[0017] In some embodiments, the feature extraction module includes a multimodal processing module; the device also includes: a multimodal prediction module. The multimodal processing module is used to map the target vehicle trajectory features, the high-level trajectory features of the surrounding vehicles related to the target vehicle, and the high-level features of the environment to obtain multiple modal results; wherein, after decoding and predicting processing and iterative motion simulation by a pre-built prediction decoding module, a plurality of trajectory prediction results corresponding to each modal result are obtained; for each trajectory prediction result, re-encoding is performed based on the encoding module to obtain the predicted trajectory features corresponding to each modality. The multimodal prediction module is used to input the predicted trajectory features, the high-level trajectory features of the surrounding vehicles, and the high-level features of the environment corresponding to each modality into a pre-built probability decoding network, and output the trajectory score of the current modality; the trajectory scores of all modalities are input into the soft maximization function to obtain the probability values ​​of multiple trajectory prediction results; the trajectory prediction result with the largest probability value is output as the final predicted trajectory.

[0018] In a fifth aspect, an embodiment of the present disclosure provides a device for constructing a trajectory prediction model that integrates kinematics and environmental cognition. The device includes: a training preprocessing module, a training feature extraction module, a training acceleration prediction module, a training kinematics simulation module, and a prediction model construction module. The training preprocessing module is used to preprocess the first training history trajectory of the target vehicle, the second training history trajectory of other vehicles, and the training traffic environment information in the vehicle driving scene. The training feature extraction module is used to extract features and interactively learn the preprocessed first training history trajectory, the second training history trajectory, and the training traffic environment information based on the encoding module to be trained, and obtain the target vehicle training trajectory features, the surrounding vehicle training high-level trajectory features related to the target vehicle, and the environment training high-level features. The training acceleration prediction module is used to input the target vehicle training trajectory features, the surrounding vehicle training high-level trajectory features, and the environment training high-level features into the prediction decoding module to be trained for decoding and prediction processing, and obtain the training prediction acceleration corresponding to the current moment. The above-mentioned training kinematic simulation module is used to input the above-mentioned training predicted acceleration, the current position and current speed of the above-mentioned target vehicle into the kinematic simulation model for iterative motion simulation, and obtain the training trajectory prediction result of the future preset time step. Wherein, during the iterative motion simulation process, the predicted speed and predicted position of a single time step are iteratively input into the above-mentioned encoding module for re-encoding to obtain the latest training trajectory characteristics of the target vehicle; the above-mentioned latest training trajectory characteristics continue to be used as the input of the above-mentioned prediction decoding module and obtain the training predicted acceleration at the next moment; iterate multiple rounds until the simulation of the preset time step is completed. The above-mentioned prediction model construction module is used to use the actual trajectory corresponding to the above-mentioned target vehicle at the above-mentioned future preset time step as the training label, and synchronously train the above-mentioned encoding module and the above-mentioned prediction decoding module; the trajectory prediction model is constructed based on the above-mentioned preprocessing module, the trained encoding module, the trained prediction decoding module and the above-mentioned kinematic simulation model.

[0019] In some embodiments, the training feature extraction module includes a multimodal processing module; the device also includes: a multimodal prediction module. The multimodal processing module is used to map the target vehicle training trajectory features, the surrounding vehicle training high-level trajectory features related to the target vehicle, and the environment training high-level features to obtain multiple modal results; wherein, after decoding and predicting processing and iterative motion simulation are performed by a pre-built prediction decoding module, a plurality of training trajectory prediction results corresponding to each modal result are obtained. For each training trajectory prediction result, re-encoding is performed based on the encoding module to obtain the training prediction trajectory features corresponding to each modality. The multimodal prediction module is used to input the training prediction trajectory features, the surrounding vehicle training high-level trajectory features, and the environment training high-level features corresponding to each modality into the probability decoding network to be trained, and output the training trajectory score of the current modality; input the training trajectory scores of all modalities into the soft maximization function to obtain the probability values ​​of multiple training trajectory prediction results; and output the training trajectory prediction result with the largest probability value as the final training prediction trajectory. The loss function is determined by the gap between the training prediction trajectory and the real trajectory, and the encoding module, the prediction decoding module and the probability decoding network are trained synchronously; the trajectory prediction model also includes a trained probability decoding network.

[0020] In the sixth aspect, an embodiment of the present disclosure provides a vehicle trajectory prediction device based on a trajectory prediction model. The above-mentioned device includes: a data acquisition module and a prediction processing module. The above-mentioned data acquisition module is used to obtain the first historical trajectory of the target vehicle, the second historical trajectory of other vehicles and traffic environment information in the vehicle driving scene. The above-mentioned prediction processing module is used to use the above-mentioned first historical trajectory, the above-mentioned second historical trajectory and the traffic environment information as inputs of the trajectory prediction model, and outputs the predicted trajectory of the above-mentioned target vehicle; wherein the above-mentioned trajectory prediction model is a model constructed by the method provided in the embodiment of the second aspect, or a model constructed by the device provided in the embodiment of the fifth aspect.

[0021] In the seventh aspect, an embodiment of the present disclosure provides an electronic device. The electronic device includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory is used to store computer programs; the processor is used to implement the trajectory prediction method for integrating kinematics and environmental cognition provided in the first aspect embodiment, the method for constructing a trajectory prediction model for integrating kinematics and environmental cognition provided in the second aspect embodiment, or the vehicle trajectory prediction method based on the trajectory prediction model provided in the third aspect embodiment when executing the program stored in the memory.

[0022] In an eighth aspect, an embodiment of the present disclosure provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the trajectory prediction method for integrating kinematics and environmental cognition provided in the first aspect embodiment, the method for constructing a trajectory prediction model for integrating kinematics and environmental cognition provided in the second aspect embodiment, or the vehicle trajectory prediction method based on the trajectory prediction model provided in the third aspect embodiment is implemented.

[0023] The above technical solution provided by the embodiments of the present disclosure has at least some or all of the following advantages:

[0024] By extracting and interactively learning the features of the first historical trajectory, the second historical trajectory and the traffic environment information after preprocessing, the trajectory features of the target vehicle, the high-level trajectory features of the surrounding vehicles related to the target vehicle and the high-level features of the environment are obtained; and based on this, the acceleration is predicted, and the motion trajectory of the target vehicle can be predicted from the perspective of the self-driving of the target vehicle, the perspective of the target vehicle being affected by the surrounding vehicles and the perspective of the target vehicle being affected by the traffic environment information. It has good interpretability and also enhances the model's ability to understand and process traffic scenes, ensuring the expected trajectory prediction performance. In addition, in the process of using the predicted acceleration input to the kinematic simulation model for iterative motion simulation, based on the autoregressive encoding method, the predicted speed and predicted position of a single time step are iteratively input to the above encoding module for re-encoding to obtain the latest trajectory features of the target vehicle; the above latest trajectory features continue to be used as the input of the above prediction decoding module and the predicted acceleration at the next moment is obtained; iterate multiple rounds until the simulation of the preset time step is completed; in this way, the acceleration and trajectory can be predicted and iterated from the perspective of step-by-step simulation from the kinematic perspective, which helps to further improve the accuracy of trajectory prediction while ensuring interpretability. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0026] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0027] Figure 1 The flowchart of the trajectory prediction method integrating kinematics and environmental cognition according to an embodiment of the present disclosure is schematically shown;

[0028] Figure 2The detailed implementation flow chart of step S120 according to an embodiment of the present disclosure is schematically shown;

[0029] Figure 3 The structure block diagram and processing diagram of the target vehicle trajectory encoding module according to an embodiment of the present disclosure are schematically shown;

[0030] Figure 4 A schematic diagram schematically shows a process of processing the pre-processed second historical trajectory based on the encoding module according to an embodiment of the present disclosure;

[0031] Figure 5 A schematic diagram schematically shows a process of processing the pre-processed traffic environment information based on the encoding module according to an embodiment of the present disclosure;

[0032] Figure 6 The detailed implementation process of the trajectory prediction method integrating kinematics and environmental cognition according to an embodiment of the present disclosure is schematically shown;

[0033] Figure 7 The structural block diagram of the electronic device provided by the embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present disclosure.

[0035] A first exemplary embodiment of the present disclosure provides a trajectory prediction method integrating kinematics and environmental cognition.

[0036] Figure 1 The flowchart of a trajectory prediction method integrating kinematics and environmental cognition according to an embodiment of the present disclosure is schematically shown.

[0037] Reference Figure 1 As shown, the trajectory prediction method integrating kinematics and environmental cognition provided by the embodiment of the present disclosure includes the following steps: S110, S120, S130 and S140.

[0038] In step S110, the first historical trajectory of the target vehicle, the second historical trajectories of other vehicles and the traffic environment information in the vehicle driving scene are preprocessed.

[0039] In some embodiments, the scene acquisition data in the vehicle driving scene can be obtained by drone aerial photography, data collection and aggregation based on roadside equipment, or vehicle sensor (at least one of radar detection or visual perception). The above scene acquisition data can be video data, or a sequence of image data collected at intervals. The first historical trajectory of the target vehicle, the second historical trajectory of other vehicles in the scene, and traffic environment information can be obtained from the above scene acquisition data. In some embodiments, the target vehicle, other vehicles, and traffic environment can be located within a preset scene range.

[0040] In the above vehicle driving scene, the preset scene range can be: with the above target vehicle as the center, vehicles within a preset size range around the above target vehicle as other vehicles, and traffic elements within a preset size range around the above target vehicle as the traffic environment. The vehicle trajectory information, traffic environment information, etc. collected in the above vehicle driving scene are all based on information in the global coordinate system.

[0041] The above-mentioned surrounding preset size range can be a rectangular range of preset sizes corresponding to the four directions of the target vehicle; or, it can be a circular range with the target vehicle as the center and the preset size as the radius; or, it can be a preset scene space divided according to regular or irregular shapes and corresponding sizes.

[0042] The duration of the first historical trajectory and the second historical trajectory is in the second level, which can generally be a collection duration corresponding to 3s to 5s. In some embodiments, the trajectory sampling frequency is 10Hz (for example, a data frame is collected every 0.1s to obtain a position point, and the information of the position point includes position and speed). The data of each frame includes: timestamp, x-coordinate, y-coordinate, x-axis speed and y-axis speed. The prediction duration corresponding to the predicted trajectory to be predicted is also a short-term prediction in the second level. Generally, the trajectory of the next 2s to 5s can be predicted. As an example, the trajectory of the target vehicle in the next 3s is predicted.

[0043] In some embodiments, all lanes are segmented according to the center lines of each lane to obtain lane segments containing a preset number (e.g., 10 to 15) of lane center points; by segmenting the lane lines (e.g., segmenting according to preset intervals), each lane is divided into a sequence of lane center line data points, which helps to improve the convenience of subsequent feature extraction and lane representation and the efficiency of data processing. For some lanes whose length is less than the preset number of segmentation points, the missing data points are filled with zeros.

[0044] It should be noted that the duration of the historical trajectory used in the model building phase (or described as the training phase) is the same as the duration of the historical trajectory used in the model use phase.

[0045] In some embodiments, the above-mentioned traffic environment includes one of the following combinations: lanes, lanes and traffic lights, lanes and roadblocks; the above-mentioned traffic environment information includes one of the following combinations: lane information, lane information and traffic light information, lane information and roadblock information.

[0046] When the traffic environment includes lanes and traffic lights, the lane information can be represented by a sequence of lane centerline data points, and the traffic light information can be represented by the following conditional expressions: allowing vehicles in the first lane to continue moving forward and pass (for example, a green light on one or more lanes at a certain intersection, indicating that they can go straight or turn left), prohibiting vehicles in the second lane from exceeding a preset indicator line (for example, a red light on one or more lanes at a certain intersection, indicating that they need to stop and wait to go straight or turn left), etc.

[0047] When the traffic environment includes lanes and roadblocks, the lane information can be represented by a sequence of lane centerline data points, and the roadblock information can be represented by the following conditional expressions: the first area section is prohibited from passing, the second area section requires a detour, etc.

[0048] In some embodiments, the first historical trajectory, the second historical trajectory, and the traffic environment information are all located in a global coordinate system.

[0049] In some embodiments, the step S110 specifically includes: performing coordinate conversion processing on the first historical trajectory, the second historical trajectory and the traffic environment information to convert them into representation results in the reference coordinate system corresponding to the observation angle of the target vehicle. By performing coordinate conversion processing, it is ensured that the model can learn trajectory prediction capabilities with translation and rotation invariance under different initial positions and directions.

[0050] Among them, the current position of the target vehicle is taken as the coordinate origin, the displacement direction of the current timestamp of the target vehicle compared to the previous timestamp is taken as the positive direction of the first direction (for example, the y-axis), and the first direction and the second direction perpendicular to the first direction (for example, the x-axis) are constructed as the reference coordinate system. In some embodiments, the current moving direction of the target vehicle is taken as the positive direction of the y-axis, the current moving direction of the target vehicle can be represented by the displacement direction of the current timestamp compared to the previous timestamp, and the direction perpendicular to the y-axis is taken as the y-axis direction, thereby constructing a reference coordinate system.

[0051] The preprocessed first historical trajectory, the second historical trajectory and the traffic environment information are representation results in a reference coordinate system.

[0052] In some embodiments, using Indicates the target vehicle in the reference coordinate system from -T h The first historical track from moment to the present moment; It represents the second historical trajectory of other vehicles located around the target vehicle at the current moment in the reference coordinate system; E represents the traffic environment information in the reference coordinate system. Indicates that the target vehicle is T after the current time f The future prediction trajectory of Indicates that the target vehicle is at T f The predicted position at the time.

[0053] In step S120, based on the pre-built encoding module, feature extraction and interactive learning are performed on the pre-processed first historical trajectory, the second historical trajectory and the traffic environment information to obtain the target vehicle trajectory features, the surrounding vehicle high-level trajectory features related to the target vehicle and the environment high-level features.

[0054] Figure 2 The detailed implementation flow chart of step S120 according to an embodiment of the present disclosure is schematically shown.

[0055] In some embodiments, reference Figure 2 As shown, the encoding module 200 includes: a target vehicle trajectory encoding module 210 , a surrounding vehicle trajectory encoding module 220 , a traffic environment encoding module 230 , a first cross-learning module 240 and a second cross-learning module 250 .

[0056] In some embodiments, the above step S120 specifically includes the following steps: S210, S220, S230, S240 and S250.

[0057] In step S210, the preprocessed first historical trajectory And the predicted trajectory p under the iteration step size t 1:t Constructing the splicing vector The above-mentioned spliced ​​vector is input into the above-mentioned target vehicle trajectory encoding module 210 for feature extraction processing (for example, based on the multi-layer perceptron MLP mapping to the high-level feature space), and the target vehicle trajectory feature is output. In the initial state, the above-mentioned predicted trajectory is an empty vector; in the iterative motion simulation process, the predicted trajectory under the above-mentioned iterative step is the result of step accumulation of the predicted position and predicted speed of a single time step corresponding to the output of each iteration round.

[0058] Figure 3The structural block diagram and processing process diagram of a target vehicle trajectory encoding module according to an embodiment of the present disclosure are schematically shown.

[0059] Reference Figure 3 As shown, the target vehicle trajectory encoding module 210 includes: a first multi-layer perceptron network 211, a long short-term memory network 212, and a second multi-layer perceptron network 213 connected in sequence, wherein the first multi-layer perceptron network (MLP) 211 is used to map each trajectory point in the input concatenated vector to a high-dimensional feature space to obtain a coded feature vector; the long short-term memory network (LSTM) 212 is based on the cell state c at the previous moment. t-1 With the hidden state h t-1 , update the cell state and hidden state through the forget gate, input gate and output gate, capture the long time series features, and finally output the latest trajectory feature vector h containing the complete sequence information t ; Then the second multi-layer perceptron network 213 performs feature extraction processing, and outputs the target vehicle trajectory feature

[0060] The input shape of the above-mentioned target vehicle trajectory encoding module 210 is (20+t,5), where the 20 in the brackets represents the number of historical trajectory frames of the target vehicle, and t represents the number of predicted trajectory frames. Each frame of trajectory data includes timestamp, x-axis coordinate, y-axis coordinate, x-axis speed, and y-axis speed, a total of 5 dimensions, so the 5 in the brackets represents the corresponding dimension. The output is a 128-dimensional feature vector. The selection of LSTM network to encode the historical trajectory of the target vehicle is mainly based on the following two important reasons: First, the LSTM network can accept historical trajectories of different lengths as input in actual scenarios, providing greater flexibility for the algorithm; secondly, the LSTM network can receive the latest trajectory position and speed information predicted by the algorithm, and re-encode the target vehicle characteristics as input for predicting the acceleration at the next moment, so that the accuracy of the predicted trajectory is further improved.

[0061] In the embodiment of the present disclosure, a multi-layer perceptron network (MLP) is used to encode the historical trajectory of the target vehicle, and the trajectory points are first mapped to a high-dimensional feature space. Then, the latest trajectory features are output through a long short-term memory network (LSTM) combined with a memory unit containing historical trajectory information, and the memory unit is updated at the same time. Since LSTM can effectively capture long-term series information, it can ensure that the dynamic change characteristics of the historical trajectory of the target vehicle are preserved and expressed.

[0062] In step S220, the preprocessed second historical trajectory is input into the surrounding vehicle trajectory encoding module 220, and features are extracted from each trajectory point in the second historical trajectory of each other vehicle, and the global trajectory features of each other vehicle are obtained based on self-attention learning between each trajectory point; then, the global trajectory features between other vehicles are interactively learned to obtain the trajectory features of surrounding vehicles.

[0063] Figure 4 The figure schematically shows a process diagram of processing the preprocessed second historical trajectory based on the encoding module according to an embodiment of the present disclosure.

[0064] Reference Figure 4 As shown, the preprocessed second historical trajectory Input to the encoding module, specifically input to the surrounding vehicle trajectory encoding module 220 for processing. The surrounding vehicle trajectory encoding module 220 includes: a third multi-layer perceptron network 221, a first self-attention network 222, a first maximum pooling layer 223 and a second self-attention network 224. First, based on the third multi-layer perceptron network 221, the historical trajectory points of all other surrounding vehicles are preliminarily encoded. Then, based on the first self-attention network 222 (Self-Attention Network), the interaction information between each trajectory point in the historical trajectory of the same other vehicle is learned. Due to the learning of the self-attention mechanism, each trajectory point in the historical trajectory of the same other vehicle can pay attention to other trajectory points, thereby capturing and learning the relationship between them. Subsequently, after the first maximum pooling layer (MaxPooling) 223 performs maximum pooling processing, the feature vector corresponding to the second historical trajectory of each other vehicle is obtained, that is, the global trajectory feature obtained based on self-attention learning between each trajectory point. In order to further simulate the global interaction between other vehicles, the interaction learning between other vehicles is carried out based on the second self-attention network 224 to obtain the surrounding vehicle trajectory feature f neigh , which allows a comprehensive understanding of the overall dynamic behavior of surrounding vehicles.

[0065] The input shape of the surrounding vehicle trajectory encoding module 220 is (n neighs ,20,5), where n neighs Indicates the total number of vehicles around the target vehicle, 20 is the number of historical trajectory frames, and 5 is the dimension corresponding to the specific trajectory point data.

[0066] In step S230, the target vehicle trajectory features and the surrounding vehicle trajectory features are input into the first cross-learning module 240 for cross-attention learning to obtain high-level trajectory features of the surrounding vehicles related to the target vehicle.

[0067] Continue to refer to Figure 4As shown, in the initial state (t has no value, that is, when t=0, when the predicted acceleration is obtained and the autoregressive iteration is performed, the value of t is a positive integer), the above-mentioned predicted trajectory is an empty vector, and the first historical trajectory of the target vehicle is input into the target vehicle trajectory encoding module 210 to obtain the target vehicle trajectory feature: The target vehicle trajectory characteristics at the initial moment and surrounding vehicle trajectory features f neigh The target vehicle trajectory characteristics at the initial moment are input to the first cross-learning module 240 for cross-learning. As the query vector q, the surrounding vehicle trajectory features f neigh As the key vector k and the value vector v, we get the high-level trajectory features f of the surrounding vehicles related to the target vehicle. neighs The high-level trajectory features of the surrounding vehicles f neighs The corresponding dimension is 128.

[0068] In step S240, the preprocessed traffic environment information is input into the above-mentioned traffic environment encoding module to perform feature extraction of local environments, internal interaction learning of each local environment, and global interaction learning between multiple local environments to obtain global environmental features.

[0069] Figure 5 The figure schematically shows a process diagram of processing the preprocessed traffic environment information based on the encoding module according to an embodiment of the present disclosure.

[0070] Reference Figure 5 As shown, the preprocessed traffic environment information E is input into the encoding module, specifically into the traffic environment encoding module 230 for processing. The traffic environment encoding module 230 includes: a fourth multi-layer perceptron network 231, a third self-attention network 232, a second maximum pooling layer 233 and a fourth self-attention network 234.

[0071] Taking the example that the traffic environment information includes lane information, each lane in the multiple lanes is regarded as a local environment. In other embodiments, when the traffic environment information includes both lane information and traffic light information, or both lane information and roadblock information, the local environment can be in the form of a combination of each lane and the light corresponding to each lane; or the local environment can be in the form of a combination of each lane and whether there is a roadblock.

[0072] Taking lane information as an example, firstly, based on the fourth multi-layer perceptron network 231, each data point of the lane centerline in the traffic environment information E is preliminarily encoded. Then, based on the third self-attention network 232, the interaction information between each data point in the same lane is learned. Due to the learning of the self-attention mechanism, each data point in the same lane can pay attention to other data points, thereby capturing and learning the relationship between them. Subsequently, the second maximum pooling layer 233 performs maximum pooling processing to obtain the feature vector corresponding to the centerline of each lane, which is the global feature of the lane obtained based on self-attention learning between each data point. In order to further simulate the global interaction between each lane, the fourth self-attention network 234 is used to perform interactive learning between different lanes to obtain the lane global feature f lane , corresponding to examples as global features of the environment.

[0073] The input shape of the traffic environment encoding module 230 is (n lanes ,10,2), where n lanes represents the total number of lanes around the target vehicle, 10 represents the total number of data points contained in the center line of each lane, and 2 represents the dimension of each data point, including the x-axis coordinate and y-axis coordinate of the lane center point.

[0074] In step S250, the target vehicle trajectory features and the global environment features are input into the second cross-learning module for cross-attention learning to obtain high-level features of the environment related to the target vehicle.

[0075] Continue to refer to Figure 5 As shown, the target vehicle trajectory characteristics at the initial moment and lane global feature f lane (as an example of global environmental features) is input to the second cross-learning module 250 for cross-attention learning to obtain high-level environmental features f related to the target vehicle. map The advanced features of this environment map The corresponding dimension is 128.

[0076] In this embodiment, based on steps S210 to S250, feature coding processing of the first historical trajectory of the target vehicle, the second historical trajectory of other vehicles and traffic environment information is implemented, and feature extraction can be performed from the perspective of the target vehicle and in combination with the attention paid to surrounding vehicles and the traffic environment during human driving, thereby improving the interpretability of the model building and prediction process, and enabling the target vehicle to perceive and understand the surrounding environment more completely, thereby improving the accuracy of the prediction.

[0077] In step S130, the target vehicle trajectory features, the surrounding vehicle high-level trajectory features and the environment high-level features are input into a pre-built prediction decoding module for decoding and prediction processing to obtain the predicted acceleration corresponding to the current moment.

[0078] Figure 6 The detailed implementation process diagram of the trajectory prediction method integrating kinematics and environmental cognition according to an embodiment of the present disclosure is schematically shown.

[0079] In some embodiments, reference Figure 6 As shown, the above prediction decoding module includes: a first decoding module 610 , a second decoding module 620 and a third decoding module 630 .

[0080] Combination Figure 2 and Figure 6 It can be seen that after the encoding module 200 performs processing, three outputs are obtained: target vehicle trajectory features, surrounding vehicle high-level trajectory features and environment high-level features; Figure 6 Only the target vehicle trajectory encoding module 210, the surrounding vehicle trajectory encoding module 220 and the traffic environment encoding module 230 are shown in FIG. Figure 6 The above step S130 specifically includes the following steps:

[0081] The target vehicle trajectory characteristics and target point characteristics are input into the first decoding module 610, and the first predicted acceleration a generated by the vehicle self-driving effect is output. self ;

[0082] The target vehicle trajectory features and the surrounding vehicle high-level trajectory features are input into the second decoding module 620, and the second predicted acceleration a generated by the surrounding vehicles affecting the target vehicle is output. neighs ;

[0083] The target vehicle trajectory characteristics and the high-level environmental characteristics are input into the third decoding module 630, and the third predicted acceleration a generated by the traffic environment affecting the target vehicle is obtained as an output. map .

[0084] The target point feature is a feature corresponding to the predicted target point position obtained by predicting the target vehicle trajectory feature and the high-level environmental feature. In some embodiments, the method further includes a step of generating target point features, specifically: inputting the target vehicle trajectory feature and the high-level environmental feature into a pre-built target point encoding module for target point prediction processing, and obtaining the target point feature corresponding to the predicted target point position.

[0085] For example, refer to Figure 6As shown, the target vehicle trajectory features output by the target vehicle trajectory encoding module 210 and the high-level environmental features obtained by cross-learning by the traffic environment encoding module 230 are input to the target point encoding module 601 for target point prediction processing, and the target point features corresponding to the predicted target point position are obtained. Then, refer to Figure 6 The target point feature and the target vehicle trajectory feature output by the target vehicle trajectory encoding module 210 are input into the first decoding module 610 for acceleration prediction to obtain the first predicted acceleration a generated by the vehicle self-driving action. self .

[0086] In some embodiments, the predicted acceleration a t The first predicted acceleration a generated by the self-driving action of the target vehicle is obtained by summing or weighting the following three accelerations: self , the second predicted acceleration a generated by the surrounding vehicles affecting the target vehicle neighs and the third predicted acceleration a generated by the traffic environment affecting the target vehicle map .

[0087] In step S140, the predicted acceleration, the current position and the current speed of the target vehicle are input into a kinematic simulation model for iterative motion simulation to obtain a trajectory prediction result of a future preset time step.

[0088] Reference Figure 6 As shown in the figure, a partial scene screenshot is used to illustrate the example. The dotted line in the center is used as an example of a lane line; the solid arrows indicate the historical trajectory of the target vehicle; and the dotted arrows indicate the historical trajectory of the surrounding vehicles. t , the current position and current speed of the target vehicle are input into the kinematic simulation model 640 for iterative motion simulation to obtain the trajectory prediction result of the future preset time step; for example, the trajectory prediction result of the future preset time step is as follows Figure 6 The circled portion in the figure may be the predicted trajectory for going straight or the predicted trajectory for turning right.

[0089] In some embodiments, the kinematic simulation model may be a uniform acceleration motion model or a motion model corresponding to other preset functions. As long as the specific functional relationship is determined, the acceleration, current position and current speed may be substituted into the functional relationship and the predicted speed and predicted position at the future moment may be output.

[0090] After the prediction decoding module outputs the latest vehicle position and speed, in order to ensure that the predicted acceleration is closely related to the trajectory data at the previous moment, in the solution provided by the embodiment of the present disclosure, the predicted latest vehicle position and speed information is returned to the target vehicle trajectory encoding module to obtain the latest target vehicle trajectory features and serve as the input features for predicting the acceleration at the next moment. This autoregressive encoding method can provide more complete historical information for acceleration prediction, which can further improve the accuracy of trajectory prediction.

[0091] The prediction decoding module can predict the future trajectory of 0.1s time step each time. After predicting multiple time steps, the complete future trajectory can be obtained.

[0092] Among them, refer to Figure 6 The iterative cycle process indicated by the double arrows in the figure illustrates that, during the iterative motion simulation process, the predicted speed and predicted position of a single time step are iteratively input to the encoding module 200 for re-encoding to obtain the latest trajectory characteristics of the target vehicle; the latest trajectory characteristics continue to be input to the prediction decoding module and the predicted acceleration at the next moment is obtained; multiple step-length iterations are performed in this way, and multiple rounds of iterations are repeated until the simulation of the preset time step is completed.

[0093] In some embodiments, the trajectory prediction result of the future preset time step can be output as the predicted trajectory.

[0094] In other embodiments, since the vehicle trajectory is multimodal, the same historical trajectory and environment may have multiple possible future trajectories. In order to provide complete prediction information for the planning module in autonomous driving, it is also necessary to have the ability to predict all potential future trajectories. At the same time, for multiple future predicted trajectories, their probability distribution must also be predicted, so as to provide more complete future information for the downstream planning system of autonomous driving.

[0095] For example, in some embodiments, the above method also includes: performing multimodal processing in the encoding module, mapping the target vehicle trajectory characteristics, the surrounding vehicle high-level trajectory characteristics related to the target vehicle, and the environmental high-level characteristics to obtain multiple modal results; wherein, after decoding prediction processing and iterative motion simulation are performed through a pre-built prediction decoding module, a plurality of trajectory prediction results corresponding to each modal result are obtained.

[0096] For each trajectory prediction result, re-encoding is performed based on the above encoding module to obtain the predicted trajectory features corresponding to each mode; the predicted trajectory features corresponding to each mode, the high-level trajectory features of surrounding vehicles and the high-level features of the environment are input into the pre-built probability decoding network, and the trajectory score of the current mode is output; the trajectory scores of all modes are input into the softmax function to obtain the probability values ​​of multiple trajectory prediction results; the trajectory prediction result with the largest probability value is output as the final predicted trajectory.

[0097] In the embodiment of the present disclosure, multimodal processing is also implemented in the encoding module 260. For example, a fully connected linear network layer 260 is introduced in the surrounding vehicle trajectory encoding module 220 and the traffic environment encoding module 230. Figure 4 and Figure 5 For simplicity, the fully connected linear network is referred to as the linear layer. Based on this fully connected linear network layer 260, the encoded high-level trajectory features of surrounding vehicles f neighs And the high-level characteristics of the environment map Map from the initial (128) dimension to the (K, 128) dimensional space, where K is the number of multimodal modes. In the embodiment of the present disclosure, the value K=6 can satisfy the prediction of each different mode. The prediction decoding module will make predictions based on these K groups of different feature vectors and output K trajectory prediction results. By predicting the corresponding probability values ​​of multiple trajectory prediction results; the trajectory prediction result with the largest probability value is output as the final predicted trajectory.

[0098] In the embodiment including the above steps S110 to S140, by extracting and interactively learning the features of the preprocessed first historical trajectory, the second historical trajectory and the traffic environment information, the target vehicle trajectory features, the high-level trajectory features of the surrounding vehicles related to the target vehicle and the high-level features of the environment are obtained; and based on this, the acceleration is predicted, and the motion trajectory of the target vehicle can be predicted from the perspective of the self-driving of the target vehicle, the angle of the target vehicle being affected by the surrounding vehicles and the angle of the target vehicle being affected by the traffic environment information. It has good interpretability and also enhances the model's ability to understand and process traffic scenes, ensuring the expected trajectory prediction performance. In addition, in the process of using the predicted acceleration input to the kinematic simulation model for iterative motion simulation, based on the autoregressive encoding method, the predicted speed and predicted position of a single time step are iteratively input to the above encoding module for re-encoding to obtain the latest trajectory features of the target vehicle; the above latest trajectory features continue to be used as the input of the above prediction decoding module and the predicted acceleration at the next moment is obtained; iterate multiple rounds until the simulation of the preset time step is completed; in this way, the acceleration and trajectory can be predicted and iterated from the perspective of kinematic step-by-step simulation, which helps to further improve the accuracy of trajectory prediction while ensuring interpretability.

[0099] A second exemplary embodiment of the present disclosure provides a method for constructing a trajectory prediction model that integrates kinematics and environmental cognition.

[0100] The method for constructing the trajectory prediction model integrating kinematics and environmental cognition includes:

[0101] Based on the preprocessing module, preprocessing is performed on the first training history trajectory of the target vehicle in the vehicle driving scene, the second training history trajectory of other vehicles, and the training traffic environment information;

[0102] Based on the encoding module to be trained, feature extraction and interactive learning are performed on the preprocessed first training history trajectory, the second training history trajectory and the training traffic environment information to obtain the target vehicle training trajectory feature, the surrounding vehicle training high-level trajectory feature related to the target vehicle and the environment training high-level feature;

[0103] Input the target vehicle training trajectory features, the surrounding vehicle training high-level trajectory features and the environment training high-level features into the prediction decoding module to be trained for decoding and prediction processing to obtain the training prediction acceleration corresponding to the current moment;

[0104] The training predicted acceleration, the current position and the current speed of the target vehicle are input into the kinematic simulation model for iterative motion simulation to obtain the training trajectory prediction result of the future preset time step; wherein, during the iterative motion simulation process, the predicted speed and predicted position of a single time step are iteratively input into the encoding module for re-encoding to obtain the latest training trajectory characteristics of the target vehicle; the latest training trajectory characteristics continue to be used as the input of the prediction decoding module to obtain the training predicted acceleration at the next moment; multiple rounds are iterated until the simulation of the preset time step is completed;

[0105] Using the actual trajectory of the target vehicle at the preset future time step as a training label, the encoding module and the prediction decoding module are synchronously trained;

[0106] A trajectory prediction model is constructed based on the preprocessing module, the trained encoding module, the trained prediction decoding module and the kinematic simulation model.

[0107] In some embodiments, the above method further comprises:

[0108] In the encoding module, multimodal processing is performed, and the target vehicle training trajectory features, the surrounding vehicle training advanced trajectory features related to the target vehicle, and the environment training advanced features are mapped to obtain multiple modal results; wherein, after decoding and predicting processing and iterative motion simulation are performed through the pre-built prediction decoding module, multiple training trajectory prediction results corresponding to each modal result are obtained; for each training trajectory prediction result, re-encoding is performed based on the above encoding module to obtain the training prediction trajectory features corresponding to each modality; the training prediction trajectory features corresponding to each modality, the surrounding vehicle training advanced trajectory features, and the environment training advanced features are input into the probability decoding network to be trained, and the training trajectory score of the current modality is output; the training trajectory scores of all modalities are input into the soft maximization (softmax) function to obtain the probability values ​​of multiple training trajectory prediction results; the training trajectory prediction result with the largest probability value is output as the final training prediction trajectory. wherein, the loss function is determined by the gap between the above training prediction trajectory and the above true trajectory, and the above encoding module, the above prediction decoding module, and the above probability decoding network are synchronously trained; the above trajectory prediction model also includes a trained probability decoding network.

[0109] For more details and beneficial effects of this embodiment, please refer to the relevant description of the first embodiment, which will not be repeated here.

[0110] A third exemplary embodiment of the present disclosure provides a vehicle trajectory prediction method based on a trajectory prediction model.

[0111] The vehicle trajectory prediction method of this embodiment includes:

[0112] Obtaining a first historical trajectory of a target vehicle, a second historical trajectory of other vehicles, and traffic environment information in a vehicle driving scenario;

[0113] The first historical trajectory, the second historical trajectory and the traffic environment information are used as inputs of a trajectory prediction model, and the predicted trajectory of the target vehicle is output; wherein the trajectory prediction model is a model constructed by the method provided in the second embodiment.

[0114] For more details and beneficial effects of this embodiment, please refer to the relevant description of the first embodiment, which will not be repeated here.

[0115] A fourth exemplary embodiment of the present disclosure provides a trajectory prediction device that integrates kinematics and environmental cognition.

[0116] The device comprises: a preprocessing module, a feature extraction module, an acceleration prediction module and a kinematics simulation module.

[0117] The preprocessing module is used to preprocess the first historical trajectory of the target vehicle, the second historical trajectories of other vehicles and the traffic environment information in the vehicle driving scene.

[0118] The feature extraction module is used to perform feature extraction and interactive learning on the preprocessed first historical trajectory, the second historical trajectory and the traffic environment information based on the pre-built encoding module, so as to obtain the target vehicle trajectory features, the surrounding vehicle high-level trajectory features related to the target vehicle and the high-level environment features.

[0119] The acceleration prediction module is used to input the target vehicle trajectory features, the surrounding vehicle high-level trajectory features and the environment high-level features into a pre-built prediction decoding module for decoding and prediction processing to obtain the predicted acceleration corresponding to the current moment.

[0120] The kinematic simulation module is used to input the predicted acceleration, the current position and the current speed of the target vehicle into the kinematic simulation model for iterative motion simulation to obtain the trajectory prediction result of the future preset time step; wherein, during the iterative motion simulation process, the predicted speed and predicted position of a single time step are iteratively input into the encoding module for re-encoding to obtain the latest trajectory characteristics of the target vehicle; the latest trajectory characteristics continue to serve as the input of the prediction decoding module and obtain the predicted acceleration at the next moment; multiple rounds are iterated until the simulation of the preset time step is completed.

[0121] In some embodiments, the feature extraction module includes a multimodal processing module; the device also includes: a multimodal prediction module. The multimodal processing module is used to map the target vehicle trajectory features, the surrounding vehicle high-level trajectory features and the environment high-level features related to the target vehicle to obtain multiple modal results; wherein, after decoding and predicting processing and iterative motion simulation by a pre-built prediction decoding module, a plurality of trajectory prediction results corresponding to each modal result are obtained; for each trajectory prediction result, re-encoding is performed based on the encoding module to obtain the predicted trajectory features corresponding to each modality. The multimodal prediction module is used to input the predicted trajectory features, the surrounding vehicle high-level trajectory features and the environment high-level features corresponding to each modality into a pre-built probability decoding network, and output the trajectory score of the current modality; the trajectory scores of all modalities are input into the soft maximization (softmax) function to obtain the probability values ​​of multiple trajectory prediction results; the trajectory prediction result with the largest probability value is output as the final predicted trajectory.

[0122] For more details and beneficial effects of this embodiment, please refer to the relevant description of the first embodiment, which will not be repeated here.

[0123] A fifth exemplary embodiment of the present disclosure provides a device for constructing a trajectory prediction model that integrates kinematics and environmental cognition.

[0124] The above-mentioned device includes: a training preprocessing module, a training feature extraction module, a training acceleration prediction module, a training kinematics simulation module and a prediction model construction module.

[0125] The training preprocessing module is used to preprocess the first training history track of the target vehicle, the second training history track of other vehicles and the training traffic environment information in the vehicle driving scene.

[0126] The above-mentioned training feature extraction module is used to perform feature extraction and interactive learning on the preprocessed first training history trajectory, the second training history trajectory and the training traffic environment information based on the encoding module to be trained, so as to obtain the target vehicle training trajectory features, the surrounding vehicle training high-level trajectory features related to the target vehicle and the environment training high-level features.

[0127] The above-mentioned training acceleration prediction module is used to input the above-mentioned target vehicle training trajectory characteristics, the above-mentioned surrounding vehicle training high-level trajectory characteristics and the environment training high-level characteristics into the prediction decoding module to be trained for decoding and prediction processing to obtain the training prediction acceleration corresponding to the current moment.

[0128] The training kinematic simulation module is used to input the training predicted acceleration, the current position and the current speed of the target vehicle into the kinematic simulation model for iterative motion simulation to obtain the training trajectory prediction result of the preset time step in the future. In the iterative motion simulation process, the predicted speed and predicted position of a single time step are iteratively input into the encoding module for re-encoding to obtain the latest training trajectory characteristics of the target vehicle; the latest training trajectory characteristics continue to be used as the input of the prediction decoding module and obtain the training predicted acceleration at the next moment; iterate multiple rounds until the simulation of the preset time step is completed.

[0129] The prediction model building module is used to use the actual trajectory corresponding to the target vehicle at the future preset time step as a training label to synchronously train the encoding module and the prediction decoding module.

[0130] A trajectory prediction model is constructed based on the preprocessing module, the trained encoding module, the trained prediction decoding module and the kinematic simulation model.

[0131] In some embodiments, the training feature extraction module includes a multimodal processing module. The apparatus further includes a multimodal prediction module.

[0132] The multimodal processing module is used to map the target vehicle training trajectory features, the surrounding vehicle training high-level trajectory features related to the target vehicle, and the environment training high-level features to obtain multiple modal results; wherein, after decoding and predicting processing and iterative motion simulation by the pre-built prediction decoding module, a plurality of training trajectory prediction results corresponding to each modal result are obtained. For each training trajectory prediction result, re-encoding is performed based on the encoding module to obtain the training prediction trajectory features corresponding to each modality.

[0133] The multimodal prediction module is used to input the training prediction trajectory features corresponding to each modality, the high-level trajectory features of the surrounding vehicles and the high-level features of the environment training into the probability decoding network to be trained, and output the training trajectory score of the current modality; the training trajectory scores of all modalities are input into the softmax function to obtain the probability values ​​of multiple training trajectory prediction results; the training trajectory prediction result with the largest probability value is output as the final training prediction trajectory. Among them, the loss function is determined by the gap between the training prediction trajectory and the real trajectory, and the encoding module, the prediction decoding module and the probability decoding network are trained synchronously; the trajectory prediction model also includes a trained probability decoding network.

[0134] A sixth exemplary embodiment of the present disclosure provides a vehicle trajectory prediction device based on a trajectory prediction model.

[0135] The above device includes: a data acquisition module and a prediction processing module.

[0136] The data acquisition module is used to acquire the first historical trajectory of the target vehicle, the second historical trajectory of other vehicles and traffic environment information in a vehicle driving scenario.

[0137] The prediction processing module is used to use the first historical trajectory, the second historical trajectory and traffic environment information as inputs of a trajectory prediction model, and output a predicted trajectory of the target vehicle; wherein the trajectory prediction model is a model constructed using the method provided in the second embodiment, or a model constructed using the device provided in the fifth embodiment.

[0138] For more details and beneficial effects of this embodiment, please refer to the relevant description of the first embodiment, which will not be repeated here.

[0139] Any number of the functional modules included in the apparatus of the fourth embodiment to the sixth embodiment above can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. At least one of the functional modules included in the apparatus of the fourth embodiment to the sixth embodiment can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the functional modules included in the apparatus of the fourth embodiment to the sixth embodiment can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0140] A seventh exemplary embodiment of the present disclosure provides an electronic device.

[0141] Figure 7 The structural block diagram of the electronic device provided by the embodiment of the present disclosure is schematically shown.

[0142] Reference Figure 7 As shown, the electronic device 700 provided by the embodiment of the present disclosure includes a processor 701, a communication interface 702, a memory 703 and a communication bus 704, wherein the processor 701, the communication interface 702 and the memory 703 communicate with each other through the communication bus 704; the memory 703 is used to store computer programs; the processor 701 is used to implement the trajectory prediction method for integrating kinematics and environmental cognition provided in the first aspect embodiment, the method for constructing a trajectory prediction model for integrating kinematics and environmental cognition provided in the second aspect embodiment, or the vehicle trajectory prediction method based on the trajectory prediction model provided in the third aspect embodiment when executing the program stored in the memory.

[0143] The eighth exemplary embodiment of the present disclosure further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the trajectory prediction method for integrating kinematics and environmental cognition provided by the first embodiment, the method for constructing a trajectory prediction model for integrating kinematics and environmental cognition provided by the second embodiment, or the vehicle trajectory prediction method based on the trajectory prediction model provided by the third embodiment is implemented.

[0144] The computer-readable storage medium may be included in the device or apparatus described in the above embodiment; or it may exist independently without being assembled into the device or apparatus. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0145] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0146] It should be noted that the collection, collection, updating, analysis, processing, use, transmission, storage and other aspects of user personal information involved in the technical solution provided by the embodiments of the present disclosure are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security and national security.

[0147] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0148] The foregoing is merely a specific embodiment of the present disclosure, which enables those skilled in the art to understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features applied herein.

Claims

1. A trajectory prediction method integrating kinematics and environmental cognition, characterized in that: include: Preprocessing the first historical trajectory of the target vehicle, the second historical trajectory of other vehicles and traffic environment information in the vehicle driving scene; Based on the pre-built encoding module, feature extraction and interactive learning are performed on the pre-processed first historical trajectory, the second historical trajectory and the traffic environment information to obtain the target vehicle trajectory features, the surrounding vehicle high-level trajectory features related to the target vehicle and the environment high-level features; Input the target vehicle trajectory features, the surrounding vehicle high-level trajectory features and the environment high-level features into a pre-built prediction decoding module for decoding and prediction processing to obtain the predicted acceleration corresponding to the current moment; The predicted acceleration, the current position and the current speed of the target vehicle are input into the kinematic simulation model for iterative motion simulation to obtain the trajectory prediction result of the future preset time step; wherein, during the iterative motion simulation process, the predicted speed and predicted position of a single time step are iteratively input into the encoding module for re-encoding to obtain the latest trajectory characteristics of the target vehicle; the latest trajectory characteristics continue to be input into the prediction decoding module to obtain the predicted acceleration at the next moment; multiple rounds are iterated until the simulation of the preset time step is completed.

2. The method according to claim 1, characterized in that The predicted acceleration is obtained by summing or weighted summing the following three accelerations: The first predicted acceleration is generated by the self-driving effect of the target vehicle, the second predicted acceleration is generated by the surrounding vehicles affecting the target vehicle, and the third predicted acceleration is generated by the traffic environment affecting the target vehicle.

3. The method according to claim 1, characterized in that The prediction decoding module includes: a first decoding module, a second decoding module and a third decoding module; The target vehicle trajectory features, the surrounding vehicle high-level trajectory features and the environment high-level features are input into a pre-built prediction decoding module for decoding and prediction processing to obtain the predicted acceleration corresponding to the current moment, including: The target vehicle trajectory feature and the target point feature are input into the first decoding module, and the first predicted acceleration generated by the vehicle self-driving action is output; wherein the target point feature is a feature corresponding to the predicted target point position obtained by predicting the target vehicle trajectory feature and the high-level environment feature; Inputting the target vehicle trajectory features and the surrounding vehicle high-level trajectory features into the second decoding module, and outputting a second predicted acceleration generated by the surrounding vehicles affecting the target vehicle; The target vehicle trajectory features and the high-level environmental features are input into the third decoding module, and a third predicted acceleration generated by the traffic environment affecting the target vehicle is output.

4. The method according to claim 3, characterized in that Also includes: The target vehicle trajectory features and the high-level environmental features are input into a pre-built target point encoding module for target point prediction processing to obtain target point features corresponding to the predicted target point position.

5. The method according to claim 1, characterized in that The encoding module includes: a target vehicle trajectory encoding module, a surrounding vehicle trajectory encoding module, a traffic environment encoding module, a first cross-learning module and a second cross-learning module; Among them, based on the pre-built encoding module, the pre-processed first historical trajectory, the second historical trajectory and the traffic environment information are subjected to feature extraction and interactive learning to obtain the target vehicle trajectory features, the surrounding vehicle high-level trajectory features related to the target vehicle and the environment high-level features, including: The preprocessed first historical trajectory and the predicted trajectory under the iteration step constitute a splicing vector, and the splicing vector is input into the target vehicle trajectory encoding module for feature extraction processing, and the target vehicle trajectory feature is output; wherein, the predicted trajectory is an empty vector in the initial state; during the iterative motion simulation process, the predicted trajectory under the iteration step is the result of step accumulation of the predicted position and predicted speed of a single time step outputted corresponding to each iteration round; The preprocessed second historical trajectory is input into the surrounding vehicle trajectory encoding module, and features are extracted from each trajectory point in the second historical trajectory of each other vehicle, and the global trajectory features of each other vehicle are obtained based on self-attention learning between each trajectory point; and the global trajectory features between other vehicles are interactively learned to obtain the trajectory features of surrounding vehicles; Inputting the target vehicle trajectory features and the surrounding vehicle trajectory features into the first cross-learning module for cross-attention learning to obtain high-level trajectory features of surrounding vehicles related to the target vehicle; The preprocessed traffic environment information is input into the traffic environment encoding module to extract the characteristics of the local environment, conduct internal interactive learning of each local environment, and conduct global interactive learning between multiple local environments to obtain the global characteristics of the environment; The target vehicle trajectory features and the global environmental features are input into the second cross-learning module for cross-attention learning to obtain high-level environmental features related to the target vehicle.

6. The method according to claim 1, characterized in that The first historical trajectory, the second historical trajectory and the traffic environment information are all located in a global coordinate system; The preprocessing of the first historical trajectory of the target vehicle, the second historical trajectory of other vehicles and the traffic environment information in the vehicle driving scene includes: Performing coordinate conversion processing on the first historical trajectory, the second historical trajectory, and the traffic environment information to convert them into representation results in a reference coordinate system corresponding to the observation angle of the target vehicle; The current position of the target vehicle is taken as the coordinate origin, the displacement direction of the current timestamp of the target vehicle compared with the previous timestamp is taken as the positive direction of the first direction, and the first direction and a second direction perpendicular to the first direction are constructed as the reference coordinate system.

7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: Multimodal processing is performed in the encoding module to map the target vehicle trajectory features, the surrounding vehicle high-level trajectory features related to the target vehicle, and the environment high-level features to obtain multiple modal results; wherein, after decoding prediction processing and iterative motion simulation are performed through a pre-built prediction decoding module, multiple trajectory prediction results corresponding to each modal result are obtained; For each trajectory prediction result, re-encoding is performed based on the encoding module to obtain the predicted trajectory features corresponding to each mode; the predicted trajectory features corresponding to each mode, the high-level trajectory features of surrounding vehicles and the high-level features of the environment are input into a pre-built probability decoding network, and the trajectory score of the current mode is output; the trajectory scores of all modes are input into the soft maximization function to obtain the probability values ​​of multiple trajectory prediction results; the trajectory prediction result with the largest probability value is output as the final predicted trajectory.

8. The method according to any one of claims 1 to 6, characterized in that The trajectory prediction result of the preset time step in the future is a short-term prediction trajectory at the second level; the final output prediction trajectory is a short-term prediction trajectory at the second level; In the vehicle driving scenario, taking the target vehicle as the center, vehicles within a preset size range around the target vehicle are regarded as other vehicles, and traffic elements within a preset size range around the target vehicle are regarded as traffic environment; The traffic environment includes one of the following combinations: lanes, lanes and traffic lights, lanes and roadblocks; the traffic environment information includes one of the following combinations: lane information, lane information and traffic light information, lane information and roadblock information.

9. A method for constructing a trajectory prediction model integrating kinematics and environmental cognition, characterized in that: include: Based on the preprocessing module, preprocessing is performed on the first training history trajectory of the target vehicle in the vehicle driving scene, the second training history trajectory of other vehicles, and the training traffic environment information; Based on the encoding module to be trained, feature extraction and interactive learning are performed on the preprocessed first training history trajectory, the second training history trajectory and the training traffic environment information to obtain the target vehicle training trajectory feature, the surrounding vehicle training high-level trajectory feature related to the target vehicle and the environment training high-level feature; Input the target vehicle training trajectory features, the surrounding vehicle training high-level trajectory features and the environment training high-level features into the prediction decoding module to be trained for decoding and prediction processing to obtain the training prediction acceleration corresponding to the current moment; The training predicted acceleration, the current position and the current speed of the target vehicle are input into the kinematic simulation model for iterative motion simulation to obtain the training trajectory prediction result of the future preset time step; wherein, during the iterative motion simulation process, the predicted speed and predicted position of a single time step are iteratively input into the encoding module for re-encoding to obtain the latest training trajectory characteristics of the target vehicle; the latest training trajectory characteristics continue to be used as the input of the prediction decoding module to obtain the training predicted acceleration at the next moment; iterate multiple rounds until the simulation of the preset time step is completed; The actual trajectory corresponding to the target vehicle at the future preset time step is used as a training label, and the encoding module and the prediction decoding module are synchronously trained; a trajectory prediction model is constructed based on the preprocessing module, the trained encoding module, the trained prediction decoding module and the kinematic simulation model.

10. The method according to claim 9, characterized in that Also includes: Multimodal processing is performed in the encoding module to map the target vehicle training trajectory features, the surrounding vehicle training high-level trajectory features related to the target vehicle, and the environment training high-level features to obtain multiple modal results; wherein, after decoding prediction processing and iterative motion simulation are performed through a pre-built prediction decoding module, multiple training trajectory prediction results corresponding to each modal result are obtained; For each training trajectory prediction result, re-encoding is performed based on the encoding module to obtain the training prediction trajectory features corresponding to each mode; the training prediction trajectory features corresponding to each mode, the surrounding vehicle training high-level trajectory features and the environment training high-level features are input into the probability decoding network to be trained, and the training trajectory score of the current mode is output; the training trajectory scores of all modes are input into the soft maximization function to obtain the probability values ​​of multiple training trajectory prediction results; the training trajectory prediction result with the largest probability value is output as the final training prediction trajectory; The loss function is determined by the gap between the training predicted trajectory and the true trajectory, and the encoding module, the prediction decoding module and the probability decoding network are trained synchronously; the trajectory prediction model also includes a trained probability decoding network.

11. A vehicle trajectory prediction method based on a trajectory prediction model, characterized in that: The method comprises: Obtaining a first historical trajectory of a target vehicle, a second historical trajectory of other vehicles, and traffic environment information in a vehicle driving scenario; The first historical trajectory, the second historical trajectory and traffic environment information are used as inputs of a trajectory prediction model, and the predicted trajectory of the target vehicle is output; wherein the trajectory prediction model is a model constructed using the method of claim 9 or 10.

12. A trajectory prediction device integrating kinematics and environmental cognition, characterized in that: include: A preprocessing module, used for preprocessing the first historical trajectory of the target vehicle, the second historical trajectory of other vehicles and the traffic environment information in the vehicle driving scene; A feature extraction module is used to extract features and interactively learn the preprocessed first historical trajectory, the second historical trajectory and the traffic environment information based on the pre-built encoding module, so as to obtain the target vehicle trajectory features, the surrounding vehicle high-level trajectory features and the environment high-level features related to the target vehicle; An acceleration prediction module is used to input the target vehicle trajectory features, the surrounding vehicle high-level trajectory features and the environment high-level features into a pre-built prediction decoding module for decoding and prediction processing to obtain a predicted acceleration corresponding to the current moment; The kinematic simulation module is used to input the predicted acceleration, the current position and the current speed of the target vehicle into the kinematic simulation model for iterative motion simulation to obtain the trajectory prediction result of the future preset time step; wherein, during the iterative motion simulation process, the predicted speed and predicted position of a single time step are iteratively input into the encoding module for re-encoding to obtain the latest trajectory characteristics of the target vehicle; the latest trajectory characteristics continue to be used as the input of the prediction decoding module to obtain the predicted acceleration at the next moment; multiple rounds are iterated until the simulation of the preset time step is completed.

13. The device according to claim 12, characterized in that The feature extraction module includes a multimodal processing module; the device also includes: a multimodal prediction module; The multimodal processing module is used to map the target vehicle trajectory features, the surrounding vehicle high-level trajectory features related to the target vehicle, and the environment high-level features to obtain multiple modal results; wherein, after decoding prediction processing and iterative motion simulation are performed by a pre-built prediction decoding module, multiple trajectory prediction results corresponding to each modal result are obtained; For each trajectory prediction result, re-encoding is performed based on the encoding module to obtain the predicted trajectory features corresponding to each mode; The multimodal prediction module is used to input the predicted trajectory features corresponding to each modality, the high-level trajectory features of surrounding vehicles and the high-level features of the environment into a pre-built probability decoding network, and output the trajectory score of the current modality; input the trajectory scores of all modalities into the soft maximization function to obtain the probability values ​​of multiple trajectory prediction results; and output the trajectory prediction result with the largest probability value as the final predicted trajectory.

14. A device for constructing a trajectory prediction model integrating kinematics and environmental cognition, characterized in that: include: A training preprocessing module, used for preprocessing a first training history track of a target vehicle, a second training history track of other vehicles and training traffic environment information in a vehicle driving scene; A training feature extraction module is used to extract features and interactively learn the preprocessed first training history trajectory, the second training history trajectory and the training traffic environment information based on the encoding module to be trained, so as to obtain the target vehicle training trajectory features, the surrounding vehicle training high-level trajectory features related to the target vehicle and the environment training high-level features; The training acceleration prediction module is used to input the target vehicle training trajectory features, the surrounding vehicle training high-level trajectory features and the environment training high-level features into the prediction decoding module to be trained for decoding and prediction processing to obtain the training prediction acceleration corresponding to the current moment; A training kinematic simulation module is used to input the training predicted acceleration, the current position and the current speed of the target vehicle into the kinematic simulation model for iterative motion simulation to obtain the training trajectory prediction result of the future preset time step; wherein, during the iterative motion simulation process, the predicted speed and predicted position of a single time step are iteratively input into the encoding module for re-encoding to obtain the latest training trajectory characteristics of the target vehicle; the latest training trajectory characteristics continue to be used as the input of the prediction decoding module to obtain the training predicted acceleration at the next moment; multiple rounds are iterated until the simulation of the preset time step is completed; A prediction model construction module is used to use the actual trajectory corresponding to the target vehicle at the future preset time step as a training label to synchronously train the encoding module and the prediction decoding module; and to obtain a trajectory prediction model based on the preprocessing module, the trained encoding module, the trained prediction decoding module and the kinematic simulation model.

15. The device according to claim 14, characterized in that The training feature extraction module includes a multimodal processing module; the device also includes: a multimodal prediction module; The multimodal processing module is used to map the target vehicle training trajectory features, the surrounding vehicle training high-level trajectory features related to the target vehicle, and the environment training high-level features to obtain multiple modal results; wherein, after decoding prediction processing and iterative motion simulation are performed by a pre-built prediction decoding module, a plurality of training trajectory prediction results corresponding to each modal result are obtained; For each training trajectory prediction result, re-encoding is performed based on the encoding module to obtain the training prediction trajectory features corresponding to each mode; The multimodal prediction module is used to input the training prediction trajectory features corresponding to each modality, the surrounding vehicle training high-level trajectory features and the environment training high-level features into the probability decoding network to be trained, and output the training trajectory score of the current modality; input the training trajectory scores of all modalities into the soft maximization function to obtain the probability values ​​of multiple training trajectory prediction results; and output the training trajectory prediction result with the largest probability value as the final training prediction trajectory; The loss function is determined by the gap between the training predicted trajectory and the true trajectory, and the encoding module, the prediction decoding module and the probability decoding network are trained synchronously; the trajectory prediction model also includes a trained probability decoding network.

16. A vehicle trajectory prediction device based on a trajectory prediction model, characterized in that: The device comprises: A data acquisition module, used to acquire a first historical trajectory of a target vehicle, a second historical trajectory of other vehicles, and traffic environment information in a vehicle driving scenario; A prediction processing module, used to use the first historical trajectory, the second historical trajectory and traffic environment information as inputs of a trajectory prediction model, and output a predicted trajectory of the target vehicle; wherein the trajectory prediction model is a model constructed using the method of claim 9 or 10, or a model constructed using the device of claim 14 or 15.

17. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory, used to store computer programs; A processor, for implementing the method according to any one of claims 1 to 11 when executing a program stored in a memory.

18. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

Citation Information

Cited By

  • Track prediction method and device for automatic driving, storage medium and program product

    CN120440076A

  • Trajectory prediction method, device, storage medium and program product for autonomous driving

    CN120440076B

  • Deep learning trajectory prediction method introducing vehicle kinematics constraint

    CN121386791A

  • A deep learning trajectory prediction method introducing vehicle kinematic constraints

    CN121386791B

  • Intelligent network connection vehicle speed prediction method and device based on vision

    CN121459599A