Intention recognition assisted vehicle trajectory prediction method, model construction method and device

Through the vehicle trajectory prediction method assisted by intention identification, combined with progressive interactive learning and driving intention, the problem of insufficient accuracy of vehicle trajectory prediction in intelligent driving is solved, and more efficient trajectory prediction is achieved.

CN119037469AInactive Publication Date: 2024-11-29JIOTAO INTELLIGENT (BEIJING) TECHNOLOGY CO LTD
View PDF 0 Cites 3 Cited by

Patent Information

Application Number
CN202411107739.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-13
Publication Date
2024-11-29
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

In the existing intelligent driving technology, the vehicle trajectory prediction model has insufficient prediction accuracy in changing environments and is difficult to compare with manual driving.

Method used

Using the vehicle trajectory prediction method assisted by intent recognition, by obtaining vehicle historical driving information and lane information, using pre-constructed intent recognition model and progressive interactive learning model, the predicted trajectory of the target vehicle is generated, and the multi-factor combination of interactive learning and driving intention is carried out. deal with.

Benefits of technology

It improves the accuracy of vehicle trajectory prediction, can more accurately consider the surrounding environment and driving intentions, and improves the predicted multiple possible trajectories and probability prediction capabilities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119037469A_ABST
    Figure CN119037469A_ABST
Patent Text Reader

Abstract

The invention relates to an intention recognition assisted vehicle track prediction method and device, and a model construction method and device, and the method comprises the steps: obtaining the historical driving information and lane information of each vehicle in a vehicle driving scene, and carrying out the preprocessing, and obtaining a historical track vector sequence and a lane vector sequence; based on a pre-constructed intention recognition model, processing the historical trajectory vector sequence of each vehicle to obtain a driving intention coding vector corresponding to each vehicle; according to the historical track vector sequence, the driving intention coding vector and the lane vector sequence, progressive interactive learning between the target vehicle and other vehicles and lanes is carried out, and a target coding vector after the target vehicle comprehensively learns surrounding vehicles and the environment corresponding to the lanes is obtained; and based on a pre-constructed decoding prediction model, performing decoding prediction processing on the target coding vector to obtain a prediction track of the target vehicle. The accuracy of trajectory prediction can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of intelligent driving and artificial intelligence technology, and in particular to an intention recognition-assisted vehicle trajectory prediction method, model building method and device. Background Art

[0002] In the field of intelligent driving, with the development of technologies such as assisted driving and autonomous driving, higher requirements have been put forward for vehicle driving safety at the implementation level. Predicting vehicle trajectories can help autonomous vehicles maintain a sufficient safe distance from surrounding vehicles and conduct safer path planning, which is one of the important technologies in the field of autonomous driving.

[0003] In the process of realizing the concept disclosed in the present invention, the inventors found that there are at least the following technical problems in the related technology: the vehicle driving trajectory is a time-series position sequence. At present, most of the existing prediction methods are mainly based on deep learning models (such as recurrent neural networks, graph neural networks, etc.) as prediction models to process the vehicle's information in time and space. However, in scenarios where the vehicle's surrounding environment is constantly changing, due to the inherent randomness in the vehicle's movement process, there is still a certain gap between the ability of these prediction models to predict the trajectory of surrounding vehicles and manual driving, so the prediction accuracy still needs to be further improved. Summary of the invention

[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the embodiments of the present disclosure provide a vehicle trajectory prediction method, a model building method and a device assisted by intention recognition.

[0005] In the first aspect, an embodiment of the present disclosure provides a vehicle trajectory prediction method assisted by intention recognition. The above method includes: obtaining the historical driving information and lane information of each vehicle in the vehicle driving scene and preprocessing it to obtain a historical trajectory vector sequence and a lane vector sequence; the above vehicle includes a target vehicle to be predicted and other vehicles within a preset scene range; based on a pre-constructed intention recognition model, the historical trajectory vector sequence of each vehicle is processed to obtain a driving intention coding vector corresponding to each vehicle; according to the above historical trajectory vector sequence, the above driving intention coding vector and the above lane vector sequence, progressive interactive learning is performed between the target vehicle and other vehicles and lanes to obtain a target coding vector after the above target vehicle has comprehensively learned the corresponding environment of the surrounding vehicles and lanes; wherein the above progressive interactive learning means: performing interactive learning between the target vehicle and other vehicles and lanes in sequence and the result of the former learning is used as the input of the latter; based on a pre-constructed decoding prediction model, the above target coding vector is decoded and predicted to obtain the predicted trajectory of the above target vehicle.

[0006] In some embodiments, based on the above-mentioned historical trajectory vector sequence, the above-mentioned driving intention coding vector and the above-mentioned lane vector sequence, progressive interactive learning is performed between the target vehicle and other vehicles and lanes to obtain the target coding vector after the above-mentioned target vehicle comprehensively learns the corresponding environment of surrounding vehicles and lanes, including: performing feature extraction on the above-mentioned lane vector sequence to obtain the lane feature coding vector; performing feature extraction on the above-mentioned historical trajectory vector sequence and the above-mentioned driving intention coding vector to obtain the trajectory feature coding vector corresponding to each vehicle; based on the above-mentioned trajectory feature coding vector, the above-mentioned lane feature coding vector and the attention mechanism, progressive interactive learning is performed between the target vehicle and other vehicles and lanes to obtain the target coding vector after the above-mentioned target vehicle comprehensively learns the corresponding environment of surrounding vehicles and lanes.

[0007] In some embodiments, based on the above-mentioned trajectory feature coding vector, the above-mentioned lane feature coding vector and the attention mechanism, progressive interactive learning is performed between the target vehicle and other vehicles and lanes, including: based on the trajectory feature coding vector of each vehicle, the above-mentioned lane feature coding vector and the cross-attention mechanism, a first interactive learning is performed between each vehicle and the lane to obtain a first high-level trajectory coding vector after each vehicle has learned the lane information; based on the first high-level trajectory coding vector of each vehicle and the self-attention mechanism, a second interactive learning is performed between each vehicle to obtain a second high-level trajectory coding vector after the target vehicle has progressively learned the driving information of other vehicles; wherein the above-mentioned second high-level trajectory coding vector is used as the target coding vector. Alternatively, in other embodiments, based on the above-mentioned trajectory feature coding vector, the above-mentioned lane feature coding vector and the attention mechanism, progressive interactive learning is performed between the target vehicle and other vehicles and lanes, including: based on the trajectory feature coding vector and the self-attention mechanism of each vehicle, a third interactive learning is performed between each vehicle to obtain a third high-level trajectory coding vector after the target vehicle has learned the driving information of other vehicles; based on the above-mentioned third high-level trajectory feature coding vector, the above-mentioned lane feature coding vector and the cross-attention mechanism, a fourth interactive learning is performed between the target vehicle and the lane to obtain a fourth high-level trajectory coding vector after the target vehicle has progressively learned the lane information; wherein the above-mentioned fourth high-level trajectory coding vector is used as the target coding vector.

[0008] In some embodiments, the above-mentioned first interactive learning between each vehicle and the lane is performed based on the trajectory feature coding vector of each vehicle, the lane feature coding vector and the cross-attention mechanism to obtain a first high-level trajectory coding vector after each vehicle has learned the lane information, including: for each vehicle, using the trajectory feature coding vector of the current vehicle as a query vector, and using the lane feature coding vector as a key vector and a value vector respectively, performing cross-attention calculation to obtain a first attention weighted trajectory feature vector after the current vehicle has learned the lane information; performing forward propagation update processing on the above-mentioned first attention weighted trajectory feature vector based on the residual module to obtain the first high-level trajectory coding vector after the current vehicle has learned the lane information.

[0009] In some embodiments, the above-mentioned first high-level trajectory coding vector and self-attention mechanism based on each vehicle are used to perform second interactive learning between each vehicle to obtain a second high-level trajectory coding vector after the target vehicle has progressively learned the driving information of other vehicles, including: splicing the first high-level trajectory coding vectors of each vehicle to obtain a first spliced ​​coding vector; using the above-mentioned first spliced ​​coding vectors as query vectors, key vectors and value vectors respectively, performing self-attention calculations, and obtaining a second high-level trajectory coding vector after the target vehicle has progressively learned the driving information of other vehicles.

[0010] In some embodiments, the third interactive learning between each vehicle is performed based on the trajectory feature coding vectors and the self-attention mechanism of each vehicle to obtain a third high-level trajectory coding vector after the target vehicle has learned the driving information of other vehicles, including: splicing the trajectory feature coding vectors of each vehicle to obtain a second spliced ​​coding vector; using the above second spliced ​​coding vector as a query vector, a key vector and a value vector respectively, performing self-attention calculation, and obtaining a third high-level trajectory coding vector after the target vehicle has learned the driving information of other vehicles.

[0011] In some embodiments, based on the third high-level trajectory feature coding vector, the lane feature coding vector and the cross-attention mechanism, a fourth interactive learning is performed between the target vehicle and the lane to obtain a fourth high-level trajectory coding vector after the target vehicle has progressively learned the lane information, including: taking the third high-level trajectory coding vector of the target vehicle as a query vector, and taking the lane feature coding vector as a key vector and a value vector respectively, performing cross-attention calculation to obtain a second attention weighted trajectory feature vector after the target vehicle has learned the lane information; performing forward propagation update processing on the second attention weighted trajectory feature vector based on a residual module to obtain a fourth high-level trajectory coding vector after the target vehicle has progressively learned the lane information.

[0012] In some embodiments, the historical driving information and lane information of each vehicle in the vehicle driving scene are obtained and preprocessed to obtain a historical trajectory vector sequence and a lane vector sequence, including: obtaining scene acquisition data in the vehicle driving scene within a preset time period before the current moment; according to the above scene acquisition data, obtaining the historical driving information of the target vehicle, the driving information of other vehicles and the lane information of multiple lanes in the environment; the above driving information is a time sequence position sequence corresponding to the time points in the above preset time period; the above lane information is a lane segment position sequence obtained by dividing each lane into multiple lane segments, each lane segment includes a starting position coordinate and an end position coordinate; for the above time sequence position sequence and the above lane segment position sequence The initial historical trajectory vector sequence and the initial lane vector sequence of each vehicle are obtained by performing difference processing on adjacent elements respectively; the initial historical trajectory vector sequence of each vehicle is determined as the historical trajectory vector sequence corresponding to each vehicle, and the above initial lane vector sequence is determined as the lane vector sequence; or, the initial historical trajectory vector sequence of the vehicle is subjected to coordinate transformation processing, and the above initial lane vector sequence is subjected to coordinate transformation processing, and both are converted into the historical trajectory vector sequence and the lane vector sequence in the reference coordinate system with the observation angle of the target vehicle at the current moment; wherein the driving direction of the target vehicle at the current moment is taken as the first direction, and the first direction and the second direction perpendicular to the first direction are constructed as the reference coordinate system corresponding to the above observation angle.

[0013] In some embodiments, the scene acquisition data corresponds to a global coordinate system. When the global coordinate system and the reference coordinate system are the same coordinate system, the following operations are performed: the initial historical trajectory vector sequence of each vehicle is determined as the historical trajectory vector sequence corresponding to each vehicle, and the initial lane vector sequence is determined as a lane vector sequence. When the global coordinate system and the reference coordinate system are different coordinate systems, one of the following operations is performed: the initial historical trajectory vector sequence of each vehicle is determined as the historical trajectory vector sequence corresponding to each vehicle, and the initial lane vector sequence is determined as a lane vector sequence; or, coordinate conversion processing is performed on the initial historical trajectory vector sequence of the vehicle, and coordinate conversion processing is performed on the initial lane vector sequence, and both are converted into a historical trajectory vector sequence and a lane vector sequence with the observation perspective of the target vehicle at the current moment as the reference coordinate system.

[0014] In some embodiments, the predicted trajectory of the target vehicle is obtained based on the prediction result of the reference coordinate system, which predicts the short-term trajectory within the future second time period; the reference coordinate system is a coordinate system corresponding to the observation perspective of the target vehicle at the current moment; wherein, the driving direction of the target vehicle at the current moment is taken as the first direction, and the first direction and the second direction perpendicular to the first direction are constructed as the reference coordinate system.

[0015] In some embodiments, the above-mentioned intention recognition model includes: a large language model, a word segmentation module and an encoding module. Among them, based on the pre-constructed intention recognition model, the historical trajectory vector sequence of each vehicle is processed to obtain the driving intention encoding vector corresponding to each vehicle, including: for each vehicle, the historical trajectory vector sequence of the current vehicle is input into the above-mentioned large language model, and the natural language text result containing the driving intention of the current vehicle is output; the above-mentioned driving intention includes: movement direction intention and movement speed change intention. Based on the above-mentioned word segmentation module, the above-mentioned natural language text result is word segmented to obtain the word element vector corresponding to the driving intention. Based on the above-mentioned encoding module, the word element vector corresponding to the above-mentioned driving intention is encoded to obtain the driving intention encoding vector corresponding to each vehicle.

[0016] In some embodiments, the large language model in the above-mentioned intention recognition model is obtained by fine-tuning a pre-trained text processing type large language model, and the above-mentioned model fine-tuning corresponds to the following processing process: the fine-tuning historical trajectory vector sequence corresponding to the vehicle with known driving intention in the historical vehicle driving scene is input into the above-mentioned text processing type large language model, and the parameters of the above-mentioned text processing type large language model are adjusted and optimized based on the known driving intentions corresponding to each input, so that the intention recognition accuracy of the fine-tuning historical trajectory vector sequence of the adjusted text processing type large language model reaches a set threshold, and the fine-tuning is considered to be completed; or, the above-mentioned large language model is a pre-trained text processing type large language model, and while the historical trajectory vector sequence of the current vehicle is input into the above-mentioned large language model, the corresponding prompt words for predicting possible results are also input, so that the above-mentioned large language model is processed based on the input and the above-mentioned prompt words and obtains a natural language text result containing the driving intention of the current vehicle.

[0017] In the second aspect, an embodiment of the present disclosure provides a method for constructing a vehicle trajectory prediction model based on intention recognition assistance. The above method includes: obtaining training historical driving information and training lane information of each vehicle in a historical vehicle driving scenario; based on a preprocessing module, preprocessing the above training historical driving information and the above training lane information to obtain a training historical trajectory vector sequence and a training lane vector sequence; the above vehicle includes a target vehicle for prediction training and other vehicles within a preset scene range; based on the intention recognition model to be trained, processing the training historical trajectory vector sequence of each vehicle to obtain a training driving intention coding vector corresponding to each vehicle; inputting the above training historical trajectory vector sequence, the above training driving intention coding vector and the above training lane vector sequence into the progressive interaction model to be trained, and performing progressive interaction between the target vehicle and other vehicles and lanes. Mutual learning is performed to obtain the target coding vector of the target vehicle after comprehensively learning the corresponding environment of the surrounding vehicles and lanes; wherein the progressive interactive learning means: the interactive learning between the target vehicle and other vehicles and lanes is performed in sequence and the result of the former learning is used as the input of the latter; based on the decoding prediction model to be trained, the target coding vector is decoded and predicted to obtain the training prediction trajectory of the target vehicle; a loss function is constructed based on the gap between the real trajectory label of the target vehicle and the training prediction trajectory, the intention recognition model, the progressive interactive model and the decoding prediction model are trained synchronously, and the preprocessing module is constructed into a vehicle trajectory prediction model with the trained intention recognition model, the progressive interactive model and the decoding prediction model.

[0018] In some embodiments, the preprocessing module has a first output and a second output; in the use phase of the vehicle trajectory prediction model, the first output is used to output a historical trajectory vector sequence, and the second output is used to output a lane vector sequence. The trained intention recognition model has a recognition input and a recognition output; the recognition input is connected to the first output, and the recognition output is used to output the driving intention encoding vector corresponding to each vehicle. The trained progressive interaction model includes: a first feature extraction model, a second feature extraction model, and one of the following model groups: a model group consisting of a first interaction model and a second interaction model, or a model group consisting of a third interaction model and a fourth interaction model; the final output of the model group is used as the output of the progressive interaction model. Among them, the first feature extraction model has a first extraction input and a first extraction output; the first extraction input is connected to both the first output and the recognition output; the first extraction output is used to output the trajectory feature encoding vector corresponding to each vehicle. The second feature extraction model has a second extraction input and a second extraction output; the second extraction input is connected to the second output; the second extraction output is used to output the lane feature encoding vector. The first interaction model is used to perform the first interaction learning between each vehicle and the lane based on the cross-attention mechanism, and has a first interaction input end and a first interaction output end; the first interaction input end is connected to both the first extraction output end and the second extraction output end; the first interaction output end is used to output the first high-level trajectory encoding vector after each vehicle has learned the lane information. The second interaction model is used to perform the second interaction learning between each vehicle based on the self-attention mechanism, and has a second interaction input end and a second interaction output end; the second interaction input end is connected to the first interaction output end; the second interaction output end is used to output the second high-level trajectory encoding vector after the target vehicle has progressively learned the driving information of other vehicles; wherein the second high-level trajectory encoding vector is used as the target encoding vector. The third interaction model is used to perform the third interaction learning between each vehicle based on the self-attention mechanism, and has a third interaction input end and a third interaction output end; the third interaction input end is connected to the first extraction output end; the third interaction output end is used to output the third high-level trajectory encoding vector after the target vehicle has learned the driving information of other vehicles. The fourth interaction model is used to perform the fourth interaction learning between the target vehicle and the lane based on the cross-attention mechanism, and has a fourth interaction input terminal and a fourth interaction output terminal; the fourth interaction input terminal is connected to the third interaction output terminal and the second extraction output terminal; the fourth interaction output terminal is used to output the fourth high-level trajectory encoding vector after the target vehicle has progressively learned the lane information; wherein the fourth high-level trajectory encoding vector is used as the target encoding vector.The trained decoding prediction model has a decoding input and a decoding output; the decoding input is connected to the output of the progressive interaction model; the decoding output is used to output the predicted trajectory of the target vehicle.

[0019] In the third aspect, the embodiments of the present disclosure provide a vehicle trajectory prediction method based on a vehicle trajectory prediction model. The method comprises: obtaining the historical driving information and lane information of each vehicle in a vehicle driving scenario; using the historical driving information and lane information as inputs to the vehicle trajectory prediction model, and outputting the predicted trajectory of the target vehicle; wherein the vehicle trajectory prediction model is a model constructed by the construction method provided in the embodiment of the second aspect.

[0020] In a fourth aspect, an embodiment of the present disclosure provides a vehicle trajectory prediction device assisted by intention recognition. The device includes: a data acquisition module, a preprocessing module, an intention recognition module, an environment interaction learning module and a prediction module. The data acquisition module is used to obtain the historical driving information and lane information of each vehicle in the vehicle driving scene. The preprocessing module is used to preprocess the historical driving information and the lane information to obtain a historical trajectory vector sequence and a lane vector sequence; the vehicle includes a target vehicle to be predicted and other vehicles within a preset scene range. The intention recognition module is used to process the historical trajectory vector sequence of each vehicle based on a pre-built intention recognition model to obtain a driving intention coding vector corresponding to each vehicle. The environment interaction learning module is used to perform progressive interaction learning between the target vehicle and other vehicles and lanes according to the historical trajectory vector sequence, the driving intention coding vector and the lane vector sequence, and obtain the target coding vector after the target vehicle comprehensively learns the corresponding environment of the surrounding vehicles and lanes; wherein the progressive interaction learning means: the interactive learning between the target vehicle and other vehicles and lanes is performed in sequence and the result of the former learning is used as the input of the latter. The prediction module is used to perform decoding prediction processing on the target coding vector based on a pre-built decoding prediction model to obtain the predicted trajectory of the target vehicle.

[0021] In a fifth aspect, an embodiment of the present disclosure provides a device for constructing a vehicle trajectory prediction model assisted by intention recognition. The above-mentioned device includes: a training data acquisition module, a training preprocessing module, a training intention recognition module, a training environment interactive learning module, a training prediction module and a prediction model construction module. The above-mentioned training data acquisition module is used to obtain the training historical driving information and training lane information of each vehicle in the historical vehicle driving scene. The above-mentioned training preprocessing module is used to preprocess the above-mentioned training historical driving information and the above-mentioned training lane information to obtain a training historical trajectory vector sequence and a training lane vector sequence; the above-mentioned vehicle includes an object vehicle for prediction training and other vehicles within a preset scene range. The above-mentioned training intention recognition module is used to process the training historical trajectory vector sequence of each vehicle based on the intention recognition model to be trained to obtain a training driving intention encoding vector corresponding to each vehicle. The training environment interactive learning module is used to input the training historical trajectory vector sequence, the training driving intention coding vector and the training lane vector sequence into the progressive interactive model to be trained, and perform progressive interactive learning between the target vehicle and other vehicles and lanes, so as to obtain the target coding vector after the target vehicle has comprehensively learned the corresponding environment of the surrounding vehicles and lanes; wherein, the progressive interactive learning means: the interactive learning between the target vehicle and other vehicles and lanes is performed in sequence, and the result of the former learning is used as the input of the latter. The training prediction module is used to perform decoding prediction processing on the target coding vector based on the decoding prediction model to be trained, so as to obtain the training prediction trajectory of the target vehicle. The prediction model construction module is used to construct a loss function based on the gap between the real trajectory label of the target vehicle and the training prediction trajectory, and simultaneously train the intention recognition model, the progressive interaction model and the decoding prediction model, and construct the training preprocessing module with the trained intention recognition model, the progressive interaction model and the decoding prediction model into a vehicle trajectory prediction model.

[0022] In the sixth aspect, an embodiment of the present disclosure provides a vehicle trajectory prediction device based on a vehicle trajectory prediction model. The above-mentioned device includes: a data acquisition module and a prediction processing module. The above-mentioned data acquisition module is used to obtain the historical driving information and lane information of each vehicle in the vehicle driving scene. The above-mentioned prediction processing module is used to use the above-mentioned historical driving information and the above-mentioned lane information as inputs of the vehicle trajectory prediction model, and output the predicted trajectory of the target vehicle; wherein the above-mentioned vehicle trajectory prediction model is a model constructed by adopting the construction method provided in the embodiment of the second aspect, or a model constructed by adopting the device provided in the embodiment of the fifth aspect.

[0023] In the seventh aspect, an embodiment of the present disclosure provides an electronic device. The electronic device includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory is used to store computer programs; the processor is used to implement the vehicle trajectory prediction method assisted by intention recognition provided in the embodiment of the first aspect, the vehicle trajectory prediction model construction method based on intention recognition assistance provided in the embodiment of the second aspect, or the vehicle trajectory prediction method based on the vehicle trajectory prediction model provided in the embodiment of the third aspect when executing the program stored in the memory.

[0024] In an eighth aspect, an embodiment of the present disclosure provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the vehicle trajectory prediction method assisted by intention recognition provided in the first aspect embodiment, the vehicle trajectory prediction model construction method based on intention recognition assistance provided in the second aspect embodiment, or the vehicle trajectory prediction method based on the vehicle trajectory prediction model provided in the third aspect embodiment is implemented.

[0025] The above technical solution provided by the embodiments of the present disclosure has at least some or all of the following advantages:

[0026] By processing the historical trajectory vector sequence of each vehicle based on a pre-built intention recognition model, the driving intention coding vector corresponding to each vehicle is obtained. This driving intention coding vector can determine the driving intention of each vehicle at the current moment (such as movement direction intentions such as going straight, turning left, turning right, changing lanes left, changing lanes right, and movement speed change intentions such as acceleration, deceleration, and constant speed) and serve as an auxiliary reference for subsequent trajectory prediction; by performing progressive interactive learning between the target vehicle to be predicted and other vehicles and lanes based on the historical trajectory vector sequence, the driving intention coding vector and the lane vector sequence, the target coding vector of the target vehicle after comprehensive learning of the corresponding environment of the surrounding vehicles and lanes is obtained; based on the pre-built decoding prediction model The target coding vector is decoded and predicted to obtain the predicted trajectory of the target vehicle. In the process of progressive interactive learning, the target vehicle is successively learned with other surrounding vehicles and lanes, and the result of the previous learning is used as the input of the subsequent interactive learning to achieve progressive interactive learning. At the same time, the driving intention of each vehicle in the interactive learning process is used as an auxiliary reference. The combination of driving intention prediction and progressive interactive learning makes the target coding vector a learning result that comprehensively considers the surrounding environment and instantaneous driving intention. By introducing multiple factors and high-dimensional processing in the coding layer, the subsequent decoding and prediction processing can realize the prediction of multiple possible trajectories and corresponding probabilities based on the driving intention, which can improve the accuracy of trajectory prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0029] Figure 1 A flowchart of a vehicle trajectory prediction method assisted by intention recognition according to an embodiment of the present disclosure is schematically shown;

[0030] Figure 2 The detailed implementation flow chart of step S110 according to an embodiment of the present disclosure is schematically shown;

[0031] Figure 3 The structure block diagram of the intent recognition model and the process diagram of performing intent recognition according to an embodiment of the present disclosure are schematically shown;

[0032] Figure 4 The detailed implementation flow chart of step S130 according to an embodiment of the present disclosure is schematically shown;

[0033] Figure 5A The detailed implementation flow chart of step S430 according to an embodiment of the present disclosure is schematically shown;

[0034] Figure 5B The detailed implementation flow chart of step S430 according to another embodiment of the present disclosure is schematically shown;

[0035] Figure 6 The flowchart of the method for constructing a vehicle trajectory prediction model based on intention recognition assistance according to an embodiment of the present disclosure is schematically shown;

[0036] Figure 7 The structure block diagram of the vehicle trajectory prediction model according to an embodiment of the present disclosure is schematically shown;

[0037] Figure 8 The structural block diagram of the electronic device provided by the embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present disclosure.

[0039] A first exemplary embodiment of the present disclosure provides a vehicle trajectory prediction method assisted by intention recognition.

[0040] Figure 1 A flowchart of a vehicle trajectory prediction method assisted by intention recognition according to an embodiment of the present disclosure is schematically shown.

[0041] Reference Figure 1 As shown, the vehicle trajectory prediction method assisted by intention recognition provided by the embodiment of the present disclosure includes the following steps: S110, S120, S130 and S140.

[0042] In step S110, historical driving information and lane information of each vehicle in the vehicle driving scenario are obtained and preprocessed to obtain a historical trajectory vector sequence and a lane vector sequence; the above-mentioned vehicles include the target vehicle to be predicted and other vehicles within the preset scenario range.

[0043] In the above vehicle driving scenario, the scene acquisition data within the preset scene range can be obtained by drone aerial photography, data collection and aggregation based on roadside equipment, or vehicle sensor (at least one of radar detection or visual perception). The above scene acquisition data can be video data, or a sequence of image data collected at intervals. The coordinate system corresponding to the scene acquisition data is described as a global coordinate system.

[0044] Figure 2 The detailed implementation flow chart of step S110 according to an embodiment of the present disclosure is schematically shown.

[0045] In some embodiments, in the above step S110, the historical driving information and lane information of each vehicle in the vehicle driving scenario are obtained and preprocessed to obtain a historical trajectory vector sequence and a lane vector sequence, including the following steps: S210, S220, S230 and one of the following: S241, S242.

[0046] In step S210, scene acquisition data of a vehicle driving scene within a preset period before the current moment is obtained.

[0047] The above historical driving information can be scene collection data within a preset period before the current moment. The duration of the preset period is, for example, 3s to 8s; the future period to be predicted can be 3s to 5s. For example, based on the scene collection data within the previous 5s, the trajectory of the target vehicle within the next 3s is predicted.

[0048] During this preset time period, the corresponding information of the target vehicle, other vehicles and lanes within the preset scene range are all covered in the scene acquisition data. The preset scene range can be a preset range space near the target vehicle, such as the preset size range corresponding to the front, rear, left and right of the target vehicle. It can cover but is not limited to at least one of the following scenarios: one or some other vehicles are traveling in the same direction as the target vehicle, one or some other vehicles are traveling in the opposite direction of the target vehicle, one or some other vehicles are perpendicular to the driving direction of the target vehicle (for example, the two are located in different lanes and the two lanes are cross lanes at an intersection), etc. The types of lanes covered by the above preset scene range can be: intersection lanes, one or more one-way lanes, two-way lanes, etc.

[0049] In step S220, based on the above-mentioned scene collection data, the historical driving information of the target vehicle, the driving information of other vehicles and the lane information of multiple lanes in the environment are obtained; the above-mentioned driving information is a time-series position sequence corresponding to the time points within the above-mentioned preset time period; the above-mentioned lane information is a lane segment position sequence obtained by dividing each lane into multiple lane segments, and each lane segment includes a starting position coordinate and an end position coordinate.

[0050] For example, data is collected in a vehicle sensor sensing manner of target vehicle A, and the scene collection data of the first 5 seconds is obtained. In this scene collection data, the time sequence position sequence of the target vehicle in the first 5 seconds is included. Taking data collection every 0.5 seconds (extracting video frames from the video, or image data obtained from the image data sequence) as an example, the time sequence position sequence corresponding to the target vehicle A is: During this period, the time sequence positions of other vehicles located within the preset scene range, for example, vehicle B, vehicle C, and vehicle D, are respectively: In the above-mentioned representation of the time sequence position, the upper right corner numbers of the horizontal and vertical coordinates represent the time sequence numbers corresponding to each position, and the lower right corner numbers represent the vehicle identification corresponding to the coordinates. If other vehicles are not within the preset scene range at one or more time points corresponding to the preset time period, that is, there is no value, zero can be used for padding.

[0052] For simplicity, the historical driving information of the i-th vehicle is expressed as follows:

[0053]

[0054] Among them, X i represents the time sequence of the position of the i-th vehicle in a preset period, where the preset period refers to the historical period corresponding to the time from tT to t-1, with a total duration of T (e.g., 5s in the previous example); i represents the serial number of each vehicle in the vehicle driving scene; Indicates the position of the i-th vehicle at the current moment; is the two-dimensional coordinate corresponding to the position of the i-th vehicle at the current time t.

[0055] In the above formula (1), the time series position sequence without the current moment is used as an example. Since the general sampling interval of trajectory data is 0.1s, excluding the current moment will only reduce a very short vector, which has a very small impact on the final performance. It can be understood that in other embodiments, the above historical driving information can also include the position corresponding to the current moment, for example

[0056] The lane information above includes lane segment position sequences corresponding to multiple lanes. For example, a total of s lanes are included, and all lanes are {L 1 ,L 2 ,L 3 ,……,L S The subscripts 1 to S in the upper right corner of L indicate the lane numbers.

[0057] A lane refers to the corresponding lane range on the navigation map covered within the preset scene range. Each lane is divided into multiple lane segments, and a total of M lane segments are divided. For example, taking the sth lane as an example, it is expressed as: L S =[l1,l2,…,l M ],in is the starting position coordinate of the jth lane segment and the end point coordinates j represents the sequence number of the lane segment, and the value of j ranges from 1 to M.

[0058] All lanes are divided into M lane segments. If a lane segment does not exist, it is padded with zero.

[0059] In step S230, adjacent elements of the time sequence position sequence and the lane segment position sequence are respectively subjected to difference processing to obtain an initial historical trajectory vector sequence and an initial lane vector sequence of each vehicle.

[0060] For example, the time series position sequence of vehicle A in the previous example is subjected to the difference processing of adjacent elements to obtain the initial historical trajectory vector sequence of vehicle A: The processing procedures for other vehicles can be understood by reference.

[0062] The lane segment position sequence {[l1,l2,…,l M ]} As an example, the adjacent elements are subtracted to obtain the initial lane vector sequence of the sth lane:

[0063] In step S241, the initial historical trajectory vector sequence of each vehicle is determined as the historical trajectory vector sequence corresponding to each vehicle, and the above initial lane vector sequence is determined as the lane vector sequence.

[0064] In some embodiments, the above-mentioned scene acquisition data corresponds to a global coordinate system.

[0065] In the case where the global coordinate system and the reference coordinate system are the same coordinate system, the operation corresponding to the above step S241 is performed. The above reference coordinate system refers to the coordinate system based on the observation angle of the target vehicle. If the data acquisition process is based on the acquisition coordinate system corresponding to the observation angle of the target vehicle, the global coordinate system and the reference coordinate system are the same coordinate system, and the initial historical trajectory vector sequence of each vehicle is directly determined as the historical trajectory vector sequence corresponding to each vehicle, and the above initial lane vector sequence is determined as the lane vector sequence.

[0066] In step S242, coordinate conversion processing is performed on the vehicle's initial historical trajectory vector sequence, and coordinate conversion processing is performed on the above-mentioned initial lane vector sequence, both of which are converted into a historical trajectory vector sequence and a lane vector sequence in a reference coordinate system with the observation perspective of the target vehicle at the current moment; wherein the driving direction of the target vehicle at the current moment is taken as the first direction, and the first direction and a second direction perpendicular to the first direction are constructed as a reference coordinate system corresponding to the above-mentioned observation perspective.

[0067] If the target vehicle is moving straight at the current moment, the first direction is the positive direction of the x-axis; if the target vehicle is turning right or left at the current moment, the direction corresponding to the turning angle is the first direction, and the lane and other vehicles have an angle compared to the first direction.

[0068] The coordinate transformation process utilizes the invariance of translation and rotation, constructs a reference coordinate system with the predicted target vehicle's current forward direction as the x-axis and its vertical direction as the y-axis, and transforms the initial historical trajectory vector sequence and the initial lane vector sequence into the reference coordinate system.

[0069] As an example, construct the rotation matrix R corresponding to the i-th vehicle i as follows:

[0070]

[0071] Among them, θ i The reference coordinate system (e.g., the direction of the x-axis: the orientation of the target vehicle) of the target vehicle is compared to the global coordinate system (e.g., the direction forward along the horizontal plane), where the counterclockwise rotation is the positive direction and the clockwise rotation is the negative direction. In other embodiments, if the clockwise rotation angle between the global coordinate system and the reference coordinate system is defined as the negative direction and the counterclockwise rotation angle is the positive direction, the specific form of the corresponding rotation matrix can be changed accordingly.

[0072] Use the rotation matrix R i Multiplying with the initial historical trajectory vector sequence to obtain the vehicle historical trajectory sequence in the reference coordinate system after preprocessing Rotation matrix R i Multiplying with the initial lane vector sequence to obtain the lane centerline sequence in the reference coordinate system after preprocessing

[0073] In the case where the global coordinate system and the reference coordinate system are different coordinate systems, step S241 or S242 is executed. The beneficial effects of coordinate conversion include: the numerical value corresponding to the coordinate represented by the observation angle of the target vehicle is smaller, the amount of calculation for prediction processing after the corresponding input to the model is smaller, and the final effect of trajectory prediction is relatively better.

[0074] In step S120, based on the pre-built intention recognition model, the historical trajectory vector sequence of each vehicle is processed to obtain the driving intention coding vector corresponding to each vehicle.

[0075] Figure 3 The structural block diagram of the intent recognition model and the process diagram of performing intent recognition according to an embodiment of the present disclosure are schematically shown.

[0076] In some embodiments, reference Figure 3 As shown, the above-mentioned intention recognition model 300 includes: a large language model 310, a word segmentation module 320 and an encoding module 330.

[0077] Among them, refer to Figure 3 As shown, based on the pre-built intention recognition model, the historical trajectory vector sequence of each vehicle is processed to obtain the driving intention encoding vector corresponding to each vehicle, including: for each vehicle, the historical trajectory vector sequence of the current vehicle is input into the above-mentioned large language model 310, and the natural language text result M containing the driving intention of the current vehicle is output. i .

[0078] The process of processing based on the large language model 310 (corresponding to the subscript LLM) is shown as follows:

[0079]

[0080] The above driving intention includes: movement direction intention and movement speed change intention. Movement direction intention includes but is not limited to: going straight, turning left, turning right, changing lanes left, changing lanes right, etc.; movement speed change intention includes but is not limited to: accelerating, decelerating, and keeping a constant speed, etc.

[0081] Based on the above word segmentation module 320, the above natural language text result M i Perform word segmentation to obtain the word element vector N corresponding to the driving intention i The word segmentation module may be any model for natural language word segmentation processing, such as a pre-trained BPE (Byte Pair Encode) word segmenter.

[0082] Based on the encoding module 330, the word element vector corresponding to the driving intention is encoded to obtain the driving intention encoding vector T corresponding to each vehicle. i The encoding module can be a linear neural network.

[0083] In some embodiments, the large language model in the above-mentioned intention recognition model is obtained by fine-tuning a pre-trained text processing type large language model (referring to an artificial intelligence (AI) model that can process a large amount of text data to understand and generate human language). The above-mentioned model fine-tuning corresponds to the following processing process: the fine-tuning historical trajectory vector sequence corresponding to the vehicle with known driving intention in the historical vehicle driving scenario is input into the above-mentioned text processing type large language model, and the parameters of the above-mentioned text processing type large language model are adjusted and optimized based on the known driving intentions corresponding to each input, so that the intention recognition accuracy of the adjusted text processing type large language model for the fine-tuning historical trajectory vector sequence reaches the set threshold, and the fine-tuning is considered to be completed.

[0084] Alternatively, in some other embodiments, the large language model is a pre-trained text processing large language model. When the historical trajectory vector sequence of the current vehicle is input into the large language model, the corresponding prompt words for predicting possible results are also input, so that the large language model is processed based on the input and the prompt words and obtains a natural language text result containing the driving intention of the current vehicle. By combining the scheme of calling the large language model in a prompt-based manner, there is no need to fine-tune the model parameters, and the training model occupies less computing resources. The prompt words can be generated based on at least one known possible result corresponding to the same historical trajectory vector sequence or a similar historical trajectory vector sequence (the degree of similarity can be represented by some distance algorithms); or in other embodiments, based on at least one known possible result and combined with the overall planned path of the current vehicle. For example, according to the overall planned path, the current vehicle is recommended to turn left and turn around, and the known possible results include: straight, right turn, left turn; uniform speed, acceleration, deceleration; combined with the planned path of left turn and turn around, turn left, uniform speed, acceleration, and deceleration are used as corresponding prompt words, so that the subsequent intention recognition process will increase the recognition probability of the intention of turning left.

[0085] In step S130, based on the above-mentioned historical trajectory vector sequence, the above-mentioned driving intention coding vector and the above-mentioned lane vector sequence, progressive interactive learning is performed between the target vehicle and other vehicles and lanes, so as to obtain a target coding vector after the above-mentioned target vehicle comprehensively learns the corresponding environment of the surrounding vehicles and lanes; wherein, the above-mentioned progressive interactive learning means: the interactive learning between the target vehicle and other vehicles and lanes is performed in sequence and the result of the former learning is used as the input of the latter.

[0086] Figure 4 The detailed implementation flow chart of step S130 according to an embodiment of the present disclosure is schematically shown.

[0087] In some embodiments, reference Figure 4 As shown, the above step S130 specifically includes: S410, S420 and S430.

[0088] In step S410, feature extraction is performed on the lane vector sequence to obtain a lane feature coding vector.

[0089] In step S420, feature extraction is performed on the historical trajectory vector sequence and the driving intention coding vector to obtain a trajectory feature coding vector corresponding to each vehicle.

[0090] There is no restriction on the execution order of step S410 and step S420, and the two can be executed in parallel or one after the other.

[0091] For example, in the above step S410, the historical trajectory vector sequence corresponding to each vehicle and driving intention encoding vector T i Input the first feature extraction model (corresponding to f vehicle ) to extract features and obtain the trajectory feature encoding vector corresponding to each vehicle

[0092]

[0093] For example, in the above step S420, the lane vector sequence of each lane Input the second feature extraction model (corresponding to Perform feature extraction processing to obtain the lane feature encoding vector h L :

[0094]

[0095] Different parameters are used to encode the vehicle trajectory and the lane position information, respectively, where f vehicle and f lane They are the functions corresponding to the two feature extraction modules, and the vector dimensions of their outputs are the same. vehicle The input is With T i The concatenated vector.

[0096] By processing the historical trajectory vector sequence of each vehicle based on a pre-built intention recognition model, the driving intention coding vector corresponding to each vehicle is obtained. This driving intention coding vector can determine the driving intention of each vehicle at the current moment (such as movement direction intentions such as going straight, turning left, turning right, changing lanes left, changing lanes right, and movement speed change intentions such as acceleration, deceleration, and constant speed) and serve as an auxiliary reference for subsequent trajectory prediction.

[0097] In step S430, based on the trajectory feature coding vector, the lane feature coding vector and the attention mechanism, progressive interactive learning is performed between the target vehicle and other vehicles and lanes to obtain a target coding vector after the target vehicle comprehensively learns the corresponding environment of the surrounding vehicles and lanes.

[0098] Figure 5A The detailed implementation flow chart of step S430 according to an embodiment of the present disclosure is schematically shown.

[0099] In some embodiments, the above step S430 specifically includes the following steps: S511 and S512.

[0100] In step S511, based on the trajectory feature coding vector of each vehicle, the lane feature coding vector and the cross-attention mechanism, the first interactive learning is performed between each vehicle and the lane to obtain the first high-level trajectory coding vector after each vehicle has learned the lane information.

[0101] In step S511, for each vehicle, the trajectory feature encoding vector of the current vehicle is used as the query vector, and the lane feature encoding vectors are used as the key vector and the value vector respectively, and cross-attention calculation is performed to obtain the first attention weighted trajectory feature vector of the current vehicle after learning the lane information; the first attention weighted trajectory feature vector is forward propagated and updated based on the residual module to obtain the first high-level trajectory encoding vector after the current vehicle has learned the lane information.

[0102] For example, specifically: Encoding vector with trajectory features As the query vector q, the lane feature encoding vector h L As the key vector K and value vector V, cross attention calculation is performed to obtain the first attention weighted trajectory feature vector g of the current vehicle that has learned the lane information i :

[0103]

[0104] Among them, d k The corresponding dimensions and With h L The dimensions are the same, d k is a scaling factor used to make the score value smoother and the gradient more stable.

[0105] The forward propagation update process based on the residual module, that is, the forward propagation corresponding to F(x)+x, can be expressed by the following expression:

[0106] h i =f forward (g i )+g i , (7)

[0107] Among them, h i represents the first high-level trajectory encoding vector after the current vehicle has learned the lane information; f forward It is the forward propagation processing function corresponding to the forward network.

[0108] The purpose of adding the residual module is to prevent the degradation problem that occurs during the training process of the deep neural network. Degradation means that as the number of layers of the deep neural network increases, the loss function gradually decreases and then tends to stabilize and reach saturation; the residual module can reduce overfitting.

[0109] In step S512, based on the first high-level trajectory coding vectors of each vehicle and the self-attention mechanism, a second interactive learning is performed between the vehicles to obtain a second high-level trajectory coding vector after the target vehicle has progressively learned the driving information of other vehicles; wherein the second high-level trajectory coding vector is used as the target coding vector.

[0110] The progressive learning here is because the cross-modal learning result outputted in step S511 is used as the input for subsequent interactions between vehicles, and double learning is performed in a progressive order, so it is described as progressive learning.

[0111] In this embodiment, the first high-level trajectory encoding vector h of each vehicle is i Splice to obtain a first spliced ​​code vector; for example, there are four first high-level trajectory code vectors h corresponding to vehicles A to D A ,h B ,h C ,h D , the first concatenated encoding vector obtained after concatenation is expressed as [h A ,h B ,h C ,h D ]; the first concatenated coding vector [h A ,h B ,h C ,h D ] are used as query vector Q, key vector K and value vector V respectively, and self-attention calculation is performed to obtain the second high-level trajectory encoding vector after the target vehicle progressively learns the driving information of other vehicles In this embodiment, is the target encoding vector.

[0112] The process of self-attention calculation is expressed as follows:

[0113]

[0114] Figure 5B The detailed implementation flow chart of step S430 according to another embodiment of the present disclosure is schematically shown.

[0115] In other embodiments, referring to Figure 5B As shown, the above step S430 specifically includes the following steps: S521 and S522.

[0116] In step S521, based on the trajectory feature coding vectors of each vehicle and the self-attention mechanism, a third interactive learning is performed between the vehicles to obtain a third high-level trajectory coding vector after the target vehicle has learned the driving information of other vehicles.

[0117] In the above step S521, the trajectory feature encoding vector of each vehicle is Splice to obtain the second spliced ​​coding vector. For example, there are four trajectory feature coding vectors corresponding to vehicles A to D. The second concatenated encoding vector obtained after concatenation is expressed as The second concatenated code vector As query vector Q, key vector K and value vector V respectively, self-attention calculation is performed to obtain the third high-level trajectory encoding vector U after the target vehicle learns the driving information of other vehicles i .

[0118] In step S522, based on the third high-level trajectory feature coding vector, the lane feature coding vector and the cross-attention mechanism, a fourth interactive learning is performed between the target vehicle and the lane to obtain a fourth high-level trajectory coding vector after the target vehicle has progressively learned the lane information; wherein the fourth high-level trajectory coding vector is used as the target coding vector.

[0119] In the above step S522, the third high-level trajectory encoding vector U of the target vehicle is i As the query vector Q, the lane feature encoding vector h L They are used as the key vector K and the value vector V respectively to perform cross-attention calculation and obtain the second attention weighted trajectory feature vector of the target vehicle that has learned the lane information.

[0120]

[0121] For the above second attention weighted trajectory feature vector q i Based on the residual module, the forward propagation update process is performed to obtain the fourth high-level trajectory encoding vector after the target vehicle has progressively learned the lane information. The specific expression is as follows:

[0122]

[0123] In this embodiment, is the target encoding vector.

[0124] In step S140, based on a pre-built decoding prediction model, the target coding vector is decoded and predicted to obtain a predicted trajectory of the target vehicle.

[0125] In some embodiments, the predicted trajectory of the target vehicle is obtained based on the prediction result of the reference coordinate system, predicting the short-term trajectory within a future time period of seconds.

[0126] For example, by decoding the target encoding vector based on the prediction model or Perform decoding prediction processing to obtain the predicted trajectory of the target vehicle

[0127] Since the output prediction trajectory In the reference coordinate system of the target vehicle, if you need to display the predicted trajectory in the global coordinate system, you can also use the translation rotation matrix Convert it to the global coordinate system, and the expression of the conversion process is as follows:

[0128]

[0129]

[0130] In an embodiment including steps S110 to S140, the historical trajectory vector sequence of each vehicle is processed based on a pre-constructed intention recognition model to obtain a driving intention coding vector corresponding to each vehicle. This driving intention coding vector can determine the driving intention of each vehicle at the current moment (such as movement direction intentions such as going straight, turning left, turning right, changing lanes left, changing lanes right, and movement speed change intentions such as acceleration, deceleration, and constant speed) and serve as an auxiliary reference for subsequent trajectory prediction; progressive interactive learning is performed between the target vehicle to be predicted and other vehicles and lanes based on the historical trajectory vector sequence, the driving intention coding vector and the lane vector sequence to obtain a target coding vector after the target vehicle comprehensively learns the corresponding environment of the surrounding vehicles and lanes; based on the pre-constructed decoding prediction model, the above-mentioned target coding vector is decoded and predicted to obtain the predicted trajectory of the target vehicle. In the process of progressive interactive learning, the target vehicle is learned successively with other surrounding vehicles and lanes, and the results of the previous learning are used as the input of the subsequent interactive learning to achieve progressive interactive learning. At the same time, the driving intention of each vehicle in the interactive learning process is used as an auxiliary reference. The combination of driving intention prediction and progressive interactive learning makes the target coding vector a learning result that comprehensively considers the surrounding environment and instantaneous driving intention. By introducing multiple factors and high-dimensional processing in the coding layer, subsequent decoding and prediction processing can realize the prediction of multiple possible trajectories and corresponding probabilities based on the driving intention, which can improve the accuracy of trajectory prediction.

[0131] The result of the above trajectory prediction can be multiple possible predicted trajectories and corresponding probability values. This trajectory prediction result can be further applied to a variety of downstream tasks, such as obstacle avoidance prompts, real-time route planning control, etc.

[0132] A second exemplary embodiment of the present disclosure provides a method for constructing a vehicle trajectory prediction model based on intention recognition assistance.

[0133] Figure 6A flowchart of a method for constructing a vehicle trajectory prediction model based on intention recognition assistance according to an embodiment of the present disclosure is schematically shown.

[0134] Reference Figure 6 As shown, the method for constructing a vehicle trajectory prediction model based on intention recognition assistance provided by an embodiment of the present disclosure includes the following steps: S610, S620, S630, S640, S650 and S660.

[0135] In step S610, historical driving information for training and lane information for training of each vehicle in a historical vehicle driving scenario are obtained.

[0136] In step S620, based on the preprocessing module 610, the above-mentioned historical driving information for training and the above-mentioned lane information for training are preprocessed to obtain a historical trajectory vector sequence for training and a lane vector sequence for training; the above-mentioned vehicle includes the target vehicle for prediction training and other vehicles within the preset scene range.

[0137] In step S630, based on the intention recognition model 620 to be trained, the training historical trajectory vector sequence of each vehicle is processed to obtain the training driving intention encoding vector corresponding to each vehicle.

[0138] In step S640, the training historical trajectory vector sequence, the training driving intention coding vector and the training lane vector sequence are input into the progressive interaction model 630 to be trained, and progressive interaction learning is performed between the target vehicle and other vehicles and lanes to obtain the target coding vector after the target vehicle comprehensively learns the corresponding environment of the surrounding vehicles and lanes.

[0139] The above-mentioned progressive interactive learning means that interactive learning between the target vehicle and other vehicles and lanes is performed in sequence and the results of the former learning are used as inputs of the latter.

[0140] In step S650, based on the decoding prediction model 640 to be trained, the target coding vector is decoded and predicted to obtain a training prediction trajectory of the target vehicle.

[0141] In step S660, a loss function is constructed based on the gap between the actual trajectory label of the target vehicle and the predicted trajectory for training, and the intention recognition model 620, the progressive interaction model 630 and the decoding prediction model 640 to be trained are trained synchronously. The preprocessing module 610 and the trained intention recognition model, the progressive interaction model and the decoding prediction model are constructed into a vehicle trajectory prediction model 700, as shown in FIG. Figure 6 Indicated by the dashed arrow.

[0142] Figure 7The structural block diagram of a vehicle trajectory prediction model according to an embodiment of the present disclosure is schematically shown.

[0143] Reference Figure 7 As shown, the vehicle trajectory prediction model 700 provided by the embodiment of the present disclosure includes: a preprocessing module 610, a trained intent recognition model 720, a trained progressive interaction model 730 and a trained decoding prediction model 740.

[0144] In some embodiments, reference Figure 7 As shown, the preprocessing module 610 has an input terminal K101, a first output terminal K102 and a second output terminal K103. In the use phase of the vehicle trajectory prediction model 700, the input terminal K101 is used to input the historical driving information and lane information of each vehicle, the first output terminal K102 is used to output the historical trajectory vector sequence, and the second output terminal K103 is used to output the lane vector sequence.

[0145] The trained intention recognition model 720 has a recognition input terminal K111 and a recognition output terminal K112; the above-mentioned recognition input terminal K111 is connected to the above-mentioned first output terminal K102, and the above-mentioned recognition output terminal K112 is used to output the driving intention coding vector corresponding to each vehicle.

[0146] The trained progressive interaction model 730 includes: a first feature extraction model 731, a second feature extraction model 732, and one of the following model groups: a model group consisting of a first interaction model 733 and a second interaction model 734, or a model group consisting of a third interaction model 735 and a fourth interaction model 736; the final output of the model group is used as the output of the above-mentioned progressive interaction model. Figure 7 The two configurations are shown separated by dashed lines.

[0147] Among them, the above-mentioned first feature extraction model 731 has a first extraction input terminal K121 and a first extraction output terminal K122; the above-mentioned first extraction input terminal K121 is connected to both the above-mentioned first output terminal K102 and the above-mentioned identification output terminal K112; the above-mentioned first extraction output terminal K122 is used to output the trajectory feature coding vector corresponding to each vehicle.

[0148] The second feature extraction model 732 has a second extraction input terminal K131 and a second extraction output terminal K132; the second extraction input terminal K131 is connected to the second output terminal K103; the second extraction output terminal K132 is used to output a lane feature coding vector.

[0149] The above-mentioned first interaction model 733 is used to perform the first interactive learning between each vehicle and the lane based on the cross-attention mechanism, and has a first interactive input terminal K141 and a first interactive output terminal K142; the above-mentioned first interactive input terminal K141 is connected to both the above-mentioned first extraction output terminal K122 and the above-mentioned second extraction output terminal K132; the above-mentioned first interactive output terminal K142 is used to output the first high-level trajectory encoding vector after each vehicle has learned the lane information.

[0150] The above-mentioned second interaction model 734 is used to perform second interactive learning between vehicles based on the self-attention mechanism, and has a second interactive input terminal K151 and a second interactive output terminal K152; the above-mentioned second interactive input terminal K151 is connected to the above-mentioned first interactive output terminal K142; the above-mentioned second interactive output terminal K152 is used to output a second high-level trajectory coding vector after the target vehicle progressively learns the driving information of other vehicles; wherein the above-mentioned second high-level trajectory coding vector is used as the target coding vector.

[0151] The third interaction model 735 is used to perform the third interaction learning between vehicles based on the self-attention mechanism, and has a third interaction input terminal K161 and a third interaction output terminal K162; the third interaction input terminal K161 is connected to the first extraction output terminal K122; the third interaction output terminal K162 is used to output the third high-level trajectory encoding vector after the target vehicle has learned the driving information of other vehicles.

[0152] The fourth interaction model 736 is used to perform the fourth interaction learning between the target vehicle and the lane based on the cross-attention mechanism, and has a fourth interaction input terminal K171 and a fourth interaction output terminal K172; the fourth interaction input terminal K171 is connected to the third interaction output terminal K162 and the second extraction output terminal K132; the fourth interaction output terminal K172 is used to output the fourth high-level trajectory coding vector after the target vehicle has progressively learned the lane information; wherein, the fourth high-level trajectory coding vector is used as the target coding vector.

[0153] The trained decoding prediction model 740 has a decoding input terminal K181 and a decoding output terminal K182; the above decoding input terminal K181 is connected to the output terminal K173 of the above progressive interaction model 730; for example, the output terminal 173 uses one of the second interaction output terminal K152 or the fourth interaction output terminal K172 as the output, referring to Figure 7 The above-mentioned decoding output terminal K182 is used to output the predicted trajectory of the target vehicle.

[0154] When the vehicle trajectory prediction model 700 is in use, the preprocessing module 610 preprocesses the historical driving information and lane information of each vehicle in the actual vehicle driving scene to obtain a historical trajectory vector sequence and a lane vector sequence. The above-mentioned vehicles include the target vehicle to be predicted and other vehicles within the preset scene range.

[0155] The input of the trained intention recognition model 720 is the above-mentioned historical trajectory vector sequence, and the output is the driving intention encoding vector corresponding to each vehicle.

[0156] Among them, the input of the above-mentioned first feature extraction model 731 is the above-mentioned historical trajectory vector sequence and the above-mentioned driving intention coding vector, and the output is the trajectory feature coding vector corresponding to each vehicle; the input of the above-mentioned second feature extraction model 732 is the above-mentioned lane vector sequence, and the output is the lane feature coding vector.

[0157] The first interaction model 733 is used to input the trajectory feature coding vector of each vehicle and the lane feature coding vector, and based on the cross-attention mechanism, performs the first interactive learning between each vehicle and the lane, and outputs the first high-level trajectory coding vector after each vehicle has learned the lane information; the output of the first interaction model 733 is used as the input of the second interaction model 734, and the second interaction model 734 is used to perform the second interactive learning between each vehicle based on the self-attention mechanism, and obtain the second high-level trajectory coding vector after the target vehicle has progressively learned the driving information of other vehicles; wherein the second high-level trajectory coding vector is used as the target coding vector. Or,

[0158] The third interaction model 735 is used to input the trajectory feature coding vectors of each vehicle, and based on the self-attention mechanism, perform the third interactive learning between the vehicles to obtain the third high-level trajectory coding vector after the target vehicle has learned the driving information of other vehicles; the output of the third interaction model 735 is used as an input of the fourth interaction model 736, and the other input of the fourth interaction model 736 is the lane feature coding vector output by the second feature extraction model 732. Based on the cross-attention mechanism, the fourth interactive learning is performed between the target vehicle and the lane to obtain the fourth high-level trajectory coding vector after the target vehicle has progressively learned the lane information; wherein, the fourth high-level trajectory coding vector is used as the target coding vector.

[0159] The trained decoding prediction model 740 performs decoding prediction processing on the target coding vector to obtain the predicted trajectory of the target vehicle.

[0160] In some embodiments, since the output predicted trajectory is in the reference coordinate system of the target vehicle, if the predicted trajectory needs to be displayed in the global coordinate system, it can also be converted to the global coordinate system through the translation and rotation matrix. For details, please refer to the coordinate conversion process example in the first embodiment, which will not be repeated here.

[0161] In some embodiments, reference Figure 3 As shown, the above-mentioned intention recognition model 720 includes: a large language model, a word segmentation module and an encoding module. The details of the large language model of this embodiment, the processing process of the specific intention recognition model, etc. can refer to the relevant description of the first embodiment, and will not be repeated here.

[0162] The construction method provided in this embodiment can be based on the progressive interactive learning between the target vehicle and other vehicles and lanes assisted by intention recognition. Through the combination of driving intention prediction and progressive interactive learning, the target coding vector is a learning result that comprehensively considers the surrounding environment and instantaneous driving intention. By introducing multiple factors and high-dimensional processing at the coding layer, subsequent decoding and prediction processing can be performed to predict multiple possible trajectories and corresponding probabilities based on driving intention. At the same time, through the training data set, each model can be trained synchronously and a vehicle trajectory prediction model can be obtained based on the trained model. The prediction accuracy of the constructed vehicle trajectory prediction model can be improved, and the construction complexity of the model can also be reduced.

[0163] A third exemplary embodiment of the present disclosure provides a vehicle trajectory prediction method based on a vehicle trajectory prediction model.

[0164] Reference Figure 7 As shown, the vehicle trajectory prediction method based on the vehicle trajectory prediction model of this embodiment includes:

[0165] Obtain the historical driving information and lane information of each vehicle in the vehicle driving scenario;

[0166] The above historical driving information and the above lane information are used as inputs of the vehicle trajectory prediction model, and the predicted trajectory of the target vehicle is output; wherein the above vehicle trajectory prediction model is a model constructed by the construction method provided in the second embodiment, for example Figure 7 Illustrated vehicle trajectory prediction model 700 .

[0167] A fourth exemplary embodiment of the present disclosure provides a vehicle trajectory prediction device assisted by intention recognition.

[0168] The above-mentioned device includes: a data acquisition module, a preprocessing module, an intention recognition module, an environment interaction learning module and a prediction module.

[0169] The data acquisition module is used to obtain the historical driving information and lane information of each vehicle in the vehicle driving scene.

[0170] The above-mentioned preprocessing module is used to preprocess the above-mentioned historical driving information and the above-mentioned lane information to obtain a historical trajectory vector sequence and a lane vector sequence; the above-mentioned vehicle includes a target vehicle to be predicted and other vehicles within a preset scene range.

[0171] The above-mentioned intention recognition module is used to process the historical trajectory vector sequence of each vehicle based on a pre-built intention recognition model to obtain the driving intention coding vector corresponding to each vehicle.

[0172] The above-mentioned environmental interactive learning module is used to perform progressive interactive learning between the target vehicle and other vehicles and lanes according to the above-mentioned historical trajectory vector sequence, the above-mentioned driving intention coding vector and the above-mentioned lane vector sequence, so as to obtain the target coding vector after the above-mentioned target vehicle comprehensively learns the corresponding environment of the surrounding vehicles and lanes; wherein, the above-mentioned progressive interactive learning means: the interactive learning between the target vehicle and other vehicles and lanes is performed in sequence and the result obtained by the former learning is used as the input of the latter.

[0173] The prediction module is used to perform decoding prediction processing on the target coding vector based on a pre-built decoding prediction model to obtain the predicted trajectory of the target vehicle.

[0174] For more details, beneficial effects, etc. of this embodiment, please refer to the relevant description of the first embodiment, which will not be repeated here.

[0175] A fifth exemplary embodiment of the present disclosure provides a device for constructing a vehicle trajectory prediction model based on intention recognition assistance.

[0176] The above-mentioned device includes: a training data acquisition module, a training preprocessing module, a training intention recognition module, a training environment interactive learning module, a training prediction module and a prediction model construction module.

[0177] The training data acquisition module is used to acquire training historical driving information and training lane information of each vehicle in a historical vehicle driving scenario.

[0178] The training preprocessing module is used to preprocess the training historical driving information and the training lane information to obtain a training historical trajectory vector sequence and a training lane vector sequence; the vehicle includes the target vehicle for prediction training and other vehicles within a preset scene range.

[0179] The training intention recognition module is used to process the training historical trajectory vector sequence of each vehicle based on the intention recognition model to be trained, so as to obtain the training driving intention encoding vector corresponding to each vehicle.

[0180] The above-mentioned training environment interactive learning module is used to input the above-mentioned training historical trajectory vector sequence, the above-mentioned training driving intention coding vector and the above-mentioned training lane vector sequence into the progressive interactive model to be trained, and perform progressive interactive learning between the target vehicle and other vehicles and lanes, so as to obtain the target coding vector of the above-mentioned target vehicle after comprehensively learning the corresponding environment of the surrounding vehicles and lanes; wherein, the above-mentioned progressive interactive learning means: the interactive learning between the target vehicle and other vehicles and lanes is performed in sequence and the result obtained by the former learning is used as the input of the latter.

[0181] The training prediction module is used to perform decoding prediction processing on the target coding vector based on the decoding prediction model to be trained, so as to obtain the training prediction trajectory of the target vehicle.

[0182] The prediction model construction module is used to construct a loss function based on the gap between the actual trajectory label of the target vehicle and the training prediction trajectory, and to train the intent recognition model, the progressive interaction model and the decoding prediction model simultaneously, and to construct the training preprocessing module and the trained intent recognition model, the progressive interaction model and the decoding prediction model into a vehicle trajectory prediction model. In this embodiment, in order to differentiate between the process of predictive model construction and the process of use, a training modifier is added to each module in the construction process. In fact, the processing logic of each module in the model training stage and the model use stage is the same and can be understood by reference to each other.

[0183] For more details, beneficial effects, etc. of this embodiment, please refer to the relevant descriptions of the first and second embodiments, which will not be repeated here.

[0184] A sixth exemplary embodiment of the present disclosure provides a vehicle trajectory prediction device based on a vehicle trajectory prediction model.

[0185] The above device includes: a data acquisition module and a prediction processing module.

[0186] The data acquisition module is used to obtain the historical driving information and lane information of each vehicle in the vehicle driving scene.

[0187] The prediction processing module is used to use the historical driving information and the lane information as inputs of a vehicle trajectory prediction model, and output a predicted trajectory of the target vehicle; wherein the vehicle trajectory prediction model is a model constructed using the construction method provided in the second embodiment, or a model constructed using the device provided in the fifth embodiment.

[0188] For more details, beneficial effects, etc. of this embodiment, please refer to the relevant descriptions of the first to third embodiments, which will not be repeated here.

[0189] Any number of the functional modules included in the apparatus of the fourth embodiment to the sixth embodiment above can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. At least one of the functional modules included in the apparatus of the fourth embodiment to the sixth embodiment above can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the functional modules included in the apparatus of the fourth embodiment to the sixth embodiment above can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0190] A seventh exemplary embodiment of the present disclosure provides an electronic device.

[0191] Figure 8 The structural block diagram of the electronic device provided by the embodiment of the present disclosure is schematically shown.

[0192] Reference Figure 8 As shown, the electronic device 800 provided in the embodiment of the present disclosure includes a processor 801, a communication interface 802, a memory 803 and a communication bus 804, wherein the processor 801, the communication interface 802 and the memory 803 communicate with each other through the communication bus 804; the memory 803 is used to store computer programs; the processor 801 is used to implement the vehicle trajectory prediction method assisted by intention recognition provided in the first embodiment, the vehicle trajectory prediction model construction method based on intention recognition assistance provided in the second embodiment, or the vehicle trajectory prediction method based on the vehicle trajectory prediction model provided in the third embodiment when executing the program stored in the memory.

[0193] The eighth exemplary embodiment of the present disclosure further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the vehicle trajectory prediction method assisted by intention recognition provided in the first embodiment, the vehicle trajectory prediction model construction method based on intention recognition assistance provided in the second embodiment, or the vehicle trajectory prediction method based on the vehicle trajectory prediction model provided in the third embodiment is implemented.

[0194] The computer-readable storage medium may be included in the device or apparatus described in the above embodiment; or it may exist independently without being assembled into the device or apparatus. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0195] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0196] It should be noted that the collection, collection, update, analysis, processing, use, transmission, storage and other aspects of user personal information involved in the technical solution provided by the embodiments of the present disclosure are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information and network security.

[0197] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0198] The foregoing is merely a specific embodiment of the present disclosure, which enables those skilled in the art to understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features applied herein.

Claims

1. A vehicle trajectory prediction method assisted by intention recognition, characterized in that: include: Acquire historical driving information and lane information of each vehicle in a vehicle driving scenario and perform preprocessing to obtain a historical trajectory vector sequence and a lane vector sequence; the vehicles include a target vehicle to be predicted and other vehicles within a preset scenario range; Based on the pre-built intention recognition model, the historical trajectory vector sequence of each vehicle is processed to obtain the driving intention encoding vector corresponding to each vehicle; According to the historical trajectory vector sequence, the driving intention coding vector and the lane vector sequence, a progressive interactive learning is performed between the target vehicle and other vehicles and lanes, so as to obtain a target coding vector after the target vehicle has comprehensively learned the corresponding environment of surrounding vehicles and lanes; wherein the progressive interactive learning means: the interactive learning between the target vehicle and other vehicles and lanes is performed in a sequential order and the result obtained by the former learning is used as the input of the latter; Based on a pre-built decoding prediction model, the target coding vector is decoded and predicted to obtain the predicted trajectory of the target vehicle.

2. The method according to claim 1, characterized in that According to the historical trajectory vector sequence, the driving intention coding vector and the lane vector sequence, progressive interactive learning is performed between the target vehicle and other vehicles and lanes to obtain a target coding vector after the target vehicle comprehensively learns the corresponding environment of surrounding vehicles and lanes, including: Performing feature extraction on the lane vector sequence to obtain a lane feature coding vector; Performing feature extraction on the historical trajectory vector sequence and the driving intention coding vector to obtain a trajectory feature coding vector corresponding to each vehicle; Based on the trajectory feature coding vector, the lane feature coding vector and the attention mechanism, progressive interactive learning is performed between the target vehicle and other vehicles and lanes to obtain a target coding vector after the target vehicle comprehensively learns the corresponding environment of surrounding vehicles and lanes.

3. The method according to claim 2, characterized in that Based on the trajectory feature encoding vector, the lane feature encoding vector and the attention mechanism, progressive interactive learning between the target vehicle and other vehicles and lanes is performed, including: Based on the trajectory feature coding vector of each vehicle, the lane feature coding vector and the cross-attention mechanism, a first interactive learning is performed between each vehicle and the lane to obtain a first high-level trajectory coding vector after each vehicle has learned the lane information; based on the first high-level trajectory coding vector of each vehicle and the self-attention mechanism, a second interactive learning is performed between each vehicle to obtain a second high-level trajectory coding vector after the target vehicle has progressively learned the driving information of other vehicles; wherein the second high-level trajectory coding vector is used as the target coding vector; or Based on the trajectory feature coding vectors of each vehicle and the self-attention mechanism, a third interactive learning is performed between the vehicles to obtain a third high-level trajectory coding vector after the target vehicle has learned the driving information of other vehicles; based on the third high-level trajectory feature coding vector, the lane feature coding vector and the cross-attention mechanism, a fourth interactive learning is performed between the target vehicle and the lane to obtain a fourth high-level trajectory coding vector after the target vehicle has progressively learned the lane information; wherein the fourth high-level trajectory coding vector is used as the target coding vector.

4. The method according to claim 3, characterized in that The first interactive learning between each vehicle and the lane is performed based on the trajectory feature coding vector of each vehicle, the lane feature coding vector and the cross attention mechanism to obtain a first high-level trajectory coding vector after each vehicle has learned the lane information, including: For each vehicle, the trajectory feature encoding vector of the current vehicle is used as the query vector, and the lane feature encoding vector is used as the key vector and the value vector respectively, and cross-attention calculation is performed to obtain the first attention weighted trajectory feature vector of the current vehicle that has learned the lane information; The first attention weighted trajectory feature vector is updated by forward propagation based on the residual module to obtain a first high-level trajectory encoding vector after the current vehicle has learned the lane information.

5. The method according to claim 3, characterized in that: The method of performing second interactive learning between the vehicles based on the first high-level trajectory encoding vectors of the vehicles and the self-attention mechanism to obtain a second high-level trajectory encoding vector after the target vehicle has progressively learned the driving information of other vehicles includes: splicing the first high-level trajectory encoding vectors of each vehicle to obtain a first spliced ​​encoding vector; The first concatenated encoding vector is used as a query vector, a key vector and a value vector respectively, and self-attention calculation is performed to obtain a second high-level trajectory encoding vector after the target vehicle progressively learns the driving information of other vehicles.

6. The method according to claim 3, characterized in that The third interactive learning between the vehicles is performed based on the trajectory feature coding vectors of the vehicles and the self-attention mechanism to obtain the third high-level trajectory coding vector after the target vehicle learns the driving information of other vehicles, including: The trajectory feature coding vectors of each vehicle are concatenated to obtain a second concatenated coding vector; The second concatenated encoding vector is used as a query vector, a key vector and a value vector respectively, and self-attention calculation is performed to obtain a third high-level trajectory encoding vector after the target vehicle learns the driving information of other vehicles.

7. The method according to claim 3, characterized in that Based on the third high-level trajectory feature encoding vector, the lane feature encoding vector and the cross attention mechanism, a fourth interactive learning is performed between the target vehicle and the lane to obtain a fourth high-level trajectory encoding vector after the target vehicle has progressively learned the lane information, including: The third high-level trajectory encoding vector of the target vehicle is used as a query vector, and the lane feature encoding vector is used as a key vector and a value vector respectively, and a cross-attention calculation is performed to obtain a second attention weighted trajectory feature vector of the target vehicle that has learned the lane information; The second attention weighted trajectory feature vector is forward propagated and updated based on the residual module to obtain a fourth high-level trajectory encoding vector after the target vehicle has progressively learned the lane information.

8. The method according to claim 1, characterized in that Obtain the historical driving information and lane information of each vehicle in the vehicle driving scene and perform preprocessing to obtain the historical trajectory vector sequence and lane vector sequence, including: Obtain scene collection data under vehicle driving scenes within a preset period before the current moment; According to the scene collection data, historical driving information of the target vehicle, driving information of other vehicles and lane information of multiple lanes in the environment are obtained; the driving information is a time sequence position sequence corresponding to time points in the preset time period; the lane information is a lane segment position sequence obtained by dividing each lane into multiple lane segments, each lane segment includes a starting position coordinate and an end position coordinate; For the time sequence position sequence and the lane segment position sequence, adjacent elements are respectively subjected to difference processing to obtain an initial historical trajectory vector sequence and an initial lane vector sequence of each vehicle; The initial historical trajectory vector sequence of each vehicle is determined as the historical trajectory vector sequence corresponding to each vehicle, and the initial lane vector sequence is determined as the lane vector sequence; or, the initial historical trajectory vector sequence of the vehicle is subjected to coordinate transformation processing, and the initial lane vector sequence is subjected to coordinate transformation processing, both of which are converted into the historical trajectory vector sequence and the lane vector sequence in a reference coordinate system with the observation angle of the target vehicle at the current moment; wherein the driving direction of the target vehicle at the current moment is taken as the first direction, and the first direction and a second direction perpendicular to the first direction are constructed as a reference coordinate system corresponding to the observation angle.

9. The method according to claim 8, characterized in that The scene acquisition data corresponds to a global coordinate system; In the case where the global coordinate system and the reference coordinate system are the same coordinate system, the following operations are performed: determining the initial historical trajectory vector sequence of each vehicle as the historical trajectory vector sequence corresponding to each vehicle, and determining the initial lane vector sequence as the lane vector sequence; When the global coordinate system and the reference coordinate system are different coordinate systems, perform one of the following operations: Determine the initial historical trajectory vector sequence of each vehicle as the historical trajectory vector sequence corresponding to each vehicle, and determine the initial lane vector sequence as the lane vector sequence; or, The initial historical trajectory vector sequence of the vehicle is subjected to coordinate conversion processing, and the initial lane vector sequence is subjected to coordinate conversion processing, and both are converted into the historical trajectory vector sequence and the lane vector sequence in a reference coordinate system with the observation angle of the target vehicle at the current moment.

10. The method according to claim 1, characterized in that The predicted trajectory of the target vehicle obtained is a prediction result based on the reference coordinate system, which predicts the short-term trajectory within a future second-level time period; the reference coordinate system is a coordinate system corresponding to the observation perspective of the target vehicle at the current moment; wherein, the driving direction of the target vehicle at the current moment is taken as the first direction, and the first direction and the second direction perpendicular to the first direction are constructed as the reference coordinate system.

11. The method according to any one of claims 1 to 10, characterized in that The intention recognition model includes: a large language model, a word segmentation module and an encoding module; The process of processing the historical trajectory vector sequence of each vehicle based on the pre-built intention recognition model to obtain the driving intention coding vector corresponding to each vehicle includes: For each vehicle, the historical trajectory vector sequence of the current vehicle is input into the large language model, and a natural language text result containing the driving intention of the current vehicle is output; the driving intention includes: movement direction intention and movement speed change intention; Based on the word segmentation module, the natural language text result is segmented to obtain a word element vector corresponding to the driving intention; Based on the encoding module, the word element vector corresponding to the driving intention is encoded to obtain the driving intention encoding vector corresponding to each vehicle.

12. The method according to claim 11, characterized in that The large language model in the intention recognition model is obtained by fine-tuning the pre-trained text processing type large language model, and the model fine-tuning corresponds to the following processing process: inputting the fine-tuning historical trajectory vector sequence corresponding to the vehicle with known driving intention in the historical vehicle driving scene into the text processing type large language model, and adjusting and optimizing the parameters of the text processing type large language model based on the known driving intention corresponding to each input, so that the intention recognition accuracy of the adjusted text processing type large language model for the fine-tuning historical trajectory vector sequence reaches a set threshold, and the fine-tuning is considered to be completed; or, The large language model is a pre-trained text processing type large language model. When the historical trajectory vector sequence of the current vehicle is input into the large language model, the corresponding prompt words for predicting possible results are also input, so that the large language model processes based on the input and the prompt words and obtains a natural language text result containing the driving intention of the current vehicle.

13. A method for constructing a vehicle trajectory prediction model based on intention recognition assistance, characterized in that: include: Obtain training historical driving information and training lane information of each vehicle in a historical vehicle driving scenario; Based on the preprocessing module, the training historical driving information and the training lane information are preprocessed to obtain a training historical trajectory vector sequence and a training lane vector sequence; the vehicle includes a target vehicle for prediction training and other vehicles within a preset scene range; Based on the intention recognition model to be trained, the training historical trajectory vector sequence of each vehicle is processed to obtain the training driving intention encoding vector corresponding to each vehicle; The training historical trajectory vector sequence, the training driving intention coding vector and the training lane vector sequence are input into the progressive interaction model to be trained, and progressive interactive learning is performed between the target vehicle and other vehicles and lanes to obtain a target coding vector after the target vehicle has comprehensively learned the corresponding environment of surrounding vehicles and lanes; wherein the progressive interactive learning means: the interactive learning between the target vehicle and other vehicles and lanes is performed in sequence and the result of the former learning is used as the input of the latter; Based on the decoding prediction model to be trained, the target coding vector is decoded and predicted to obtain a training prediction trajectory of the target vehicle; A loss function is constructed based on the gap between the real trajectory label of the target vehicle and the predicted trajectory for training, the intention recognition model, the progressive interaction model and the decoding prediction model are trained synchronously, and the preprocessing module and the trained intention recognition model, progressive interaction model and decoding prediction model are constructed into a vehicle trajectory prediction model.

14. The method according to claim 13, characterized in that The preprocessing module has a first output end and a second output end; in the use phase of the vehicle trajectory prediction model, the first output end is used to output a historical trajectory vector sequence, and the second output end is used to output a lane vector sequence; The trained intention recognition model has a recognition input end and a recognition output end; the recognition input end is connected to the first output end, and the recognition output end is used to output the driving intention encoding vector corresponding to each vehicle; The trained progressive interaction model includes: a first feature extraction model, a second feature extraction model, and one of the following model groups: a model group consisting of the first interaction model and the second interaction model, or a model group consisting of the third interaction model and the fourth interaction model; the final output of the model group is used as the output of the progressive interaction model; The first feature extraction model has a first extraction input terminal and a first extraction output terminal; the first extraction input terminal is connected to both the first output terminal and the identification output terminal; the first extraction output terminal is used to output the trajectory feature encoding vector corresponding to each vehicle; The second feature extraction model has a second extraction input terminal and a second extraction output terminal; the second extraction input terminal is connected to the second output terminal; the second extraction output terminal is used to output a lane feature encoding vector; The first interaction model is used to perform first interaction learning between each vehicle and the lane based on a cross attention mechanism, and has a first interaction input end and a first interaction output end; the first interaction input end is connected to both the first extraction output end and the second extraction output end; the first interaction output end is used to output a first high-level trajectory encoding vector after each vehicle has learned the lane information; The second interaction model is used to perform second interaction learning between each vehicle based on the self-attention mechanism, and has a second interaction input end and a second interaction output end; the second interaction input end is connected to the first interaction output end; the second interaction output end is used to output a second high-level trajectory encoding vector after the target vehicle progressively learns the driving information of other vehicles; wherein the second high-level trajectory encoding vector is used as the target encoding vector; The third interaction model is used to perform third interaction learning between various vehicles based on the self-attention mechanism, and has a third interaction input end and a third interaction output end; the third interaction input end is connected to the first extraction output end; the third interaction output end is used to output a third high-level trajectory encoding vector after the target vehicle learns the driving information of other vehicles; The fourth interaction model is used to perform a fourth interaction learning between the target vehicle and the lane based on the cross attention mechanism, and has a fourth interaction input end and a fourth interaction output end; the fourth interaction input end is connected to the third interaction output end and the second extraction output end; the fourth interaction output end is used to output a fourth high-level trajectory encoding vector after the target vehicle progressively learns the lane information; wherein the fourth high-level trajectory encoding vector is used as the target encoding vector; The trained decoding prediction model has a decoding input end and a decoding output end; the decoding input end is connected to the output end of the progressive interaction model; and the decoding output end is used to output the predicted trajectory of the target vehicle.

15. A vehicle trajectory prediction method based on a vehicle trajectory prediction model, characterized in that: The method comprises: Obtain the historical driving information and lane information of each vehicle in the vehicle driving scenario; The historical driving information and the lane information are used as inputs of a vehicle trajectory prediction model, and the predicted trajectory of the target vehicle is output; wherein the vehicle trajectory prediction model is a model constructed using the method of claim 13 or 14.

16. A vehicle trajectory prediction device assisted by intention recognition, characterized in that: include: A data acquisition module is used to obtain the historical driving information and lane information of each vehicle in the vehicle driving scene; A preprocessing module, used for preprocessing the historical driving information and the lane information to obtain a historical trajectory vector sequence and a lane vector sequence; the vehicle includes a target vehicle to be predicted and other vehicles within a preset scene range; The intention recognition module is used to process the historical trajectory vector sequence of each vehicle based on the pre-built intention recognition model to obtain the driving intention coding vector corresponding to each vehicle; An environment interactive learning module is used to perform progressive interactive learning between the target vehicle and other vehicles and lanes according to the historical trajectory vector sequence, the driving intention coding vector and the lane vector sequence, and obtain a target coding vector after the target vehicle has comprehensively learned the surrounding vehicle and lane corresponding environment; wherein the progressive interactive learning means: the interactive learning between the target vehicle and other vehicles and lanes is performed in sequence and the result of the former learning is used as the input of the latter; The prediction module is used to perform decoding prediction processing on the target coding vector based on a pre-built decoding prediction model to obtain the predicted trajectory of the target vehicle.

17. A vehicle trajectory prediction model construction device based on intention recognition assistance, characterized in that: include: A training data acquisition module is used to acquire training historical driving information and training lane information of each vehicle in a historical vehicle driving scenario; A training preprocessing module, used to preprocess the training historical driving information and the training lane information to obtain a training historical trajectory vector sequence and a training lane vector sequence; the vehicle includes a target vehicle for prediction training and other vehicles within a preset scene range; A training intention recognition module is used to process the training historical trajectory vector sequence of each vehicle based on the intention recognition model to be trained to obtain the training driving intention encoding vector corresponding to each vehicle; A training environment interactive learning module, used to input the training historical trajectory vector sequence, the training driving intention coding vector and the training lane vector sequence into the progressive interactive model to be trained, and perform progressive interactive learning between the target vehicle and other vehicles and lanes, so as to obtain a target coding vector after the target vehicle has comprehensively learned the surrounding vehicle and lane corresponding environment; wherein the progressive interactive learning means: performing interactive learning between the target vehicle and other vehicles and lanes in sequence and the result of the former learning is used as the input of the latter; A training prediction module, used to perform decoding prediction processing on the target coding vector based on the decoding prediction model to be trained, so as to obtain a training prediction trajectory of the target vehicle; A prediction model construction module is used to construct a loss function based on the gap between the real trajectory label of the target vehicle and the training predicted trajectory, synchronously train the intention recognition model, the progressive interaction model and the decoding prediction model, and construct the training preprocessing module with the trained intention recognition model, the progressive interaction model and the decoding prediction model into a vehicle trajectory prediction model.

18. A vehicle trajectory prediction device based on a vehicle trajectory prediction model, characterized in that: The device comprises: A data acquisition module is used to obtain the historical driving information and lane information of each vehicle in the vehicle driving scene; A prediction processing module, used to use the historical driving information and the lane information as inputs of a vehicle trajectory prediction model, and output a predicted trajectory of the target vehicle; wherein the vehicle trajectory prediction model is a model constructed using the method of claim 13 or 14, or a model constructed using the device of claim 17.

19. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory, used to store computer programs; A processor, for implementing the method according to any one of claims 1 to 15 when executing a program stored in a memory.

20. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 15 is implemented.

Citation Information

Cited By

  • Track generation method and system for autonomous vehicle and storage medium

    CN120840665A

  • Vehicle intention and trajectory prediction method based on large language model

    CN121148143A

  • Trajectory prediction method and apparatus, and vehicle

    WO2026157298A1