Vehicle optimal lane changing opportunity prompting method and system based on pre-training LLMs trajectory prediction

Abstract

The invention relates to a vehicle optimal lane changing opportunity prompting method and system based on pre-training LLMs trajectory prediction, and belongs to the field of intelligent driving and intelligent traffic systems. The method comprises the following steps: establishing vehicle motion trajectory data through a trajectory prediction module, and generating an information joint encoding feature vector through an information joint encoder; processing the joint coding feature vector through a pre-trained LLMs and a multi-layer vector machine to obtain an advanced interactive representation feature vector; a long-short-term memory neural network of a multi-modal trajectory prediction module is established, and trajectory information of the target vehicle and surrounding vehicles in a future time domain is obtained through prediction; future trajectory information is input into a lane changing intention prediction module, and a lane changing scene coding module extracts vehicle lane changing intention features; the surrounding vehicle interaction modeling module is used for modeling and extracting context feature information of a lane changing scene; the lane changing intention output module outputs the maximum lane changing probability; and if the target vehicle needs to change the lane, the optimal lane changing opportunity is predicted through the optimal lane changing opportunity prediction module.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent driving and intelligent transportation systems, and relates to a method and system for prompting a vehicle's optimal lane-changing timing based on pre-trained LLMs trajectory prediction. Background Art

[0002] With the continuous growth of urbanization and the number of cars, the pressure on the traffic network has increased dramatically, urban traffic has gradually become more dense and complex, and the incidence of traffic accidents has greatly increased. At the same time, with the rapid development of autonomous driving technology, accurately predicting the best time for vehicles to change lanes has become one of the key points in the safety of autonomous driving. For the driving behavior of changing lanes, the lane change of the target vehicle will not only affect itself but also other vehicles, which is easy to cause traffic accidents. Therefore, the lane changing process needs to consider not only the current position of the surrounding vehicles, but also the history and future trajectories of the two and the influence between them. Changing lanes on urban roads is one of the main causes of collision accidents, and changing lanes on highways can easily cause car crashes and deaths. Therefore, being able to accurately find the best time to change lanes, reduce the impact on other vehicles, and maximize safety has become one of the keys to intelligent driving.

[0003] Existing trajectory prediction methods are mainly divided into two methods: traditional and deep learning-based. Traditional methods such as physical models, kinematic models, and Kalman filters usually rely on mature mathematical and statistical techniques to make predictions based on historical data and predefined models. However, these trajectory prediction methods have certain limitations when dealing with complex scenarios with intricate interactions and uncertainties. In recent years, deep learning-based methods have shown broader prospects. Deep learning methods use the powerful functions of artificial neural networks to learn complex patterns and relationships from large amounts of data. Generally, convolutional neural networks or recurrent neural networks are used to extract features from the historical trajectory of the vehicle and the surrounding road environment, and then predict the future driving logic. Although deep learning-based methods have made certain progress, existing methods still have certain limitations: First, there is a lack of cognition and understanding of scenes with complex traffic semantics. Existing methods are characterized by time dependence and spatial interaction, and pay more attention to learning the movement logic of vehicles from visual data. It is difficult to use more advanced semantic information such as road environment and traffic signal information, which is crucial to the accuracy of trajectory prediction; second, there is a lack of a clear mechanism for multimodal fusion to integrate various information and fully understand and utilize their complex interactions.

[0004] In addition, existing lane change intention recognition models such as physical models cannot capture long-term time series information well, and the lane change decision is essentially a long-term behavior that requires a certain time interval to complete. It is necessary to consider the driving behavior of surrounding vehicles in the future, and the same is true for finding the best lane change time. Therefore, an accurate prediction of the vehicle's future trajectory is needed as the basis for lane change intention recognition and lane change timing prompts, so as to achieve the best lane change timing. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a method and system for prompting the optimal lane changing timing of vehicles based on pre-trained LLMs trajectory prediction, which adopts a large language model to effectively process different data features, and perform more accurate, practical, and real multimodal trajectory prediction of traffic distribution, so as to accurately identify the lane changing intention for the predicted trajectory and provide the optimal lane changing timing for the target vehicle that needs to change lanes.

[0006] To achieve the above objectives, one aspect of the present invention provides a vehicle optimal lane change timing prompt system based on pre-trained LLMs trajectory prediction, the system comprising a trajectory prediction module, a lane change intention prediction module and a lane change optimal timing prediction module.

[0007] The trajectory prediction module includes an information joint coding submodule, an interactive information modeling submodule and a multimodal trajectory prediction submodule. The information joint coding submodule encodes the vehicle historical trajectory information and motion information of the target vehicle and surrounding vehicles to obtain a joint coding feature vector; the interactive information modeling submodule receives the joint coding feature vector, generates an interactive representation feature vector through pre-trained LLMs, and matches the dimension of the interactive representation feature vector with the dimension of the joint coding feature vector through a multi-layer perceptron to obtain a high-level interactive representation feature vector; the multimodal trajectory prediction submodule generates the predicted trajectory of the target vehicle and surrounding vehicles in the future time domain according to the high-level interactive representation feature vector.

[0008] The lane-changing intention prediction module predicts the behavior intention of the target vehicle based on the predicted trajectories of the target vehicle and surrounding vehicles in the future time domain; the lane-changing optimal timing prediction module finds the optimal lane-changing time through a Markov decision process when the behavior intention is to change lanes, and issues a prompt.

[0009] Furthermore, the lane-changing intention prediction module includes a lane-changing scene encoding submodule, a surrounding vehicle interaction modeling submodule and a lane-changing intention output submodule.

[0010] Among them, the lane-changing scenario encoding submodule encodes the predicted trajectory and extracts the vehicle's lane-changing behavior intention features; the surrounding vehicle interaction modeling submodule is used to represent the spatial relationship between the surrounding vehicles and the target vehicle, model the impact of the surrounding vehicles on the target vehicle, and further extract features; the lane-changing intention output submodule predicts the probability of the current target vehicle changing lanes to the left, changing lanes to the right, or staying straight based on the feature vector extracted by the surrounding vehicle interaction modeling submodule, and outputs the behavior intention with the highest probability.

[0011] Furthermore, the information joint encoding submodule encodes the historical trajectory information of the vehicle through a trajectory encoder, encodes the motion information of the vehicle through a motion encoder, and fuses the encoding vectors of the trajectory encoder and the motion encoder to obtain the joint encoding feature vector.

[0012] Another aspect of the present invention provides a method for prompting a vehicle's optimal lane change timing based on pre-trained LLMs trajectory prediction, the method comprising:

[0013] S1, obtaining the motion trajectory data set and historical time domain trajectory information of the target vehicle and surrounding vehicles through the vehicle-mounted sensor;

[0014] S2, jointly encode the motion trajectory data and the historical time domain trajectory information to obtain a jointly encoded feature vector;

[0015] S3, input the joint encoding feature vector into the pre-trained LLMs to generate an interactive representation feature vector, and match the dimension of the interactive representation feature vector with the dimension of the joint encoding feature vector through a multi-layer perceptron to generate a high-level interactive representation feature vector;

[0016] S4, combining historical data, the current state of the target vehicle and the traffic state around the target vehicle, decoding the high-level interaction representation feature vector, and generating the predicted trajectory of the target vehicle and surrounding vehicles in the future time domain;

[0017] S5. Input the predicted trajectories of the target vehicle and surrounding vehicles in the future time domain into the Transformer encoder to obtain the vehicle lane change intention feature, input the obtained vehicle lane change intention feature into the GCN network, and use the GCN network to represent the spatial relationship between the surrounding vehicles and the target vehicle, and further extract features;

[0018] S6. The features extracted by the GCN network are further input into the lane-changing intention output module composed of a linear fully connected layer and an activation function to obtain the target vehicle's behavior intention with the highest probability among the three behavior intentions of changing lanes to the left, changing lanes to the right, and keeping the current road straight;

[0019] S7. If the target vehicle has the highest probability of lane-changing intention, the Markov decision process is used to obtain the maximum expected long-term return, find the best lane-changing time point, and issue a prompt.

[0020] Furthermore, in step S2, the motion trajectory data and the spatial information are jointly encoded, specifically, the historical trajectory information of the vehicle is encoded by a trajectory encoder, and the motion information of the vehicle is encoded by a motion encoder, wherein both the trajectory encoder and the motion encoder are encoded using a gated recurrent unit. The encoding results of the trajectory encoder and the motion encoder are then connected in series to obtain a jointly encoded feature vector.

[0021] Furthermore, in step S7, in the process of maximizing the expected long-term return through the Markov decision process, for each strategy π of the agent, its corresponding action state value function Q π are the only solutions to the agent’s behavior and state at that time, so the optimal strategy π is solved by the greedy optimization algorithm. * :

[0022] π * (s,a)=argmaxQ π (s,a)

[0023]

[0024] The optimal strategy π is * The corresponding action state value function The time t in is the best time to change lanes.

[0025] Among them, s represents the state of the agent, a represents the action of the agent, N represents the final moment, γ represents the discount factor, and r t+k represents the direct reward obtained at time t+k, t represents the current time, s t and a t They represent the state and action of the agent at time t respectively.

[0026] The beneficial effects of the present invention are:

[0027] (1) The present invention introduces a large language model for interactive modeling, makes full use of high-level semantic information, and integrates semantic information and scene visual information. By integrating various data streams, it can integrate multiple prediction capabilities under various traffic conditions.

[0028] (2) The present invention uses Transformer as an encoder, which can not only consider the timing and continuity characteristics of the lane changing process, but also extract deeper lane changing behavior intention features. At the same time, the present invention uses GCN network modeling to enhance the dependency relationship between vehicles.

[0029] (3) To address the problem of inaccurate lane change timing selection, the present invention predicts lane change intention and provides the best lane change timing based on the vehicle's future trajectory, making the prediction and the best lane change timing more consistent with the actual situation.

[0030] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0032] Figure 1 A structural block diagram of a vehicle optimal lane-changing timing prompt system provided by an embodiment of the present invention;

[0033] Figure 2 It is a schematic diagram of the trajectory prediction module structure;

[0034] Figure 3 Schematic diagram of the process of fine-tuning parameters of large language models;

[0035] Figure 4 This is a schematic diagram of the lane change intention prediction module structure;

[0036] Figure 5 Schematic diagram of modeling the scene information representation module GCN;

[0037] Figure 6 Computational block diagram of a long short-term memory neural network unit. DETAILED DESCRIPTION

[0038] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0039] like Figure 1As shown, a vehicle optimal lane change timing prompt system based on pre-trained LLMs trajectory prediction is provided in one embodiment of the present invention. The system mainly includes a trajectory prediction module, a lane change intention prediction module and a lane change optimal timing prediction module.

[0040] Among them, the trajectory prediction module includes information joint encoding submodule, interactive information modeling submodule, and multimodal trajectory prediction submodule. The specific implementation process is as follows: Figure 2 As shown:

[0041] ① The information joint encoding submodule encodes the vehicle's historical trajectory information and motion information through the trajectory encoder and motion encoder respectively, and fuses them to generate a joint encoding feature vector g i The trajectory encoder focuses on the spatial collinear coordinates (x, y) of the vehicle, and the motion encoder is used to process the dynamic state of the vehicle, including speed and acceleration.

[0042] ② The interactive information modeling submodule uses the powerful understanding and reasoning capabilities of large language models (LLMs) and the NLP pre-trained Transformer architecture to calculate and interpret the potential interactions and relationships between traffic lights and vehicle trajectories, and performs advanced interactive modeling to generate interactive representation feature vectors z i Finally, the multi-layer perceptron MLP layer generates a high-level interactive representation feature vector s i .

[0043] ③The multimodal trajectory prediction submodule uses a long short-term memory neural network to fuse multiple data streams and integrate multiple prediction capabilities under various traffic conditions. It can fully understand vehicle dynamics and reliably and accurately predict vehicle trajectories. This module converts the high-level interactive representation feature vector s i As the input of the long short-term memory neural network, a sequence with temporal consistency and social acceptability is finally generated, and the trajectory information of the target vehicle and its surrounding vehicles in the future time domain is predicted.

[0044] The lane-changing intention prediction module includes a lane-changing scene encoding submodule, a vehicle-surrounding interaction modeling submodule, and a lane-changing intention output submodule. The specific implementation process is as follows: Figure 4 As shown:

[0045] ① The lane-changing scene encoding submodule uses the Transformer encoder to encode the predicted trajectory obtained by the multimodal trajectory prediction module. Considering the temporal and continuous characteristics of the lane-changing process, it not only extracts the hidden features of the vehicle's lane-changing intention, but also extracts deeper lane-changing behavior intention features. The lane-changing scene encoding submodule uses the Transformer's self-attention mechanism to allow each vehicle to notice the impact of the surrounding vehicle positions on its current lane-changing intention at each time step. The self-attention mechanism is expressed as follows:

[0046]

[0047] In the formula, Q represents the query vector, K and V are a set of key-value pairs, and d k is the input feature dimension.

[0048] The multi-head attention mechanism in the Transformer encoder is expressed as:

[0049] MultiHead(Q,K,V)=concat(head 1 ,head 2 ,...,head h )W O

[0050] head i =Attention(QW i Q ,KW i K ,VW i V )

[0051] Where W is the parameter matrix.

[0052] ② The surrounding vehicle interaction modeling submodule considers the impact of surrounding vehicles on the target vehicle’s driving trajectory, represents the spatial relationship between surrounding vehicles and the target vehicle through a graph convolutional network (GCN), and extracts contextual feature information of the lane-changing scene. The GCN modeling process of the surrounding vehicle interaction modeling submodule is as follows: Figure 5 shown.

[0053] The weekly vehicle interaction modeling submodule uses graph convolutional networks to enhance the interdependence between vehicles. When building the GCN model, spectral domain graph convolution is used to represent the spatial position information of the vehicle. The calculation formula is as follows:

[0054]

[0055] Among them, θ is the convolution sum, *g is the spectral domain graph convolution, x is the state information of the vehicle, σ is the activation function, k is the size of the graph convolution kernel, δ j is the convolution kernel parameter, L j is the normalized Laplacian matrix used to construct the graph.

[0056] ③ The lane-changing intention output submodule includes a linear fully connected layer and an activation function, where the fully connected layer outputs the predicted lane-changing behavior intention, and the Softmax activation function outputs the probability of changing lanes to the left, changing lanes to the right, and staying straight on the current road. This module ultimately uses the behavior operation with the highest probability as the predicted driving intention.

[0057]

[0058] Among them, U = 3, which means that there are only 3 output categories, namely, changing lanes to the left, changing lanes to the right, and staying straight on the current road. j , W u represents the corresponding weight matrix, u=0, 1, 2 represents the probability of judging the three categories of changing lanes to the left, changing lanes to the right, and keeping the current road straight once, x T is the transpose of the GCN output vector.

[0059] The optimal lane-changing timing prediction module uses a Markov decision process (MDP) to solve sequential decision problems in a dynamic environment, and is used to simulate the stochastic strategies and rewards that can be achieved by an intelligent agent in an environment where the system state has a Markov property. The module finds the optimal lane-changing timing by obtaining the maximum expected long-term reward.

[0060] Another embodiment of the present invention provides a method for prompting the best lane change timing of a vehicle based on pre-trained LLMs trajectory prediction, and the method is implemented in the following steps:

[0061] 1. Based on the roadside sensor equipment, obtain and establish a data set of vehicle movement trajectories at the intersection. The vehicle trajectory in the roadside area is composed of a series of discrete points, and the vehicle trajectory information includes lanes, historical trajectories and other information.

[0062] 2. Based on the information from the on-board sensors, the historical time-domain trajectory information of the target vehicle and surrounding vehicles is obtained, including longitudinal and transverse positions, lateral positions, vehicle speed, and acceleration.

[0063] Among them, the vehicle motion trajectory data set and historical time domain trajectory information can be collected through on-board sensors and processed on the vehicle side.

[0064] 3. The vehicle motion trajectory data and historical time domain trajectory information in step 1 and step 2 are jointly encoded through the information joint encoding module to obtain the information joint encoding feature vector g i .

[0065] Specifically, the trajectory encoder uses the gated recurrent unit GRU for encoding to capture the temporal changes in the vehicle's motion trajectory data, thereby gaining a deep understanding of the vehicle's motion trajectory:

[0066] E traj =GRU traj (x,y)

[0067] Among them, x and y are vectors composed of trajectory coordinate points, x represents the lateral position, and y represents the lateral position;

[0068] The calculation steps of GRU are as follows:

[0069]

[0070] Among them, x t Enter information for the current moment, h t-1 is the hidden state of the previous moment, h t is the hidden state passed to the next moment, To select the hidden state, r t and z t are reset gate and update gate respectively, σ is sigmoid function, tanh(·) is tanh function, W z , W r , W is the weight matrix.

[0071] The motion encoder uses a gated recurrent unit (GRU) to encode the motion state of the vehicle, providing insight into the real-time effects of vehicle motion:

[0072] E motion =GRU state (V,A)

[0073] Where V and A are the vectors composed of vehicle velocity and vehicle acceleration respectively. In the motion encoder, the calculation process of GRU is consistent with that of the trajectory encoder.

[0074] The feature E traj and E motion Connect them in series to get the joint encoding feature vector g i :

[0075] g i =Concat(E traj ,E motion )

[0076] Where Concat(·) represents the concatenation function.

[0077] 4. The information joint encoding feature vector in step 3 is input into the advanced interaction modeling layer (i.e., large language model) for further modeling calculations to explain the potential interactions and relationships between vehicle mutual motion and vehicle historical trajectories. The advanced interaction representation feature vector s is obtained through the large language model LLMs and multi-layer perceptron MLP i .

[0078] 1) If Figure 3 As shown, fine-tune the parameters Q and V of the large language model LLMs, first freeze all pre-trained parameters of LLMs;

[0079] 2) The required new trainable parameters are injected through the real-time low-rank adaptation technology LoRA, which is applied to the query vector Q and key value vector V of the attention layer; among them, for the pre-trained weight matrix W0 +ΔW=W 0 +BA, freeze parameter W during training 0 , only train the parameters in A and B;

[0080] 3) Pass the required trainable parameters into the network for presetting;

[0081] 4) Update the LoRA approximation in step 2) to: W f =W+BA,W f represents the updated weight matrix, W represents the pre-trained weight matrix, and B and A represent the trainable parameters;

[0082] 5) During the training process, W∈R d×k Keeping frozen, B and A are treated as trainable parameters, B is initialized to zero and A is initialized to a Gaussian distribution; therefore, the forward transfer function of LoRA can be concisely expressed as: a j represents the pre-trained forward transfer function, represents the updated forward transfer function;

[0083] 6) The joint encoding feature vector g i Passed to LLMs after the new trainable parameters have been injected via LoRA;

[0084] 7) Based on the pre-trained Transformer module and the new trainable parameters, the LLMs are trained first, and then the interaction representation feature vector z is generated through the trained LLMs i =LLMs(g i );

[0085] 8) z i After passing through the multi-layer perceptron MLP and the input g i The dimensions of the s are matched, and finally a high-level interactive representation feature vector s is generated. i The multi-layer perceptron MLP is a generalization of the perceptron. Its main feature is that it has multiple neuron layers, usually including an input layer, one or more hidden layers, and an output layer. The implementation process of MLP is as follows:

[0086]

[0087] Where w is the weight matrix, b is the bias vector, x is the input of the network, y is the output of the network, u is the intermediate vector, and sign is the activation function.

[0088] 5. Represent the high-level interaction feature vector s in step 4 iThe input is decoded into the multimodal trajectory prediction module, which is implemented by the LSTM framework and introduces a certain degree of randomness r, so that the model can generate a series of possible future trajectories. The multimodal trajectory prediction module is constructed as follows:

[0089] LSTM generates temporally consistent and socially acceptable sequences based on historical data, current ego vehicle status, and surrounding traffic conditions. This dynamic process can be summarized as:

[0090] Trajectory future =LSTM([s i ; r])

[0091] Trajectory future In order to generate a sequence of predicted trajectories with temporal consistency and social acceptability, r is an introduced vector, which is used to introduce a certain degree of randomness so that the model can generate a series of possible future trajectories, thereby improving its robustness.

[0092] like Figure 6 As shown, the calculation process of LSTM is as follows:

[0093]

[0094] Among them, i t is the input gate, f t For the forget gate, t is the output gate, c t is the memory state, h t is the output state of the network, h t-1 and c t-1 They are the network output state and memory unit at the previous moment, W xi , W hi , W ci , W xf , W hf , W cf , W xo , W ho , W co , W xc , W hc is the weight matrix, b i , b f , b o , b c is the bias vector.

[0095] 6. The future trajectory of the target vehicle and its surrounding vehicles is input into the Transformer encoder to obtain the vehicle lane change intention feature.

[0096] 7. The obtained vehicle lane change intention features are used through a graph convolutional network to obtain the close relationship between vehicles, and at the same time, the impact of surrounding vehicles on the current target vehicle is modeled to further extract features;

[0097] 8. The feature vector extracted by the graph convolutional network is input into the lane change intention output submodule, which outputs the probabilities of changing lanes to the left, changing lanes to the right, and staying straight on the current road through a fully connected layer and an activation function. The behavior operation with the maximum output probability will be regarded as the driving intention predicted by the model.

[0098] 9. If the current probability of the vehicle changing lanes is high, the Markov decision process (MDP) is used to obtain the maximum expected long-term return based on the lane-changing optimal timing prediction module, find the optimal lane-changing time point, and finally issue a prompt.

[0099] MDP can be expressed as a five-tuple (S, A, T, γ, R), where S is the state space, A is the behavior space, T is the state transition function, γ is the discount factor, and R is the feedback value of the benefit evaluation. π To evaluate the quality of the agent's strategy, as follows:

[0100]

[0101] Among them, t represents the current time, N represents the final time, r t+k Represents the direct reward obtained at time t+k.

[0102] For each strategy π of the agent, its corresponding action-state value function Q π are the only solutions to the agent’s behavior and state at that time. A greedy optimization algorithm is used to solve the optimal strategy π * :

[0103] π * (s,a)=argmaxQ π (s,a)

[0104] The optimal strategy π is * The corresponding action state value function The time t in is the best time to change lanes.

[0105] In the above embodiment, the specific training parameters of the trajectory prediction module and the lane change intention prediction module are described as follows:

[0106] 1. Trajectory prediction module

[0107] ① Hyperparameter setting: The batch size and initial learning rate are set to 132 and 0.001 respectively, and the model is trained for 50 epochs. First, the information joint encoding module is trained, followed by the interactive information modeling module, and finally the multimodal trajectory prediction module. The hidden dimension of all feature vectors is uniformly configured to 128.

[0108] ② Evaluation indicators: For the trajectory prediction module, the average displacement error (ADE) and the final displacement error (FDE) are used as evaluation indicators of trajectory accuracy. The definitions of the two are as follows:

[0109]

[0110] Where N is the total number of trajectories, T is the prediction range, is the actual ground coordinate, is the predicted coordinate of the ith trajectory at time t.

[0111] ③Loss function setting: Use smooth-L1 loss function:

[0112]

[0113] The smooth-L1 loss function is a loss function widely used in computer vision tasks such as target detection, object recognition, and face recognition. Compared with the L1 and L2 loss functions, the smooth-L1 loss function has the advantages of better robustness, faster convergence rate, and smoother gradient.

[0114] 2. Lane-changing intention prediction module

[0115] ① Hyperparameter settings: The batch size and initial learning rate are set to 64 and 0.001 respectively, and the model is trained for 50 epochs. The Transformer encoding module is trained first, followed by the GCN modeling module. The number of GCN network layers is set to 2, and the number of attention heads of the Transformer multi-head attention layer is set to 8.

[0116] ② Loss function setting: The smooth-L1 loss function is also used.

[0117] ③Model training: Use Pytorch to implement the framework of this module, and set the model seed to 1. Use mixed precision training to reduce memory consumption and speed up training.

[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A vehicle optimal lane change timing prompt system based on pre-trained LLMs trajectory prediction, characterized in that: The system includes a trajectory prediction module, a lane-changing intention prediction module, and a lane-changing optimal timing prediction module; The trajectory prediction module includes an information joint encoding submodule, an interactive information modeling submodule and a multimodal trajectory prediction submodule; the information joint encoding submodule encodes the vehicle historical trajectory information and motion information of the target vehicle and the surrounding vehicles to obtain a joint encoding feature vector; the interactive information modeling submodule receives the joint encoding feature vector, and generates an interactive representation feature vector through pre-trained LLMs, and matches the dimension of the interactive representation feature vector with the dimension of the joint encoding feature vector through a multi-layer perceptron to obtain a high-level interactive representation feature vector; the multimodal trajectory prediction submodule generates a predicted trajectory of the target vehicle and the surrounding vehicles in the future time domain according to the high-level interactive representation feature vector; The lane change intention prediction module predicts the behavior intention of the target vehicle based on the predicted trajectories of the target vehicle and surrounding vehicles in the future time domain; When the behavior intention is to change lanes, the optimal lane-changing timing prediction module finds the optimal lane-changing moment through a Markov decision process and issues a prompt.

2. The system according to claim 1, characterized in that The lane-changing intention prediction module includes a lane-changing scene encoding submodule, a surrounding vehicle interaction modeling submodule and a lane-changing intention output submodule; The lane-changing scenario encoding submodule encodes the predicted trajectory to extract the vehicle lane-changing behavior intention feature; The surrounding vehicle interaction modeling submodule is used to represent the spatial relationship between the surrounding vehicles and the target vehicle, model the impact of the surrounding vehicles on the target vehicle, and further extract features; The lane-changing intention output submodule predicts the probability of the current target vehicle changing lanes to the left, changing lanes to the right, or staying straight based on the feature vector extracted by the surrounding vehicle interaction modeling submodule, and outputs the behavior intention with the highest probability.

3. The system according to claim 1, characterized in that The information joint encoding submodule encodes the historical trajectory information of the vehicle through a trajectory encoder, encodes the motion information of the vehicle through a motion encoder, and fuses the encoding vectors of the trajectory encoder and the motion encoder to obtain the joint encoding feature vector.

4. A method for prompting the best lane-changing timing for vehicles based on pre-trained LLMs trajectory prediction, characterized in that: The method comprises: obtaining a motion trajectory data set and historical time domain trajectory information of a target vehicle and surrounding vehicles through an on-board sensor; Jointly encode the motion trajectory data and the historical time-domain trajectory information to obtain a joint encoding feature vector; Inputting the joint encoding feature vector into the pre-trained LLMs to generate an interactive representation feature vector, and matching the dimension of the interactive representation feature vector with the dimension of the joint encoding feature vector through a multi-layer perceptron to generate a high-level interactive representation feature vector; Combining historical data, the current state of the target vehicle and the traffic state around the target vehicle, decoding the high-level interactive representation feature vector, and generating predicted trajectories of the target vehicle and surrounding vehicles in the future time domain; The predicted trajectories of the target vehicle and surrounding vehicles in the future time domain are input into the Transformer encoder to obtain the vehicle lane change intention features. The obtained vehicle lane change intention features are input into the GCN network. The spatial relationship between the surrounding vehicles and the target vehicle is represented by the GCN network to further extract features. The features extracted by the GCN network are further input into a lane-changing intention output module composed of a linear fully connected layer and an activation function, so as to obtain the behavior intention with the highest probability among the three behavior intentions of the target vehicle, namely, changing lanes to the left, changing lanes to the right, and keeping the current road straight; If the probability of the target vehicle's lane-changing intention is the largest, the Markov decision process is used to obtain the maximum expected long-term return, find the best lane-changing time point, and issue a prompt.

5. The method according to claim 4, characterized in that The joint encoding of the vehicle motion trajectory data and the spatial information includes encoding the historical trajectory information of the vehicle through a trajectory encoder and encoding the motion information of the vehicle through a motion encoder; the trajectory encoder and the motion encoder both use a gated recurrent unit for encoding; The encoding results of the trajectory encoder and the motion encoder are connected in series to obtain the joint encoding feature vector.

6. The method according to claim 4, characterized in that In the process of maximizing the expected long-term return through the Markov decision process, for each strategy π of the agent, the corresponding action-state value function Q π are the only solutions to the agent’s behavior and state at that time, so the optimal strategy π is solved by the greedy optimization algorithm. * : p * (s,a)=argmaxQ π (s,a) The optimal strategy π is * The corresponding action state value function The time t in is the optimal time to change lanes; where s represents the state of the agent, a represents the action of the agent, N represents the final moment, γ represents the discount factor, and r t+k represents the direct reward obtained at time t+k, t represents the current time, s t and a t They represent the state and action of the agent at time t respectively.

Images