Large language model-based interpretable pedestrian trajectory prediction method and device

Abstract

The invention relates to the field of pedestrian trajectory prediction, in particular to an interpretable pedestrian trajectory prediction method and device based on a large language model. The method comprises the following steps: predicting a pedestrian historical trajectory by using a pre-trained deep learning model to obtain an initial pedestrian prediction trajectory; textualizing the pedestrian historical trajectory and the initial pedestrian prediction trajectory, and constructing a prompt template TP in combination with task requirements and response requirements; performing interpretable expansion on the prompt template TP to obtain a prompt template TPERG, and performing interpretable reasoning on the pedestrian trajectory by adopting an LLM model I based on the prompt template TPERG; performing trajectory optimization expansion on the prompt template TP to obtain a prompt template TPTPR, and based on the prompt template TPTPR, adopting an LLM model 2 to optimize an initial pedestrian prediction trajectory; and performing illusion check on the optimized pedestrian prediction trajectory to obtain a final pedestrian prediction trajectory. According to the invention, the interpretability of the prediction result of the deep learning model can be improved, and the pedestrian trajectory prediction precision is improved.

Description

Technical Field

[0001] The present invention relates to the field of pedestrian trajectory prediction, and in particular to a large language model-based and interpretable pedestrian trajectory prediction method and device. Background Art

[0002] Trajectory prediction aims to predict the future trajectory of an individual by analyzing their historical motion patterns. This technology plays a vital role in autonomous systems such as autonomous driving, behavioral analysis, and robotic path planning. In the field of intelligent driving, accurately predicting the future trajectory of surrounding traffic participants is a fundamental prerequisite for ensuring the safety of the driving system and the effectiveness of decision-making. Autonomous driving systems must not only accurately perceive the road environment and static obstacles, but also analyze and predict the movement trajectories of dynamic traffic participants (including pedestrians, vehicles, cyclists, etc.) in real time, so as to pre-plan the optimal path and implement necessary avoidance strategies.

[0003] With the increasing maturity of deep learning technology, deep learning methods have become the mainstream paradigm in the field of pedestrian trajectory prediction. Researchers have widely adopted deep learning architectures such as recurrent neural networks (RNNs), attention mechanisms, and graph neural networks to effectively extract temporal features from observed trajectories and model spatial interactions between neighboring pedestrians. Given the inherent randomness of pedestrian motion, multimodal prediction methods have gradually gained attention. Such methods aim to generate diverse future trajectories to better capture the diversity of pedestrian motion. Deep learning-based methods have achieved state-of-the-art results in trajectory prediction tasks and can predict longer time horizons than physics-based methods and classic machine learning-based methods. However, their "black box" nature still limits the interpretability and transparency of the models.

[0004] Therefore, there is an urgent need to solve the above technical problems to achieve both accuracy and interpretability. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to overcome the defects of the existing technology and provide an interpretable pedestrian trajectory prediction method based on a large language model, which can improve the interpretability of the prediction results of the deep learning model and improve the accuracy of pedestrian trajectory prediction.

[0006] In order to solve the above technical problems, the technical solution of the present invention is: a pedestrian trajectory prediction method based on a large language model and interpretable, comprising:

[0007] Use the pre-trained deep learning model to predict pedestrian historical trajectories and obtain the initial pedestrian prediction trajectory;

[0008] The pedestrian history trajectory and the initial pedestrian prediction trajectory are converted into text respectively to construct the pedestrian trajectory context C H, then combined with the task requirements I Q , response request R A Build into prompt template T P ;

[0009] Tip template T P Perform interpretability expansion to obtain the prompt template T P_ERG , based on the prompt template T P_ERG ,LLM model 1 is used to perform explanatory reasoning on pedestrian trajectories;

[0010] Tip template T P Perform trajectory optimization expansion to obtain the prompt template T P_TPR , based on the prompt template T P_TPR ,LLM model 2 is used to optimize the initial pedestrian prediction trajectory;

[0011] Perform hallucination check on the optimized pedestrian prediction trajectory to obtain the final pedestrian prediction trajectory.

[0012] Furthermore, the pedestrian trajectory context C H Includes historical trajectories, predicted trajectories, coordinates, speed, direction, trajectory collection of neighboring pedestrians, and motion pattern data, all in the format of text strings.

[0013] Furthermore, for the prompt template T P Expand interpretability; specifically include:

[0014] Pedestrian trajectory context C H Expand: Add natural language descriptions for deep learning models;

[0015] Task Requirements I Q Expand: Add natural language description for explanatory reasoning tasks and guide LLM model to consider pedestrian trajectory context C H Instructions on factors affecting the

[0016] Response requirements R A To expand: add instructions to instruct LLM to only output content related to explanatory reasoning and require the output to conform to a predefined format.

[0017] Furthermore, for the prompt template T P Conduct trajectory optimization expansion; specifically including:

[0018] Task Requirements I Q Expand: Use natural language to assign different weights to each influencing factor and limit the scope of optimization;

[0019] Response requirements R AExtension: Generate optimized prediction trajectories for a preset number of frames and output them in a predefined format.

[0020] Furthermore, the optimized pedestrian prediction trajectory is subjected to hallucination inspection, including:

[0021] Check frame by frame whether the Euclidean distance change between the optimized pedestrian prediction trajectory and the initial pedestrian prediction trajectory is within the threshold range;

[0022] Overall check whether the total deviation between the optimized pedestrian prediction trajectory and the initial pedestrian prediction trajectory is within the preset range;

[0023] If both the frame-by-frame inspection and the overall inspection meet the requirements, the optimized pedestrian prediction trajectory is used as the final pedestrian prediction trajectory;

[0024] Otherwise, the initial pedestrian prediction trajectory is used as the final pedestrian prediction trajectory.

[0025] Furthermore, the deep learning model is a TUTR model or a Social-GAN model.

[0026] The present invention also relates to an interpretable pedestrian trajectory prediction device based on a large language model, comprising:

[0027] The initial prediction module is used to predict pedestrian historical trajectories using a pre-trained deep learning model to obtain the initial pedestrian prediction trajectory;

[0028] Tip Template T P The construction module is used to convert the pedestrian history trajectory and the initial pedestrian prediction trajectory into text, and construct the pedestrian trajectory context C H , and combined with the task requirements I Q , response request R A Build into prompt template T P ;

[0029] Explanatory reasoning generation module for prompt template T P Perform interpretability expansion to obtain the prompt template T P_ERG , and based on the prompt template T P_ERG ,LLM model 1 is used to perform explanatory reasoning on pedestrian trajectories;

[0030] Trajectory prediction optimization module is used to optimize the prompt template T P Perform trajectory optimization expansion to obtain the prompt template T P_TPR , and based on the prompt template T P_TPR ,LLM model 2 is used to optimize the initial pedestrian prediction trajectory;

[0031] The hallucination check module is used to perform hallucination check on the optimized pedestrian prediction trajectory to obtain the final pedestrian prediction trajectory.

[0032] The present invention also relates to a device comprising:

[0033] memory for storing computer programs;

[0034] A processor is used to execute the computer program, and when the computer program is executed by the processor, the steps of the pedestrian trajectory prediction method based on a large language model and being interpretable are implemented.

[0035] The present invention also relates to a readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the large language model-based and interpretable pedestrian trajectory prediction method.

[0036] The present invention also relates to a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the large language model-based and interpretable pedestrian trajectory prediction method.

[0037] After adopting the above technical solution, the present invention has the following beneficial effects:

[0038] (1) Improving the interpretability of deep learning model prediction results: To address the problem of insufficient interpretability of pedestrian trajectory prediction due to the “black box” nature of deep learning models, unlike traditional methods such as latent layer analysis and sensitivity analysis, this paper proposes a method that reconstructs the numerical pedestrian trajectory prediction task into a textual natural language processing (NLP) task and applies LLM to perform explanatory reasoning on the prediction results of the deep learning model, which significantly enhances the transparency of the model.

[0039] (2) Improving pedestrian trajectory prediction accuracy: To further improve the prediction accuracy of deep learning models, this paper innovatively integrates LLM into traditional deep learning models, unlike methods such as LMTrj that use large models for direct prediction. It designs a language-based trajectory prediction result optimization method, which uses the time series prediction ability of LLMs to optimize the preliminary prediction results and significantly improves the prediction accuracy of deep learning models.

[0040] (3) Modularity and “plug-and-play” features: The present invention can be applied to different deep learning pedestrian trajectory prediction models in a “plug-and-play” manner without adjusting the internal structure or training process of the original model, which greatly reduces the deployment cost and adaptation difficulty, and improves the versatility and scalability of the framework.

[0041] (4) Improving the decision-making capabilities and safety of intelligent driving systems: This invention can provide intelligent driving systems with highly accurate and interpretable pedestrian trajectory prediction results, thereby assisting in driving decision-making. The interpretability of the prediction results enhances user trust in autonomous driving technology and further improves the safety of autonomous driving systems, which is of great significance to the promotion of autonomous driving technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 Flowchart of the large language model-based and interpretable pedestrian trajectory prediction method of the present invention;

[0043] Figure 2 This is a framework diagram of the large language model-based and interpretable pedestrian trajectory prediction method of the present invention;

[0044] Figure 3 A design drawing of the prompt template of the present invention;

[0045] Figure 4 This is a qualitative analysis diagram of the present invention;

[0046] Figure 5 A comparative analysis diagram for the explanatory reasoning of the present invention;

[0047] Figure 6 This is a diagram of the illusion phenomenon of the present invention. DETAILED DESCRIPTION

[0048] In order to make the contents of the present invention more clearly understood, the present invention is further described in detail below based on specific embodiments in conjunction with the accompanying drawings.

[0049] Example 1

[0050] like Figure 1 and Figure 6 As shown in FIG, a pedestrian trajectory prediction method based on a large language model and interpretable, including:

[0051] Step S1: Use a pre-trained deep learning model to predict pedestrian historical trajectories and obtain an initial pedestrian prediction trajectory;

[0052] Among them, the pedestrian trajectory prediction task involves inferring the future position information of the subject from the historical coordinate sequence. This task can generally be regarded as a time series to time series mapping process. Specifically, the deep learning model can be but not limited to the TUTR model or the Social-GAN model. Suppose that given a scene and a length of T obs The pedestrian history trajectory is used as the input of the pre-trained deep learning model, and the output is a length of T pred The initial pedestrian prediction trajectory;

[0053]

[0054] Step S2: Format the pedestrian’s historical trajectory and the initial pedestrian’s predicted trajectory into text, and construct the pedestrian trajectory context C. H , then combined with the task requirements I Q , response request R A Build into prompt template T P , specifically Figure 3 shown; among them,

[0055] S NLobs =TT(S obs );

[0056] S NLpred =TT(S pred );

[0057] TT() indicates text operation, S NLobs represents the textualized pedestrian history trajectory, S NLpred Represents the initial pedestrian prediction trajectory after textualization.

[0058] Pedestrian trajectory context C H Includes historical trajectories, predicted trajectories, coordinates, speed, direction, trajectory sets of nearby pedestrians, and motion pattern data, all in the form of text strings;

[0059] Specifically, floating-point trajectory coordinates are converted to decimal text strings to ensure compatibility with the natural language input requirements of the LLM. Trajectory coordinates are rounded to two decimal places to strike a balance between numerical accuracy and computational efficiency, while reducing prompt length to optimize the processing power of the LLM. These coordinates are formatted as a structured time series, where each data point is separated by a comma and enclosed in square brackets, enabling the LLM to efficiently parse temporal patterns. To fully account for model-specific contextual factors, this operation is consistently applied to historical trajectories, predicted trajectories, and adjacent trajectories. In addition, the average speed and direction of historical trajectories are used as additional input to provide the LLM with enhanced contextual representation, leading to a deeper understanding of pedestrian motion dynamics.

[0060] Mission Requirements I Q Serves as a prompt to provide task requirements to the LLM to prompt it to generate explanations for the prediction results of the deep learning model;

[0061] Response Request R A It is used to specify the format of LLM responses. Through this question-and-answer template, the pedestrian trajectory prediction task is successfully reconstructed into a natural language processing task, allowing LLM to effectively explain and optimize the prediction results of the deep learning model.

[0062] Specifically, the traditional pedestrian trajectory prediction task is redefined as a prompt-based natural language processing (NLP) task to fully utilize the prior knowledge of pre-trained LLM and deeply explore its potential in the field of time series prediction.

[0063] Step S3: P Perform interpretability expansion to obtain the prompt template T P_ERG , based on the prompt template T P_ERG ,LLM model 1 is used to perform explanatory reasoning on pedestrian trajectories;

[0064] This step innovatively separates the interpretable reasoning task in trajectory prediction to minimize the interfering context, thereby fully utilizing the LLM's ability to handle complex contextual relationships and conducting in-depth analysis of the implicit logic of the model prediction under given input conditions. In addition, this step develops a customized explanation strategy for the LLM model to match its specific influencing factors, thereby improving the adaptability and accuracy of the explanation. This work modifies the prompt template T P T P_ERG , providing customized input to LLM model 1, thus maximizing its reasoning ability. H 、Task Requirements I Q , response request R A The expansion is:

[0065] C H :This step expands C H , so that the LLM model can be explained based on the factors and architecture of the deep learning model. For the deep learning model, natural language is used to describe the data flow and incorporate it into the C H , to enhance the understanding of the LLM model over a pair of baseline models.

[0066] I Q :This step expands I Q , so that the LLM model can accurately identify the task of explanatory reasoning. It contains the natural language description of the explanatory reasoning task and the guidance of the LLM model to consider C H Instructions on factors affecting the

[0067] R A :In this step, R is expanded A , to strictly limit the output content and prevent the hallucination phenomenon in LLM model 1. A These include: (1) instructions for LLM to only output content related to explanatory reasoning. (2) requirements that the output must conform to a predefined JSON format to ensure better controllability and accuracy of the reasoning process.

[0068] Step S4: Prompt template T P Perform trajectory optimization expansion to obtain the prompt template T P_TPR , based on the prompt template T P_TPR ,LLM model 2 is used to optimize the initial pedestrian prediction trajectory;

[0069] This step is used to optimize the prediction results of the deep learning model, prompting template T P_TPR Pedestrian trajectory context C in H With prompt template T P Pedestrian trajectory context C in H This step expands I according to the new task Q and R A , as follows:

[0070] I Q :This step expands I Q , so that LLM model 2 can accurately and reasonably optimize the predicted trajectory. In addition to the task requirements, I Q It also includes: (1) using natural language to assign different weights to each influencing factor, such as using "most important" to identify the key factor for optimization, or "only as supplementary information" to identify less important factors. (2) limiting the optimization scope. The optimization scope can be limited to 10% to avoid extreme unreasonable adjustments.

[0071] R A :This step expands R A , to reduce the hallucination effect of LLM model 2 during the optimization of the initial forecast and provide standardized output to facilitate the extraction of information from the LLM response. A The generated 12-frame optimized trajectory points must be output in JSON format. Then, extract the "refinedtrajectory" part from the JSON format response generated by LLM to obtain the optimized trajectory.

[0072] Step S5: Perform hallucination check on the optimized pedestrian prediction trajectory to obtain the final pedestrian prediction trajectory.

[0073] This ensures that the optimized trajectory is within a reasonable range and avoids deviations caused by extreme adjustments. This step uses a double verification mechanism, first checking whether the change in the Euclidean distance frame by frame is within the threshold range, and then measuring whether the total deviation of the optimized trajectory is reasonable based on the overall change ratio. The specific process is as follows:

[0074] (1) Check frame by frame: For the Euclidean distance of each frame, let p i=(x i ,y i ) represents the coordinates of the i-th frame in the initial pedestrian prediction trajectory, r i =(x′ i ,y′ i ) represents the coordinates of the i-th frame in the optimized predicted trajectory. The frame-by-frame Euclidean distance is defined as (eq.5):

[0075]

[0076] Among them, if d i >τ (threshold), the optimization is judged to be beyond the reasonable range and does not meet the standard, thereby avoiding negative trajectory optimization caused by extreme adjustments of a single frame.

[0077] (2) Overall change rate check: First, calculate the total distance D of the initial pedestrian prediction trajectory orig The total distance D from the optimized predicted trajectory refined , N represents the total number of frames of the initial pedestrian prediction trajectory, which are calculated as follows:

[0078]

[0079] Then calculate the total rate of change R:

[0080]

[0081] If R>δ (total change rate threshold), the optimization is considered to be beyond the reasonable range and does not meet the standard; this avoids the drastic deviation of the trajectory caused by extreme adjustments to the total distance.

[0082] If both the frame-by-frame inspection and the overall inspection meet the requirements, the optimized pedestrian prediction trajectory is used as the final pedestrian prediction trajectory; otherwise, the initial pedestrian prediction trajectory is used as the final pedestrian prediction trajectory.

[0083] The method in this embodiment constructs a two-stage enhancement framework. The LLM interacts with existing deep learning pedestrian trajectory prediction models through a standardized interface. This modular design enables the framework to be applied to different deep learning models in a plug-and-play manner, effectively improving the interpretability and accuracy of prediction results without adjusting the internal structure or training process of the original model. Its core advantage lies in: on top of further improving the high prediction accuracy of deep learning models, the introduction of the LLM significantly enhances the interpretability of prediction results. Specifically, this embodiment introduces a zero-shot learning paradigm to perform explanatory reasoning and optimize predicted trajectories, thereby improving the transparency and accuracy of predictions. This improvement eliminates the incomprehensible "black box" nature of prediction results and enables clear, natural language explanations, such as "The pedestrian is accelerating along the positive X-axis and the negative Y-axis, and there are no obstacles in front of him." This provides more reliable decision-making for intelligent driving systems and enhances user trust in autonomous driving technology, which is crucial for improving autonomous driving safety.

[0084] The solutions involved in the above embodiments are verified in conjunction with specific experiments below.

[0085] 1. Dataset

[0086] In order to fairly compare the framework proposed in this paper, we strictly follow the benchmark of pedestrian trajectory prediction and use ETH / UCY as the evaluation dataset.

[0087] 2. Evaluation indicators

[0088] To evaluate the model's generalization and practical application, we followed a standard train-validate-test split strategy and employed a leave-one-out training approach, training the model on four subsets and testing it on the remaining subset. The diversity and complexity of this dataset provided a rigorous testing environment for the model's performance in handling diverse social interaction scenarios.

[0089] In the experiment, the present invention uses common evaluation metrics, including average displacement error (ADE) and final displacement error (FDE), to quantify the accuracy performance of the model at different time steps and prediction endpoints.

[0090] 3. Ablation research

[0091] Table 1: Ablation experiments of pedestrian prediction trajectory optimization (TPR) and hallucination checking (HC) in the Social-GAN model and the TUTR (Trajectory Unified Transformer for Pedestrian Trajectory Prediction) model. Columns 2-6 in the table are the ADE / FDE indicators of the five scenes in the ETH / UCY dataset, column 7 AVG is the average ADE / FDE, and column 8 is the accuracy improvement rate.

[0092]

[0093] 4. Quantitative analysis

[0094] To evaluate the contribution of Trajectory Optimization for Pedestrian Prediction (TPR), we conducted an ablation experiment. Table 1 shows a performance comparison of baseline models with and without TPR. TPR significantly improves trajectory prediction accuracy. For Social-GAN, ADE and FDE are reduced by 7% and 4.98%, respectively. For Social-GAN-(p) using pooling, the reductions are 3.73% and 3.27%, respectively. Even for the baseline model TUTR, which exhibits strong performance, ADE and FDE are reduced by 0.91% and 1.65%, respectively. This demonstrates that TPR, combined with two-stage prediction using LLM, effectively improves trajectory prediction accuracy. TPR leverages the powerful reasoning capabilities of LLM to process complex contextual information expressed in natural language. LLM is capable of processing multi-dimensional and interconnected data, enabling it to more accurately model both intrinsic motion patterns (e.g., speed, direction, or acceleration) and extrinsic social interactions (e.g., group behavior or avoidance strategies).

[0095] The “hallucination” phenomenon of large language models (LLMs) is a key factor affecting optimization results. During the optimization process, LLMs will inevitably produce unreasonable predictions, such as Figure 6Empty trajectories or unreasonable adjustments are shown, which severely impacts the evaluation metrics. Despite the optimization process's well-defined logical constraints and output rules, this problem persists and negatively impacts some prediction results. To mitigate the negative impact of hallucinations, we introduce hallucination checking (HC) into trajectory optimization for pedestrian prediction (TPR). To evaluate the effectiveness of HC, we conducted an ablation experiment comparing performance with and without HC. Table 1 shows the performance comparison with and without the HC module. After introducing HC, the ADE / FDE of Social-GAN was reduced by 7.82% / 6.02%, respectively. The ADE / FDE of Social-GAN-(p) was reduced by 4.85% / 4.42%. For TUTR, the ADE and FDE were reduced by 1.84% and 2.20%, respectively. These results demonstrate that the addition of HC further improves prediction accuracy. This demonstrates that HC can effectively address hallucination phenomena such as empty trajectories and extreme adjustments, thereby ensuring the physical plausibility of the optimized trajectories.

[0096] 5. Qualitative analysis

[0097] like Figure 4 As shown in (a), the optimized predictions show high consistency with the true trajectory in both dimensions, effectively capturing the dynamic characteristics of human motion, such as maintaining a constant velocity in two dimensions. Figure 4 In (b), the lack of motion pattern data makes it difficult for pedestrian prediction trajectory optimization (TPR) to accurately predict sudden changes in motion state. The TUTR model compensates for this limitation by providing more accurate trajectory data, which can capture transitions such as acceleration and deceleration. By utilizing LLM, pedestrian prediction trajectory optimization (TPR) shows a stronger ability to recognize motion features. Figure 4 As shown in (c), the optimized trajectory is closer to the actual trajectory than the baseline, demonstrating superior adherence to real-world motion dynamics. Qualitative analysis demonstrates that TPR based on the lightweight LLM effectively improves trajectory prediction performance in complex scenarios. Qualitative analysis shows that while TPR based on the LLM cannot actively adjust the trajectory using information beyond the influencing factors provided by the deep learning model, it can more effectively capture the motion information embedded in these factors compared to the deep learning model, thereby improving trajectory prediction performance.

[0098] 6. Interpretability Analysis

[0099] This paper verifies the validity of interpretable reasoning of trajectory prediction generated by LLM on two deep learning models. Figure 5As shown, LLM is able to generate personalized and reliable reasoning based on key influencing factors and model architecture. The explanatory reasoning for the Social-GAN model is more reliable than that for the TUTR model. This may be because the Social-GAN architecture is simpler and involves fewer influencing factors, which makes it easier for LLM to provide accurate explanations. Although the output format of LLM is clearly specified in the prompt template to alleviate the hallucination phenomenon, this problem still exists. LLM sometimes ignores key factors and provides unreasonable responses in low-probability scenarios. The present invention also compares the reasoning outputs of multiple LLMs using the same prompt template. Models with similar parameter scale to Llama3.1-8B, such as Qwen2-7B, ChatGLM3-6B and Phi-3.5-mini-instruct, mainly capture the overall trajectory trend, but lack precise kinematic analysis, resulting in lower reliability. In contrast, GPT-4, which has a larger parameter scale, performs well in extracting contextual information and performing mathematical reasoning, and can reveal hidden motion patterns and interactions. In summary, compared with other models of the same size, Llama3.1-8B significantly reduces hardware resource requirements and computational complexity while providing similar or even better interpretability. Compared with larger-scale models (such as GPT-4), although its explanatory power is slightly insufficient in some complex reasoning scenarios, its advantages in hardware and computational efficiency are more prominent, making it more feasible in practical applications.

[0100] Example 2

[0101] A large language model-based and interpretable pedestrian trajectory prediction device, comprising:

[0102] The initial prediction module is used to predict pedestrian historical trajectories using a pre-trained deep learning model to obtain the initial pedestrian prediction trajectory;

[0103] Tip Template T P The construction module is used to convert the pedestrian history trajectory and the initial pedestrian prediction trajectory into text, and construct the pedestrian trajectory context C H , and combined with the task requirements I Q , response request R A Build into prompt template T P ;

[0104] Explanatory reasoning generation module for prompt template T P Perform interpretability expansion to obtain the prompt template T P_ERG , and based on the prompt template T P_ERG ,LLM model 1 is used to perform explanatory reasoning on pedestrian trajectories;

[0105] Trajectory prediction optimization module is used to optimize the prompt template TP Perform trajectory optimization expansion to obtain the prompt template T P_TPR , and based on the prompt template T P_TPR ,LLM model 2 is used to optimize the initial pedestrian prediction trajectory;

[0106] The hallucination check module is used to perform hallucination check on the optimized pedestrian prediction trajectory to obtain the final pedestrian prediction trajectory.

[0107] Example 3

[0108] A device comprising:

[0109] memory for storing computer programs;

[0110] A processor is used to execute the computer program, and when the computer program is executed by the processor, the steps of the pedestrian trajectory prediction method based on a large language model and being interpretable as described in Example 1 are implemented.

[0111] Example 4

[0112] A readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the large language model-based and interpretable pedestrian trajectory prediction method as described in Example 1.

[0113] Example 5

[0114] A computer program product includes a computer program, which, when executed by a processor, implements the steps of the large language model-based and interpretable pedestrian trajectory prediction method as described in Example 1.

[0115] With the above-described preferred embodiments of the present invention as a guide, and with reference to the above description, relevant personnel are fully capable of making various changes and modifications without departing from the technical scope of this invention. The technical scope of this invention is not limited to the contents of the specification and must be determined according to the scope of the claims.

Claims

1. A large language model-based and interpretable pedestrian trajectory prediction method, characterized by: include: Use the pre-trained deep learning model to predict pedestrian historical trajectories and obtain the initial pedestrian prediction trajectory; The pedestrian history trajectory and the initial pedestrian prediction trajectory are converted into text respectively to construct the pedestrian trajectory context C H , then combined with the task requirements I Q , response request R A Build into prompt template T P ; Tip template T P Perform interpretability expansion to obtain the prompt template T P_ERG , based on the prompt template T P_ERG ,LLM model 1 is used to perform explanatory reasoning on pedestrian trajectories; Tip template T P Perform trajectory optimization expansion to obtain the prompt template T P_TPR , based on the prompt template T P_TPR ,LLM model 2 is used to optimize the initial pedestrian prediction trajectory; Perform hallucination check on the optimized pedestrian prediction trajectory to obtain the final pedestrian prediction trajectory.

2. The interpretable pedestrian trajectory prediction method based on a large language model according to claim 1, characterized in that Pedestrian trajectory context C H Includes historical trajectories, predicted trajectories, coordinates, speed, direction, trajectory collection of neighboring pedestrians, and motion pattern data, all in the format of text strings.

3. The interpretable pedestrian trajectory prediction method based on a large language model according to claim 1, characterized in that Tip template T P Expand interpretability; specifically include: Pedestrian trajectory context C H Expand: Add natural language descriptions for deep learning models; Task Requirements I Q Expand: Add natural language description for explanatory reasoning tasks and guide LLM model to consider pedestrian trajectory context C H Instructions on factors affecting the Response requirements R A To expand: add instructions to instruct LLM to only output content related to explanatory reasoning and require the output to conform to a predefined format.

4. The interpretable pedestrian trajectory prediction method based on a large language model according to claim 1, characterized in that Tip template T P Conduct trajectory optimization expansion; specific include: Task Requirements I Q Expand: Use natural language to assign different weights to each influencing factor and limit the scope of optimization; Response requirements R A Extension: Generate optimized prediction trajectories for a preset number of frames and output them in a predefined format.

5. The interpretable pedestrian trajectory prediction method based on a large language model according to claim 1, characterized in that: Perform hallucination checks on the optimized pedestrian prediction trajectory, including: Check frame by frame whether the Euclidean distance change between the optimized pedestrian prediction trajectory and the initial pedestrian prediction trajectory is within the threshold range; Overall check whether the total deviation between the optimized pedestrian prediction trajectory and the initial pedestrian prediction trajectory is within the preset range; If both the frame-by-frame inspection and the overall inspection meet the requirements, the optimized pedestrian prediction trajectory is used as the final pedestrian prediction trajectory; Otherwise, the initial pedestrian prediction trajectory is used as the final pedestrian prediction trajectory.

6. The interpretable pedestrian trajectory prediction method based on a large language model according to claim 1, characterized in that The deep learning model is the TUTR model or the Social-GAN model.

7. A pedestrian trajectory prediction device based on a large language model and capable of interpretation, characterized in that: include: The initial prediction module is used to predict pedestrian historical trajectories using a pre-trained deep learning model to obtain the initial pedestrian prediction trajectory; Tip Template T P The construction module is used to convert the pedestrian history trajectory and the initial pedestrian prediction trajectory into text, and construct the pedestrian trajectory context C H , and combined with the task requirements I Q , response request R A Build into prompt template T P ; Explanatory reasoning generation module for prompt template T P Perform interpretability expansion to obtain the prompt template T P_ERG ; and based on the prompt template T P_ERG ,LLM model 1 is used to perform explanatory reasoning on pedestrian trajectories; Trajectory prediction optimization module is used to optimize the prompt template T P Perform trajectory optimization expansion to obtain the prompt template T P_TPR , and based on the prompt template T P_TPR ,LLM model 2 is used to optimize the initial pedestrian prediction trajectory; The hallucination check module is used to perform hallucination check on the optimized pedestrian prediction trajectory to obtain the final pedestrian prediction trajectory.

8. A device, characterized in that include: memory for storing computer programs; A processor is used to execute the computer program, and when the computer program is executed by the processor, the steps of the large language model-based and interpretable pedestrian trajectory prediction method according to any one of claims 1 to 6 are implemented.

9. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the large language model-based and interpretable pedestrian trajectory prediction method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the large language model-based and interpretable pedestrian trajectory prediction method according to any one of claims 1 to 6 are implemented.

Images