Traffic prediction method based on large model

Abstract

The invention provides a traffic prediction method based on a large model, and belongs to the technical field of intelligent traffic. The invention aims to realize efficient prediction of traffic data through an innovative space-time graph word segmentation device (STG-Tokenizer) and an adapter (STG-Adapter) by utilizing the strong semantic understanding and reasoning capability of the LLMs. The method mainly comprises the following parts: STG construction, an STG word segmentation device, an STG adapter, parameter fine tuning, model prediction, and performance evaluation and application. The word segmentation device module converts complex space-time graph data into simple marks and captures corresponding space and time relations. And the adapter module bridges the marked data with the LLMs through a linear coding layer and a decoding layer, so that the LLMs can understand the semantics of the traffic data. And the prediction model carries out prediction analysis on the traffic data by using the finely adjusted LLMs in combination with the STG-Tokenizer and the STG-Adapter. By designing the traffic prediction method based on the LLMs, the accuracy of traffic prediction is improved, and the method has wide application prospects and practical application value.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent transportation technology, and specifically relates to a method for predicting traffic flow based on capturing spatiotemporal correlations based on a large model. Background Art

[0002] With the acceleration of urbanization, intelligent transportation systems have developed rapidly, providing important support for solving problems such as traffic congestion and road planning, and effectively alleviating urban traffic pressure. In intelligent transportation systems, traffic flow prediction is a crucial link, which aims to use historical observation data to predict future road traffic conditions. Due to the complex spatiotemporal correlation, it is extremely challenging to mine this spatiotemporal correlation to achieve accurate predictions. In addition, in practical applications, route planning, traffic scheduling, and various data analyses are often required based on long-term prediction results. Therefore, a method and system for traffic flow prediction is urgently needed. Summary of the invention

[0003] In view of the shortcomings of the prior art, the present invention provides a traffic prediction method based on a large model. The present invention mainly includes several parts, including a real-time data collection module, a distributed traffic database, data preprocessing, a prediction model, an application platform and user feedback. The real-time data collection module is used to collect real-time road network information, traffic data and other external information (weather information, road section attributes, etc.), and store the collected data in a distributed traffic database. The data preprocessing module is used to extract data, remove noise and provide appropriate input for the prediction module. The prediction model is used to output predictions, provide support for the route planning, congestion prediction and other functions of the application platform, and accept user feedback to modify the model.

[0004] In order to achieve the above object, the present invention adopts the following technical solution:

[0005] Step 1: Represent traffic data as a graph structure, where nodes represent key locations in the traffic network, edges represent the connection relationships between nodes, and each node is accompanied by time series data.

[0006] Step 2: Convert complex spatial-temporal graph data into concise tags, each of which contains the time series data of the corresponding node and temporal semantic information.

[0007] Step 3: Through the linear encoding layer and decoding layer, the tokenized data is bridged with the understanding ability of the large model (LLM), so that the LLM can understand the semantics of the traffic data.

[0008] Step 4: Only a small set of parameters in the adapter are fine-tuned to effectively grasp the semantics of the tokens generated by the STG-Tokenizer while retaining the original natural language understanding ability of the LLM.

[0009] Step 5: Use the fine-tuned LLM in combination with STG-Tokenizer and STG-Adapter to perform predictive analysis on traffic data.

[0010] Step 6: Experiments are conducted on multiple traffic prediction benchmark datasets to verify the effectiveness of the proposed method and compare its performance with existing state-of-the-art methods.

[0011] Step 7: Apply this method to actual traffic prediction tasks, such as urban traffic flow prediction, Giti congestion prediction, etc., to achieve accurate prediction and traffic management decision support.

[0012] The specific steps of the second, third and fifth steps of the traffic prediction method based on the large model are as follows

[0013] Step 1: Assign time series data and time semantic information to each node to form an initial label.

[0014]

[0015] T g Represented by spatiotemporal data X t-L+1:t The generated token, Represents the token corresponding to node i.

[0016] Step 2: Add additional information to each token through embedding strategies, such as embedding for a specific time of day and embedding for a specific day of the week.

[0017]

[0018]

[0019] Among them B td Represents the time embedding of a node within a day, which divides a day into K 1 time, B dw Represents the day-of-week embedding of a node, which divides a week into k 2 Days, T t represents the time embedding of the node, E o Represents external information encoding. t It consists of two parts: one is Indicates a time of day, and the other part is Indicates the day of the week.

[0020] Step 3: Map the tokens generated by STG-Tokenizer to a dimensional space that LLM can understand through a linear encoding layer.

[0021] T q =W 1T e +b 1

[0022] in Indicates the tokens that the encoding has processed. D represents the trainable weight matrix, bias, and LLM embedding dimension respectively

[0023] Step 4: Combine the mapped tags with the prompt information related to the traffic prediction task to form the final tags input to the LLM.

[0024] T=T p ||T q

[0025] in Represents the combined token, Represents the tokens generated by the Text-Tokenizer from the prompt, and M represents the number of prompt tokens.

[0026] Step 5: Use LLM to process the labels combined with the prompt information to obtain a global representation of the transportation network.

[0027] H=LLM(T)

[0028] in Represents the final hidden state of the large model.

[0029] Step 6: The global representation of LLM is converted into a prediction of future traffic conditions through the decoding layer.

[0030]

[0031] in represents the prediction for the next P time steps, and represents the trainable weight matrix, and Represents trainable bias.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] (1) Provide a system for traffic flow prediction models. This system can convert complex spatiotemporal data into concise tags through STG-Tokenizer to help the prediction model achieve higher prediction accuracy, enable the model to better assist the application platform to complete downstream tasks, and optimize the model based on user feedback to improve user experience.

[0034] (2) The use of non-textual spatiotemporal data in the STG-Adapter model enables the large model to understand and process non-textual spatiotemporal data, improving the model's generalization ability to different traffic scenarios and data sets.

[0035] (3) Through overall system design and model optimization, the task of traffic flow prediction can be better accomplished. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a schematic diagram of the overall structure of the traffic flow prediction method designed by the present invention.

[0037] Figure 2 It is a schematic diagram of the traffic flow prediction model designed by the present invention. Specific implementation methods

[0039] The technical solution of the present invention will be further described below in conjunction with specific embodiments and drawings. The embodiments are only a part of the present solution and are only used to illustrate the present invention, but cannot be used to limit the scope of the present invention.

[0040] A traffic prediction method based on a large model, such as Figure 1 As shown, the steps are as follows:

[0041] Step 1: Construct a space-time graph to represent traffic data as a graph structure, where nodes and edges represent locations and connection relationships in the traffic network, respectively.

[0042] Step 2: Design STG-Tokenizer to convert space-time graph data into tags, each of which contains the time series data and temporal semantic information of the node.

[0043] Step 3: Implement STG-Adapter to bridge the tokenized data with LLM through the linear encoding layer and decoding layer.

[0044] Step 4: Fine-tune the parameters in STG-Adapter to understand the semantics of the tokens generated by STG-Tokenizer.

[0045] Step 5: Use the fine-tuned LLM in combination with STG-Tokenizer and STG-Adapter to perform predictive analysis on traffic data.

[0046] Step 6: Experiments are conducted on multiple traffic prediction benchmark datasets to verify the effectiveness of the method and compare its performance with existing methods.

[0047] Step 7: Apply this method to actual traffic prediction tasks to provide functions such as traffic flow prediction and traffic congestion prediction.

[0048] The second, third and fifth steps are based on large-scale traffic prediction methods, such as Figure 2 As shown, the specific steps are as follows:

[0049] Step 1: Assign time series data and time semantic information to each node to form an initial label.

[0050]

[0051] T g Represented by spatiotemporal data X t-L+1:t The generated token, Represents the token corresponding to node i.

[0052] Step 2: Add additional information to each token through embedding strategies, such as embedding for a specific time of day and embedding for a specific day of the week.

[0053]

[0054] Among them B td Represents the time embedding of a node within a day, which divides a day into K 1 time, B dw Represents the day-of-the-week embedding of a node, which divides a week into K 2 Days, T t represents the time embedding of the node, E o Represents external information encoding. t It consists of two parts: one is Indicates a time of day, and the other part is Indicates the day of the week.

[0055] Step 3: Map the tokens generated by STG-Tokenizer to a dimensional space that LLM can understand through a linear encoding layer.

[0056] T q =W 1 T e +b 1

[0057] in Indicates the tokens that the encoding has processed. D represents the trainable weight matrix, bias, and LLM embedding dimension respectively

[0058] Step 4: Combine the mapped tags with the prompt information related to the traffic prediction task to form the final tags input to the LLM.

[0059] T=T p ||T q

[0060] in Represents the combined token, Represents the tokens generated by the Text-Tokenizer from the prompt, and M represents the number of prompt tokens.

[0061] Step 5: Use LLM to process the labels combined with the prompt information to obtain a global representation of the transportation network.

[0062] H=LLM(T)

[0063] in Represents the final hidden state of the large model.

[0064] Step 6: The global representation of LLM is converted into a prediction of future traffic conditions through the decoding layer.

[0065]

[0066] in represents the prediction for the next P time steps, and represents the trainable weight matrix, and Represents trainable bias.

Claims

1. A traffic prediction method based on a large model, characterized by The following steps are involved: S1. Construct a space-time graph (STG): Represent traffic data as a graph structure, where nodes represent key locations in the traffic network, edges represent the connection relationships between nodes, and each node is accompanied by time series data. S2. Design a space-time graph tokenizer (STG-Tokenizer): Convert complex space-time graph data into concise tokens, each of which contains the time series data of the corresponding node and temporal semantic information. S3. Implementing the Space-Time Graph Adapter (STG-Adapter): Through the linear encoding layer and the decoding layer, the tokenized data is bridged with the understanding ability of the large model (LLM), so that the LLM can understand the semantics of the traffic data. S4. Fine-tuning parameters: Only a small set of parameters in the adapter are fine-tuned to effectively grasp the semantics of the tokens generated by the STG-Tokenizer while retaining the original natural language understanding ability of the LLM. S5. Traffic data prediction: Use the fine-tuned LLM in combination with STG-Tokenizer and STG-Adapter to perform predictive analysis on traffic data. S6.Performance Evaluation: Experiments are conducted on multiple traffic prediction benchmark datasets to verify the effectiveness of the proposed method and compare its performance with existing state-of-the-art methods. S7. Application Implementation: Apply this method to actual traffic prediction tasks, such as urban traffic flow prediction, Giti congestion prediction, etc., to achieve accurate prediction and traffic management decision support.

2. A traffic prediction method based on a large model as described in claim 1, characterized in that The space-time graph tokenizer (STG-Tokenizer) described in step S2 further includes the following sub-steps: Step 1: Assign time series data and time semantic information to each node to form an initial label. T g Represented by spatiotemporal data X t-L+1:t The generated token, Represents the token corresponding to node i. Step 2: Add additional information to each token through embedding strategies, such as embedding for a specific time of day and embedding for a specific day of the week. Among them B td represents the time embedding of a node within a day, which divides a day into K1 time periods, B dw Represents the embedding of the days of the week of a node, which divides a week into K2 days, T t represents the time embedding of the node, E o Represents external information encoding. t It consists of two parts: one is Indicates a time of day, and the other part is Indicates the day of the week.

3. A traffic prediction method based on a large model as described in claim 1, characterized in that The space-time graph adapter (STG-Adapter) described in step S3 further comprises the following sub-steps: Step 1: Map the tokens generated by STG-Tokenizer to a dimensional space that LLM can understand through a linear encoding layer. T q =W1T e +b1 in Indicates the tokens that the encoding has processed. D represents the trainable weight matrix, bias, and LLM embedding dimension respectively Step 2: Combine the mapped tags with the prompt information related to the traffic prediction task to form the final tags input to the LLM. T=T p ||T q in Represents the combined token, Represents the tokens generated by the Text-Tokenizer from the prompt, and M represents the number of prompt tokens.

4. A traffic prediction method based on a large model as described in claim 1, characterized in that The traffic data prediction described in step S5 further includes the following sub-steps: Step 1: Use LLM to process the labels combined with the prompt information to obtain a global representation of the transportation network. H=LLM(T) in Represents the final hidden state of the large model. Step 2: The global representation of LLM is converted into a prediction of future traffic conditions through the decoding layer. in represents the prediction for the next P time steps, and represents the trainable weight matrix, and Represents trainable bias.

Images