A traffic flow prediction method, apparatus, electronic device, and storage medium

Abstract

This application discloses a traffic flow prediction method, apparatus, electronic device, and storage medium. The method involves acquiring a traffic flow sequence within a first historical time period at a target intersection; standardizing each first traffic flow data point in the sequence to determine corresponding second traffic flow data; quantizing each second traffic flow data point within a preset historical window to determine corresponding first quantized characters; inputting each first quantized character into a large language model to determine second quantized characters for at least one sub-time period within the prediction window; and finally, dequantizing and destandardizing the second quantized characters to determine the corresponding predicted traffic flow data. This method achieves traffic flow prediction based on a large language model, avoiding the high generalization cost and poor scalability issues of existing traffic flow prediction models based on statistical, machine learning, and deep learning. It improves the generalization performance of traffic flow prediction models.

Description

Technical Field

[0001] This application relates to the field of traffic forecasting and analysis technology, and in particular to a traffic flow forecasting method, device, electronic device, and storage medium. Background Technology

[0002] Currently, intelligent transportation systems driven by big data and artificial intelligence technologies are evolving from perceptual intelligence to predictive and early warning intelligence. Traffic managers can assess the situation and make accurate decisions based on the prediction results. Therefore, accurate and efficient traffic flow prediction has a significant impact on reasonable traffic decisions, command and dispatch, and safety management.

[0003] Currently, mainstream prediction models in the transportation industry include statistical prediction, machine learning prediction, and deep learning prediction models. Statistical prediction models are simple and easy to use, but they are not effective at depicting complex traffic patterns. Machine learning prediction models are limited by manual feature dependency modeling, resulting in poor prediction performance. While end-to-end prediction models based on deep learning can improve the issues of feature dependency and prediction performance, deep learning prediction models heavily rely on scenario data. Due to the complexity and irregularity of traffic flow prediction scenarios, the accuracy of traffic flow prediction models based on deep learning cannot be guaranteed. Existing traffic flow prediction models based on statistical, machine learning, and deep learning suffer from poor generalization performance. Summary of the Invention

[0004] This application provides a traffic flow prediction method, apparatus, electronic device, and storage medium to address the problem of poor generalization performance in existing traffic flow prediction models based on statistics, machine learning, and deep learning.

[0005] In a first aspect, this application provides a traffic flow prediction method, the method comprising:

[0006] Obtain the traffic flow sequence within the first historical time period of the target intersection; wherein, the traffic flow sequence includes first traffic flow data within each sub-time period arranged in chronological order; standardize each first traffic flow data to determine the corresponding second traffic flow data;

[0007] The second traffic flow data within the preset historical window are quantized to determine the corresponding first quantization characters; wherein, the first quantization characters are characters that the large language model can recognize.

[0008] Each of the first quantized characters is input into the large language model, and the second quantized character within at least one sub-time period in the prediction window is determined based on the large language model; the second quantized character is dequantized and denormalized to determine the corresponding predicted traffic data.

[0009] The above technical solution has the following advantages or beneficial effects:

[0010] To address the poor accuracy of existing traffic flow prediction technologies, and considering the advantages of large language models, this application proposes a traffic flow prediction method based on a large language model. First, the traffic flow sequence within a first historical time period of the target intersection is obtained. Then, each first traffic flow data point in the sequence is standardized to determine the corresponding second traffic flow data. Next, each second traffic flow data point within a preset historical window is quantized to determine the corresponding first quantized characters. These first quantized characters are then input into the large language model, which determines the second quantized characters for at least one sub-time period within the prediction window. Finally, the second quantized characters are dequantized and destandardized to determine the corresponding predicted traffic flow data. This achieves traffic flow prediction based on a large language model, avoiding the high generalization cost and poor scalability issues of existing traffic flow prediction models based on statistics, machine learning, and deep learning. It improves the generalization performance of the traffic flow prediction model.

[0011] In one optional implementation, the standardization of each first traffic flow data point to obtain the corresponding second traffic flow data includes:

[0012] Determine the mean and standard deviation of each of the first traffic flow data; for each of the first traffic flow data, determine the difference between the first traffic flow data and the mean, and determine the ratio of the difference to the standard deviation as the standardized second traffic flow data corresponding to the first traffic flow data.

[0013] The above technical solution has the following advantages or beneficial effects:

[0014] In this application, when standardizing each first traffic flow data point, the mean and standard deviation of each first traffic flow data point are first determined. Then, for each first traffic flow data point, the difference between that first traffic flow data point and the mean is calculated. Finally, the ratio of this difference to the standard deviation is calculated, and this ratio is determined as the standardized second traffic flow data point corresponding to that first traffic flow data point. This improves the accuracy of traffic flow data standardization.

[0015] In one optional implementation, quantizing each second traffic flow data within a preset historical window and determining the corresponding first quantization characters includes:

[0016] Based on the first quantity of each second traffic flow data in the preset historical window and the second quantity of the preset predicted traffic flow data in the prediction window, the number of segmentation intervals is determined; based on the number of segmentation intervals and the percentile function of the standard normal distribution, each segmentation interval and the quantization character corresponding to each segmentation interval are determined.

[0017] For each of the second traffic flow data, a first segmentation interval to which the second traffic flow data belongs is determined; the quantization character corresponding to the first segmentation interval is determined as the first quantization character corresponding to the second traffic flow data.

[0018] The above technical solution has the following advantages or beneficial effects:

[0019] First, obtain the first quantity of each second traffic flow data point within the preset historical window. For example, if the preset historical window is 40 minutes and the sub-time period is 5 minutes, then the first quantity of each second traffic flow data point within the preset historical window is 40 / 5 = 8. Then, obtain the second quantity of the preset predicted traffic flow data within the prediction window. For example, if it is predetermined that traffic flow will be predicted every 5 minutes within the next 10 minutes based on the historical 40 minutes of data, then the second quantity of the preset predicted traffic flow data within the prediction window is determined to be 10 / 5 = 2. Determine the number of segmentation intervals based on the first and second quantities. Optionally, the sum of the first and second quantities can be used to determine the number of segmentation intervals. Based on the number of segmentation intervals and the percentile function of the standard normal distribution, determine each segmentation interval and its corresponding quantization character. Specifically, based on the number of segmentation intervals and the percentile function of the standard normal distribution, the number of segmentation intervals minus one boundary value can be determined. These boundary values, arranged from smallest to largest, yield the segmentation intervals. The smallest segmentation interval is from negative infinity to the smallest boundary value, and the largest segmentation interval is from the largest boundary value to positive infinity. For example, if there are 10 segmented intervals, the quantization characters for these 10 intervals, ordered from smallest to largest, are 1 to 10. That is, the smallest segmented interval corresponds to a quantization character of 1, and the largest segmented interval corresponds to a quantization character of 10. It should be noted that if there are more than 10 segmented intervals, the quantization character for the largest segmented interval will be a value greater than 10. For each second traffic flow data point, firstly, the first segmented interval to which the second traffic flow data belongs is determined; then, the quantization character corresponding to that first segmented interval is determined as the first quantization character for that second traffic flow data. This improves the accuracy of traffic flow data quantization.

[0020] In one optional implementation, the process of dequantizing the second quantized character includes:

[0021] Determine the second segmentation interval corresponding to the second quantized character; determine the two boundary values ​​of the second segmentation interval; and determine the average value of the two boundary values ​​as the value after the second quantized character is dequantized.

[0022] The above technical solution has the following advantages or beneficial effects:

[0023] In this application, each first quantized character is input into a large language model. After determining the second quantized character within at least one sub-time period in the prediction window based on the large language model, for at least one second quantized character, the second segmentation interval corresponding to the second quantized character is first determined; then, the two boundary values ​​of the second segmentation interval are determined, and the average value of the two boundary values ​​is determined. This average value is then used as the value after dequantization of the second quantized character. This improves the accuracy of dequantization of the second quantized character. It should be noted that for the smallest segmentation interval, that is, the segmentation interval from negative infinity to the smallest boundary value, the smallest boundary value can be used as the average value corresponding to the smallest segmentation interval; for the largest segmentation interval, that is, the segmentation interval from the largest boundary value to positive infinity, the largest boundary value can be used as the average value corresponding to the largest segmentation interval.

[0024] In one optional implementation, the value obtained by dequantizing the second quantized character is denormalized to determine the corresponding predicted traffic data, including:

[0025] Determine the product of the inverse quantized value and the standard deviation; the sum of the product and the average value is determined as the corresponding predicted flow data.

[0026] The above technical solution has the following advantages or beneficial effects:

[0027] In this application, for each second quantized character, after determining the inverse quantized value corresponding to that character, the product of the inverse quantized value and the standard deviation is calculated; then, the sum of this product and the mean is determined as the predicted traffic flow data corresponding to the inverse quantized value. This achieves traffic flow prediction based on a large language model. It should be noted that the standard deviation and mean are the mean and standard deviation of the determined first traffic flow data.

[0028] In one optional implementation, the training process of the large language model includes:

[0029] Obtain sample traffic flow sequences corresponding to each intersection within the second historical time period, and determine the training sample set based on the sample traffic flow sequences; wherein, the sample traffic flow sequences include first sample traffic flow data within each sub-time period arranged in chronological order;

[0030] For each sample traffic flow sequence in the training sample set, the traffic flow data of each first sample in the sample traffic flow sequence is standardized to determine the corresponding second sample traffic flow data; the second traffic flow data within the preset historical window and prediction window are quantized to determine the corresponding first sample quantized characters; wherein, the first sample quantized characters are characters that the large language model can recognize.

[0031] Each first sample quantized character within the preset historical window is input into the large language model to be trained. Based on the generative pre-trained model and low-rank adaptation network in the large language model, the semantic vectors of each first sample quantized character are extracted. Based on the semantic vectors, the predicted quantized character for at least one sub-time period within the prediction window is determined. Based on the predicted quantized character and the first sample quantized character for at least one sub-time period within the prediction window, a loss value is determined. The large language model is trained based on the loss value.

[0032] The above technical solution has the following advantages or beneficial effects:

[0033] When training a large language model, sample traffic flow sequences corresponding to each intersection within a second historical time period are obtained, and a training sample set is determined based on these sequences. Optionally, the sample traffic flow sequences can be directly used as the sample traffic flow sequences in the training sample set. Then, for each sample traffic flow sequence in the training sample set, the first sample traffic flow data in each sequence is standardized to determine the corresponding second sample traffic flow data. Next, the second traffic flow data within the preset historical window and prediction window are quantized to determine the corresponding first sample quantized characters. The standardization and quantization processes are similar to those described above for the first traffic flow data and will not be elaborated further. Then, the first sample quantized characters within the preset historical window are input into the large language model to be trained. Based on the generative pre-training model and low-rank adaptation network in the large language model, semantic vectors of each first sample quantized character are extracted. Based on the semantic vectors, predicted quantized characters for at least one sub-time period within the prediction window are determined. Then, based on the predicted quantized characters and the first sample quantized characters for at least one sub-time period within the prediction window, a loss value is determined. Finally, the large language model is trained based on the loss value. This enables large language models to predict traffic flow.

[0034] In one alternative implementation, training the large language model based on the loss value includes:

[0035] During training, the parameters of the generative pre-trained model are fixed, and the model parameters of the low-rank adaptation network are updated according to the loss value.

[0036] The above technical solution has the following advantages or beneficial effects:

[0037] To improve the efficiency of large language model training, this application trains the large language model through fine-tuning. Specifically, during training, the parameters of the generative pre-trained model are fixed, and the model parameters of the low-rank adapting network are updated based on the loss value. The parameters of the generative pre-trained model are pre-trained using an open-source large language model, and the low-rank adapting network is a network structure added to the large language model in this application.

[0038] In one alternative implementation, the process of determining the training sample set includes:

[0039] Obtain the sample traffic flow sequence for each intersection within the second historical time period;

[0040] Based on the traffic flow sequences of each sample, and based on the weight values ​​of each group, the traffic flow data at the corresponding time are weighted and summed to obtain the traffic flow sequences of each enhanced sample; based on the traffic flow sequences of each sample and the traffic flow sequences of each enhanced sample, the training sample set is determined; wherein, for each group of weight values, each weight value of the group is greater than 0, and the sum of the weight values ​​of the group is 1.

[0041] The above technical solution has the following advantages or beneficial effects:

[0042] To improve the predictive performance of the trained large language model, this application first augments the sample traffic flow sequences corresponding to each intersection within the acquired second historical time period. Specifically, based on each sample traffic flow sequence and its corresponding weight values, the traffic flow data for that time period are weighted and summed to obtain augmented sample traffic flow sequences. The training sample set is then determined based on these sample and augmented sample traffic flow sequences. For each weight value group, all weight values ​​are greater than 0, and the sum of all weight values ​​in that group is 1. This ensures that the augmented sample traffic flow sequences satisfy non-negativity and normalization conditions. The large language model is then trained based on the sample traffic flow sequences and augmented sample traffic flow sequences in the training sample set, thereby improving the predictive performance of the large language model.

[0043] Secondly, this application provides a traffic flow prediction device, the device comprising:

[0044] The acquisition module is used to acquire the traffic flow sequence within a first historical time period of the target intersection; wherein, the traffic flow sequence includes first traffic flow data within each sub-time period arranged in chronological order; and the first traffic flow data is standardized to determine the corresponding second traffic flow data.

[0045] The determination module is used to quantize each second traffic flow data within a preset historical window and determine the corresponding first quantization characters; wherein, the first quantization characters are characters that the large language model can recognize;

[0046] The prediction module is used to input the first quantized characters into the large language model, determine the second quantized characters of at least one sub-time period within the prediction window based on the large language model, and perform dequantization and denormalization on the second quantized characters to determine the corresponding predicted traffic data.

[0047] In one optional implementation, the acquisition module is specifically used to determine the average value and standard deviation of each of the first traffic flow data; for each of the first traffic flow data, determine the difference between the first traffic flow data and the average value, and determine the ratio of the difference to the standard deviation as the standardized second traffic flow data corresponding to the first traffic flow data.

[0048] In one optional implementation, the determining module is specifically configured to: determine the number of segmentation intervals based on the first quantity of each second traffic flow data within the preset historical window and the second quantity of preset predicted traffic flow data within the prediction window; determine each segmentation interval and the quantization character corresponding to each segmentation interval based on the number of segmentation intervals and the percentile function of the standard normal distribution; determine the first segmentation interval to which each second traffic flow data belongs for each second traffic flow data; and determine the quantization character corresponding to the first segmentation interval as the first quantization character corresponding to the second traffic flow data.

[0049] In one optional implementation, the prediction module is specifically used to determine the second segmentation interval corresponding to the second quantized character; determine the two boundary values ​​of the second segmentation interval; and determine the average value of the two boundary values ​​as the value after the second quantized character is dequantized.

[0050] In one optional implementation, the prediction module is specifically used to determine the product of the inversely quantized value and the standard deviation; and to determine the sum of the product and the average value as the corresponding predicted flow data.

[0051] In one alternative embodiment, the device further includes:

[0052] The training module is used to acquire sample traffic flow sequences corresponding to each intersection within a second historical time period, and determine a training sample set based on the sample traffic flow sequences. The sample traffic flow sequences include first sample traffic flow data within each sub-time period arranged in chronological order. For each sample traffic flow sequence in the training sample set, the first sample traffic flow data in the sequence is standardized to determine the corresponding second sample traffic flow data. The second traffic flow data within a preset historical window and a prediction window are quantized to determine the corresponding first sample quantized characters. The first sample quantized characters are characters that the large language model can recognize. The first sample quantized characters within the preset historical window are input into the large language model to be trained. Based on the generative pre-trained model and low-rank adaptation network in the large language model, semantic vectors of the first sample quantized characters are extracted. Based on the semantic vectors, predicted quantized characters for at least one sub-time period within the prediction window are determined. A loss value is determined based on the predicted quantized characters and the first sample quantized characters for at least one sub-time period within the prediction window. The large language model is trained based on the loss value.

[0053] In one alternative implementation, the training module is specifically used to update the model parameters of the low-rank adaptation network based on the loss value while the parameters of the generative pre-trained model are fixed during training.

[0054] In one optional implementation, the training module is further configured to acquire sample traffic flow sequences corresponding to each intersection within a second historical time period; based on each sample traffic flow sequence and each set of weight values, to perform a weighted summation of the traffic flow data for the corresponding time period to obtain each enhanced sample traffic flow sequence; and to determine the training sample set based on the sample traffic flow sequences and the enhanced sample traffic flow sequences; wherein, for each set of weight values, each weight value in the set is greater than 0, and the sum of each weight value in the set is 1.

[0055] Thirdly, this application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0056] Memory, used to store computer programs;

[0057] A processor, used to execute a program stored in memory, implements the method described.

[0058] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described herein.

[0059] Fifthly, this application provides a computer program product comprising an executable program that is executed by a processor to implement the method described. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 This application provides a schematic diagram of the first traffic flow prediction process.

[0062] Figure 2 This is a schematic diagram of the second traffic flow prediction process provided in this application;

[0063] Figure 3 A schematic diagram illustrating the process of determining each first quantization character provided in this application;

[0064] Figure 4 A schematic diagram of the third traffic flow prediction process provided in this application;

[0065] Figure 5 A schematic diagram illustrating the training process of the large language model provided in this application;

[0066] Figure 6 A schematic diagram of the traffic flow prediction model based on the large language fine-tuning model provided in this application;

[0067] Figure 7 A schematic diagram of traffic sequence tokenization provided in this application;

[0068] Figure 8 A schematic diagram of the framework for fine-tuning the LLaMA2 model based on LoRA provided in this application;

[0069] Figure 9 A schematic diagram of the traffic flow prediction device provided in this application;

[0070] Figure 10 A schematic diagram of the electronic device structure provided in this application. Detailed Implementation

[0071] To make the objectives and implementation methods of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the exemplary embodiments described are only some embodiments of this application, and not all embodiments.

[0072] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0073] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0074] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0075] The term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0076] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0077] For ease of explanation, the above description has been provided in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Various modifications and variations can be obtained based on the above teachings. The selection and description of the above embodiments are for the purpose of better explaining the principles and practical applications, thereby enabling those skilled in the art to better utilize the described embodiments and various different variations of embodiments suitable for specific use considerations.

[0078] This application proposes a traffic flow prediction method based on a large language model (LLM), innovatively exploring a traffic flow prediction approach based on large language model technology. Considering that LLM lacks the ability to model traffic flow data features and cannot directly input traffic flow sequence data, a strategy for fine-tuning LLM is proposed to construct a large language model suitable for traffic flow prediction. First, this application proposes a traffic sample augmentation mechanism based on time-series hybrid cross-validation. Targeting the different trend characteristics of traffic flow data from multiple intersections, time-series hybrid augmentation technology is used to expand a rich sample dataset with different traffic flow evolution characteristics, serving as the training sample set for fine-tuning the LLM. Second, a quantization strategy is proposed to convert traffic flow sequence data into token sequence data that can be recognized by LLM. This strategy utilizes the distribution characteristics of traffic sample data to achieve Gaussian distribution with equal probability segmentation of intervals. Then, the original traffic flow sequence data can be mapped to intervals and converted into characters, thereby achieving token sequence conversion. Finally, to enable the large model to have traffic flow feature modeling capabilities, a lightweight LoRA fine-tuning method for the LLaMA2 large model (denoted as T-LLaMA2) is used to train a prediction model adapted to traffic flow data. In the predictive inference stage, after obtaining the traffic character prediction results based on T-LLaMA2, the predicted characters are mapped to the distribution space of real traffic flow data through inverse quantization and inverse standardization operations, thereby obtaining the final traffic prediction results.

[0079] This application proposes a tokenization method for traffic flow sequence data, achieving equal-probability segmentation and quantization based on Gaussian distribution quantiles, effectively supporting accurate traffic flow sequence prediction based on LLM. It also proposes a traffic flow prediction method based on a large-scale language model, constructing a large-scale traffic flow sequence prediction model through fine-tuning training. This improves the generalization adaptability of traditional small-scale prediction models in different road environments, avoiding the high cost and scalability difficulties of training different prediction models for different scenario data, and providing zero-shot prediction capability.

[0080] In recent years, with the vigorous development of Large Language Model (LLM) technology represented by ChatGPT, LLM, with its zero-shot learning capability, has shown great potential in the field of time series prediction, stemming from the similarity between the mechanisms of time series prediction and word sequence prediction. Furthermore, LLM's inherent ability to model complex contextual dependencies can elevate the performance of time series prediction to a new level. However, traffic flow time series data and word sequence data are inherently heterogeneous, making it impractical to directly input traffic flow time series data into LLM. Therefore, this application proposes an adaptation mechanism to realize an LLM-based traffic flow prediction method. It achieves the construction of a training dataset based on mixed-sample augmentation, converts traffic flow sequences into token sequences through a Gaussian distribution quantization mechanism, and fine-tunes the LLM using a Low-rank Adaptation (LoRA) strategy to form a large-scale traffic flow sequence prediction model. This results in a unified traffic flow prediction model based on fine-tuned LLM, which can be generalized to any intersection for prediction.

[0081] Figure 1 The first traffic flow prediction process diagram provided for this application includes the following steps:

[0082] S101: Obtain the traffic flow sequence within the first historical time period of the target intersection; wherein, the traffic flow sequence includes first traffic flow data within each sub-time period arranged in chronological order; standardize each first traffic flow data to determine the corresponding second traffic flow data;

[0083] S102: Quantize each second traffic flow data within the preset historical window and determine the corresponding first quantized characters; wherein, the first quantized characters are characters that the large language model can recognize;

[0084] S103: Input each of the first quantized characters into the large language model, and determine the second quantized characters of at least one sub-time period within the prediction window based on the large language model; perform dequantization and denormalization on the second quantized characters to determine the corresponding predicted traffic data.

[0085] The traffic flow prediction method provided in this application is applied to electrochromic devices, which can be PCs, computers, servers, or other similar devices.

[0086] In this application, the intersection for which traffic flow prediction is performed is designated as the target intersection. First, the traffic flow sequence for the first historical time period of the target intersection is obtained. This sequence includes first traffic flow data for each sub-time period arranged chronologically. For example, a sub-time period might be 5 minutes, and the first historical time period could be the time period of one day prior to the current time. This results in a traffic flow sequence containing 288 first traffic flow data points arranged chronologically. Then, each first traffic flow data point is standardized to determine the corresponding second traffic flow data. In other words, standardizing these 288 first traffic flow data points yields 288 corresponding second traffic flow data points.

[0087] The electronic device quantizes each second traffic flow data point within a preset historical window to determine the corresponding first quantized characters; wherein the first quantized characters are characters that the large language model can recognize. The preset historical window is, for example, a 40-minute period prior to the current time, thus including eight second traffic flow data points. The electronic device quantizes these eight second traffic flow data points to determine the corresponding eight first quantized characters. The electronic device is equipped with a trained large language model for traffic flow prediction. Each first quantized character is input into the large language model, and based on the large language model, the second quantized character for at least one sub-time period within the prediction window is determined. The prediction window is, for example, 10 minutes after the current time, thus the large language model determines the second quantized characters corresponding to two sub-time periods within the prediction window, resulting in two second quantized characters. For at least one second quantized character, it is dequantized and denormalized to determine the predicted traffic data corresponding to the sub-time period of that second quantized character.

[0088] To address the poor accuracy of existing traffic flow prediction technologies, and considering the advantages of large language models, this application proposes a traffic flow prediction method based on a large language model. First, the traffic flow sequence within a first historical time period of the target intersection is obtained. Then, each first traffic flow data point in the sequence is standardized to determine the corresponding second traffic flow data. Next, each second traffic flow data point within a preset historical window is quantized to determine the corresponding first quantized characters. These first quantized characters are then input into the large language model, which determines the second quantized characters for at least one sub-time period within the prediction window. Finally, the second quantized characters are dequantized and destandardized to determine the corresponding predicted traffic flow data. This achieves traffic flow prediction based on a large language model, avoiding the high generalization cost and poor scalability issues of existing traffic flow prediction models based on statistics, machine learning, and deep learning. It improves the generalization performance of the traffic flow prediction model.

[0089] Figure 2 The second traffic flow prediction process provided in this application includes the following steps:

[0090] S201: Obtain the traffic flow sequence within the first historical time period of the target intersection; wherein, the traffic flow sequence includes first traffic flow data within each sub-time period arranged in chronological order; determine the average value and standard deviation of each first traffic flow data; for each first traffic flow data, determine the difference between the first traffic flow data and the average value, and determine the ratio of the difference to the standard deviation as the standardized second traffic flow data corresponding to the first traffic flow data;

[0091] S202: Quantize each second traffic flow data within the preset historical window and determine the corresponding first quantized characters; wherein, the first quantized characters are characters that the large language model can recognize;

[0092] S203: Input each of the first quantized characters into the large language model, and determine the second quantized characters of at least one sub-time period within the prediction window based on the large language model; perform dequantization and denormalization on the second quantized characters to determine the corresponding predicted traffic data.

[0093] In this application, when standardizing each first traffic flow data point, the mean and standard deviation of each first traffic flow data point are first determined. Then, for each first traffic flow data point, the difference between that first traffic flow data point and the mean is calculated. Finally, the ratio of this difference to the standard deviation is calculated, and this ratio is determined as the standardized second traffic flow data point corresponding to that first traffic flow data point. This improves the accuracy of traffic flow data standardization.

[0094] For example, a sub-time period of 5 minutes, and the first historical time period, such as the time period of one day prior to the current time, would result in a traffic flow sequence containing 288 first traffic flow data points arranged chronologically. The mean and standard deviation of these 288 first traffic flow data points are determined. For each of these 288 first traffic flow data points, the difference between that first traffic flow data point and the mean is determined. The ratio of this difference to the standard deviation is used to determine the standardized second traffic flow data point corresponding to that first traffic flow data point. This results in 288 standardized second traffic flow data points.

[0095] Figure 3 The process diagram for determining each first quantization character provided in this application includes the following steps:

[0096] S301: Determine the number of segmentation intervals based on the first quantity of each second traffic flow data in the preset historical window and the second quantity of the preset predicted traffic flow data in the prediction window; determine each segmentation interval and the quantization character corresponding to each segmentation interval based on the number of segmentation intervals and the percentile function of the standard normal distribution.

[0097] S302: For each of the second traffic flow data, determine the first segmentation interval to which the second traffic flow data belongs; and determine the quantization character corresponding to the first segmentation interval as the first quantization character corresponding to the second traffic flow data.

[0098] First, obtain the first quantity of each second traffic flow data point within the preset historical window. For example, if the preset historical window is 40 minutes and the sub-time period is 5 minutes, then the first quantity of each second traffic flow data point within the preset historical window is 40 / 5 = 8. Then, obtain the second quantity of the preset predicted traffic flow data within the prediction window. For example, if it is predetermined that traffic flow will be predicted every 5 minutes within the next 10 minutes based on the historical 40 minutes of data, then the second quantity of the preset predicted traffic flow data within the prediction window is determined to be 10 / 5 = 2. Determine the number of segmentation intervals based on the first and second quantities. Optionally, the sum of the first and second quantities can be used to determine the number of segmentation intervals. Based on the number of segmentation intervals and the percentile function of the standard normal distribution, determine each segmentation interval and its corresponding quantization character. Specifically, based on the number of segmentation intervals and the percentile function of the standard normal distribution, the number of segmentation intervals minus one boundary value can be determined. These boundary values, arranged from smallest to largest, yield the segmentation intervals. The smallest segmentation interval is from negative infinity to the smallest boundary value, and the largest segmentation interval is from the largest boundary value to positive infinity. For example, if there are 10 segmented intervals, the quantization characters for these 10 intervals, ordered from smallest to largest, are 1 to 10. That is, the smallest segmented interval corresponds to a quantization character of 1, and the largest segmented interval corresponds to a quantization character of 10. It should be noted that if there are more than 10 segmented intervals, the quantization character for the largest segmented interval will be a value greater than 10. For each second traffic flow data point, firstly, the first segmented interval to which the second traffic flow data belongs is determined; then, the quantization character corresponding to that first segmented interval is determined as the first quantization character for that second traffic flow data. This improves the accuracy of traffic flow data quantization.

[0099] Figure 4 The third traffic flow prediction process provided in this application includes the following steps:

[0100] S401: Obtain the traffic flow sequence within the first historical time period of the target intersection; wherein, the traffic flow sequence includes first traffic flow data within each sub-time period arranged in chronological order; determine the average value and standard deviation of each first traffic flow data; for each first traffic flow data, determine the difference between the first traffic flow data and the average value, and determine the ratio of the difference to the standard deviation as the standardized second traffic flow data corresponding to the first traffic flow data;

[0101] S402: Quantize each second traffic flow data within the preset historical window and determine the corresponding first quantized characters; wherein, the first quantized characters are characters that the large language model can recognize;

[0102] S403: Input each of the first quantized characters into the large language model, and determine the second quantized character of at least one sub-time period within the prediction window based on the large language model; determine the second segmentation interval corresponding to the second quantized character; determine the two boundary values ​​of the second segmentation interval; determine the average value of the two boundary values ​​as the value after inverse quantization of the second quantized character; determine the product of the inverse quantized value and the standard deviation; determine the sum of the product and the average value as the corresponding predicted flow data.

[0103] In this application, each first quantized character is input into a large language model. After determining the second quantized character within at least one sub-time period in the prediction window based on the large language model, for at least one second quantized character, the second segmentation interval corresponding to the second quantized character is first determined; then, the two boundary values ​​of the second segmentation interval are determined, and the average value of the two boundary values ​​is determined. This average value is then used as the value after dequantization of the second quantized character. This improves the accuracy of dequantization of the second quantized character. It should be noted that for the smallest segmentation interval, that is, the segmentation interval from negative infinity to the smallest boundary value, the smallest boundary value can be used as the average value corresponding to the smallest segmentation interval; for the largest segmentation interval, that is, the segmentation interval from the largest boundary value to positive infinity, the largest boundary value can be used as the average value corresponding to the largest segmentation interval.

[0104] In this application, for each second quantized character, after determining the inverse quantized value corresponding to that character, the product of the inverse quantized value and the standard deviation is calculated; then, the sum of this product and the mean is determined as the predicted traffic flow data corresponding to the inverse quantized value. This achieves traffic flow prediction based on a large language model. It should be noted that the standard deviation and mean are the mean and standard deviation of the determined first traffic flow data.

[0105] Figure 5A schematic diagram illustrating the training process of the large language model provided in this application includes the following steps:

[0106] S501: Obtain the sample traffic flow sequence corresponding to each intersection within the second historical time period, and determine the training sample set based on the sample traffic flow sequence; wherein, the sample traffic flow sequence includes the first sample traffic flow data within each sub-time period arranged in chronological order.

[0107] S502: For each sample traffic flow sequence in the training sample set, standardize each first sample traffic flow data in the sample traffic flow sequence to determine the corresponding second sample traffic flow data; quantize each second traffic flow data in the preset historical window and prediction window to determine the corresponding first sample quantized characters; wherein, the first sample quantized characters are characters that the large language model can recognize.

[0108] S503: Input each first sample quantized character within the preset historical window into the large language model to be trained; extract the semantic vector of each first sample quantized character based on the generative pre-trained model and low-rank adaptation network in the large language model; determine the predicted quantized character for at least one sub-time period within the prediction window based on the semantic vector; determine the loss value based on the predicted quantized character and the first sample quantized character for at least one sub-time period within the prediction window; train the large language model based on the loss value.

[0109] When training a large language model, sample traffic flow sequences corresponding to each intersection within a second historical time period are obtained, and a training sample set is determined based on these sequences. Optionally, the sample traffic flow sequences can be directly used as the sample traffic flow sequences in the training sample set. Then, for each sample traffic flow sequence in the training sample set, the first sample traffic flow data in each sequence is standardized to determine the corresponding second sample traffic flow data. Next, the second traffic flow data within the preset historical window and prediction window are quantized to determine the corresponding first sample quantized characters. The standardization and quantization processes are similar to those described above for the first traffic flow data and will not be elaborated further. Then, the first sample quantized characters within the preset historical window are input into the large language model to be trained. Based on the generative pre-training model and low-rank adaptation network in the large language model, semantic vectors of each first sample quantized character are extracted. Based on the semantic vectors, predicted quantized characters for at least one sub-time period within the prediction window are determined. Then, based on the predicted quantized characters and the first sample quantized characters for at least one sub-time period within the prediction window, a loss value is determined. Finally, the large language model is trained based on the loss value. This enables large language models to predict traffic flow.

[0110] In this application, training the large language model based on the loss value includes:

[0111] During training, the parameters of the generative pre-trained model are fixed, and the model parameters of the low-rank adaptation network are updated according to the loss value.

[0112] To improve the efficiency of large language model training, this application trains the large language model through fine-tuning. Specifically, during training, the parameters of the generative pre-trained model are fixed, and the model parameters of the low-rank adapting network are updated based on the loss value. The parameters of the generative pre-trained model are pre-trained using an open-source large language model, and the low-rank adapting network is a network structure added to the large language model in this application.

[0113] In this application, the process of determining the training sample set includes:

[0114] Obtain the sample traffic flow sequence for each intersection within the second historical time period;

[0115] Based on the traffic flow sequences of each sample, and based on the weight values ​​of each group, the traffic flow data at the corresponding time are weighted and summed to obtain the traffic flow sequences of each enhanced sample; based on the traffic flow sequences of each sample and the traffic flow sequences of each enhanced sample, the training sample set is determined; wherein, for each group of weight values, each weight value of the group is greater than 0, and the sum of the weight values ​​of the group is 1.

[0116] To improve the predictive performance of the trained large language model, this application first augments the sample traffic flow sequences corresponding to each intersection within the acquired second historical time period. Specifically, based on each sample traffic flow sequence and its corresponding weight values, the traffic flow data for that time period are weighted and summed to obtain augmented sample traffic flow sequences. The training sample set is then determined based on these sample and augmented sample traffic flow sequences. For each weight value group, all weight values ​​are greater than 0, and the sum of all weight values ​​in that group is 1. This ensures that the augmented sample traffic flow sequences satisfy non-negativity and normalization conditions. The large language model is then trained based on the sample traffic flow sequences and augmented sample traffic flow sequences in the training sample set, thereby improving the predictive performance of the large language model.

[0117] The traffic flow prediction process provided in this application will be described in detail below with reference to the accompanying drawings.

[0118] Figure 6 The traffic flow prediction model based on the large language fine-tuning model provided in this application includes the following steps:

[0119] S1: Traffic data sample enhancement;

[0120] S2: Traffic data scaling and quantification;

[0121] S21: Standard normalization of traffic flow data; S22: Quantification interval delineation of Gaussian distribution quantiles; S23: Symbol conversion of traffic flow sequence data;

[0122] S3: LoRA fine-tuning training of large language models;

[0123] S4: Predicted output dequantization and descaledation.

[0124] This application proposes a traffic flow prediction method based on a large language model (LLM). Considering the lack of traffic flow data features in LLM and the inability to directly input traffic flow sequence data, a strategy for fine-tuning LLM is proposed to construct a large language model suitable for traffic flow prediction. First, this application proposes a traffic sample augmentation mechanism based on time-series hybrid cross-validation. Targeting the different trend characteristics of traffic flow data from multiple intersections, time-series hybrid augmentation techniques are used to expand the dataset with rich sample data possessing different traffic flow evolution characteristics, serving as the training sample set for fine-tuning the LLM. Second, a quantization strategy is proposed to convert traffic flow sequence data into token sequence data recognizable by LLM. This strategy utilizes the distribution characteristics of traffic sample data to achieve equal probability segmentation of intervals based on Gaussian distribution, and then maps the original traffic flow sequence data to interval characters, thereby achieving token sequence conversion. Finally, to enable the large model to model traffic flow features, a lightweight LoRA-based fine-tuning of the LLaMA2 large model (denoted as T-LLaMA2) is used to train a large prediction model adapted to traffic flow data. In the prediction inference stage, after obtaining the traffic character prediction results based on T-LLaMA2, the predicted characters are mapped to the distribution space of real traffic flow data through inverse quantization and inverse scaling operations, thereby obtaining the final traffic prediction results.

[0125] The method provided in this application explores the application of large language models in intelligent transportation systems, constructs a large-scale traffic prediction model suitable for the field of traffic prediction, and achieves more efficient, accurate and generalized traffic prediction through a unified model, making traffic management more intelligent.

[0126] The purpose of this application is to provide a traffic flow prediction method based on a large-scale language model, which improves upon the poor generalization performance of traditional traffic prediction models due to their limitation by specific models. Specifically, the traffic flow prediction method based on a large-scale language model involved in this application includes the following steps:

[0127] Assuming a traffic network consisting of n intersections, each equipped with traffic flow monitoring sensors (e.g., electronic police, checkpoints, etc.), this application proposes a traffic flow prediction method based on a large-scale language model to construct a unified model for efficient and accurate prediction of traffic flow at the n intersections. This method achieves large-scale model training and inference prediction through traffic flow data augmentation, traffic flow data quantization and labeling, large-scale model fine-tuning training, and prediction output dequantization and descaling.

[0128] S1: Traffic data sample enhancement.

[0129] During the training phase, the limited number of sample sequences that can be collected due to the number of sensors may result in weak predictive power for large models. To train a traffic flow prediction model with strong generalization ability, this application employs time-series hybrid augmentation techniques to augment the original multi-intersection traffic data corpus.

[0130] Specifically, to enhance the predictive generalization ability of large models, a traffic data sample augmentation technique based on time series hybrid augmentation is implemented to expand the original traffic data X. 0 Forming enhanced sample X 1 This leads to the formation of a rich set of traffic sample data X = {X} 0 ,X 1}. Where X 0 Composed of n intersection traffic flow sequences, X 1 It consists of m enhanced sample flow sequences.

[0131] The core concept of time series mixing is to randomly sample several segments from different traffic flow time series, and then mix them together in a certain proportion to generate a new traffic flow series. This process can be formalized as follows: in arrive It consists of k time series segments sampled from the original traffic dataset, λ1 to λ2. k These are the corresponding mixing coefficients, satisfying the non-negativity and normalization conditions. By adjusting k and λ, time series with varying diversity and complexity can be generated.

[0132] Although the traffic sequences generated using time series mixing techniques are artificially synthesized, they can simulate the time series evolution patterns of various traffic trends in the real world. Adding this synthesized, enhanced data to a pre-training corpus can improve the generalization ability and robustness of large models in traffic prediction.

[0133] S2: Traffic data scaling and quantification.

[0134] General large-scale language models (LLMs) are deep learning prediction models trained on massive amounts of text data, and their input adaptation format is text corpus data. However, traffic flow data is traffic flow time series data that cannot be directly input into LLMs for prediction. Therefore, this application proposes a new traffic flow data quantization strategy, which converts traffic flow data into a token sequence suitable for LLM input through scaling and quantization, thereby enabling traffic flow prediction using (fine-tuned) large-scale language models.

[0135] S21: Traffic data standard normalization.

[0136] First, to reduce subsequent computational complexity and decrease the training and inference costs of large models, a scaling strategy can be used to map traffic flow data at different scales to a suitable range. This application employs the z-score normalization method to scale the original training sample sequence into a new sample x′:

[0137]

[0138] Where μ and σ represent the mean and standard deviation of the feature dimension to which the training sample belongs.

[0139] S22: Quantitative intervals for Gaussian distribution quantiles.

[0140] Assume the scaled sequence is x′ 1:H+F =[x ′ 1,…,x ′ H ,…,x′ H+F ], where H represents the historical correlation time step and F represents the prediction time step. Initial scaled sequence x′ 1:H+F These are still real-valued numbers and cannot be directly processed by the language model. To convert these real-valued numbers into discrete symbols, this application employs quantization techniques to convert the real-valued data into a token sequence.

[0141] Mapping continuous real values ​​to discrete token IDs is equivalent to dividing the real number axis into several intervals (bins). Assume C bin centers (b) are selected in the real number domain. 1:C Satisfying b1<… C There are C-1 edges c i Divide these buckets to satisfy b i <c i i+1 ,i∈{1,…,C-1}.

[0142] To ensure that the bins are equally distributed, Gaussian quantiles are used to define the partitioning intervals. The standardized time series is assumed to have a normal distribution, satisfying x... ′ Since the interval is ~N(0,1), the discrete interval can be constructed using the quantiles of the normal distribution.​​

[0143] Specifically, this application uses the percentile point function of the standard normal distribution to obtain the Z-score (cutoff point location). For C bin centers, C-1 quantiles p are calculated. 1:C-1 Then, for each quantile, calculate the number of percentiles in the standard normal distribution, i.e., the number of dividing edges c. i .

[0144] φ(c i )≤p i ;φ(·) uses the norm object from the scipy.stats library to calculate the percentile number.

[0145] For example, assuming the interval is C = 10, the calculated percentile numbers are as follows:

[0146] c 1:9 ={-1.28,-0.84,-0.52,-0.25,0,0.25,0.52,0.84,1.28}.

[0147] The 10 intervals divided by quantiles are...

[0148] S23: Flow sequence data symbol conversion.

[0149] After obtaining the quantized segmentation interval, the scaled flow sequence x can be... ′ The formula for converting quantization into a string sequence is as follows:

[0150]

[0151] In addition to the list of tokens {1,2,…,C} in the flow sequence, a special marker EOS representing the end of the sequence is added to facilitate understanding by large language models.

[0152] Finally, the original sequence x′ 1:H+F The tokenization will transform it into a character sequence z. 1:H+F This is for use in training and predictive inference of subsequent LLM traffic prediction models.

[0153] In summary, this application can tokenize the original traffic sequence. Figure 7This is a schematic diagram illustrating the traffic sequence tokenization provided in this application. It is assumed that the number of buckets is 10. The original traffic sequence is {23, 35, 46, 123, 222, 245, 102, 33, 22, 28}; the scaled traffic sequence (normalized) is {-0.81, -0.66, -0.52, 0.43, 1.67, 1.96, 0.17, -0.68, -0.82, -0.74}; the quantized character sequence is {3, 3, 4, 7, 10, 10, 6, 3, 3, 3}; the output character sequence is {3, 3, 4, 7, 10, 10, 6, 3, 3, 3, EOS}.

[0154] S3: LoRA fine-tuning training of large language models.

[0155] To obtain a large language model capable of understanding the evolutionary features of traffic flow sequences, this application fine-tunes the open-source LLaMA2 large language model, specifically using the LoRA method to train the model and obtain a large language prediction model for traffic flow sequences.

[0156] LoRA is a novel technique that addresses the problem of efficiently fine-tuning large language models. By freezing the weights of the pre-trained model and injecting trainable layers (rank decomposition matrices) into each Transformer block, LoRA significantly reduces the number of training parameters and memory requirements.

[0157] Assume the sequence data after flow label conversion is z 1:H+F Then the model receives the input data z 1:H Then, the prediction result can be output. H+1:H+F By comparing the predicted value with the actual value z H+1:H+F By minimizing the interval error, a trained model can be obtained.

[0158] This application is a fine-tuning of the open-source large model LLaMA2. The parameters that need to be updated are as follows:

[0159] W0+ΔW;

[0160] Here, W0 represents the initialization parameters of the pre-trained model LLaMA2, and ΔW represents the parameters that need to be fine-tuned and updated. To achieve lightweight model fine-tuning, W0 can be frozen while only ΔW is updated.

[0161] Assume the LLaMA2 pre-training matrix is The process of fine-tuning the model parameters can then be represented as:

[0162]

[0163] Where the rank r << min(d,k). Then, the model training process can be completed by minimizing the error through backpropagation.

[0164] Figure 8 A schematic diagram of the framework for fine-tuning the LLaMA2 model based on LoRA provided in this application is shown below. Figure 8 As shown, the LlaMA2 pre-trained weights are the weights of the generative pre-trained model, and the dimensionality reduction matrix A and the dimensionality increase matrix B are the weights of the low-rank adaptation network. Traffic character sequence data {z1,…,z...} H ,…,z H+F The inputs are fed into a generative pre-trained model and a low-rank adapting network, respectively. Based on the generative pre-trained model and the low-rank adapting network, semantic vectors of the traffic character sequence data are extracted. Based on the semantic vectors, the traffic character prediction data {o} is determined. H+1 ,…,o H+F}

[0165] S4: Predicted output dequantization and descaledation.

[0166] After the above fine-tuning process, the large language model can predict traffic character sequence data. After obtaining the character sequence output, it is necessary to perform inverse quantization and inverse scaling operations to map the character sequence to the real traffic data.

[0167] First, based on the Gaussian distribution quantization interval segmentation rule, the inverse quantization operation is defined as follows:

[0168]

[0169] Among them, b j This represents the center of the j-th bucket, i.e., the bucket center position. The predicted character sequence can be converted into a numerical sequence d through dequantization.

[0170] Then, the inverse standardization technique is used to map the prediction results to traffic data at the original scale that conforms to the real scenario.

[0171]

[0172] Using the above approach, the finely tuned T-LLaMA2 large language model can be used to effectively predict traffic flow at any intersection in a large-scale traffic network.

[0173] Figure 9 This is a schematic diagram of the traffic flow prediction device provided in this application. The device includes:

[0174] The acquisition module 21 is used to acquire the traffic flow sequence within a first historical time period of the target intersection; wherein, the traffic flow sequence includes first traffic flow data within each sub-time period arranged in chronological order; and the first traffic flow data is standardized to determine the corresponding second traffic flow data.

[0175] The determination module 22 is used to quantize each second traffic flow data within a preset historical window and determine the corresponding first quantization characters; wherein, the first quantization characters are characters that the large language model can recognize;

[0176] Prediction module 23 is used to input the first quantized characters into the large language model, determine the second quantized characters of at least one sub-time period within the prediction window based on the large language model, and perform dequantization and denormalization on the second quantized characters to determine the corresponding predicted traffic data.

[0177] In one optional implementation, the acquisition module 21 is specifically used to determine the average value and standard deviation of each of the first traffic flow data; for each of the first traffic flow data, determine the difference between the first traffic flow data and the average value, and determine the ratio of the difference to the standard deviation as the standardized second traffic flow data corresponding to the first traffic flow data.

[0178] In one optional implementation, the determining module 22 is specifically configured to: determine the number of segmentation intervals based on the first quantity of each second traffic flow data within the preset historical window and the second quantity of preset predicted traffic flow data within the prediction window; determine each segmentation interval and the quantization character corresponding to each segmentation interval based on the number of segmentation intervals and the percentile function of the standard normal distribution; determine the first segmentation interval to which each second traffic flow data belongs for each second traffic flow data; and determine the quantization character corresponding to the first segmentation interval as the first quantization character corresponding to the second traffic flow data.

[0179] In one optional implementation, the prediction module 23 is specifically used to determine the second segmentation interval corresponding to the second quantized character; determine the two boundary values ​​of the second segmentation interval; and determine the average value of the two boundary values ​​as the value after the second quantized character is dequantized.

[0180] In one optional implementation, the prediction module 23 is specifically used to determine the product of the inversely quantized value and the standard deviation; and to determine the sum of the product and the average value as the corresponding predicted flow data.

[0181] In one alternative embodiment, the device further includes:

[0182] Training module 24 is used to acquire sample traffic flow sequences corresponding to each intersection within a second historical time period, and determine a training sample set based on the sample traffic flow sequences. The sample traffic flow sequences include first sample traffic flow data within each sub-time period arranged in chronological order. For each sample traffic flow sequence in the training sample set, the first sample traffic flow data in the sample traffic flow sequence is standardized to determine the corresponding second sample traffic flow data. The second traffic flow data within a preset historical window and a prediction window are quantized to determine the corresponding first sample quantized characters. The first sample quantized characters are characters that the large language model can recognize. The first sample quantized characters within the preset historical window are input into the large language model to be trained. Based on the generative pre-trained model and the low-rank adaptation network in the large language model, semantic vectors of the first sample quantized characters are extracted. Based on the semantic vectors, predicted quantized characters for at least one sub-time period within the prediction window are determined. A loss value is determined based on the predicted quantized characters and the first sample quantized characters for at least one sub-time period within the prediction window. The large language model is trained based on the loss value.

[0183] In one alternative implementation, the training module 24 is specifically used to update the model parameters of the low-rank adaptation network based on the loss value while the parameters of the generative pre-trained model are fixed during training.

[0184] In an optional implementation, the training module 24 is further configured to acquire sample traffic flow sequences corresponding to each intersection within a second historical time period; based on each sample traffic flow sequence and each group of weight values, to perform a weighted summation of the traffic flow data at the corresponding time to obtain each enhanced sample traffic flow sequence; and to determine the training sample set based on the each sample traffic flow sequence and the each enhanced sample traffic flow sequence; wherein, for each group of weight values, each weight value of the group is greater than 0, and the sum of each weight value of the group is 1.

[0185] This application also provides an electronic device, such as Figure 10 As shown, it includes: processor 31, communication interface 32, memory 33 and communication bus 34, wherein processor 31, communication interface 32 and memory 33 communicate with each other through communication bus 34;

[0186] The memory 33 stores a computer program, which, when executed by the processor 31, causes the processor 31 to perform any of the above method steps.

[0187] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0188] Communication interface 32 is used for communication between the above-mentioned electronic device and other devices.

[0189] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0190] The processors mentioned above can be general-purpose processors, including central processing units, network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits, field-programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0191] This application also provides a computer-readable storage medium storing a computer program executable by an electronic device, which, when run on the electronic device, causes the electronic device to perform any of the above method steps.

[0192] This application provides a computer program product, which includes an executable program that, when executed by a processor, implements the method described herein.

[0193] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0194] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A traffic flow prediction method, characterized in that, The method includes: Obtain the traffic flow sequence of the target intersection within a first historical time period; wherein the traffic flow sequence includes first traffic flow data within each sub-time period arranged in chronological order; determine the average value and standard deviation of each first traffic flow data; for each first traffic flow data, determine the difference between the first traffic flow data and the average value, and determine the ratio of the difference to the standard deviation as the standardized second traffic flow data corresponding to the first traffic flow data; Based on the first quantity of each second traffic flow data within a preset historical window and the second quantity of preset predicted traffic flow data within a prediction window, the number of segmentation intervals is determined; based on the number of segmentation intervals and the percentile function of the standard normal distribution, each segmentation interval and the quantized character corresponding to each segmentation interval are determined; for each second traffic flow data, the first segmentation interval to which the second traffic flow data belongs is determined; the quantized character corresponding to the first segmentation interval is determined as the first quantized character corresponding to the second traffic flow data; wherein, the first quantized character is a character that the large language model can recognize; Each first quantized character is input into the large language model, and a second quantized character for at least one sub-time period within the prediction window is determined based on the large language model; the second quantized character is dequantized and denormalized to determine the corresponding predicted traffic data.

2. The method as described in claim 1, characterized in that, The process of dequantizing the second quantized character includes: Determine the second segmentation interval corresponding to the second quantized character; determine the two boundary values ​​of the second segmentation interval; and determine the average value of the two boundary values ​​as the value after the second quantized character is dequantized.

3. The method as described in claim 2, characterized in that, The value obtained by dequantizing the second quantized character is denormalized to determine the corresponding predicted traffic data, including: Determine the product of the inverse quantized value and the standard deviation; the sum of the product and the average value is determined as the corresponding predicted flow data.

4. The method as described in claim 1, characterized in that, The training process of the large language model includes: Obtain sample traffic flow sequences corresponding to each intersection within the second historical time period, and determine the training sample set based on the sample traffic flow sequences; wherein, the sample traffic flow sequences include first sample traffic flow data within each sub-time period arranged in chronological order; For each sample traffic flow sequence in the training sample set, the traffic flow data of each first sample in the sample traffic flow sequence is standardized to determine the corresponding second sample traffic flow data; the second traffic flow data within the preset historical window and prediction window are quantized to determine the corresponding first sample quantized characters; wherein, the first sample quantized characters are characters that the large language model can recognize. Each first sample quantized character within the preset historical window is input into the large language model to be trained. Based on the generative pre-trained model and low-rank adaptation network in the large language model, the semantic vectors of each first sample quantized character are extracted. Based on the semantic vectors, the predicted quantized character for at least one sub-time period within the prediction window is determined. Based on the predicted quantized character and the first sample quantized character for at least one sub-time period within the prediction window, a loss value is determined. The large language model is trained based on the loss value.

5. The method as described in claim 4, characterized in that, Training the large language model based on the loss value includes: During training, the parameters of the generative pre-trained model are fixed, and the model parameters of the low-rank adaptation network are updated according to the loss value.

6. The method as described in claim 4, characterized in that, The process of determining the training sample set includes: Obtain the sample traffic flow sequence for each intersection within the second historical time period; Based on the traffic flow sequences of each sample, and based on the weight values ​​of each group, the traffic flow data at the corresponding time are weighted and summed to obtain the traffic flow sequences of each enhanced sample; based on the traffic flow sequences of each sample and the traffic flow sequences of each enhanced sample, the training sample set is determined; wherein, for each group of weight values, each weight value of the group is greater than 0, and the sum of the weight values ​​of the group is 1.

7. A traffic flow prediction device, characterized in that, The device includes: The acquisition module is used to acquire the traffic flow sequence within a first historical time period of the target intersection; wherein the traffic flow sequence includes first traffic flow data within each sub-time period arranged in chronological order; determine the average value and standard deviation of each first traffic flow data; for each first traffic flow data, determine the difference between the first traffic flow data and the average value, and determine the ratio of the difference to the standard deviation as the standardized second traffic flow data corresponding to the first traffic flow data; The determination module is used to determine the number of segmentation intervals based on the first quantity of each second traffic flow data within a preset historical window and the second quantity of preset predicted traffic flow data within a prediction window; determine each segmentation interval and the quantized character corresponding to each segmentation interval based on the number of segmentation intervals and the percentile function of the standard normal distribution; determine the first segmentation interval to which each second traffic flow data belongs for each second traffic flow data; and determine the quantized character corresponding to the first segmentation interval as the first quantized character corresponding to the second traffic flow data; wherein, the first quantized character is a character that the large language model can recognize; The prediction module is used to input each first quantized character into the large language model, determine the second quantized character of at least one sub-time period within the prediction window based on the large language model, and perform dequantization and denormalization on the second quantized character to determine the corresponding predicted traffic data.

8. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in any one of claims 1-6.

Images