Traffic flow prediction method, system and device based on hierarchical semantic prompt large language model

Abstract

The present application belongs to the technical field of intelligent transportation system, and discloses a traffic flow prediction method, system and device based on hierarchical semantic prompt large language model. The present application comprises decomposing multivariate traffic time series data into trend, seasonality and residual components; independently encoding each component into a component embedding sequence and injecting double-layer semantic information into it: FBP vectors capturing the instance-level dynamic characteristics of each component, CSP vectors providing the overall background of the dataset; assembling the CSP vectors, FBP vectors and the corresponding component embedding sequences into a unified input sequence, and sending it into the large language model for reasoning; decoding the high-dimensional output of the model, fusing it through a dynamic gating mechanism, and generating the final traffic flow prediction result. The present application can solve the "modal gap" problem faced when applying large language models to traffic time series prediction, effectively guiding the large language model to deeply understand the internal patterns and context of traffic numerical sequences, thereby achieving efficient traffic management.

Description

Technical Field

[0001] This invention belongs to the field of intelligent transportation systems technology, specifically relating to a traffic flow prediction method, system, and device based on a hierarchical semantic prompting large language model. Background Technology

[0002] Accurate traffic flow prediction is the core and prerequisite for modern Intelligent Transportation Systems (ITS) to achieve efficient traffic management and control (such as dynamic signal timing, ramp control, and travel route planning). Urban traffic flow data is collected by a large number of sensors deployed on the road network or through floating car technology. It is a typical multivariate time series and faces multiple challenges, such as capturing long-term dependencies, handling non-stationary distributions caused by morning and evening peak hours, holidays, and emergencies (such as traffic accidents), and modeling complex spatial relationships between upstream and downstream road segments.

[0003] Traditional forecasting methods, such as using statistical models like ARIMA (Autoregressive Integral Moving Average), perform well when dealing with stationary univariate sequences with strong linear relationships, but they are difficult to effectively capture nonlinear dependencies between multivariate sequences, and their models have strong assumptions and limited applicability.

[0004] With the development of deep learning technology, recurrent neural networks (RNNs) and their variants, such as long short-term memory networks (LSTMs), have been widely used in sequence modeling, as they are better able to learn nonlinear patterns in time-series data. However, the inherent serial computation method of RNN / LSTM structures makes them prone to gradient vanishing or exploding problems when processing long sequences, and they are difficult to capture ultra-long-term dependencies spanning a whole day or even a week.

[0005] In recent years, models represented by the Transformer architecture have achieved revolutionary success in the field of Natural Language Processing (NLP). Its core self-attention mechanism can compute the dependencies between any two positions in a sequence in parallel, thus demonstrating a significant advantage in capturing long-term dependencies. Inspired by this, a series of Transformer-based prediction models, such as Informer, Autoformer, and PatchTST, have been proposed. These models effectively reduce the quadratic computational complexity of the self-attention mechanism by introducing innovative attention mechanisms or employing patching strategies to transform long sequences into short token sequences, achieving leading performance on multiple traffic flow prediction benchmarks.

[0006] However, these Transformer-based approaches still have inherent limitations. First, the permutation-invariance of the self-attention mechanism inherently lacks awareness of temporal order, and even with the introduction of positional encoding, the capture of fine-grained local temporal dynamics may be weakened. Second, and more importantly, although these models borrow from NLP architectures, they internally still process purely numerical representations, failing to fully leverage the powerful capabilities of modern Large Language Models (LLMs) in understanding, reasoning, and context learning.

[0007] With the rise of large language models such as GPT (Generative Pre-trained Transformer), the technology field has begun to explore the application of LLM to various tasks beyond natural language processing, and using LLM for traffic flow prediction has become a new and highly promising research direction. However, a fundamental technical challenge exists—the modality gap. LLM pre-training is based on massive text corpora, and its internal knowledge and reasoning capabilities are built around natural language symbols; while time series data are continuous, semantically ambiguous numerical streams. How to effectively "translate" purely numerical traffic flow data into inputs with rich contextual information (e.g., distinguishing between "weekend periodic congestion" and "sudden congestion caused by accidents") that LLM can deeply understand and process is the key bottleneck to unlocking the predictive potential of LLM.

[0008] Currently, there is a lack of systematic solutions in the field of intelligent transportation that can effectively bridge the gap between traffic numerical sequences and the semantic space of Large Language Models (LLMs). Specifically, there is a lack of methods that can not only structurally decompose and encode traffic time series, but also dynamically and adaptively inject rich, multi-layered semantic information into the encoded sequences, thereby guiding LLMs to focus their powerful contextual understanding and reasoning capabilities on traffic flow prediction tasks involving complex traffic conditions. Therefore, developing a new method that can bridge the modal gap through hierarchical semantic cues, thereby efficiently utilizing large language models for traffic flow prediction, has significant theoretical and practical value. Summary of the Invention

[0009] The purpose of this invention is to address the shortcomings of existing technologies by providing a traffic flow prediction method, system, and device based on a hierarchical semantic cueing large language model (LLM), aiming to solve the "modal gap" problem encountered when applying LLM to traffic flow time series prediction. More specifically, this invention aims to solve the technical challenge of lacking a systematic approach to transforming semantically nuanced traffic numerical sequences into context-rich input paradigms that LLM can deeply understand, thereby fully unleashing the potential of LLM in complex traffic condition prediction tasks.

[0010] Specifically, the present invention is implemented using the following technical solutions.

[0011] On the one hand, this invention provides a traffic flow prediction method based on a hierarchical semantic cueing large language model, comprising:

[0012] Component generation step: Receive the input multivariate traffic time series and decompose it into components that include at least one trend component, one seasonal component and one residual component.

[0013] Component encoding steps: The trend component, seasonal component, and residual component are independently encoded into their respective component embedding sequences;

[0014] Hierarchical prompt generation and assembly steps: By generating two-layer semantic prompts and assembling them with the component embedding sequence obtained in the component encoding step, a unified input sequence for the large language model is constructed;

[0015] Model inference steps: Input the unified input sequence of the large language model into the large language model for inference, and output a high-dimensional representation;

[0016] Prediction decoding and fusion steps: The high-dimensional features corresponding to each component in the high-dimensional representation are decoded into preliminary prediction values ​​through an independent decoding head; the preliminary prediction values ​​of each component are fused to generate the final prediction result.

[0017] Furthermore, the step of generating components specifically includes:

[0018] By comparing with the input multivariate traffic time series Each channel The corresponding independent RNN encoder captures the temporal context information of that channel. Generate high-dimensional features based on the number of channels. : Through a linear shrinkage layer shared by all channels and a nonlinear activation function The high-dimensional features are mapped to intermediate representation tensors. ;

[0019] Each component is calculated using a channel-mixed MLP with shared time steps; wherein, the trend component... The intermediate representation tensor is obtained by an MLP The MLP is obtained through processing at each time step. The following nonlinear transformation is applied to the characteristics of all channels:

[0020] ;

[0021] in, This indicates that the trend component is in the batch. Data from all channels at time step t; Indicates batch index; Indicates the time step index; This indicates retrieving data from all channels within the specified dimension; Let be the intermediate representation tensor. Represents the intermediate representation tensor In, corresponding to batch Time step Data from all channels; Number of channels; Trend component MLP;

[0022] The seasonal component The detrended sequence is obtained from another independent MLP, i.e. The MLP is obtained through processing at each time step. The characteristics of all channels are subjected to nonlinear transformation;

[0023] The residual components The result is obtained through addition closure calculation:

[0024] .

[0025] Furthermore, the traffic flow prediction method based on a hierarchical semantic prompting large language model further includes, before the component generation step, processing the input multivariate traffic time series... Perform a reversible instance normalization process, including:

[0026] For each instance and each channel within each batch, normalization is performed independently, and its mean is calculated. and standard deviation :

[0027]

[0028] in, The input is a multivariate traffic time series. The mean, Standard deviation, and Let L and L represent the length of the input multivariate traffic time series along the time dimension, respectively. Calculate the mean and variance. For batch size, Number of channels;

[0029] Using a set of learnable affine transformation parameters , The input multivariate traffic time series Mapping to the normalized space yields the normalized sequence. :

[0030] ;

[0031] in, This indicates element-wise multiplication; It is a small constant used to prevent division by zero;

[0032] And after generating the final prediction result, the final prediction result generated in the normalized space is restored to the original numerical scale through the corresponding inverse normalization operation:

[0033] ;

[0034] in, To restore the final prediction results to the original scale; This represents the final prediction result generated in the normalized space; The mean is calculated in the normalization step. To calculate the standard deviation in the normalization step, and These are learnable affine transformation parameters; This indicates element-wise multiplication; It is a small constant used to prevent division by zero, and its value is consistent with the value used during normalization.

[0035] Furthermore, for any component , , The component encoding step specifically includes:

[0036] 2-1) Data partitioning: dividing the data into blocks Divided along the time axis There are data blocks, each with a length of . The sliding step size is Reshape the data ;

[0037] 2-2) Intra-block feature extraction: Using a TCN as the backbone network, the temporal features within each data block are extracted independently to obtain the intra-block feature representation. TCN is composed of multiple residual layers stacked together, each residual layer containing causal convolution and dilated convolution; before extracting the temporal features within each data block, the data block dimension is converted to the channel dimension through an axis remapping operation to adapt to the input format of TCN and achieve parallel processing.

[0038] 2-3) Feature Projection: The intra-block features output by the TCN are represented by a projection head module. Feature dimensions from data block length Target dimension required for mapping to a large language model .

[0039] Furthermore, the specific steps for generating and assembling the hierarchical prompts are as follows:

[0040] 3-1) Generate FBP vectors, which provide instance-level guidance for the large language model regarding the specific behavioral patterns of each component;

[0041] 3-2) Generate CSP vectors, which provide the large language model with high-level dataset-level context about the overall background of the prediction task;

[0042] 3-3) Sequence Assembly: The generated FBP vector, CSP vector, and component embedding sequence obtained from the component encoding step are structurally assembled: The CSP vector is broadcast C times to match the dimension of BC, where BC is the product of batch size B and number of channels C, and placed at the beginning of the sequence. Subsequently, the FBP vector of each component and its corresponding component embedding sequence are concatenated sequentially. The blocks formed ultimately create the unified input sequence for the large language model.

[0043] Furthermore, for each component and each instance The process of generating the FBP vector includes:

[0044] 3-1-1) Feature Extraction and Textification: Extract the numerical statistical features of this component and fill the numerical statistical features into a preset text template. This generates a structured text description.

[0045] 3-1-2) Information Summary: The structured text description is input into the word segmenter and embedding layer of the large language model to obtain its key vector. Sum value vector Then, using a set of learnable query vectors Information summarization of the text embedding is performed through a cross-attention module to generate FBP vectors. :

[0046] ;

[0047] in, It is the length of the FBP vector. It is the model dimension; A block diagonal mask is used during batch computation to maintain the independence of each component's information; k represents the index of the corresponding component, and k is... hour, The FBP vector representing the trend component, where k is... hour, The FBP vector representing the seasonal component, where k is... hour, The FBP vector represents the residual components.

[0048] Furthermore, the input to the CSP vector generation process is global metadata about the traffic dataset corresponding to the current traffic flow prediction task, which is converted into a text description. The text description is input into the word segmenter and embedding layer of the large language model to obtain its key vector. Sum value vector ; Using a learnable query vector through a cross-attention module. Regarding the Sum value vector Perform a summary and generate a CSP vector:

[0049] ;

[0050] in, This represents the final generated CSP vector; Represents the cross-attention function; It is a learnable query vector; It is a key vector. It is a value vector; B represents all learnable parameters contained in the cross-attention module; B is the batch size. is the length or number of generated CSP vectors; D is the hidden state or embedding dimension of the large language model.

[0051] Furthermore, the large language model adopts a decoder-only Transformer architecture; the backbone network of the large language model consists of several stacked Transformer layers; each layer contains a multi-head self-attention module and a feedforward network module, and adopts a structure based on root mean square layer normalization and residual connections; the computation process of the first layer is represented as follows:

[0052] ;

[0053] ;

[0054] in, Represents the index of the current Transformer layer; Representing the The input sequence of the layer, i.e., the first layer -1 layer output; Represents a multi-head self-attention module; This represents the element-wise addition operation that implements residual joins; This represents the root mean square layer normalization function; Represents the feedforward network module; Representative at the In the layer, the intermediate output after passing through the MSA module and the first residual connection; Representing the The final output sequence of the layer, as the first layer +1 layer input.

[0055] On the other hand, the present invention also provides a traffic flow prediction system based on a hierarchical semantic prompting large language model, which realizes the above-mentioned traffic flow prediction method based on a hierarchical semantic prompting large language model. The traffic flow prediction system based on a hierarchical semantic prompting large language model includes a neural decomposition module, a component encoding module, a hierarchical prompting generation and assembly module, a large language model, and a prediction decoding and fusion module.

[0056] The neural decomposition module decomposes the input multivariate traffic time series into several components, including trend components, seasonal components, and residual components.

[0057] The component encoding module independently encodes each component into a component embedding sequence;

[0058] The hierarchical prompt generation and assembly module generates and injects two layers of semantic information into the component embedding sequence: FBP vector and CSP vector; it then structurally assembles the CSP vector, FBP vector, and corresponding component embedding sequence into a unified input sequence and feeds it into the large language model for inference.

[0059] The large language model comprehensively processes the global context from CSP, the instance-level behavior from FBP vectors, and the local numerical dynamics from the component embedding sequence in the unified input sequence to generate and output a high-dimensional representation containing future prediction information.

[0060] The prediction decoding and fusion module decodes the high-dimensional representation output by the large language model into preliminary predicted values ​​of each component through a decoding head network, and then fuses them to generate the final multivariate time series traffic flow prediction result.

[0061] In another aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the above-described traffic flow prediction method based on a hierarchical semantic prompting large language model.

[0062] The beneficial effects of the traffic flow prediction method, system, and device based on a hierarchical semantic prompting large language model of the present invention are as follows:

[0063] Effectively bridging the modality gap: This invention's unique hierarchical semantic prompting framework injects instance-level and dataset-level semantic information into pure numerical sequences through FBP (Fine-grained Behavioral Prompt) vectors and CSP (Corpus-level Semantic Prior) vectors, constructing an effective bridge from the numerical space to the LLM semantic space. This allows LLM to no longer process meaningless numbers, but to reason based on understanding their "behavior" and "context," fundamentally solving the modality mismatch problem.

[0064] Significantly improves prediction accuracy: By successfully guiding LLM to apply its powerful contextual understanding and logical reasoning capabilities to prediction tasks, this invention can capture deeper, more complex patterns and cross-variable dependencies that are difficult for traditional models to discover, thereby achieving higher prediction accuracy in various complex traffic prediction scenarios (such as holiday travel prediction and traffic accident impact assessment).

[0065] It enhances the model's adaptability and generalization ability: Fine-grained behavioral cue (FBP) vectors are data-driven and dynamically generated, enabling LLM to adaptively handle traffic time series instances with different statistical properties and non-stationarity. Meanwhile, corpus-level semantic priors (CSP) provide LLM with cross-task meta-knowledge, helping to improve its generalization ability on different city or road network datasets.

[0066] This invention provides a systematic end-to-end solution: It offers a complete and systematic end-to-end framework from data preprocessing, decomposition, encoding, semantic injection, LLM inference to result fusion. Each module of this framework has a clearly defined function, low coupling, and is easy to implement and extend, providing a standardized and reproducible technical paradigm for using LLM to solve traffic time series problems. Attached Figure Description

[0067] Figure 1 This is a system structure block diagram of an embodiment of the present invention.

[0068] Figure 2 This is a schematic diagram of the method flow according to an embodiment of the present invention. Detailed Implementation

[0069] The present invention will now be described in further detail with reference to the embodiments and the accompanying drawings.

[0070] One embodiment of the present invention is a traffic flow prediction system based on a hierarchical semantic prompting large language model, such as... Figure 1 As shown, it includes a neural decomposition module, a component encoding module, a hierarchical prompt generation and assembly module, a large language model, and a prediction decoding and fusion module.

[0071] The neural decomposition module decomposes the input multivariate traffic time series into several components, including trend components, seasonal components, and residual components.

[0072] The component encoding module independently encodes each component into a component embedding sequence.

[0073] The hierarchical prompt generation and assembly module generates and injects two layers of semantic information into the component embedding sequence: first, an FBP vector that captures the instance-level dynamic characteristics of each component (such as morning and evening rush hour patterns); second, a CSP that provides the overall dataset-level context (such as "urban main road data"). Then, the CSP vector, FBP vector, and corresponding component embedding sequence are structurally assembled into a unified input sequence, which is then fed into a large language model for inference.

[0074] The large language model integrates the global context from CSP vectors, the instance-level behavior from FBP vectors, and the local numerical dynamics from component embedding sequences in the unified input sequence to generate and output a high-dimensional representation containing future prediction information.

[0075] The prediction decoding and fusion module decodes the high-dimensional representation output by the large language model into preliminary predicted values ​​of each component through the decoding head network, and then fuses them to generate the final multivariate time series traffic flow prediction results.

[0076] The traffic flow prediction method based on a hierarchical semantic prompting large language model of the present invention is based on observed multivariate traffic time series. Where L is the length of the input multivariate traffic time series (i.e., the length of the historical observation series), and C is the number of variables (or channels), this method constructs a prediction function. This function has learnable parameters. To predict traffic state values ​​over the next H time steps. This process can be represented as:

[0077] ;

[0078] In model training or batch prediction, data is typically organized in mini-batches, with the input being... The goal is , where B is the batch size.

[0079] The traffic flow prediction method based on a hierarchical semantic prompting large language model of the present invention, such as... Figure 2 As shown, it includes the following steps:

[0080] I. Generating Components

[0081] Received input multivariate traffic time series Decompose it into at least one trend component. A seasonal component and a residual component The components. Here, , and These are the trend component, seasonal component, and residual component obtained by decomposition on the original numerical scale:

[0082] .

[0083] Preferably, in another embodiment, to address potential distribution offsets between different instances and between different channels (i.e., variables), the input multivariate traffic time series is further processed before generating the components. Perform a reversible instance normalization (RevIN) process to obtain the normalized multivariate traffic time series. .

[0084] Reversible instance normalization processing includes:

[0085] Reversible instance normalization performs independent normalization on each instance and each channel within each batch, and calculates its mean. and standard deviation :

[0086] ;

[0087] in, The input is a multivariate traffic time series. The mean, Standard deviation, and These represent the calculation of the mean and variance along the time dimension L, respectively. L represents the batch size, and L represents the length of the input multivariate traffic time series (i.e., the length of the historical observation series). This represents the number of channels.

[0088] Then, using a set of learnable affine transformation parameters , The input multivariate traffic time series Mapping to the normalized space yields the normalized sequence. :

[0089] ;

[0090] in, It is a small constant used to prevent division by zero. This indicates element-wise multiplication.

[0091] Subsequent steps are all performed in the normalized space. An architecture of Channel-Independent-then-Channel-Mixing (CI-then-CM) is adopted, implementing additive decomposition in the normalized space:

[0092] .

[0093] at this time, , and These are the trend component, seasonal component, and residual component generated in the normalized space, respectively.

[0094] The channel-independent post-hybrid architecture first extracts the temporal dynamics of each variable in a channel-independent manner, and then models the relationships between variables through a channel blending mechanism to generate trend components. Seasonal component S and residual component .

[0095] The specific steps for generating components include:

[0096] 1-1) Extracting channel-independent features

[0097] By comparing with the input multivariate traffic time series Each channel The corresponding independent recurrent neural network (RNN) encoder (such as a long short-term memory network LSTM or a gated recurrent unit GRU) captures the temporal context information of the channel and generates high-dimensional features. :

[0098] ;

[0099] in, B is the input multivariate traffic time series; L is the batch size; and L is the length of the input multivariate traffic time series. An independent recurrent neural network encoder representing the c-th channel; This represents the multivariate traffic time series of the input. The middle corresponds to the first Channel data for each channel; It is the hidden state dimension of the RNN; This indicates that data from all channels within the specified dimension will be retrieved.

[0100] Through a linear shrinking layer shared by all channels and a nonlinear activation function (such as ReLU or GELU), mapping high-dimensional features to intermediate representation tensors. :

[0101] ;

[0102] ;

[0103] in, This represents the intermediate representation of the c-th channel after linear shrinkage and nonlinear activation, whose dimension is compressed to... ; Indicates the batch size; L is the length of the input multivariate traffic time series (i.e., the length of the historical observation series). The high-dimensional feature sequence of the c-th channel is the input to the linear shrinking layer; the shrinkage parameter... and Shared across all channels. These are the weights of the linear shrinking layer. It is a bias parameter; This represents a non-linear activation function applied element-wise; This represents the final generated intermediate representation tensor; This represents a tensor stacking operation; This represents the total number of channels for the input data.

[0104] 1-2) Components of the synthesis channel mixture:

[0105] The intermediate representation tensor is processed by a multilayer perceptron (MLP) with shared time steps. Calculate each component (trend component) Seasonal components and residual components Among them, the trend component A channel-based hybrid MLP shared by a single time step (denoted as...) ) for intermediate representation tensor The MLP is obtained through processing at each time step. The following nonlinear transformation is applied to the characteristics of all channels:

[0106] ;

[0107] in, This indicates that the trend component is in the batch. Data from all channels at time step t; Indicates batch index; Indicates the time step index. This indicates retrieving data from all channels within the specified dimension. Let be the intermediate representation tensor. Represents the intermediate representation tensor In the middle, the data corresponds to all channels of batch b and time step t; Number of channels; Trend component MLP.

[0108] Seasonal components By another independent MLP (denoted as Time series after detrended components The same process as the above formula is performed to obtain the result.

[0109] residual components The result is obtained through addition closure calculation:

[0110] .

[0111] Preferably, in another embodiment, in order to address the potential distribution offset between different instances and between different channels (i.e., variables), before performing step one, the input multivariate traffic time series is first processed. Perform a reversible instance normalization (RevIN) process. This includes:

[0112] II. Component Coding.

[0113] The trend components generated in step one are encoded using the component encoding module. Seasonal components and residual components Each component is independently encoded as its own embedded sequence (i.e., a numerical token). , , .

[0114] Preferably, in another embodiment, the component encoding module employs a Temporal Convolutional Patch Encoder (TC-Patch). To capture the unique dynamic characteristics of each component, preferably, in another embodiment, an independent, weight-free TC-Patch encoder is instantiated for each component. For any component... , , The specific steps of component encoding are as follows:

[0115] 2-1) Data Patching:

[0116] Components Divided along the time axis There are data blocks, each with a length of . The sliding step size is This operation will reshape the data to... .

[0117] 2-2) Intra-block feature extraction:

[0118] A temporal convolutional network (TCN) is used as the backbone network to independently extract the temporal features within each data block, thus obtaining the intra-block feature representation. TCN consists of multiple stacked residual layers, each containing causal convolutions and dilated convolutions to expand the receptive field without leaking future information. Before extracting the temporal features within each data block, an axis remapping operation is needed to convert the data block dimension to the channel dimension to adapt to the TCN's input format and enable parallel processing.

[0119] 2-3) Feature projection:

[0120] The intra-block feature representation output by TCN is represented by a projection head module. Feature dimensions from data block length Target dimension required for mapping to a large language model Specifically, the projection head module may include linear projection layers, parallel residual connections, and depthwise convolutional layers.

[0121] Among them, the linear projection layer will reduce the feature dimensions. From mapping to dimension Parallel residual connections that upsample the original signal to dimension 1 through a transposed convolution. The output of the linear projection layer is then added to the output of the linear projection layer. The depthwise separable convolutional layer independently operates on the features of each data block along the channel dimension (i.e., the data block index dimension), maintaining the independence of the representation of each data block while performing feature calibration. Finally, the projection head module outputs the final result for each component. Generate the corresponding component embedding sequence .

[0122] III. Hierarchical prompt generation and assembly.

[0123] By generating two-layer semantic prompts and assembling them with the component embedding sequence obtained in step two, the final input of the large language model, i.e., the unified input sequence of the large language model, is constructed. The specific steps of hierarchical prompt generation and assembly are as follows:

[0124] 3-1) Generate FBP (Fine-grained Behavioral Prompt):

[0125] FBP aims to provide LLM with instance-level guidance on the specific "behavioral patterns" of each component. For each component... and each instance The FBP generation process includes:

[0126] 3-1-1) Feature Extraction and Textification: Extract the numerical statistical features of this component, such as minimum, maximum, mean, variance, the K lags with the strongest autocorrelation, trend strength (e.g., discretized description of the linear regression slope), etc. Then, input these numerical statistical features into a preset text template. This generates a structured text description.

[0127] 3-1-2) Information Summary: The structured text description is input into the tokenizer and embedding layer of the large language model to obtain its key vector. Value vector Then, using a set of learnable query vectors The text embedding is summarized using a cross-attention module to generate an FBP vector. :

[0128] ;

[0129] in, It is the length of FBP; It is the model dimension; A block diagonal mask is used during batch computation to maintain the independence of each component's information; k represents the index of the corresponding component, for example, k can take the value of (representing the trend component), S (representing the seasonal component), and (Represents the residual components).

[0130] 3-2) Generate CSP (Corpus-level Semantic Prior):

[0131] CSP provides LLM with a high-level dataset-level context regarding the overall background of the prediction task. Its generation process is similar to FBP, but the input is global metadata about the traffic dataset corresponding to the current prediction task, such as "this data comes from the expressway network of a certain city" and "the sampling frequency is 5 minutes," which is then converted into a text description. The text description is input into the word segmenter and embedding layer of the large language model to obtain its key vector. Sum value vector Similarly, through the cross-attention module, a dedicated learnable query is used. Its key vector Sum value vector Perform a summary and generate a CSP vector:

[0132] ;

[0133] in, This represents the final generated corpus-level semantic suggestion (CSP) vector; Represents the cross-attention function; It is a learnable query vector whose role is to proactively extract key global information from the text description of the dataset; It is a key vector. These are value vectors, all of which are obtained by transforming the text description T_data of the dataset through a text embedding model; This represents all learnable parameters (such as weights and biases) contained in the cross-attention module; B is the batch size. is the length or number of generated CSP vectors; D is the hidden state or embedding dimension of the large language model.

[0134] 3-3) Sequence assembly:

[0135] The CSP vector generated above and FBP vector The component embedding sequence obtained in step two is then structurally assembled. The CSP is broadcast C times to match the dimension of BC, where BC is the product of batch size B and channel number C, and placed at the beginning of the sequence. Subsequently, the FBP of each component and its corresponding component embedding sequence are sequentially concatenated. The resulting blocks. The unified input sequence of the final large language model. The structure is as follows:

[0136] ;

[0137] The definitions of the variables and symbols in the formula are as follows:

[0138] This represents the unified input sequence for the large language model. This sequence integrates global semantic priors, local behavioral cues for each component, and numerical information for each component, serving as the input for subsequent prediction modules.

[0139] For corpus priors, it represents corpus-level semantic cues. It is a vector or sequence learned from a large-scale dataset that contains global high-level semantic information, providing the model with prior knowledge about the overall characteristics of the time series.

[0140] This indicates a broadcasting or replicating operation. Specifically, it will represent a single... Duplicate C times (C being the number of channels) to align it with the data in each channel.

[0141] This represents the sequence concatenation operation. It connects different vectors or sequences sequentially along a preset dimension (usually the sequence length dimension) to form a longer sequence.

[0142] , , These represent fine-grained behavioral prompts for the trend component, seasonal component, and residual component, respectively. These prompts are customized for each specific component to guide the model to focus on the specific dynamic behavior of that component. , , These are component embedding sequences representing the trend component, seasonal component, and residual component, respectively. They are generated in step one (component generation) and represent the specific representation of each component in the numerical space.

[0143] Based on the above definition, a unified input sequence can be constructed by concatenating the following blocks: the trend component block consists of the trend components... and its component embedding It is pieced together; the seasonal component blocks are composed of seasonal components. and its component embedding It is pieced together; the residual component block is composed of the residual components. and its component embedding It was pieced together.

[0144] IV. Model Reasoning.

[0145] Unify the input sequence The input is fed into a large language model (LLM) for inference. Preferably, the LLM employs a decoder-only Transformer architecture, such as the LLaMA, GPT, and other related models. To efficiently utilize the knowledge of the pre-trained model and reduce training costs, parameter-efficient fine-tuning techniques, such as Low-Rank Adaptation (LoRA), can be used.

[0146] The backbone network of the large language model consists of several (e.g., N, where N is a natural number) stacked Transformer layers. Each layer contains a multi-head self-attention (MSA) module and a feedforward network (FFN) module, employing a pre-norm normalization and residual connection structure, where root mean square (RMS) normalization can be used. The computation process of the first layer can be represented as follows:

[0147] ;

[0148] ;

[0149] in, Represents the index of the current Transformer layer. Representing the The input sequence of the layer, i.e., the first layer Output of layer -1. This represents the Multi-Head Self-Attention module. This represents the element-wise addition operation that implements residual joins. This represents the root mean square layer normalization function. This represents the Feed-Forward Network module. Representative at the In the layer, the intermediate output is after passing through the MSA module and the first residual connection. Representing the The final output sequence of the layer will serve as the first... +1 layer input.

[0150] V. Predictive Decoding and Fusion.

[0151] Decoding the high-dimensional representation of the LLM output into the final prediction result. Specifically, this includes:

[0152] 5-1) Component decoding:

[0153] Each component is configured with an independent decoding head, which decodes the features corresponding to that component from the high-dimensional representation output by the large language model (which is essentially composed of a series of feature tokens) into preliminary predicted values. For example, to achieve prediction from a variable-length high-dimensional representation output by a large language model to a fixed-length H, a two-layer residual MLP performs a nonlinear transformation on all feature tokens, followed by a final projection layer that aggregates the information of all feature tokens and maps them to a length of H. The predictive horizon.

[0154] 5-2) Predictive fusion:

[0155] Preliminary predicted values ​​for each component , , The data is fused to generate the final prediction in the normalized space. .

[0156] Preferably, in another embodiment, a Dynamic Gated Fusion mechanism is employed for prediction fusion. The core idea of ​​this mechanism is not to simply add the components together, but rather to dynamically and independently generate a set of fusion weights for each channel of the traffic time series based on the information inherent in the predicted values ​​of each component, and then perform a weighted summation. This approach can adaptively adjust the contribution of each component to the final prediction, thereby more flexibly modeling complex temporal dynamics.

[0157] In one specific embodiment of the present invention, to achieve this dynamic gating fusion, the fusion weights consist of two parts: one part is a set of static baseline weights shared by all channels and learned through model training; the other part is a dynamic weight adjustment term. This dynamic weight adjustment term is generated by a gating network (e.g., a multilayer perceptron), whose input is a combination of the preliminary predicted values ​​of all components of the current channel (e.g., their sum or concatenation). The final fusion weights are the sum of the static baseline weights and the dynamic weight adjustment term. In this way, the model retains a general prior on the importance of each component while flexibly adjusting the contribution of each component according to the specific context of the current prediction (e.g., determining whether the current situation is a random fluctuation during off-peak hours or a severe congestion caused by peak hours and accidents).

[0158] This completes the entire process from inputting multivariate traffic time series data to the final prediction results.

[0159] If reversible instance normalization is applied to the input multivariate traffic time series before decomposition, the final prediction result in the generated normalized space will be... Then, it needs to be restored to the original numerical scale through the corresponding inverse normalization operation, specifically for the final prediction result. Perform the following inverse normalization operation to obtain the final prediction result restored to the original scale. .

[0160] ;

[0161] in, To restore the final prediction results to the original scale; This represents the final prediction result generated in the normalized space; The mean is calculated in the normalization step. This is the standard deviation calculated in the normalization step; This indicates element-wise multiplication; and To be Mapped to normalized space A set of learnable affine transformation parameters; It is a small constant used to prevent division by zero, and its value is consistent with the value used during normalization.

[0162] Through the above steps, this invention constructs a complete, end-to-end framework. By employing an innovative hierarchical semantic prompting mechanism, it successfully introduces the powerful capabilities of large language models into complex multivariate traffic time series prediction tasks, effectively solving the modality gap problem.

[0163] Those skilled in the art should understand that the embodiments of the present invention are not limited to the specific structures and parameters described above. The number of network layers, convolutional kernel size, activation function, etc., can be adjusted and modified according to specific application scenarios and needs, and these modifications do not depart from the scope of protection claimed by the present invention.

[0164] In some embodiments, certain aspects of the above-described techniques may be implemented by one or more processors of a processing system executing software. The software includes one or more sets of executable instructions stored or otherwise tangibly implemented on a non-transitory computer-readable storage medium. The software may include instructions and certain data that, when executed by one or more processors, manipulate one or more processors to perform one or more aspects of the above-described techniques. The non-transitory computer-readable storage medium may include, for example, magnetic or optical disk storage devices, solid-state storage devices such as flash memory, cache, random access memory (RAM), or other non-volatile memory devices. The executable instructions stored on the non-transitory computer-readable storage medium may be source code, assembly language code, object code, or other instruction formats interpreted or otherwise executed by one or more processors.

[0165] Computer-readable storage media can include any storage medium or combination of storage media that can be accessed by a computer system during use to provide instructions and / or data to the computer system. Such storage media can include, but are not limited to, optical media (e.g., optical discs (CDs), digital versatile optical discs (DVDs), Blu-ray discs), magnetic media (e.g., floppy disks, magnetic tapes, or magnetic hard disks), volatile memory (e.g., random access memory (RAM) or cache), non-volatile memory (e.g., read-only memory (ROM) or flash memory), or microelectromechanical systems (MEMS) based storage media. Computer-readable storage media can be embedded in a computing system (e.g., system RAM or ROM), fixedly attached to a computing system (e.g., a magnetic hard disk drive), removably attached to a computing system (e.g., an optical disc or universal serial bus-based (USB) flash memory), or coupled to a computer system via a wired or wireless network (e.g., network-accessible storage (NAS)).

Claims

1. A traffic flow prediction method based on a hierarchical semantic prompting large language model, characterized in that, include: Component generation step: Receive the input multivariate traffic time series and decompose it into components that include at least one trend component, one seasonal component and one residual component. Component encoding steps: The trend component, seasonal component, and residual component are independently encoded into their respective component embedding sequences; Hierarchical prompt generation and assembly steps: By generating two-layer semantic prompts and assembling them with the component embedding sequence obtained in the component encoding step, a unified input sequence for the large language model is constructed; Model inference steps: Input the unified input sequence of the large language model into the large language model for inference, and output a high-dimensional representation; Prediction decoding and fusion steps: The high-dimensional features corresponding to each component in the high-dimensional representation are decoded into preliminary prediction values ​​through an independent decoding head; the preliminary prediction values ​​of each component are fused to generate the final prediction result; The specific steps for generating and assembling the hierarchical prompts are as follows: 3-1) Generate FBP vectors, which provide instance-level guidance for the large language model regarding the specific behavioral patterns of each component; 3-2) Generate CSP vectors, which provide the large language model with high-level dataset-level context about the overall background of the prediction task; 3-3) Sequence Assembly: The generated FBP vector, CSP vector, and component embedding sequence obtained from the component encoding step are structurally assembled: The CSP vector is broadcast C times to match the dimension of BC, where BC is the product of batch size B and number of channels C, and placed at the beginning of the sequence. Subsequently, the FBP vector of each component and its corresponding component embedding sequence are concatenated sequentially. The blocks formed ultimately create the unified input sequence for the large language model; For each component and each instance The process of generating the FBP vector includes: 3-1-1) Feature Extraction and Textification: Extract the numerical statistical features of this component and fill the numerical statistical features into a preset text template. This generates a structured text description. 3-1-2) Information Summary: The structured text description is input into the word segmenter and embedding layer of the large language model to obtain its key vector. Sum value vector Then, using a set of learnable query vectors Information summarization of the text embedding is performed through a cross-attention module to generate FBP vectors. : ； in, It is the length of the FBP vector. It is the model dimension; A block diagonal mask is used during batch computation to maintain the independence of each component's information; k represents the index of the corresponding component, and k is... hour, The FBP vector representing the trend component, where k is... hour, The FBP vector representing the seasonal component, where k is... hour, The FBP vector representing the residual components; The input to the CSP vector generation process is global metadata about the traffic dataset corresponding to the current traffic flow prediction task, which is converted into a text description. The text description is input into the word segmenter and embedding layer of the large language model to obtain its key vector. Sum value vector ; Using a learnable query vector through a cross-attention module. For the key vector Sum value vector Perform a summary and generate a CSP vector: ； in, This represents the final generated CSP vector; Represents the cross-attention function; It is a learnable query vector; It is a key vector. It is a value vector; B represents all learnable parameters contained in the cross-attention module; B is the batch size. is the length or number of generated CSP vectors; D is the hidden state or embedding dimension of the large language model; The large language model employs a decoder-only Transformer architecture; the backbone network of the large language model consists of several stacked Transformer layers; each layer contains a multi-head self-attention module and a feedforward network module, and adopts a structure based on root mean square layer normalization and residual connections; the computation process of the first layer is represented as follows: ； ； in, Represents the index of the current Transformer layer; Representing the The input sequence of the layer, i.e., the first layer -1 layer output; Represents a multi-head self-attention module; This represents the element-wise addition operation that implements residual joins; This represents the root mean square layer normalization function; Represents the feedforward network module; Representative at the In the layer, the intermediate output after passing through the MSA module and the first residual connection; Representing the The final output sequence of the layer, as the first layer +1 layer input.

2. The traffic flow prediction method based on a hierarchical semantic prompting large language model according to claim 1, characterized in that, The component generation step specifically includes: By comparing with the input multivariate traffic time series Each channel The corresponding independent RNN encoder captures the temporal context information of the channel and generates high-dimensional features. , The number of channels; through a linear shrinkage layer shared by all channels and a nonlinear activation function. The high-dimensional features are mapped to intermediate representation tensors. ; Each component is calculated using a channel-mixed MLP with shared time steps; wherein, the trend component... The intermediate representation tensor is obtained by an MLP The MLP is obtained through processing at each time step. The following nonlinear transformation is applied to the characteristics of all channels: ； in, This indicates that the trend component is in the batch. Data from all channels at time step t; Indicates batch index; Indicates the time step index; This indicates retrieving data from all channels within the specified dimension; Let be the intermediate representation tensor. Represents the intermediate representation tensor In, corresponding to batch Time step Data from all channels; Number of channels; Trend component MLP; The seasonal component The detrended sequence is obtained from another independent MLP, i.e. The MLP is obtained through processing at each time step. The characteristics of all channels are subjected to nonlinear transformation; The residual components The result is obtained through addition closure calculation: 。 3. The traffic flow prediction method based on a hierarchical semantic prompting large language model according to claim 2, characterized in that, It also includes processing the input multivariate traffic time series before the component generation step. Perform a reversible instance normalization process, including: For each instance and each channel within each batch, normalization is performed independently, and its mean is calculated. and standard deviation : ； in, The input is a multivariate traffic time series. The mean, Standard deviation, and Let L and L represent the length of the input multivariate traffic time series along the time dimension, respectively. Calculate the mean and variance. For batch size, Number of channels; Using a set of learnable affine transformation parameters , The input multivariate traffic time series Mapping to the normalized space yields the normalized sequence. : ； in, This indicates element-wise multiplication; It is a small constant used to prevent division by zero; And after generating the final prediction result, the final prediction result generated in the normalized space is restored to the original numerical scale through the corresponding inverse normalization operation: ； in, To restore the final prediction results to the original scale; This represents the final prediction result generated in the normalized space; The mean is calculated in the normalization step. To calculate the standard deviation in the normalization step, and These are learnable affine transformation parameters; This indicates element-wise multiplication; It is a small constant used to prevent division by zero, and its value is consistent with the value used during normalization.

4. The traffic flow prediction method based on a hierarchical semantic prompting large language model according to claim 3, characterized in that, For any component , , The component encoding step specifically includes: 2-1) Data partitioning: dividing the data into blocks Divided along the time axis There are data blocks, each with a length of . The sliding step size is Reshape the data ; 2-2) Intra-block feature extraction: Using a TCN as the backbone network, the temporal features within each data block are extracted independently to obtain the intra-block feature representation. TCN is composed of multiple residual layers stacked together, each residual layer containing causal convolution and dilated convolution; before extracting the temporal features within each data block, the data block dimension is converted to the channel dimension through an axis remapping operation to adapt to the input format of TCN and achieve parallel processing. 2-3) Feature Projection: The intra-block features output by the TCN are represented by a projection head module. Feature dimensions from data block length Target dimension required for mapping to a large language model .

5. A traffic flow prediction system based on a hierarchical semantic cueing large language model, implementing the traffic flow prediction method based on a hierarchical semantic cueing large language model as described in any one of claims 1-4, characterized in that, It includes a neural decomposition module, a component encoding module, a hierarchical prompt generation and assembly module, a large language model, and a prediction decoding and fusion module; The neural decomposition module decomposes the input multivariate traffic time series into several components, including trend components, seasonal components, and residual components. The component encoding module independently encodes each component into a component embedding sequence; The hierarchical prompt generation and assembly module generates and injects two layers of semantic information into the component embedding sequence: FBP vector and CSP vector; it then structurally assembles the CSP vector, FBP vector, and corresponding component embedding sequence into a unified input sequence and feeds it into the large language model for inference. The large language model comprehensively processes the global context from CSP, the instance-level behavior from FBP vectors, and the local numerical dynamics from the component embedding sequence in the unified input sequence to generate and output a high-dimensional representation containing future prediction information. The prediction decoding and fusion module decodes the high-dimensional representation output by the large language model into preliminary predicted values ​​of each component through a decoding head network, and then fuses them to generate the final multivariate time series traffic flow prediction result.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the traffic flow prediction method based on a hierarchical semantic prompting large language model as described in any one of claims 1-4.

Images