A lightweight large-scale multi-element time series prediction model and a training method thereof

Abstract

The application provides a lightweight large-scale multi-element time sequence prediction model and a training method thereof, the training method comprising: obtaining a training set comprising a plurality of traffic flow samples and corresponding labels; obtaining an initially constructed traffic flow prediction model, which comprises: a storage body for storing a plurality of source representation vectors, each source representation vector being a prototype vector representing a spatial attribute, an encoder for extracting time sequence features of each time sequence, and a decoder for determining traffic flow of a corresponding road at one or more future time points according to the time sequence features of each time sequence, a spatial identification vector of each time sequence, and a corresponding time attribute when the time sequence is collected, wherein the spatial identification vector of each time sequence is obtained by weighting each source representation vector according to the attention value of the source representation vector and the time sequence features of each time sequence; and performing multiple iterations of training on the traffic flow prediction model using the training set.

Description

Technical Field

[0001] This invention relates to the field of information technology, specifically to the field of neural network technology, and more specifically, to a lightweight large-scale multivariate time series prediction model and its training method. Background Technology

[0002] A time series is a sequence of data collected at multiple points in time that has a temporal relationship. It can come from various fields and is an important tool for studying and predicting various phenomena.

[0003] In the real world, time series data are ubiquitous, widely present in various fields, including finance, meteorology, energy, transportation, industrial manufacturing, the internet, and social media. For example, in finance, stock prices and trading volumes are time series data, their changes influenced by numerous factors such as company financial conditions, macroeconomic factors, and market trends. In meteorology, temperature, rainfall, and humidity are all time series data, their variations related to time and seasonal changes. In manufacturing, various parameters of production lines, such as temperature, pressure, speed, and vibration, are also time series data.

[0004] Applying time series forecasting techniques can serve multiple purposes, including but not limited to:

[0005] 1. Optimized Decision Making: Time series forecasting technology can provide predictions of future trends and changes, helping people make better decisions. For example, in the financial field, analyzing time series data such as exchange rates and stock prices can assist investors in formulating investment strategies.

[0006] 2. Resource Planning: Time series forecasting technology can help people predict future demand and supply, thereby better planning resources. For example, in the transportation sector, time series forecasting of traffic flow can optimize the allocation and scheduling of transportation resources.

[0007] Existing time series forecasting techniques mainly employ the following models:

[0008] (1) Traditional statistical models: such as linear regression, ARIMA, VAR, etc. These models can use historical data to predict future data, but their prediction effect is not ideal for dynamically changing, non-linear time series data.

[0009] (2) Traditional machine learning models, such as support vector machines and random forests, have a certain ability to process dynamic and nonlinear time series data compared with traditional statistical models. They can also make predictions by learning patterns in historical data. However, the learning ability of traditional machine learning models is sometimes insufficient when faced with larger datasets.

[0010] (3) Deep Learning Models: With the explosive growth of computing power, deep learning models, which have stronger representation capabilities and adaptability, have become the mainstream method in the field of time series prediction. Convolutional neural networks, recurrent neural networks, graph neural networks and other technologies have achieved good results in the field of time series prediction. In particular, models based on a special type of graph neural network: spatiotemporal graph neural network, have become the mainstream in related work in recent years.

[0011] Spatiotemporal graph neural networks hold such a crucial position in the field of time series forecasting because the way time series are generated naturally aligns with the graph data structure: time series values ​​are influenced by multiple factors, not only from themselves but also from other similar data sources. Consider a traffic system as an example: sensors in a traffic system are typically installed at intersections or junctions, and each sensor records a time series, such as a traffic flow series. Because roads are interconnected, the data recorded by different sensors are actually related. Therefore, to accurately predict the time series of a sensor, it is necessary to consider not only the historical data of that sensor but also the historical data of other sensors. Based on this reasoning, researchers treat sensors as nodes in a graph and the connections between roads as edges, thus representing the traffic system as a graph structure. Furthermore, traffic flow data has spatiotemporal characteristics—that is, the changes in traffic flow over time and space. These spatiotemporal characteristics can be considered as features of graph data, while the nodes and edges in the traffic network can be considered as elements of the graph.

[0012] Spatiotemporal graphical neural networks typically consist of two components: a graph convolutional neural network (GCN) and a temporal module. A GCN is a special type of neural network used to perform convolutional operations on a graph structure. The temporal module is a model for processing time series data, such as a convolutional neural network (CNN) or a recurrent neural network (RNN). By combining these two components, spatiotemporal graphical neural networks can process spatiotemporal graph data and model time series data while preserving spatial structure.

[0013] For the reasons mentioned above, spatiotemporal graph neural networks, with their ability to exchange and transmit information between nodes and edges in a graph, thus better capturing the spatiotemporal characteristics of time series data, have become the mainstream method in the field of time series prediction. A classic model is the Diffusion Convolutional Recurrent Neural Network (DCRNN). [1] ), Graph WaveNet [2] wait.

[0014] However, there's no such thing as a free lunch. More powerful models often come with higher complexity, and spatiotemporal graph neural networks are no exception. In fact, as the number of nodes in the graph increases, the time and space complexity of spatiotemporal graph neural networks rises rapidly, and the resources required for network training and inference also increase significantly. With the arrival of the big data era, the volume of time series data has exploded, making it possible to model even time series within a certain range into a massive graph. How to perform time series prediction on such large graphs has attracted the attention of some researchers.

[0015] To address this problem, current research focuses on methods and techniques for optimizing graph size, such as graph partitioning. [3] and parallel computing [4] Methods such as these improve the computational efficiency of graph neural networks. This approach enhances the usability of graph neural networks for time series prediction tasks in large-scale scenarios to some extent, but it still doesn't completely abandon the graph neural network tool. This approach brings the following potential concerns:

[0016] 1. The model requires a large amount of storage and computing resources; in large-scale scenarios, the problem of high time and space complexity of graph neural network models has not been fundamentally solved, and the required computing resources and training time are still considerable.

[0017] 2. The design of graph partitioning technology is quite difficult; graph partitioning technology has high requirements for data preprocessing, and it is difficult to design a good graph partitioning method.

[0018] The aforementioned prior art documents are as follows:

[0019] 1.Yaguang Li,Rose Yu,Cyrus Shahabi,and Yan Liu.2018.DiffusionConvolutional Recurrent Neural Network:Data-Driven Traffic Forecasting.InICLR.

[0020] 2.Z Wu,S Pan,G Long,J Jiang,and C Zhang.2019.Graph WaveNet for DeepSpatial-Temporal Graph Modeling.In IJCAI.

[0021] 3. Mallick, T., Balaprakash, P., Rask, E., & Macfarlane, J. (2020). Graph-partitioning-based diffusion convolutional recurrent neural network for large-scale traffic forecasting. Transportation Research Record, 2674(9), 473-488.

[0022] 4. Liu, Y., Li, H., Zhang, G., Zeng, Summary of the Invention

[0023] Therefore, the purpose of this invention is to overcome the shortcomings of the prior art and provide a training method for traffic flow prediction models.

[0024] The objective of this invention is achieved through the following technical solution:

[0025] According to a first aspect of the present invention, a training method for a traffic flow prediction model is provided, comprising: acquiring a training set, which includes multiple traffic flow samples and corresponding labels, wherein each traffic flow sample is a multivariate time series, the multivariate time series includes time series collected by multiple sensors at different roads in a preset traffic system at the same time, each time series includes traffic flow data at multiple historical moments collected by sensors set at the intersection of a road, and the label is the ground truth traffic flow value of the intersection of the road corresponding to each time series in the traffic flow sample at one or more future moments; acquiring an initially constructed traffic flow prediction model, which includes: a storage for storing multiple source representation vectors, each source representation vector representing a... The prototype vector of spatial attributes is used by the encoder to extract the temporal features of each time series from the input multivariate time series. The decoder is used to determine the traffic flow of the road corresponding to each time series at one or more future times based on the temporal features, spatial identifier vector, and the temporal attributes corresponding to the time series when it was collected. The spatial identifier vector of each time series is obtained by weighting each source representation vector according to the temporal features of each time series and the attention value of each source representation vector in the storage. Based on the initially constructed traffic flow prediction model, the traffic flow prediction model is trained iteratively multiple times using the training set. During training, the trainable parameters of the encoder and decoder and the multiple source representation vectors in the storage are updated.

[0026] Optionally, the spatial identifier vector of each time series is calculated as follows: the attention value between the time series feature and each source representation vector in the storage is calculated to obtain multiple attention values; each of the multiple attention values ​​is used to perform a weighted summation on the corresponding source representation vector in each source representation vector to obtain the spatial identifier vector of the time series feature.

[0027] Optionally, the spatial identifier vector for each time series is calculated as follows:

[0028]

[0029] Where Q represents the time series characteristics, K T d represents the transpose of multiple source representation vectors in the storage, d represents the feature dimension of the source representation vectors, and V represents the multiple source representation vectors in the storage.

[0030] Optionally, the number of multiple source representation vectors is set to be less than the number of time series in the multivariate time series.

[0031] Optionally, the encoder is a multilayer perceptron, and the decoder is a multilayer perceptron.

[0032] Optionally, the traffic flow includes the total number of vehicles passing through the road per unit time and / or the average vehicle speed.

[0033] According to a second aspect of the present invention, a traffic flow prediction method is provided, comprising: acquiring data to be predicted, the data to be predicted including traffic flow data of multiple roads at multiple historical times; inputting the data to be predicted into a traffic flow prediction model trained by the method of the first aspect, and predicting the traffic flow of the multiple roads corresponding to the data to be predicted at one or more future times.

[0034] According to a third aspect of the present invention, a traffic light control method is provided, comprising: acquiring traffic flow data for multiple roads at one or more future times, predicted by the method of the second aspect; and controlling corresponding traffic lights based on the predicted traffic flow data for the multiple roads at one or more future times, so as to reduce road congestion time.

[0035] According to a fourth aspect of the present invention, an electronic device is provided, comprising: one or more processors; and a memory for storing executable instructions; wherein the one or more processors are configured to implement the steps of the methods of the first aspect, the second aspect, and / or the third aspect by executing the executable instructions. Attached Figure Description

[0036] The embodiments of the present invention will be further described below with reference to the accompanying drawings, wherein:

[0037] Figure 1 This is a schematic diagram of the traffic flow prediction model according to an embodiment of the present invention. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the invention.

[0039] As mentioned in the background section, using graph neural networks to predict traffic flow presents challenges such as the large storage and computational resources required for the model and the significant design difficulties associated with graph partitioning techniques. To address these issues, the applicant has designed a traffic flow prediction model containing a storage unit for multiple source representation vectors. Each source representation vector represents a prototype vector of a spatial attribute. When predicting traffic flow, a multivariate time series is directly input. The decoder, in addition to using the temporal attributes of the time series and extracting temporal features from the encoder, also uses the temporal features of each time series and the attention values ​​of each source representation vector in the storage unit to weight each source representation vector, obtaining a spatial identifier vector for each time series for prediction. Since the source representation vectors in the storage unit are also learned, the spatial dependencies between data sources can be modeled efficiently and with low computational cost using the storage unit. This allows the spatial identifier vector of the time series to be calculated using the temporal features corresponding to the time series and the source representation vectors. Traffic flow prediction is then performed using the temporal features, temporal attributes, and spatial identifier vectors of each time series in the multivariate time series, improving model performance. Furthermore, this model is not a graph neural network and does not require graph partitioning techniques, thus eliminating dependence on graph structures and achieving efficient modeling of spatial dependencies.

[0040] Before describing the embodiments of the present invention in detail, some of the terms used therein are explained as follows:

[0041] The data source is a sensor placed at a road intersection to measure traffic flow.

[0042] Multivariate time series refers to multiple time series originating from different data sources. A time series is ordered data recorded chronologically according to certain time intervals. In this invention, each multivariate time series is a sequence of traffic flow data for a road at multiple historical moments.

[0043] To facilitate understanding of the overall implementation process of this invention, the overall process is first simplified as follows:

[0044] Training Phase A:

[0045] A1. Construct an initial traffic flow prediction model;

[0046] A2. Obtain the dataset and extract a subset from it as the training set. The training set includes multiple traffic flow samples, that is, each traffic flow sample includes traffic flow sequences of multiple intersections (corresponding to multivariate time series).

[0047] A3. Based on the preset loss function, train the initial traffic flow prediction model using the training set, update the parameters of the traffic flow prediction model, and obtain the trained traffic flow prediction model.

[0048] Test Phase B:

[0049] B1. Input the traffic flow samples in the test set into the trained traffic flow prediction model and output the prediction results, that is, the traffic flow of the road corresponding to each time series in the multivariate time series at one or more future times.

[0050] B2. Calculate the prediction performance based on the prediction results and labels corresponding to the traffic flow samples in the test set.

[0051] The testing phase works almost exactly the same way as the training phase, except that the testing phase does not update the model's parameters based on the loss function, but instead uses the test set to test the model trained on the training set.

[0052] Application Phase C:

[0053] C1. Obtain the multivariate time series to be predicted, which is a multivariate time series consisting of traffic flow on multiple roads;

[0054] C2. Input the multivariate time series data to be predicted into the trained traffic flow prediction model and output the prediction results.

[0055] To better understand this invention, the following sections will provide a detailed description of its model structure, training samples, training process, and application scenarios.

[0056] I. Model Structure

[0057] See Figure 1 Traffic flow prediction models (i.e., lightweight large-scale multivariate time series prediction models) can include storage, encoders, and decoders. Among them:

[0058] The memory bank is used to store multiple source representation vectors. According to one embodiment of the invention, each source representation vector is a prototype vector representing a spatial attribute. Spatial relationships (or spatial associations) between sensors are also indispensable. Modeling of spatial relationships is indirectly achieved using the memory bank. This is because the source representation vectors in the memory bank will be used subsequently to generate spatial identifier vectors for each time series in the multivariate time series. For example, the memory bank stores a source representation vector table (two-dimensional vector table) of dimension M*d, consisting of M source representation vectors of length d (which can be called memory). Its function is to form stable prototype representations, i.e., source representation vectors, representing a spatial attribute during training. Illustratively, M is set to 10 or 20, and d is set to 32, 64, or 128.

[0059] To avoid excessive storage space and computational resources required for subsequent calculations, according to one embodiment of the present invention, the number of multiple source representation vectors is set to be less than the number of time series in the multivariate time series (each time series originating from a different sensor, i.e., a different data source). For example, in a multivariate time series with more than 1000 time series, the number of source representation vectors M is set to 10, 20, 50, 100, etc. It should be understood that this is only illustrative, and implementers can train multiple times and optimize the value of M with a validation set according to the specific application scenario to set the value of M in the specific implementation scenario.

[0060] An encoder (or temporal feature encoder) is used to extract temporal features of each time series from an input multivariate time series. According to one embodiment of the invention, the encoder is a multilayer perceptron (MLP). The encoder uses a multilayer perceptron with 3-5 layers, for example, a 3-layer or 4-layer perceptron. Illustratively, the dimension of the encoder's input data is B*N*(L*C), where B refers to the number of traffic flow samples input at one time during training (i.e., batch size), N refers to the number of time series in the traffic flow samples (or the number of time series in the multivariate time series, corresponding to the number of data sources in the traffic system to be predicted; each data source corresponds to a sensor collecting traffic flow data from one road (e.g., an intersection), L refers to a time series containing L time points of traffic flow data, meaning the model needs to make predictions about the future based on data from the past L time points, and C refers to the dimension of the traffic flow data at each time point. For example, C may be 1-dimensional (e.g., containing only the total number of vehicles passing through the road per unit time or the average vehicle speed) or 2-dimensional (e.g., including both the total number of vehicles passing through the road per unit time and the average vehicle speed). However, before training, it is necessary to define the specific value of C and the specific type of traffic flow data. For example, if C is limited to 1 and the specific type of traffic flow data is the total number of vehicles passing through the road per unit time, then the training will only use the total number of vehicles passing through the road per unit time for prediction, and the predicted traffic flow data will also be the total number of vehicles passing through the road per unit time at multiple future time points. For illustration, B is set to 8, 16, 32, 64, or 128; N is actually set according to the number of data sources in the traffic system; for example, if there are 1000 data sources, then N is set to 1000; L is set to 12, 24, or 36.

[0061] In other words, in the field of multivariate time series forecasting, a time series is generated by a single variable (or sensor). Taking a traffic system as an example, if this system contains N sensors recording data, then it contains N time series. Training samples consist of subsequences extracted from the time series; they represent only a short portion of the time series. B is defined as the number of training samples read into the model at a time during training.

[0062] According to one embodiment of the invention, the encoder is a linear projection used to map each time series of length L*C to a latent space: in, and These are the trainable parameters, W represents the parameters of each layer of the perceptron, and b represents the bias parameters. Let represent the variable at time point j within the i-th time series of a multivariate time series. E is the vector output by the encoder (i.e., temporal features), where d is the dimension of the latent space. After the operation of the temporal feature encoder, the input of dimension B*N*(L*C) is transformed into a temporal feature representation of dimension B*N*d, denoted as E. T E T It includes the temporal characteristics of each time series in a multivariate time series.

[0063] In addition to the temporal features, the spatial identifier vector of each time series and the temporal attribute corresponding to the time series when it was collected are also required. This can be achieved by calling some preset interfaces. After the calculation is completed, the temporal features, spatial identifier vector and temporal attribute corresponding to the time series when it was collected are transmitted to the decoder for subsequent calculation.

[0064] According to one embodiment of the present invention, the spatial identifier vector of each time series is obtained by weighting each source representation vector based on the temporal features of each time series and the attention values ​​of each source representation vector in the memory. Thus, a spatial identifier vector for distinguishing different time series is obtained from the source representation vectors. Preferably, the spatial identifier vector of each time series is calculated as follows: the attention values ​​between the temporal features of the time series and each source representation vector in the memory are calculated respectively to obtain multiple attention values; the corresponding source representation vectors in each source representation vector are weighted and summed using each of the multiple attention values ​​to obtain the spatial identifier vector of that temporal feature. For example, the spatial identifier vector of each time series is calculated as follows:

[0065]

[0066] Where Q represents the time series feature (i.e., using the time series feature as the query vector Q), K T d represents the transpose of multiple source representation vectors in the storage, d represents the feature dimension of the source representation vectors, and V represents the multiple source representation vectors in the storage.

[0067] During computation, a query is performed on the M*d source representation vector table in the storage using time-series features generated by a data source. This yields the similarity (i.e., multiple attention values) between the time-series features of each time series and the various source representation vectors in the table. Based on these similarities, the source representation vectors are weighted and summed to obtain a spatial identifier vector (or identity vector). The distance between spatial identifier vectors from different data sources reflects the spatial relationships between them.

[0068] The process described in the previous paragraph specifically relies on an attention mechanism. The following section introduces an illustrative process for generating spatial identifier vectors for temporal features based on storage and an attention mechanism:

[0069] The attention mechanism mainly involves three key vectors: Q, K, and V, representing the query, key, and value, respectively. K and V are identical vectors. Unlike traditional attention mechanisms, this invention does not require additional transformation matrices for K and V. In this invention, Q is the temporal feature output by the encoder, with dimensions B*N*d as mentioned above, while K and V are source representation vector tables of dimension m*d.

[0070] The multiple attention values ​​mentioned above are denoted as W (including w1, w2...w). m The weighted representation E is calculated from Q and K, and then W is used to weight the m vectors in V accordingly. S This refers to the spatial identification vector. For example, the spatial identification vector is calculated as follows:

[0071] E S =WV=w1*v1+w2*v2+w3*v3+……+w m *v m

[0072] Among them, w1~w m Let v1 and vm represent the temporal features of a given time series and the attention values ​​of the 1st to mth source representation vectors, respectively. m These represent the first to m source representation vectors, respectively.

[0073] In addition to the above, time attributes of the time series need to be introduced, as this can improve the model's predictive performance. According to one embodiment of the present invention, the time attributes of each time series include a weekday attribute and / or a time period attribute. The weekday attribute indicates which day of the week the data was collected on, and the time period attribute indicates which time period of the day the data was collected on. The time attributes can be extracted from the time information corresponding to the time series in the original dataset. The time attribute is denoted as E. time.

[0074] Feature integration is the last stage before the decoder; this stage involves integrating the various features E T E S E time The features to be decoded are then integrated, and the resulting integrated features are used as input to the decoder. Integration could, for example, involve combining E... T E S E time The features E to be decoded are obtained by splicing them together.

[0075] The decoder is used to determine the traffic flow of the road corresponding to each time series at one or more future times based on the temporal features, spatial identifier vector, and time attributes corresponding to the time series when it was collected. According to one embodiment of the invention, the decoder's structure is similar to an encoder, also being a multilayer perceptron (MLP). Preferably, the decoder uses a multilayer perceptron with 3-5 layers. The input to the decoder is the feature E to be decoded obtained in the preceding steps. The decoder is responsible for mapping the feature E to be decoded into a predicted value. Assuming the dimension of the input feature E to be decoded is B*N*e, and the model wants to predict data for H future time points, the decoder can use a linear transformation to map E into a vector of dimension B*N*H as the prediction result. The value of H can be the same as or different from the number of time points L in the input time series; for example, H can be set to 6, 12, 24, or 36.

[0076] In the field of time series forecasting, spatial dependencies exist between different variables, and the modeling method of spatial dependencies can significantly impact the forecasting results. However, in large-scale scenarios, neither graph convolution nor predefined graphs are practical for spatial dependency modeling. Therefore, influenced by the data-driven paradigm, this invention proposes a storage-based spatial dependency modeling approach. By leveraging storage, the model autonomously learns the spatial relationships between data sources and generates a vector as its spatial identifier for each time series, eliminating reliance on graph structures and achieving efficient spatial dependency modeling. This enables a lightweight time series encoding / decoding framework based on MLP, retaining only the two essential modules for time series forecasting: a time series encoder (Embedding Layer) and a time series decoder, along with a storage unit. This simplifies the network structure by eliminating the need for complex neural networks, reducing the computational resources required by the model, and enabling multivariate time series forecasting in large-scale scenarios.

[0077] II. Training Samples

[0078] According to one embodiment of the present invention, the original dataset includes one or more original multivariate time series collected from various roads in the traffic system. The original multivariate time series are relatively long time series and require segmentation using a sliding window to obtain the desired traffic flow samples. For example, the original dataset is the LargeST dataset (see dataset link: https: / / github.com / liuxu77 / LargeST) or its sub-datasets GLA, GBA, and SD. Another example is the California speed.h5 dataset or the California flow.h5 dataset from https: / / github.com / tanwimallick / graph_partition_based_DCRNN.

[0079] To illustrate, assume that the traffic system corresponding to the original dataset has N data sources, each data source generating one time series from the original multivariate time series. Before the training and testing phases, the samples in the training and testing sets must first be constructed.

[0080] According to one embodiment of the present invention, the construction of traffic flow samples involves segmenting each original multivariate time series using a sliding window to obtain segmented multivariate time series. Each time series in the segmented multivariate time series includes L+H data points. Correspondingly, the first multivariate time series of length L is used as the traffic flow sample, and the second multivariate time series of length H is used as the label. Assume that a dataset is to be constructed to predict the future L data points based on the historical L data points in each time series of the multivariate time series; in other words, L is the input and output length used by the traffic flow prediction model. That is, the task of the traffic flow prediction model is to make predictions of length L using an input of length L. The sample construction process is as follows: Let the i-th time series in the original multivariate time series be Si. Divide the input sequence Si into P segments of length 2*L (the input sequence is obtained by a sliding window of length 2*L on the original multivariate time series, where the first L data points are used as input; the second L data points are used as labels, i.e., the true values ​​of the future L data points). The j-th segment can be represented as... C represents the number of variables in the traffic flow at each time point in the time series (each sensor may collect multiple variables, such as the total number of vehicles passing through the road per unit time and the average vehicle speed). By performing the above operation on all time series, a dataset containing multiple traffic flow samples (length L) and corresponding labels (length L) for each traffic flow sample can be constructed. Next, the dataset can be divided into training, testing, and validation sets (optional) according to the ratio set by the implementer (e.g., 7:2:1 or 8:1:1). The training, testing, and validation sets each include multiple traffic flow samples and corresponding labels for each sample. Each traffic flow sample is a multivariate time series, which includes time series collected from multiple data sources at different roads within a pre-defined traffic system at the same time. Each time series includes traffic flow data from multiple historical moments collected by sensors at intersections of a road. The label is the ground truth traffic flow value for the road corresponding to each time series in the traffic flow sample at one or more future moments.

[0081] III. Training Process

[0082] During training, the traffic flow prediction model is trained iteratively multiple times using the training set, based on the initially constructed traffic flow prediction model. During training, the trainable parameters of the encoder and decoder, as well as multiple source representation vectors in the storage are updated.

[0083] According to one embodiment of the present invention, each training session includes:

[0084] Obtain all traffic flow samples currently being trained (e.g., all samples in a batch size) and their corresponding labels.

[0085] Input all the traffic flow samples currently being trained into the current traffic flow prediction model (the initial traffic flow prediction model during the first training, and the updated traffic flow prediction model from the previous training) and output the traffic flow of the road corresponding to each time series in all the traffic flow samples currently being trained at one or more future times.

[0086] Based on the traffic flow and corresponding label of each road at one or more future times in all traffic flow samples of the current training, the loss value is calculated using a preset loss function. The gradient is calculated based on the loss value and backpropagation is used to update the trainable parameters of the encoder and decoder, as well as multiple source representation vectors in the storage.

[0087] According to one embodiment of the present invention, the preset loss function is an objective function used to guide the optimization of model parameters. For example, the Mean Absolute Error (MAE), a loss function commonly used in the field of time series forecasting, is adopted as the loss function. The MAE loss function measures the average absolute error between the model's predicted values ​​and the actual values. It is insensitive to outliers and preserves the sign information of the error. The formula for calculating the MAE of a single sample is expressed as:

[0088]

[0089] Where L represents the length of the traffic flow sample used in the current training, N represents the number of time series in the traffic flow sample, and C represents the number of variables contained in the traffic flow at each time point in the time series. Indicates the predicted value. This represents the predicted value of the k-th variable at the j-th time point in the i-th time series. It's a label, Y ijk Let L represent the label value of the k-th variable at the j-th time point in the i-th time series. If B traffic flow samples are used in one training session, then the L value for each traffic flow sample is calculated using the above formula. mae Then, the average value is calculated to obtain the loss value for this training.

[0090] IV. Training Equipment

[0091] Based on the aforementioned method, a corresponding training device can be constructed. This device aims to enable the method of the present invention to be better promoted and applied. Next, the functional interfaces and details of the device will be described in Table 1.

[0092] Table 1

[0093]

[0094]

[0095]

[0096]

[0097] V. Application Scenarios

[0098] According to an embodiment of the present invention, a traffic flow prediction method is provided, comprising: acquiring data to be predicted, the data to be predicted including traffic flow data of multiple roads at multiple historical times; inputting the data to be predicted into a traffic flow prediction model trained by the training method for traffic flow prediction models of the aforementioned embodiment, and predicting the traffic flow of the multiple roads corresponding to the data to be predicted at one or more future times. In the field of transportation, by predicting the time series of traffic flow, the traffic flow at one or more future times can facilitate city managers to optimize the allocation and scheduling of traffic resources.

[0099] According to one embodiment of the present invention, a traffic light control method is provided, characterized by comprising: acquiring traffic flow forecasts for multiple roads at one or more future times, obtained by the traffic flow forecasting method of the aforementioned embodiment; and controlling corresponding traffic lights based on the predicted traffic flow for the multiple roads at one or more future times, thereby reducing road congestion time. Controlling corresponding traffic lights based on the predicted traffic flow for multiple roads at one or more future times to reduce road congestion time is a technique already well-established in the art, and implementers can employ existing techniques or their customized methods; this invention does not impose any limitations on this.

[0100] To verify the effectiveness of the technical solution of the present invention, the inventors also conducted comparative experiments.

[0101] (1) The existing techniques for comparative experiments are as follows:

[0102] HI, the corresponding literature is: Historical inertia: A neglected but powerful baseline for long sequence time-series forecasting)

[0103] MLP, encoder and decoder use only a 3-layer perceptron structure (no memory);

[0104] D-Linear, the corresponding literature is: Are transformers effective for time series forecasting;

[0105] STGCN, the corresponding paper is: Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting;

[0106] Pyraformer, the corresponding literature is: Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting;

[0107] The corresponding literature for STID is: Spatial-temporal identity: A simple yet effective baseline for multivariate time series forecasting.

[0108] (2) Comparative experiment of the structure of the present invention

[0109] The traffic flow prediction model (MemoMLP) constructed in this invention includes a storage unit, an encoder, and a decoder. The encoder and decoder are 3-layer MLPs, and the storage unit adopts a 10*64 (M*d) source representation vector table.

[0110] (3) Dataset for the comparative experiment

[0111] The dataset for the comparative experiment comes from:

[0112] The California speed.h5 dataset, located at https: / / github.com / tanwimallick / graph_partition_based_DCRNN, is a large-scale vehicle speed dataset with 11,160 variables. The experimentally segmented samples are historical time series of length 12, and the model's output, i.e., the prediction, is a future time series of length 12. Specifically, in this dataset, it predicts the speed of vehicles at intersections in the next 12 time points based on the past 12 time points.

[0113] (4) Comparison of experimental results

[0114] The results of the comparative experiment are shown in Table 2:

[0115] Table 2

[0116]

[0117] As can be seen from Table 2, although the present invention does not use a complex neural network, but adopts a lightweight model structure, the present invention has the smallest mean absolute error of the predicted values ​​at multiple time points and the average predicted values ​​at 1-12 time points, and the prediction performance is the best.

[0118] In summary, this invention learns multiple prototype vectors for calculating the spatial identifier vector corresponding to each time series through a storage unit. Then, the spatial identifier vector of each time series is determined by the multiple prototype vectors in the storage unit. This efficiently and with low computational cost represents the spatial relationship between time series, eliminating the dependence on graph structures and achieving efficient modeling of spatial dependencies. Furthermore, it still achieves superior prediction performance even when the encoder and decoder only use a simple multilayer perceptron.

[0119] It should be noted that although the steps are described in a specific order above, it does not mean that the steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently, or even in a different order, as long as the required function can be achieved.

[0120] This invention can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the invention.

[0121] Computer-readable storage media can be tangible devices that hold and store instructions for use by an instruction execution device. Computer-readable storage media can be, for example, including but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof.

[0122] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A training method for a traffic flow prediction model, characterized in that, include: Obtain a training set, which includes multiple traffic flow samples and corresponding labels. Each traffic flow sample is a multivariate time series, which includes time series collected by multiple sensors on different roads in a preset traffic system at the same time. Each time series includes traffic flow data collected by sensors set at the intersection of a road at multiple historical moments. The label is the true value of traffic flow at one or more future moments for the road corresponding to each time series in the traffic flow sample. Obtain the initially constructed traffic flow prediction model, which includes: A storage volume used to store multiple source representation vectors, each of which is a prototype vector representing a spatial attribute. An encoder used to extract temporal features of each time series from an input multivariate time series. A decoder for determining the traffic flow of a road at one or more future times for each time series based on its temporal features, spatial identifier vector, and the time attribute corresponding to the time series when it was collected. The spatial identifier vector of each time series is obtained by weighting each source representation vector based on the temporal features of each time series and the attention values ​​of each source representation vector in the storage. The spatial identifier vector is calculated as follows: the attention values ​​between the temporal features of the time series and each source representation vector in the storage are calculated to obtain multiple attention values. Each of the multiple attention values ​​is then used to perform a weighted summation of the corresponding source representation vectors in each source representation vector to obtain the spatial identifier vector of the temporal feature. Based on the initially constructed traffic flow prediction model, the traffic flow prediction model is trained iteratively multiple times using the training set. During training, the trainable parameters of the encoder and decoder, as well as multiple source representation vectors in the storage are updated.

2. The method according to claim 1, characterized in that, The spatial identifier vector for each time series is calculated as follows: in, Representing the temporal characteristics of a time series, This represents the transpose of multiple source representation vectors in the memory. This represents the feature dimension of the source representation vector. This represents multiple source representation vectors in the memory.

3. The method according to claim 1, characterized in that, The number of multiple source representation vectors is set to be less than the number of time series in the multivariate time series.

4. The method according to claim 1, characterized in that, The encoder is a multilayer perceptron, and the decoder is a multilayer perceptron.

5. The method according to any one of claims 1-4, characterized in that, The traffic flow includes the total number of vehicles passing through the road per unit time and / or the average speed of the vehicles.

6. A traffic flow prediction method, characterized in that, include: Obtain the data to be predicted, which includes traffic flow data of multiple roads at multiple historical times; The traffic flow prediction model is trained by the method described in any one of claims 1-5 by inputting the data to be predicted into the traffic flow prediction model, and the traffic flow of multiple roads corresponding to the data to be predicted is predicted at one or more future times.

7. A traffic light control method, characterized in that, include: Obtain the traffic flow of multiple roads predicted by the method according to claim 6 at one or more future times; Based on the predicted traffic flow of multiple roads at one or more future times, the corresponding traffic lights are controlled to reduce road congestion time.

8. A computer-readable storage medium, characterized in that, It stores a computer program that can be executed by a processor to implement the steps of the method according to any one of claims 1 to 7.

9. An electronic device, characterized in that, include: One or more processors; as well as Memory, wherein the memory is used to store executable instructions; The one or more processors are configured to implement the steps of the method according to any one of claims 1 to 7 by executing the executable instructions.

Images