A method for processing traffic flow data based on data segmentation

Abstract

The invention discloses a vehicle flow data processing method based on data segmentation, comprising the following steps: step 1, data processing and segmentation, the data is vehicle flow data and is collected by sensors deployed on a highway, and a data set is defined as X; step 2, model construction: step 3, pre-task design and pre-training loss function definition, an MTP pre-task is designed to train the model in step 2 to learn the intrinsic characteristics of the vehicle flow data and obtain a representation vector of the data; step 4, fine-tuning according to downstream tasks, in the fine-tuning stage, the last fully connected layer is replaced with a corresponding output layer according to the objectives of different downstream tasks; the invention trains the model by means of self-supervised learning, the data does not need to be centrally stored and annotated by professionals, the time and professional knowledge required for data annotation are saved, manpower and material resources are saved, and mass data can be fully and efficiently utilized, and a model with generalization is obtained.

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicle flow data analysis in intelligent traffic system (ITS), and in particular, relates to a vehicle flow data processing method based on data segmentation. Specifically, the method uses the self-supervised learning (SSL) method in deep learning, and draws on the idea of ​​word segmentation in natural language processing (NLP). By training a pre-trained model that is unrelated to downstream tasks, the token-based feature representation in the vehicle flow data is automatically learned, and good performance is achieved through fine-tuning in different downstream tasks. Background Art

[0002] Traffic flow refers to the number of vehicles passing a fixed point within a certain time interval. The rapid development of urbanization has brought about a surge in urban population and also posed new challenges to the mobility and stability of urban transportation. Among these challenges, ITS has become one of the most active research areas due to its potential to promote system effectiveness and decision-making. In intelligent transportation systems, traffic flow data can intuitively reflect traffic conditions and play an important role in route planning and traffic light control. Therefore, accurate traffic flow data analysis is the key to the stability and efficiency of many ITS applications. As a time series, traffic flow data has a natural time correlation, and its nonlinear and random characteristics also bring difficulties to data processing. At the same time, traffic flow data will also show different characteristics due to different factors. For example, in different blocks of the city, due to the different external conditions of the density of roadside buildings and the width of the streets, traffic flow data will show different characteristic trends; on the other hand, under real conditions, the irregular jitter caused by traffic accidents or sudden road repairs makes the processing and analysis of traffic flow data challenging, especially in the continuous long-term prediction. The accuracy is difficult to improve.

[0003] In recent years, with the development of deep neural networks (DNN), the analysis and processing of traffic flow data has gradually transitioned from traditional methods to DNN methods with good results. Traffic flow processing tasks are divided into three categories: traffic flow classification, traffic flow prediction, and traffic flow completion. Among them, traffic flow prediction is the basis for the development of various ITS applications. Self-supervised learning uses a large amount of unlabeled data to train a pre-trained model that can learn the deep representation of data and further fine-tune it for different downstream tasks through a small amount of labeled data. It can also achieve good performance while accelerating the training of downstream tasks.

[0004] The patent with application number 202110190389.0 discloses a method and device for vehicle data processing, which is applied to a cloud server, wherein the method includes: obtaining the original collected data of each vehicle object; mining the target data from the original collected data; determining the label information of the target data, and adding the label information to the target data, wherein the label information includes one or a combination of the following: vehicle perception label, decision planning label, path label, positioning label, map label; using the label information as an index to store the target data.

[0005] In the field of NLP, BERT's large-scale pre-trained model has achieved better performance in different tasks through self-supervised learning training methods. The pre-training tasks designed for the BERT model are Masked Language Model (MLM) and Next Sentence Prediction (NSP), providing a powerful pre-training model for various NLP tasks. Inspired by the success of self-supervised learning in NLP, researchers further explored its ability to learn image visual representations in the field of Computer Vision (CV): In the self-supervised pre-training stage, the input image is input into the model for training in a task-independent manner without classification labels, in order to obtain representation information that is not suitable for a specific classification task. Larger and deeper models can better utilize unlabeled data to obtain higher classification accuracy on downstream tasks. With the advent of big data and the Internet of Everything, various terminal devices and a large number of deployed sensors can collect massive amounts of data for us in real time. Among them, traffic flow data can be obtained in real time by traffic probes and sensors throughout the road network, and provides a basis for using machine learning and deep learning to study ITS. However, conventional machine learning / deep learning methods require a large amount of labeled data for the model to imitate and learn. However, the data labeling work requires technical personnel with professional knowledge to do it manually, which is time-consuming and labor-intensive and difficult to obtain on a large scale, making it difficult to use these unlabeled raw data efficiently and accurately. The analysis and processing of traffic flow data is not only focused on the single task of prediction, but also on other tasks such as classification and completion. Different tasks also contain different task requirements, such as short-term prediction, medium-term prediction, long-term prediction, binary classification, multi-classification, continuous missing completion or random missing completion. This requires that a matching model needs to be trained separately for each specific task, that is, the prediction model trained for the traffic flow prediction task cannot complete other tasks of traffic flow classification or completion; and when the previous task is updated or replaced, a new model needs to be retrained from scratch.

[0006] At present, the processing of traffic flow data takes a time step as the smallest unit of analysis, rather than considering a sequence of several consecutive time points as a unified whole. Models trained on a point-by-point basis focus more on short-term connections and high-frequency features in traffic flow data, making them difficult to apply to long-term prediction and continuous completion tasks.

[0007] Existing methods for processing traffic flow data all view traffic flow data as a time series consisting of individual points. This means that when training the model, it can only focus on short-term, high-frequency features between the data, which will lead to poor performance in downstream tasks.

[0008] Traditional methods and current deep learning methods based on Convolutional Neural Network (CNN) and Long-Short Term Memory (LSTM) perform poorly when performing long-term prediction or completion. Summary of the invention

[0009] The purpose of the present invention is to provide a traffic flow data processing method TSSN (Token-based Self-Supervised Network) based on data segmentation that can overcome the above-mentioned technical problems. The method of the present invention specifically includes data processing and segmentation, a task-independent pre-trained model and a task-related downstream fine-tuning model, wherein "segmentation" comes from the operation of word segmentation of a sentence in NLP. TSSN draws on this idea and uses this expression, that is, in TSSN, "segmentation" refers to dividing a segment of traffic flow data into multiple sub-segments, each segment is analogous to a "word" in NLP. The present invention aims to solve the problem that the existing method of processing traffic flow data using deep learning models requires a large amount of difficult-to-obtain labeled data, and different models need to be trained for different downstream tasks.

[0010] The method of the present invention comprises the following steps:

[0011] Step 1: Data processing and word segmentation. The data is traffic flow data collected by sensors deployed on the road. The data set is defined as X, and the length of each data is N. Define x i ∈X, i=1,…, N is the i-th point in a piece of data. The steps of data preprocessing and word segmentation are as follows:

[0012] Step 1.1, data preprocessing, normalizing the data, i.e., the traffic flow data, using the maximum-minimum normalization method to adjust all data to between [0, 1], as shown in the following formula (1):

[0013]

[0014] Step 1.2, data segmentation, define each word contains K points, each data contains M = N / K words, define the i-th word in each data as t j ,j∈[1,M], as shown in the following formula (2):

[0015] t j :=x (j-1)K:jK ={x (j-1)K+1 ,x (j-1)K+2 ,...,x jK}, ... (2),

[0016] The original data is normalized and then segmented, that is, K consecutive points are treated as a unified whole. Each sentence has a total of M words as the original input into the model;

[0017] Step 2, model construction:

[0018] Step 2.1, Token Embedding. Set the input after tokenization in step 1 as X, with a dimension of M×K, that is, M words, each word contains K points. Multiply X by a weight matrix to obtain a matrix consisting of the embedding vector of each word and use it to map the input vector to a high-dimensional latent space to facilitate the training of subsequent models. The dimension of the high-dimensional latent space is defined as D, which is the same as the dimension of the subsequent Transformer encoder, as shown in the following formula (3):

[0019]

[0020] in, D is the word embedding dimension;

[0021] Step 2.2, Positional Encoding. The word embedding matrix should be added to the positional encoding to integrate the positional relationship information between words. The positional information here is at the word level rather than the single point level. The position of the input data is encoded by a set of sine and cosine functions of different frequencies. The positional encoding matrix is ​​given by the following formula (4):

[0022]

[0023] Among them, j∈[1,M],d∈[1,D], represents the element in the jth row and dth column of the position encoding matrix P. Finally, the position encoding matrix and the word embedding matrix are added, as shown in the following formula (5):

[0024]

[0025] Step 2.3, self-attention mechanism, the word embedding matrix will pass through the encoder of Y L-head attention transformer to obtain the representation vector of each word in the traffic flow data. Transformer directly adopts the existing model and defines the Transformer encoder as The matrix composed of the word representation vector is shown in the following formula (6):

[0026]

[0027] in, The dimension is M×D. Finally, the representation matrix passes through a fully connected layer and undergoes dimension transformation to obtain the final output, as shown in the following formula (7):

[0028]

[0029] in, F(N) means flattening the output into an N-dimensional vector. The final model output dimension is consistent with the initial input dimension, which is an N-dimensional vector.

[0030] Step 3: Pre-task design and pre-training loss function definition. Design the MTP pre-task to train the model described in step 2 to learn the intrinsic characteristics of the traffic flow data and obtain the data representation vector. The specific steps are as follows:

[0031] Step 3.1, MTP pre-task, the pre-training task is to predict the randomly masked words. Randomly select a part of the words in a sentence and superimpose a uniformly distributed noise, as shown in the following formula (8):

[0032]

[0033] Among them, ξ j ∈[0,1] is the random number associated with the jth word, and α is the probability of masking;

[0034] There is a probability that positive noise is added to the superimposed noise to simulate the situation where the road condition improves due to human control and the traffic volume increases in a short period of time. The remaining probability is to superimpose a negative noise to simulate the situation where the road condition deteriorates due to road congestion and the traffic volume decreases in a short period of time.

[0035] The loss function is defined as the mean squared error (MSE) between the predicted result and the true result, and only the position of the masked word is involved in the calculation. The position of the masked word is defined as shown in the following formula (9):

[0036]

[0037] Among them, m(j-1)K:jK represents the (j-1)Kth to jKth continuous elements in the vector m (left open and right closed). The MTP loss function of the pre-task is designed as shown in the following formula (10):

[0038]

[0039] Among them, θ represents all trainable parameters of the model, is the element-wise multiplication operator;

[0040] Step 3.2, pre-training, the masked words are randomly selected, which makes the extracted features more abstract. By forcing the pre-trained model to use a large amount of unlabeled traffic data to solve the pre-task in a self-supervised learning manner, the model extracts the intrinsic correlation of traffic data to facilitate fine-tuning of any downstream tasks directly used for traffic data. The pre-task is that the minimum unit of random masking is a whole word containing a string of continuous points rather than a single point, so that the model pays more attention to the medium- and long-term features within the data;

[0041] Step 4: Fine-tune according to the downstream tasks. During the fine-tuning stage, replace the last fully connected layer with the corresponding output layer according to the objectives of different downstream tasks. Use a small amount of task-related and labeled data as input. Get a consistent representation of the traffic flow data in the pre-training stage and then get the output of the model under the task through different task-specific output layers.

[0042] The method of the present invention has the following beneficial effects:

[0043] 1. The token-based self-supervised network TSSN (Token-based Self-Supervised Network) proposed by the method of the present invention can efficiently use the unlabeled traffic flow data collected by road sensors at all times in the pre-training stage, and train the model through self-supervised learning. The data does not need to be centrally stored and labeled by professionals, which saves the time and professional knowledge required for data labeling, saves manpower and material resources, and can make full and efficient use of massive data, and obtain a generalized model;

[0044] 2. In the fine-tuning stage for different downstream tasks, the method of the present invention only needs to use a few task-related model layers to replace the final output layer of the pre-trained model. That is, when fine-tuning the task, the pre-trained model only outputs the learned representation of the traffic flow data, and only a small amount of task-related labeled data is needed for fine-tuning training. During the fine-tuning process, the parameters of the pre-trained layer are kept unchanged, and only the parameters of the subsequent task-related layers are updated for model training. In this way, there is no need to start model training from scratch, and since the required training samples are small and the model structure involved in parameter update is simple, a lot of time is saved compared with starting model training from scratch.

[0045] 3. The method of the present invention can enable the model to deeply mine the internal consistency characteristics of traffic flow data during the self-supervised pre-training process through the pre-training task of traffic flow data and can obtain superior performance after fine-tuning using a small amount of task-related labeled data in different downstream tasks. The same pre-trained model obtained for traffic flow data through the self-supervised learning method can be applied to many different ITS applications, requiring only a small amount of data fine-tuning and no need to train different models from scratch for different applications;

[0046] 4. The method of the present invention uses a word segmentation operation to make the minimum unit of the input model a word (Token) containing multiple continuous points, which can pay more attention to the medium- and long-term characteristics of the data and better mine the connections within the data. The semantic information between words is learned through the attention mechanism in the model, so that the model can obtain a more accurate representation of the traffic flow data;

[0047] 5. Aiming at the characteristics of traffic flow data and the problem that existing methods cannot extract length features therein, the method of the present invention proposes for the first time to segment traffic flow data and then input it into an efficient task-independent model, and learn the hidden layer representation of a series of continuous points through a self-supervisory method, so as to improve the performance in various ITS applications;

[0048] 6. The method of the present invention randomly masks a part of the words and enables the pre-trained model to infer the true value of the position based on the semantic information around the masked words, thereby forcing the model to fully explore the deep features of the traffic flow data;

[0049] 7. The method of the present invention is aimed at various applications of ITS. The hidden layer representation of the traffic flow data extracted through the pre-training process can be directly applied to various downstream tasks, such as: traffic flow data classification, traffic flow data prediction and traffic flow data completion. Good performance can be achieved by using a small amount of data for a small amount of training;

[0050] 8. The method described in the present invention is the first to apply word segmentation and self-supervised learning to traffic flow data processing. The word segmentation-based algorithm is more capable of mining the macro and long-term connections in traffic flow data than the point-based algorithm. At the same time, the self-supervised learning method breaks the boundaries between different tasks during model training. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a schematic diagram of the principle of the method of the present invention;

[0052] Figure 2 is a schematic diagram of the performance results of the vehicle flow classification task of the method described in the present invention;

[0053] Figure 3 is a schematic diagram of the performance results of the vehicle flow prediction task of the method of the present invention;

[0054] Figure 4 7-day prediction result display diagram when P=1 according to the method of the present invention;

[0055] Figure 5 7-day prediction result display diagram when P=6 according to the method of the present invention;

[0056] Figure 6 7-day prediction result display diagram when P=12 according to the method of the present invention;

[0057] Figure 7 7-day prediction result display diagram when P=36 according to the method of the present invention;

[0058] Figure 8 12-point vehicle flow completion results under different models of the method of the present invention are schematic diagrams;

[0059] Fig. 9 36-point vehicle flow completion results under different models of the method of the present invention are schematic diagrams;

[0060] Fig.10 It is a schematic diagram of the performance results of the vehicle flow completion task of the method described in the present invention. DETAILED DESCRIPTION

[0061] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0062] The method of the present invention comprises the following steps:

[0063] Step 1: Data processing and word segmentation. The data is traffic flow data collected by sensors deployed on the road. The data set is defined as X, and the length of each data is N. Define x i ∈X, i=1,..., N is the i-th point in a piece of data. The steps of data preprocessing and word segmentation are as follows:

[0064] Step 1.1, data preprocessing, normalizing the data, i.e., the traffic flow data, using the maximum-minimum normalization method to adjust all data to between [0, 1], as shown in the following formula (1):

[0065]

[0066] Step 1.2, data segmentation, the segmentation diagram is as follows Figure 3 As shown, each word is defined to contain K points, each piece of data contains M = N / K words, and the i-th word in each piece of data is defined as t j,j∈[1,M], as shown in the following formula (2):

[0067] t j :=x (j-1)K:jK ={x (j-1)K+1 ,x (j-1)K+2 ,...,x jK}, ... (2),

[0068] The original data is normalized and then segmented, that is, K consecutive points are treated as a unified whole. Each sentence has a total of M words as the original input into the model;

[0069] Step 2, model construction:

[0070] Step 2.1, Token Embedding. Set the input after tokenization in step 1 as X, with a dimension of M×K, that is, M words, each word contains K points. Multiply X by a weight matrix to obtain a matrix consisting of the embedding vector of each word and use it to map the input vector to a high-dimensional latent space to facilitate the training of subsequent models. The dimension of the high-dimensional latent space is defined as D, which is the same as the dimension of the subsequent Transformer encoder, as shown in the following formula (3):

[0071]

[0072] in, D is the word embedding dimension;

[0073] Step 2.2, Positional Encoding. The word embedding matrix should be added to the positional encoding to integrate the positional relationship information between words. The positional information here is at the word level rather than the single point level. The position of the input data is encoded by a set of sine and cosine functions of different frequencies. The positional encoding matrix is ​​given by the following formula (4):

[0074]

[0075] Among them, j∈[1,M],d∈[1,D], represents the element in the jth row and dth column of the position encoding matrix P. Finally, the position encoding matrix and the word embedding matrix are added, as shown in the following formula (5):

[0076]

[0077] Step 2.3, self-attention mechanism, the word embedding matrix will pass through the encoder of Y L-head attention transformer to obtain the representation vector of each word in the traffic flow data. Transformer directly adopts the existing model and defines the Transformer encoder as The matrix composed of the word representation vector is shown in the following formula (6):

[0078]

[0079] in, The dimension is M×D. Finally, the representation matrix passes through a fully connected layer and undergoes dimension transformation to obtain the final output, as shown in the following formula (7):

[0080]

[0081] in, F(N) means flattening the output into an N-dimensional vector. The final model output dimension is consistent with the initial input dimension, which is an N-dimensional vector.

[0082] Step 3: Pre-task design and pre-training loss function definition. Design the MTP pre-task to train the model described in step 2 to learn the intrinsic characteristics of the traffic flow data and obtain the data representation vector. The specific steps are as follows:

[0083] Step 3.1, MTP pre-task, the pre-training task is to predict the randomly masked words. Randomly select a part of the words in a sentence and superimpose a uniformly distributed noise, as shown in the following formula (8):

[0084]

[0085] Among them, ξ j ∈[0,1] is the random number associated with the jth word, and α is the probability of masking;

[0086] There is a probability that positive noise is added to the superimposed noise to simulate the situation where the road condition improves due to human control and the traffic volume increases in a short period of time. The remaining probability is to superimpose a negative noise to simulate the situation where the road condition deteriorates due to road congestion and the traffic volume decreases in a short period of time.

[0087] The loss function is defined as the mean squared error (MSE) between the predicted result and the true result, and only the position of the masked word is involved in the calculation. The position of the masked word is defined as shown in the following formula (9):

[0088]

[0089] Among them, m (j-1)K:jK represents the (j-1)Kth to jKth continuous elements in the vector m (left open and right closed). The MTP loss function of the pre-task is designed as shown in the following formula (10):

[0090]

[0091] Among them, θ represents all trainable parameters of the model, is the element-wise multiplication operator;

[0092] Step 3.2, pre-training, the masked words are randomly selected, which makes the extracted features more abstract. By forcing the pre-trained model to use a large amount of unlabeled traffic data to solve the pre-task in a self-supervised learning manner, the model extracts the intrinsic correlation of traffic data to facilitate fine-tuning of any downstream tasks directly used for traffic data. The pre-task is that the minimum unit of random masking is a whole word containing a string of continuous points rather than a single point, so that the model pays more attention to the medium- and long-term features within the data;

[0093] Step 4: Fine-tune according to the downstream tasks. During the fine-tuning stage, replace the last fully connected layer with the corresponding output layer according to the objectives of different downstream tasks. Use a small amount of task-related and labeled data as input. Get a consistent representation of the traffic flow data in the pre-training stage and then get the output of the model under the task through different task-specific output layers.

[0094] The schematic diagram of the method principle of the present invention is as follows Figure 1 As shown, the method of the present invention is divided into two stages: a pre-training stage and a fine-tuning stage. In the pre-training stage, TSSN uses a large amount of unlabeled traffic data and solves the previous task - Masked Token Prediction (MTP) to train a pre-training model that is unrelated to the downstream task, and obtains a representation vector for the traffic segmentation data; in the fine-tuning stage, TSSN uses a small amount of task-related labeled data on the basis of the representation vector to perform a small amount of training, that is, it can complete fine-tuning for different downstream tasks, that is, the pre-training model obtained through a large amount of unlabeled data and a large amount of training can be applied to various different traffic downstream tasks. While accelerating the training of downstream tasks, there is no need to train multiple different models from scratch for different application scenarios.

[0095] All simulation experiments of the method embodiments of the present invention are run on NVIDIA Tesla graphics processors (Graphics Processing Units, GPUs) equipped with CUDA 10.

[0096] In order to verify the performance of TSSN, the method embodiment of the present invention is designed to verify five different word lengths: K=1 (5min), K=2 (10min), K=6 (0.5h), K=12 (1h) and K=24 (2h). The embedding layer dimension and the hidden layer dimension of the Transformer encoder are both 64. Three Transformer encoding blocks are stacked. The multi-head attention layer of each block consists of 8 parallel point product attentions. The hidden layer representation output by the TSSN pre-training model is M*64 (M=N / K).

[0097] The dataset used for pre-training is the traffic flow data collected by the California Department of Transportation (California Traffic Management Bureau) from some highway probes across the state (Performance Measurement System, PeMS). The real-time collected data is aggregated into a point every 5 minutes, that is, a time series of 12 o'clock every hour and 288 points every day. The original collected data is discrete points. After smoothing and normalization, the dataset is divided into 80% for model training and 20% for performance verification. Then the word segmentation is performed, and 15% of the words are randomly masked. The superimposed noise is uniformly distributed in [0,0.5]. For each word length K, the model uses 1*10 -4 The training was performed for 50 rounds with a learning rate of , and validation was performed after each round of training.

[0098] In the fine-tuning stage, the initial learning rate is lowered to 2*10 because it is easier to converge the model based on the model parameters saved by pre-training. -5 , and verify the following three downstream tasks:

[0099] Traffic flow data on weekdays and weekends show different patterns. The dataset used in this task is traffic flow data collected in the Seattle area. The collection and preprocessing methods are the same as those used in pre-training, and the original data is labeled with corresponding classification labels based on the actual date.

[0100] The fine-tuning model inputs the hidden layer representation into two fully connected layers, and finally outputs the classification result through the Softmax layer, and calculates the cross-entropy loss function with the true category;

[0101] At the same time, two basic deep learning models that have advantages in processing time series data: Three-layer Fully Connected Network (FC) and Gated Recurrent Unit (GRU) are used for comparison, and the following indicators are statistically analyzed: classification accuracy, precision, recall rate, F1 index and Kappa coefficient.

[0102] The results of the classification task are as follows Figure 2 As shown in the figure: the performance is best when the word segmentation length K=2; the classification accuracy of TSSN with all word segmentation lengths K is higher than that of the two comparison models, indicating that TSSN can significantly improve the performance of classification tasks; the Kappa coefficient shows that TSSN is effective and reliable.

[0103] TSSN's prediction tasks are divided into short-term prediction and long-term prediction. Predicting 1 point, 6 points, and 12 points in the future (representing 5 minutes, half an hour, and an hour, respectively) is a short-term prediction; predicting 36 points and 72 points in the future (representing 3 hours and 6 hours, respectively) is a long-term prediction.

[0104] In order to accurately predict future data using known traffic flow data, two fully connected layers are connected after the TSSN representation layer, and the output dimension of the last layer is the same as the number of future points to be predicted. , The MSE between the predicted value and the true value is calculated.

[0105] The Bidirectional Long-Short Term Memory (Bi-LSTM) and Transformer models were used for comparison, and the mean absolute error (MAE), root mean square error (RMSE), mean absolute percentage error (MAPE) and R 2 coefficient.

[0106] For traffic flow prediction, P≤12(1h) is a short-term prediction, and P>12 is a long-term prediction. When MAPE is used as an evaluation indicator, MAPE=20% is regarded as the dividing line of whether the model is usable. The prediction results are as follows: Figure 3 shown.

[0107] When P=1, the performance of TSSN is inferior to Bi-LSTM under all word segmentation lengths. This is because Bi-LSTM can better capture the short-term characteristics of data, while TSSN focuses more on medium- and long-term characteristics. The Transformer model and K=1 TSSN without word segmentation are not suitable for single-point traffic flow prediction because of MAPE close to 20%. The prediction results are shown in Figure 2. Figure 4 As shown;

[0108] When P=6 and P=12, TSSN starts to perform better than Bi-LSTM in terms of MAPE, indicating that TSSN can indeed capture the representation between traffic flow words and extract richer internal information from traffic flow data. However, the results do not confirm that the prediction performance is better when the word segmentation length K is longer. The TSSN model after word segmentation performs better than TSSN (K=1) and the comparison model. The prediction results are shown in the figure below. Figure 5 and Figure 6 As shown;

[0109] When P=36, the MAPE of TSSN based on word segmentation is less than 20%, among which TSSN (K=12) has the best performance, while TSSN (K=1) has a MAPE of more than 20%, indicating the effectiveness of the pre-trained model based on word segmentation in long-term prediction. The other two comparison models are no longer suitable for this task. The prediction results are shown in Figure 2. Figure 7 shown.

[0110] The difficulty of 72-point prediction is significantly higher than that of 36-point prediction. The best performing model is TSSN (K=12), but its MAPE is slightly more than 20%. Except for TSSN (K=12) and TSSN (K=24), the performance of other models is too poor for the 72-point prediction task, especially the two comparison models.

[0111] The statistical trend of the evaluation indicators for all point predictions is as follows: Bi-LSTM has the best performance at the beginning (P=1), but the model performs poorly in subsequent long-term predictions; the Transformer model and TSSN (K=1) have poor performance in prediction tasks of all lengths (the only difference between the two is that TSSN (K=1) has been pre-trained), which shows that the transformer-type model structure is not fully applicable to the task of traffic flow prediction, and pre-training alone cannot improve its performance; TSSN models of other word segmentation lengths perform similarly and are better than Transformer and TSSN (K=1), which fully confirms that the word segmentation-based traffic flow processing method proposed by the method described in the present invention can effectively improve the performance of the transformer-type structure pre-training model.

[0112] In order to cope with the situation of road sensor failure, traffic flow completion uses known surrounding information to infer missing values. The positions of missing values ​​are random and continuous. The method embodiment of the present invention evaluates the data completion performance of two missing lengths, namely 12 points (1 hour) and 36 points (3 hours). Since traffic flow data also has different patterns and obvious characteristics in a day, three positions (uphill, peak and downhill) are randomly selected for performance verification in each missing length. The two-layer fully connected layer outputs the completed position value and calculates the MSE with the true value. Different from the pre-task during pre-training, the missing value is represented by a specific value instead of superimposing noise on the true value. The three-layer FC and Transformer models are used for comparison, and the MAE, RMSE, MAPE and R are statistically analyzed. 2 coefficient.

[0113] For the 12-point and 36-point completion, we defined three modes: upslope, stationary, and downslope according to the external form of traffic flow data for a consecutive day. We randomly selected one day from the entire test set to test the completion results. The results for 12-point and 36-point are as follows: Figure 8 and Fig. 9 As shown, all completion performance test results are as follows Fig.10 shown.

[0114] Among all the completion results, TSSN (K=24) has the best performance. The results show that TSSNs can improve the performance of the vehicle flow completion task. For the three modes, since the uphill and downhill modes are approximately linear without too much jitter, and the flat mode has some severe jitter, it is difficult for the model to extract the features, resulting in the completion results being inferior to the uphill and downhill modes.

[0115] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with the technical field within the scope disclosed in the present invention should be covered by the protection scope of the claims of the present invention.

Claims

1. A method for processing vehicle flow data based on data segmentation, characterized in that: The following steps are involved: Step 1: Data processing and word segmentation. The data is traffic flow data collected by sensors deployed on the highway. The data set is defined as The length of each data is recorded as N, and the definition is the i-th point in a piece of data; Step 2, model construction; Step 3, pre-task design and pre-training loss function definition, design the MTP pre-task to train the model described in step 2 to learn the intrinsic characteristics of the traffic flow data and obtain the representation vector of the data, MTP pre-task, the pre-training task is to predict the randomly masked words, the masked words are randomly selected, by forcing the pre-training model to use a large amount of unlabeled traffic flow data to solve the pre-task in a self-supervised learning manner, the pre-task is that the smallest unit of random masking is a whole word containing a string of continuous points rather than a single point; Step 4: Fine-tune according to the downstream tasks. During the fine-tuning stage, replace the last fully connected layer with the corresponding output layer according to the objectives of different downstream tasks. Use a small amount of task-related and labeled data as input. Get a consistent representation of the traffic flow data in the pre-training stage and then get the output of the model under the task through different task-specific output layers.

2. The vehicle flow data processing method based on data segmentation according to claim 1 is characterized in that: The step 1 comprises the following steps: Step 1.1, data preprocessing, normalizing the data, i.e., the traffic flow data, using the maximum-minimum normalization method to adjust all data to between [0, 1], as shown in the following formula (1): Step 1.2, data segmentation, define each word contains K points, each data contains M = N / K words, define the i-th word in each data as t j ,j∈[1,M], as shown in the following formula (2): t j :=x (j-1)K:jK ={x (j-1)K+1 ,x (j-1)K+2 ,...,x jK },……(2), The original data is normalized and then segmented, that is, K consecutive points are treated as a unified whole. Each sentence has a total of M words as the original input into the model.

3. The vehicle flow data processing method based on data segmentation according to claim 1 is characterized in that: The step 2 comprises the following steps: Step 2.1, word embedding. Set the input after word segmentation in step 1 as X, with a dimension of M×K, that is, M words, each word contains K points. Multiply X by a weight matrix to obtain a matrix consisting of the embedding vector of each word and use it to map the input vector to a high-dimensional latent space to facilitate the training of subsequent models. The dimension of the high-dimensional latent space is defined as D, which is the same as the dimension of the subsequent Transformer encoder, as shown in the following formula (3): in, D is the word embedding dimension; Step 2.2, position encoding, the word embedding matrix should be added to the position encoding to integrate the position relationship information between words. The position information here is at the word level rather than the single point level. The position of the input data is encoded by a set of sine and cosine functions of different frequencies. The position encoding matrix is ​​given by the following formula (4): Among them, j∈[1,M],d∈[1,D], represents the element in the jth row and dth column of the position encoding matrix P. Finally, the position encoding matrix and the word embedding matrix are added, as shown in the following formula (5): Step 2.3, self-attention mechanism, the word embedding matrix will pass through the encoder of Y L-head attention Transformer to obtain the representation vector of each word in the traffic flow data. Transformer directly adopts the existing model and defines the Transformer encoder as f(·). Then the matrix composed of the word representation vector is shown in the following formula (6): in, The dimension is M×D. Finally, the representation matrix passes through a fully connected layer and undergoes dimension transformation to obtain the final output, as shown in the following formula (7): in, F(N) means flattening the output into an N-dimensional vector. The final model output dimension is consistent with the initial input dimension, which is an N-dimensional vector.

4. The method for processing vehicle flow data based on data segmentation according to claim 1, characterized in that: The step 3 comprises the following steps: Step 3.1, MTP pre-task, the pre-training task is to predict the randomly masked words. Randomly select a part of the words in a sentence and superimpose a uniformly distributed noise, as shown in the following formula (8): Among them, ξ j ∈[0,1] is the random number associated with the jth word, and α is the probability of masking; There is a probability that positive noise is added to the words of superimposed noise to simulate the situation where the road condition improves due to human control and the traffic volume increases in a short period of time. The remaining probability is to add a negative noise to simulate the situation where the road condition deteriorates due to road congestion and the traffic volume decreases in a short period of time. The loss function is defined as the mean square error between the predicted result and the true result, and only the position of the masked word is involved in the calculation. The position of the masked word is defined as shown in the following formula (9): Among them, m (j-1)K:jK represents the (j-1)Kth to jKth continuous elements in the vector m. The MTP loss function of the pre-task is designed as shown in the following formula (10): Among them, θ represents all trainable parameters of the model, and ⊙ is the element-wise multiplication operator; Step 3.2, pre-training, the masked words are randomly selected, which makes the extracted features more abstract. By forcing the pre-trained model to use a large amount of unlabeled traffic data to solve the pre-task in a self-supervised learning manner, the model extracts the intrinsic correlation of the traffic data to facilitate fine-tuning of any downstream tasks directly used for traffic data. The pre-task is that the smallest unit of random masking is a whole word containing a string of continuous points rather than a single point so that the model can pay more attention to the medium- and long-term features within the data.

Images