A method for processing periodic traffic data based on an adaptive aggregation and prediction framework of bidirectional GRU

Abstract

The present invention discloses a method for processing periodic traffic data based on an adaptive aggregation and prediction framework of a bidirectional GRU, including the following operations: according to the correlation characteristics of periodic data, a data set containing different data feature patterns is obtained, the data set is divided into different time periods using an ordered sample clustering method, and a segmented data set is obtained, and according to the characteristics of the segmented data set, different time periods are assigned as peak areas, fluctuation areas, low peak areas, and stable areas, respectively, to form an adaptive aggregation data set. The adaptive aggregation data set is substituted into an adaptive prediction framework, and after data expansion based on local predictions and data splicing based on global predictions, a neural network framework based on a bidirectional GRU is constructed, and traffic prediction is performed on the processed data to achieve aggregation and prediction of periodic traffic data. The present invention improves data processing efficiency and prediction accuracy, and has extremely high application value.

Description

Technical Field

[0001] The present invention relates to the technical field of big data analysis and mining, and in particular to a method for processing periodic traffic data based on a bidirectional GRU-based adaptive aggregation and prediction framework. Background Art

[0002] As one of the key issues in data processing, the study of periodic traffic data is of great value. Accurate, timely and detailed periodic traffic forecasting plays a vital role in traffic flow planning, wind power generation and photovoltaic power generation forecasting and planning of power grids, etc. Although unexpected factors such as emergencies and weather lead to uncertainty and randomness, periodic traffic data still has some periodic, spatial and temporal characteristics that are hidden by noise. In addition, with the advancement of statistical techniques and data analysis techniques, the availability of a large amount of periodic traffic data has created opportunities for short-term predictions based on historical data.

[0003] A large number of methods have been introduced in the prior art to improve the accuracy and time efficiency of predictions, but most of the research focuses on developing prediction models rather than data preprocessing. Fixed time aggregation intervals are usually used for predictions, without considering the characteristics of traffic data. For example, although the most commonly used time intervals are 5 minutes and 15 minutes, other time intervals (such as 2 minutes, 3 minutes or longer time intervals such as 30 minutes, 60 minutes) have been selected in different studies to process raw data for prediction, but data predictions with longer time intervals are limited in their ability to reflect the characteristics of dynamic periodic data, while if the time interval is too short and the amount of data is large, the calculation takes a long time and the results are often unstable. Therefore, although there are many lengths of time aggregation intervals, and the most suitable one with the highest prediction accuracy can be found, there is no fixed time interval that is suitable for all situations.

[0004] In view of this, it makes sense to adaptively select time aggregation intervals of different lengths according to the characteristics of periodic traffic data, which has an important impact on the accuracy and efficiency of the prediction. Summary of the invention

[0005] The purpose of the present invention is to provide a method for processing periodic traffic data based on an adaptive aggregation and prediction framework of a bidirectional GRU, so as to solve the problems that there is no fixed time aggregation interval suitable for all situations in the existing periodic traffic data preprocessing stage, and that different time data require different aggregation time differentiation standards to be established. The algorithm framework of the present invention is designed for general periodic traffic data, and can adaptively select the optimal aggregation interval according to the different characteristics of the data. Compared with the fixed time aggregation, the data processed by the present invention highlights the peak period in the adaptive periodic traffic data sequence, and the amount of adaptive periodic traffic data is reduced by about 33% on average, while maintaining the prediction accuracy of the original algorithm. The algorithm framework of the present invention plays a vital role in the processing of periodic data such as wind power generation, photovoltaic power generation and traffic flow in power systems, and improves the utilization rate of periodic traffic data.

[0006] In order to achieve the above object, the present invention is implemented by the following technical solutions:

[0007] A method for processing periodic traffic data based on a bidirectional GRU adaptive aggregation and prediction framework includes the following operations:

[0008] Obtain periodic traffic data within a preset number of days, calculate the Pearson correlation coefficient of different days to analyze the similarity of periodic traffic data, classify the periodic traffic data into different data patterns, extract periodic traffic data with the same data pattern to form a v-dimensional periodic traffic data set;

[0009] The daily periodic flow data set is divided into different time periods by an ordered sample clustering method; each of the different time periods is divided into a peak area, a fluctuation area, a low peak area and a stable area, and the corresponding aggregation time is obtained by cross-validation mean square error to generate a processed adaptive aggregation data set;

[0010] An adaptive prediction framework is constructed to process the adaptive aggregated data set, perform local prediction and global prediction, and output a final prediction result.

[0011] Further, as a preferred embodiment of the present invention, the step of obtaining the periodic flow data within a preset number of days, calculating the Pearson correlation coefficient of different days to analyze the similarity of the periodic flow data, classifying the periodic flow data into different data modes, and extracting the periodic flow data with the same data mode to form a v-dimensional periodic flow data set includes the following operations:

[0012] Periodic flow data is collected every m min within the preset number of days to form a time series {x1, x2, ..., x p}, where x prepresents the pth periodic flow data, where p is the total amount of periodic flow data;

[0013] For any number of days q, Represents the periodic flow data at the starting time point of the qth day, defined as represents all periodic flow data on the qth day, where v represents the number of time points collected each day, then v = 1440 / m;

[0014] Calculate the Pearson correlation coefficient of the periodic traffic data of any two days, define the periodic traffic data corresponding to the days when the Pearson correlation coefficient exceeds the preset threshold to have the same data pattern, and extract the periodic traffic data of n days with the same data pattern to form a new data set:

[0015]

[0016] in, represents all periodic traffic data on day 1 of n days with the same data pattern, represents all periodic traffic data on the second day of n days with the same data pattern, Represents all periodic traffic data on the nth day of n days with the same data pattern;

[0017] Extract the periodic flow data at the same time point every day in the new data set, and for each time point d, calculate the mean of the periodic flow data for n days in, Represents the periodic traffic data at time point d on the eth day of n days, and combines the means of all time points into a v-dimensional daily periodic traffic data set X a .

[0018] Further, as a preferred embodiment of the present invention, the step of dividing the daily periodic flow data set into different time periods by using an ordered sample clustering method specifically includes:

[0019] For a daily periodic traffic data set X a And classification collection Calculate the center point of each cluster And diameter Among them, a, i and j are the indexes of time points;

[0020] Calculate the Loss function L[b(v, k)] for a specific segmentation method b(v, k):

[0021]

[0022] In the formula, i c 、ic+1 They represent the cth and c+1th segmentation points respectively, k is the number of time periods into which the periodic flow data of a day is divided, that is, the number of segmentation points, and D is the diameter; the segmentation points {i1,…,i k}In a daily periodic traffic data set X a The corresponding data in is recorded as The negative correlation curve between the Loss function L[b(v, k)] and the number of segmentation points k is obtained by the ordered sample clustering method, and the second-order derivative is calculated. The minimum integer segment number k0 is obtained by the elbow method, and the second-order derivative corresponding to the minimum integer segment number k0 is less than the given threshold m0;

[0023] Substitute the minimum integer segment number k0 into the ordered sample clustering method and output the optimal segmentation method Make in is the segmented set, I j ,I j+1 is the time point, and the segmentation point index is recorded as The data in the daily periodic flow data set corresponding to each segment point index is recorded as Index the segment points Adjust to the nearest hour or half hour.

[0024] Further, as a preferred embodiment of the present invention, each of the different time periods is divided into a peak area, a fluctuation area, a low peak area and a stable area, and a processed adaptive aggregation data set is generated, including the following operations:

[0025] Calculate any set after segmentation and the data point X in the daily periodic flow data set p The distance between Introducing K-means clustering to set Divide into M clusters K i , calculate the corresponding sum of squares error objective function SSE:

[0026]

[0027] In the formula, c i is the cluster K i the center of

[0028] Set state set and clustering flag set Among them, 1, 2, 3, and 4 correspond to the peak area, fluctuation area, low peak area, and stable area respectively. As a clustering mark, the time period is divided into four time zones;

[0029] According to the cross-validation mean square error results, the aggregation time of periodic traffic data in the peak area is set to the best aggregation time obtained by the model, and the aggregation time intervals of the other three time zones are set longer than the peak area value to highlight the peak period;

[0030] The aggregation interval of each time series data is determined according to the time period classification, thereby generating a processed adaptive aggregation data set.

[0031] Further, as a preferred embodiment of the present invention, the dividing of the time period into four time zones includes the following operations: calculating the cluster K i Average flow rate in the medium period Clustering the minimum mean The corresponding flag function It is recorded as number 3, which is the low peak area;

[0032] For the three clusters K that have not yet been classified into time zones i , calculate and compare the three clustering K i The average periodic flow rate With the periodic flow average indicator If with clustering The corresponding periodic flow average Greater than the average periodic flow index Then cluster Corresponding It is recorded as number 1, which is the peak area;

[0033] Calculate the standard deviation of the remaining two undivided clusters The standard deviation Large clusters Corresponding sign It is recorded as number 2, which is the fluctuation area;

[0034] The last cluster Corresponding sign It is recorded as number 4, which means the stable area.

[0035] Further, as a preferred embodiment of the present invention, the step of constructing an adaptive prediction framework to process the adaptive aggregated data set, perform local prediction and global prediction, and output a final prediction result includes the following operations:

[0036] Perform local prediction on the preprocessed adaptive aggregate dataset, extract data from four time periods of the adaptive aggregate dataset and expand them to form four new datasets;

[0037] The data expansion GRU algorithm is used to expand the ascending or descending sequence into the current time series to supplement the information lost in the time division. In the time series data, the downstream data is ignored and only the upstream data needs to be extracted; the extended part is the ascending or descending sequence. In the process of splicing the new sequence, each data sequence and the corresponding extended sequence are extracted and spliced ​​into a new independent data sequence, specifically:

[0038] make:

[0039]

[0040] Where D t-1 is the time series at time t-1, is the data at position r in the sequence, is the data at position f in the sequence;

[0041] From D t-1 Extract the ascending or descending subsequence M from t-1 , subsequence M t-1 The expression is:

[0042]

[0043] In the formula, the subsequence M t-1 satisfy or in is the data at position k1 in the sequence, is the data at position k2 in the sequence, is the position k in the sequence n Data at

[0044] The extracted subsequence M t-1 Expand to the current time series D t In the tth moment, a new sequence N is formed t , the expression is:

[0045] N t =[M t _1,D t ];

[0046] Combined with the results of local prediction, the data is predicted globally. The method of dynamically adjusting the historical mean is adopted to adaptively adjust the historical mean data. The indicator F(t) is introduced to represent the fluctuation of periodic traffic data:

[0047]

[0048] In the formula, y(ti) is the periodic flow data at time ti, y(ti-1) is the periodic flow data at time ti-1, and F(t) is the average of the first-order differences of the first four periodic flow data;

[0049] The historical mean H is introduced to reflect the characteristics of historical data. The maximum and minimum values ​​of the historical mean H(t+1) at time t+1 are:

[0050] H(t+1)m ax =y(t)+F(t)

[0051] H(t+1) min =y(t)-F(t)

[0052] Where y(t) is the periodic flow data at time t;

[0053] The domain of the constructed mean is M = [H(t+1) min ,H(t+1) max ], through the mean domain, determine whether the historical mean H(t+1) at time t+1 is within the domain M, and change the historical mean H(t+1) to:

[0054] H(t+1), if H(t+1)∈M

[0055] H(t+1) min ,if H(t+1) <H(t+1) min

[0056] H(t+1) max , if H(t+1)>H(t+1) max

[0057] When y(t+1)<10, the periodic traffic data at this moment is small, and there is no need to adjust the historical mean H(t+1). The adjusted H(t+1) is concatenated with the original periodic traffic data X input to form a new input, and the dimension w of the periodic traffic data is concatenated with the dynamic historical mean H i Concatenate to form a new vector of dimension w+1, and add the vector of dimension w+1 to the time segment feature Z i Splice into a (w+1)*s matrix, where Z i It is a (w+1)*(s-1) matrix, where s is the length of the time series;

[0058] The concatenated (w+1)*s matrix is ​​used as input, and data adaptive bidirectional GRU is used to predict data flow.

[0059] The concatenated (w+1)*s matrix is ​​used as input, and data adaptive bidirectional GRU is used to perform data flow prediction, which includes the following operations:

[0060] During the encoding process, the hidden state h = [h1...h t ], during the decoding process, for each time t, the decoder input and the hidden state output of the previous time t-1 are used to calculate the hidden state s at the current time t t , through s t With each encoding process h t The inner product of the attention score e is obtained t ;

[0061] Use Softmax to convert the attention score e t Convert to discrete probability distribution, and convert hidden state into weighted sum c according to the probability distribution t , the output Pr at the corresponding time is obtained through the decoder t , the calculation process is as follows:

[0062]

[0063] In the formula, (Softmax(e t ))l is the Softmax conversion e t The lth element of h l is the hidden state of the encoder at time step l, m is the total number of hidden states, is the weight matrix of the decoder output layer, is the bias vector of the decoder output layer; Pr t It is the prediction result of periodic data.

[0064] The beneficial effects of the present invention are:

[0065] 1. The algorithm framework of the present invention is designed for general periodic traffic data and can adaptively select the optimal aggregation interval according to different characteristics of the data.

[0066] 2. Compared with the fixed time aggregation, the data processed by the method of the present invention highlights the peak period in the adaptive periodic traffic data sequence, and the amount of adaptive periodic traffic data is reduced by about 33% on average, while maintaining the prediction accuracy of the original algorithm.

[0067] 3. The algorithm framework of the present invention plays a vital role in the processing of periodic data such as wind power generation, photovoltaic power generation and traffic flow in power systems, and improves the utilization rate of periodic flow data. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0069] in:

[0070] Figure 1 It is a schematic diagram of the algorithm framework and symbol assumptions of the adaptive periodic traffic data aggregation algorithm in an embodiment of the present invention;

[0071] Figure 2 Schematic diagram of Pearson correlation coefficient values ​​of periodic flow data within one month in an embodiment of the present invention;

[0072] Figure 3 , 4 Schematic diagram of Pearson correlation coefficient values ​​of periodic flow data within a week in an embodiment of the present invention;

[0073] Figure 5 Schematic diagram of the Loss value function that changes negatively with the number of segments in an embodiment of the present invention;

[0074] Figure 6 Schematic diagram of an optimal time segmentation method in an embodiment of the present invention;

[0075] Figure 7 A schematic diagram of time zone division corresponding to time periods in an embodiment of the present invention;

[0076] Figure 8 Schematic diagram of the original periodic flow data sequence in the embodiment of the present invention;

[0077] Fig. 9 Schematic diagram of an adaptive periodic flow data sequence corresponding to the processing in an embodiment of the present invention;

[0078] Fig.10 Schematic diagram of the local prediction algorithm framework in an embodiment of the present invention;

[0079] Fig.11 This is an example diagram of a sequence splicing method for peak period data in a global prediction algorithm in an embodiment of the present invention;

[0080] Fig.12 The data splicing method and algorithm flow chart of the global prediction algorithm in the embodiment of the present invention;

[0081] Fig.13 Schematic diagram of a data-adaptive bidirectional GRU framework in an embodiment of the present invention. DETAILED DESCRIPTION

[0082] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. The present invention applies the periodic flow data time adaptive aggregation and prediction framework to the traffic flow data set. First, the relevant data used in this case are introduced:

[0083] The data experiment mainly uses the dataset HwMine obtained from the Regional Transportation Management Center (RTMC) under the Minnesota Department of Transportation (MnDOT). The original dataset is collected every 30 seconds from more than 400 circular detectors and Wavetronix radar sensors installed on the Twin Cities Metro Freeway and around Rochester, 24 / 7. RTMC traffic data includes flow, occupancy, and speed. This case uses volume data from 3 detectors (ID.3, 6908, 6909) in 2016 and 2021.

[0084] The adaptive aggregation framework for traffic flow data aims to select smaller time aggregation intervals for traffic flow data during peak hours and select larger time aggregations to improve the prediction accuracy during peak hours while reducing the amount of data during rest periods. Figure 1 As shown, the following steps are included:

[0085] S1, obtain the periodic traffic data within a preset number of days, calculate the Pearson correlation coefficient of different days to analyze the similarity of the periodic traffic data, classify the periodic traffic data into different data modes, extract the periodic traffic data with the same data mode to form a v-dimensional daily periodic traffic data, including the following operations:

[0086] Define {x1, x2, …, x p} is the original traffic flow data collected every m min. For a particular day q, Indicates the time of zero o'clock on that day. Represents the data of the day, where v = 1440 / m;

[0087] Calculate the Pearson correlation coefficient of the periodic traffic data of any two days. Days with high PCC values ​​can be regarded as having the same data pattern; extract n days with the same data pattern separately to form a new data set:

[0088]

[0089] Extract the data at the same time point every day in the collection and calculate the mean Thus, a v-dimensional daily traffic flow data set is obtained.

[0090] S2, for periodic traffic data with the same data pattern, calculate the Loss function, output the optimal segmentation method according to the ordered sample clustering method, and divide the data into several time periods, including the following operations:

[0091] For data X a And classification collection Where a is the data sequence number of the periodic traffic data, i and j are days, and v is the time. Calculate the center point of each set And diameter Among them, a, i and j are the indexes of time points;

[0092] Calculate the Loss function of a specific segmentation method b(v, k) where i c 、i c+1 They represent the cth and c+1th segmentation points respectively, k is the number of time periods into which the periodic flow data of a day is divided, that is, the number of segmentation points, D is the diameter, b(v, k) is the specific segmentation method, and L is the Loss function; the second-order difference of the Loss function relative to the number of segments is used as the criterion, the new data set is divided by the ordered sample clustering method, and the minimum integer segmentation number k0 is obtained by using the elbow rule, which satisfies that the corresponding second-order derivative is less than the given threshold m0=0.1;

[0093] Substitute the minimum integer segment number k0 and traffic flow data into the ordered sample clustering method to obtain the segmentation points of the data set At the same time, subscript the segment points Make adjustments by whole or half points to make the segment points more reasonable.

[0094] S3, divide each time period into peak area, fluctuation area, low peak area and stable area, obtain the corresponding aggregation time through the cross-validated mean square error, and generate the processed adaptive aggregation data set, including the following operations:

[0095] Calculate any set after segmentation and periodic flow data points X p The distance between Introducing K-means clustering to set Divide into M clusters K i Calculate the corresponding squared error objective function c i K i The center of cluster K i The number of is any set after segmentation, K i For clustering, Any set after segmentation With periodic flow data point X p The distance between

[0096] Set state set and clustering flag set Among them, 1, 2, 3, and 4 correspond to the peak area, fluctuation area, low peak area, and stable area respectively. For clustering marks, the time period is divided into four time zones by following the steps below:

[0097] Calculate cluster K i Average flow rate in the medium period Clustering the minimum mean The corresponding flag function As the number 3, that is, the low peak area;

[0098] For the three clusters K that have not yet been classified into time zones i , calculate and compare the three clustering K i The average periodic flow rate The average value of periodic flow Indicator, if with clustering Corresponding Greater than Cluster Corresponding as number 1, which is the peak area;

[0099] Calculate the standard deviation of the remaining two undivided clusters will have greater Clustering Corresponding sign As the number 2, that is, the fluctuation zone;

[0100] The last cluster Corresponding sign As the number 4, that is, the stable zone;

[0101] According to the cross-validation mean square error results, the aggregation time of the data in the peak area is set to the best aggregation time obtained by the model, and the aggregation time intervals of the other three time zones are set longer than the peak area value to highlight the peak period;

[0102] The specific aggregation interval of each time series data is determined according to the time period classification, thereby generating the processed adaptive traffic flow dataset D.

[0103] S4, after generating the adaptive aggregated data set, constructing an adaptive prediction framework to process the adaptive aggregated data set, performing local prediction and global prediction, and outputting the final prediction result, including the following operations:

[0104] Perform local prediction on the preprocessed data. Local prediction is mainly divided into four steps, such as Fig.10 As shown. From the data set Extract data from four time periods in the dataset and expand them to form four new datasets;

[0105] In order to supplement the information lost in the time division, part of the rising or falling sequence is extended to the current time series. In the time series data, the downstream data can be ignored naturally, and only the upstream information needs to be extracted. The extended part should satisfy the rising or falling sequence to better reflect the upstream change trend. The data of each period is extended, such as the splicing process of the new sequence of the peak period. Fig.11 As shown. represents the time series corresponding to time t-1, where is the data at position r in the sequence, is the data at position f in the sequence, from D t-1 Extract M t-1 , where M t-1 The specific definition is as follows:

[0106]

[0107] So that the sequence is descending or rising Get the extended sequence M of period t t-1 Finally, let N t =[M t-1 , D t ] represents the tth new sequence;

[0108] Combine the results of local prediction to make global prediction of the data. The flowchart of global prediction is as follows: Fig.12 First, the value of the indicator F(t) is calculated to represent the fluctuation of traffic flow:

[0109]

[0110] Where y(t) is the traffic data at time t. F(t) is also regarded as the average of the first-order differences of the previous four traffic flow cycles. The larger F(t) is, the larger the current traffic flow range is. The indicator F(t) is then used to calculate the historical mean. The maximum and minimum values ​​of the historical mean H(t+1) are defined as:

[0111] H(t+1) max =y(t)+F(t),

[0112] H(t+1) min =y(t)-F(t).

[0113] Based on this, the domain of the corresponding constructed mean is M = [H(t+1) min ,H(t+1) max ]. The purpose of constructing the mean domain is to make the historical mean more meaningful, to determine whether the historical mean at time t+1 is within the domain M, and then to change the historical mean H(t+1) to:

[0114] H(t+1), if H(t+1)∈M,

[0115] H(t+1) min ,if H(t+1) <H(t+1) min ,

[0116] H(t+1) max , if H(t+1)>H(t+1) max .

[0117] When y(t+1)<10, the periodic traffic data at this moment is small, and there is no need to adjust the historical mean H(t+1). The adjusted H(t+1) is concatenated with the original periodic traffic data X input to form a new input, and the dimension w of the periodic traffic data is concatenated with the dynamic historical mean H i Concatenate to form a new vector of dimension w+1, and add the vector of dimension w+1 to the time segment feature Z i Splice into a (w+1)*s matrix, where Z i It is a (w+1)*(s-1) matrix, where s is the length of the time series;

[0118] The concatenated (w+1)*s matrix is ​​used as input, and the data adaptive bidirectional GRU is used to predict the data flow. The overall structure is shown in the figure below: Fig.13 As shown, it specifically includes the following detailed steps:

[0119] During the encoding process, the hidden state h = [h1...h t ], during the decoding process, for each time t, the decoder input and the hidden state output of the previous time t-1 are used to calculate the hidden state s at the current time t t , through s t With each encoding process h t The inner product of the attention score e is obtained t .

[0120] Get the attention score t Finally, Softmax is used to convert the attention score into a discrete probability distribution. According to the probability distribution, the hidden state is converted into a weighted sum c t Finally, the decoder obtains the output Pr of the corresponding time. t The calculation process is as follows:

[0121]

[0122] Where (Softmax(e t ))l is the Softmax conversion e t The lth element of h l is the hidden state of the encoder at time step, m is the total number of hidden states, is the weight matrix of the decoder output layer, is the bias vector of the decoder output layer; Pr t It is the prediction result of periodic data.

[0123] In order to verify the method, this test example simulates the aggregation process of the traffic flow dataset HwMine through Python simulation. In order to verify the effectiveness of the method of the present invention in applying to traffic flow data, this test aggregates the data from July 9, 2016 to August 7, 2016 into 10 minutes for similarity analysis.

[0124] Figure 2 The PCC for 30 days is shown, indicating some results of the similarity analysis. It is not difficult to see that the PCC of traffic flow on different weekdays and the PCC on weekends are both around 0.9. However, the PCC between weekdays and weekends is lower than 0.75. Therefore, traffic flow on weekends and weekdays is considered to follow two different patterns. In order to focus on traffic flow data on ordinary weekdays with the same regularity, the data on weekends and special weekdays in 2016 and 2021 are excluded from the dataset of this experiment.

[0125] Specifically, the PCC of the data from July 4, 2016 to July 17, 2016 is as follows: Figure 3 , 4 As shown. In fact, July 9, 10, 16, and 17 are weekends, and the remaining 10 days are weekdays. The PCC value on weekdays is above 0.75, and on weekends it is close to 1. It is not difficult to see that the PCC values ​​on weekdays and weekends are relatively small. Figure 3 , 4 It can also be seen that the traffic flow data on July 4th is quite different from the data on other working days, indicating that the traffic flow on July 4th presents a new pattern. It can be concluded that holidays, accidents, detector failures and other special factors can be detected through similarity analysis. Therefore, the average value of traffic data with the same pattern is used to obtain one-day traffic flow data, which is then put into the adaptive traffic flow sequence processing framework.

[0126] The samples obtained by 10-minute aggregation are divided into time periods, each day containing 144 data items, and then the new set is divided using the ordered sample clustering method. Figure 5The loss of the number of segments is shown, and the loss decreases as the number of segments increases. As described in the previous technical solution, the second-order difference of Loss relative to the number is used as the measurement standard, and the threshold m0 is taken as 0.1. Therefore, after selecting the segment as the number of segments in the data set, the number of road sections and traffic data are substituted into the ordered sample clustering method to obtain the road section points of the data set.

[0127] In addition, to make the segmentation points more reasonable, the segmentation points need to be adjusted by whole points or half points. Figure 6 An example of the resulting time segmentation is shown. The segmentation is divided into four time zones with different aggregation time intervals, which are called peak zone, fluctuation zone, stable zone and low peak zone. After the steps introduced in the previous technical solution, the 3rd and 5th segments are divided into peak periods, the 2nd and 6th segments are regarded as fluctuation periods, the 1st segment is a stable period, and the 4th segment is a fluctuation period. Figure 5 The time period of each segment is shown, and their corresponding flag set is F = {3, 2, 1, 4, 1, 2}.

[0128] According to the cross-validation mean square error, the optimal aggregation interval during peak hours is determined to be 5 minutes, and the intervals of 10 minutes, 15 minutes, and 20 minutes are applied to the flow data during the fluctuation period, the flat period, and the stable period, respectively. Then the original data is aggregated to different aggregation times and compared with the original data aggregated for 10 minutes to obtain the adaptive traffic flow sequence, such as Figure 8 , 9 Table 1 shows the specific time periods and corresponding aggregation times of the obtained adaptive traffic flow sequences.

[0129] Table 1 Adaptive periodic traffic data corresponding time period and timetable

[0130]

[0131] Generally, the framework is designed for general traffic flow data and can adaptively select the optimal aggregation interval according to different characteristics of the data.

[0132] Taking a normal working day as an example, the amount of data in the adaptive traffic flow sequence is reduced by 1 / 3 compared with the amount of data aggregated in 5 minutes. In order to compare the aggregation effect of the adaptive traffic flow data processing framework mentioned in the technical solution compared with the fixed 5-minute aggregation, Table 2 shows the detailed volume reduction in four periods in this experiment. It can be seen from the table that the volume in the low peak area, peak area, fluctuation area, and stable area is reduced by 81.7%, 0%, 50%, and 66.7%, respectively, with significant effects. In addition, the proposed adaptive traffic flow sequence processing framework can be generally applied to other traffic flow data.

[0133] Table 2 Compared with the fixed aggregation time of 5 minutes, the reduction of data volume in four different time zones by the adaptive periodic traffic data aggregation algorithm

[0134]

[0135] Finally, the original GRU algorithm is applied to predict the adaptive data and the 5-minute aggregated data. Fig. 9 It is not difficult to see from the table that the accuracy of the two data sets is almost the same, while the amount of adaptive data is much smaller than the amount of data aggregated in 5 minutes.

[0136] In summary, the time-adaptive aggregation and prediction framework of periodic traffic data proposed by the method of the present invention solves the problems of large data calculation amount, lack of real-time performance, weak model adaptability, etc. in short-term data prediction. Compared with fixed-time aggregation, the periodic data sequence after adaptive processing highlights the peak period and reduces the amount of data, but the prediction accuracy remains basically unchanged, as shown in Table 3. At the same time, the prediction accuracy of the GRU based on the data processed by the adaptive framework in each local period is higher than that of the original GRU. It can be concluded that among the existing global prediction algorithms, the GRU algorithm based on the data processed by the adaptive framework performs the best, which brings inspiration for the future research on a lighter periodic traffic data aggregation framework.

[0137] Table 3 Comparison of the predicted MAPE values ​​of the data set generated by the 5-minute fixed aggregation time and the data set generated by the adaptive periodic traffic data aggregation algorithm

[0138]

[0139] It should be noted that the above content only illustrates the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. For ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications all fall within the protection scope of the claims of the present invention.

Claims

1. A method for processing periodic traffic data based on a bidirectional GRU adaptive aggregation and prediction framework, characterized by: The following operations are included: Obtain periodic traffic data within a preset number of days, calculate the Pearson correlation coefficient of different days to analyze the similarity of periodic traffic data, classify periodic traffic data into different data patterns, extract periodic traffic data with the same data pattern to form a v-dimensional daily periodic traffic data set; The daily periodic flow data set is divided into different time periods by an ordered sample clustering method; each of the different time periods is divided into a peak area, a fluctuation area, a low peak area and a stable area, and the corresponding aggregation time is obtained by cross-validation mean square error to generate a processed adaptive aggregation data set; Constructing an adaptive prediction framework to process the adaptive aggregated data set, perform local prediction and global prediction, and output a final prediction result; The step of constructing an adaptive prediction framework to process the adaptive aggregated data set, perform local prediction and global prediction, and output a final prediction result includes the following operations: Perform local prediction on the preprocessed adaptive aggregate dataset, extract data from four time periods of the adaptive aggregate dataset and expand them to form four new datasets; The data expansion GRU algorithm is used to expand the ascending or descending sequence into the current time series to supplement the information lost in the time division. In the time series data, the downstream data is ignored and only the upstream data needs to be extracted; the extended part is the ascending or descending sequence. In the process of splicing the new sequence, each data sequence and the corresponding extended sequence are extracted and spliced ​​into a new independent data sequence, specifically: make: Where D t-1 is the time series at time t-1, is the data at position r in the sequence, is the data at position f in the sequence; From D t-1 Extract the ascending or descending subsequence M from t-1 , subsequence M t-1 The expression is: In the formula, the subsequence M t-1 satisfy or in is the data at position k1 in the sequence, is the data at position k2 in the sequence, is the position k in the sequence n The data at; extract the subsequence M t-1 Expand to the current time series D t In the tth moment, a new sequence N is formed t , the expression is: N t =[M t-1 ,D t ]; Combined with the results of local prediction, the data is predicted globally. The method of dynamically adjusting the historical mean is adopted to adaptively adjust the historical mean data. The indicator F(t) is introduced to represent the fluctuation of periodic traffic data: In the formula, y(ti) is the periodic flow data at time ti, y(ti-1) is the periodic flow data at time ti-1, and F(t) is the average of the first-order differences of the first four periodic flow data; The historical mean H is introduced to reflect the characteristics of historical data. The maximum and minimum values ​​of the historical mean H(t+1) at time t+1 are: H(t+1) max =y(t)+F(t) H(t+1) min =y(t)-F(t) Where y(t) is the periodic flow data at time t; The domain of the constructed mean is M = [H(t+1) min ,H(t+1) max ], through the mean domain, determine whether the historical mean H(t+1) at time t+1 is within the domain M, and change the historical mean H(t+1) to: H(t+1), if H(t+1)∈M H(t+1) min ,if H(t+1)<H(t+1) min H(t+1) max ,if H(t+1)>H(t+1) max When y(t+1)<10, the periodic traffic data at this moment is small, and there is no need to adjust the historical mean H(t+1). The adjusted H(t+1) is concatenated with the original periodic traffic data X input to form a new input, and the dimension w of the periodic traffic data is concatenated with the dynamic historical mean H i Concatenate to form a new vector of dimension w+1, and add the vector of dimension w+1 to the time segment feature Z i Splice into a (w+1)*s matrix, where Z i It is a (w+1)*(s-1) matrix, where s is the length of the time series; The concatenated (w+1)*s matrix is ​​used as input, and data adaptive bidirectional GRU is used to predict data flow.

2. According to claim 1, a method for processing periodic traffic data based on a bidirectional GRU adaptive aggregation and prediction framework is characterized by: The step of obtaining periodic flow data within a preset number of days, calculating the Pearson correlation coefficient of different days to analyze the similarity of periodic flow data, classifying the periodic flow data into different data modes, and extracting periodic flow data with the same data mode to form a v-dimensional periodic flow data set for one day includes the following operations: Periodic flow data is collected every m min within the preset number of days to form a time series {x1, x2, ..., x p }, where x p represents the pth periodic flow data, where p is the total amount of periodic flow data; For any number of days q, Represents the periodic flow data at the starting time point of the qth day, defined as represents all periodic flow data of the qth day, where v represents the number of time points collected every day, then v = 1440 / m; calculate the Pearson correlation coefficient of the periodic flow data of any two days, define the periodic flow data corresponding to the days whose Pearson correlation coefficient exceeds the preset threshold to have the same data pattern, and extract the periodic flow data of n days with the same data pattern to form a new data set: in, represents all periodic traffic data on day 1 of n days with the same data pattern, represents all periodic traffic data on the second day of n days with the same data pattern, Represents all periodic traffic data on the nth day of n days with the same data pattern; Extract the periodic flow data at the same time point every day in the new data set, and for each time point d, calculate the mean of the periodic flow data for n days in, Represents the periodic traffic data at time point d on the eth day of n days, and combines the means of all time points into a v-dimensional daily periodic traffic data set X a .

3. The method for processing periodic traffic data based on a bidirectional GRU adaptive aggregation and prediction framework according to claim 1, characterized in that: The step of dividing the daily periodic flow data set into different time periods by using the ordered sample clustering method specifically includes: For a daily periodic traffic data set X a And classification collection Calculate the center point of each cluster And diameter Among them, a, i and j are the indexes of time points; Calculate the Loss function L[b(v, k)] for a specific segmentation method b(v, k): In the formula, i c 、i c+1 They represent the cth and c+1th segmentation points respectively, k is the number of time periods into which the periodic flow data of a day is divided, that is, the number of segmentation points, and D is the diameter; the segmentation points {i1,…,i k }In a daily periodic traffic data set X a The corresponding data in is recorded as The negative correlation curve between the Loss function L[b(v, k)] and the number of segmentation points k is obtained by the ordered sample clustering method, and the second-order derivative is calculated. The minimum integer segment number k0 is obtained by the elbow method, and the second-order derivative corresponding to the minimum integer segment number k0 is less than the given threshold m0; Substitute the minimum integer segment number k0 into the ordered sample clustering method and output the optimal segmentation method Make in is the segmented set, I j ,I j+1 is the time point, and the segmentation point index is recorded as The data in the daily periodic flow data set corresponding to each segment point index is recorded as Index the segment points Adjust to the nearest hour or half hour.

4. The method for processing periodic traffic data based on a bidirectional GRU adaptive aggregation and prediction framework according to claim 1, characterized in that: Dividing each of the different time periods into a peak area, a fluctuation area, a low peak area, and a stable area, and generating a processed adaptive aggregation data set includes the following operations: Calculate any set after segmentation and the data point X in the daily periodic flow data set p The distance between Introducing K-means clustering to set Divide into M clusters K i , calculate the corresponding sum of squares error objective function SSE: In the formula, c i is the cluster K i the center of; Defining State Sets Among them, 1, 2, 3, and 4 correspond to the peak area, fluctuation area, low peak area, and stable area, respectively, as well as the cluster mark set in Represents cluster K i The corresponding time zones, thus dividing the time period into four time zones; According to the cross-validation mean square error results, the aggregation time of periodic traffic data in the peak area is set to the best aggregation time obtained by the model, and the aggregation time intervals of the other three time zones are set longer than the peak area value to highlight the peak period; The aggregation interval of each time series data is determined according to the time period classification, thereby generating a processed adaptive aggregation data set.

5. The method for processing periodic traffic data based on a bidirectional GRU adaptive aggregation and prediction framework according to claim 4 is characterized in that: The dividing of the time period into four time zones includes the following operations: Calculate cluster K i Average flow rate during the medium period Clustering the minimum mean The corresponding flag function It is recorded as number 3, which is the low peak area; For the three clusters K that have not yet been classified into time zones i , calculate and compare the three clustering K i The average periodic flow rate With the periodic flow average indicator If with clustering The corresponding periodic flow average Greater than the average periodic flow index Then cluster Corresponding It is recorded as number 1, which is the peak area; Calculate the standard deviation of the remaining two undivided clusters The standard deviation Large clusters Corresponding sign It is recorded as number 2, which is the fluctuation area; The last cluster Corresponding sign It is recorded as number 4, which means the stable area.

6. The method for processing periodic traffic data based on a bidirectional GRU adaptive aggregation and prediction framework according to claim 1, characterized in that: The concatenated (w+1)*s matrix is ​​used as input, and data adaptive bidirectional GRU is used to perform data flow prediction, which includes the following operations: During the encoding process, the hidden state h = [h1...h t ], during the decoding process, for each time t, the decoder input and the hidden state output of the previous time t-1 are used to calculate the hidden state s at the current time t t , through s t With each encoding process h t The inner product of the attention score e is obtained t ; Use Softmax to convert the attention score e t Convert to discrete probability distribution, and convert hidden state into weighted sum c according to the probability distribution t , the output Pr at the corresponding time is obtained through the decoder t , the calculation process is as follows: In the formula, (Softmax(e t ))l is the Softmax conversion e t The lth element of h l is the hidden state of the encoder at time step, m is the total number of hidden states, is the weight matrix of the decoder output layer, is the bias vector of the decoder output layer; Pr t It is the prediction result of periodic data.

Images