Expressway traffic flow missing data complementation method based on space-time multi-scale matrix decomposition

Abstract

The invention discloses an expressway traffic flow missing data complementing method based on space-time multi-scale matrix factorization, which comprises the following steps: acquiring expressway multi-source heterogeneous traffic data, and constructing an observation matrix, wherein N represents the number of spatial positions, and T represents the number of time steps; decomposing the observation matrix into a product of a multi-scale space factor matrix and a time factor matrix by adopting Bayesian time sequence matrix decomposition; splitting the multi-scale space factor matrix into road section-level, region-level and network-level feature matrixes, and introducing dynamic evolution of a multi-order vector autoregression process modeling time factor matrix; performing model parameter learning through a Bayesian inference framework and a Gibbs sampling algorithm; and reconstructing an observation matrix by using W and X obtained by Bayesian inference, and outputting a complemented data matrix. According to the method, the problems of weak expression ability, high calculation complexity, limited missing value processing ability and the like of a traditional interpolation method are effectively solved, and the data completion precision is remarkably improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of traffic monitoring data processing, and particularly relates to a highway traffic flow missing data completion method based on space-time multi-scale matrix decomposition. BACKGROUND

[0002] With the development of urbanization and Internet of Things technology, the highway has multi-source heterogeneous massive traffic data, including dynamic data such as toll station card swiping data, gantry ETC data, monitoring camera data, and key vehicle GPS data, and static data such as highway network, entrance and exit, interflow, and stake segment. These data have the characteristics of high dimensionality, high frequency, but in actual application, due to factors such as device failure, communication interruption, and data packet loss, the original data often has a large number of missing values.

[0003] Therefore, in the prior art, there are technical problems of low precision in missing value completion of multi-source heterogeneous traffic data, poor processing efficiency, and inability to effectively capture space-time correlation. SUMMARY

[0004] In order to make up for the deficiencies of the prior art, the application aims to provide a highway traffic flow missing data completion method based on space-time multi-scale matrix decomposition, to solve the problems of low data completion precision and insufficient capture of space-time correlation in the prior art.

[0005] To solve the above problems, the application can be implemented through the following specific technical solutions:

[0006] The highway traffic flow missing data completion method based on space-time multi-scale matrix decomposition comprises the following steps:

[0007] Step 1, acquire multi-source heterogeneous traffic data of the highway, and construct an observation matrix Wherein, N represents the number of spatial positions, and T represents the number of time steps;

[0008] Step 2, use Bayesian time series matrix decomposition to approximate the observation matrix Y as the product of two low-rank matrices: Y≈W T X, wherein, is a spatial factor matrix, is a time factor matrix, and R is the number of latent factors;

[0009] Step 3, split the spatial factor matrix into road segment-level, regional-level, and network-level spatial feature matrices;

[0010] Step 4, introduce a multi-order vector autoregressive process to model the dynamic evolution of the time factor matrix;

[0011] Step 5, establish a Bayesian inference framework, and set a prior distribution for the parameters of the Bayesian inference model;

[0012] Step 6, Bayesian inference model inference and parameter learning are performed by using Gibbs sampling algorithm;

[0013] Step 7, the observed matrix is reconstructed by using the inferred W and X to fill in the missing data and output the completed data matrix.

[0014] Further, in the step 3, the observed matrix Y is expressed as:

[0015] Y≈[W local ,W regional ,W global ] T X,

[0016] wherein, is a road section level spatial feature, is a regional level spatial feature, is a network level spatial feature, is a time factor matrix, R is a latent factor number, R=R1+R2+R3;

[0017] The observed elements are subject to Gaussian distribution:

[0018] Further, in the step 4, the time factor satisfies:

[0019]

[0020] wherein, x t+1 is traffic flow data (flow, speed, etc.) at t+1, A k is a VAR coefficient matrix, h k is a lag step number, ∈ t is Gaussian white noise.

[0021] The spatial factor W is set to be a multivariate Gaussian prior:

[0022]

[0023] wherein, μ local , μ regional , μ global are prior mean matrices of the spatial factors at different levels, are prior covariance matrices of the spatial factors at different levels, Ψ local , Ψ regional , Ψ global are inter-row covariance matrices of the spatial factors at different levels.

[0024] Further, in step 5, the Bayesian inference framework includes a multivariate Gaussian prior with spatial factors, a matrix normal-inverse Wishart joint prior for VAR parameters, and a Gamma prior for observation error precision;

[0025] VAR parameter prior: (A k ,B k ) ~ MN(M k ,V k ,S k ),∑ ε ~ IW(v,Ψ);

[0026] Observation error precision prior: τ i ~ Gamma(a τ ,b τ );

[0027] where MN denotes matrix normal distribution, IW denotes inverse Wishart distribution, M k , V k , S k are prior hyperparameters for VAR parameters, v, Ψ are degree of freedom and scale matrix parameters for inverse Wishart distribution, a τ , b τ are shape and scale parameters for Gamma distribution.

[0028] Further, in step 6, the Gibbs sampling algorithm is used to update the spatial factor W, the temporal factor X, the VAR parameters and the error precision τ in turn.

[0029] Further, for each scale of spatial factor, the conditional posterior distribution is:

[0030] p(W scale |X,τ,Y) ∝ p(Y|W scale ,X,τ)·p(W scale ),

[0031] where, The update is obtained by solving normal equations; W scale is the spatial factor matrix of a specific scale, τ is the observation precision parameter vector, is the updated spatial factor prior mean matrix, is the updated spatial factor prior covariance matrix, is the updated spatial factor inter-row covariance matrix.

[0032] Further, the temporal factor update uses the conditional posterior distribution, the formula is:

[0033] p(X|W,A,∑ εp(Y|W,X,τ)∝p(Y|W,X,τ)·p(X|A,∑ ε ),

[0034] Efficiently compute by Kalman filtering and smoothing algorithm:

[0035]

[0036] where ∑ ε is the noise covariance matrix of VAR model, is the updated time factor prior mean matrix, is the updated time factor prior covariance matrix, is the updated time factor inter-row covariance matrix.

[0037] Further, the VAR parameter is updated as follows:

[0038] The conditional posterior of coefficient matrix:

[0039]

[0040] The conditional posterior of covariance matrix:

[0041]

[0042] where, is the updated VAR coefficient matrix prior mean, is the updated VAR coefficient matrix prior row covariance, is the updated VAR coefficient matrix prior column covariance, ∑ ε is the noise covariance matrix of VAR model, is the updated inverse Wishart distribution degree of freedom parameter, is the updated inverse Wishart distribution scale matrix parameter.

[0043] Further, the precision parameter τ is updated as follows:

[0044]

[0045] where τ i is the observation precision parameter of the i-th spatial position, is the updated Gamma distribution shape parameter, is the updated Gamma distribution scale parameter.

[0046] Compared with the prior art, the present application has the following advantages: the method of the present application processes multi-dimensional traffic data by modeling existing sensor data, increases the dimension of the traffic data, changes the traffic data matrix into the product of two low-rank matrices, splits the time factor matrix W into a road section level local space feature matrix, a regional level space feature matrix and a network level global space feature matrix, introduces a multi-order vector autoregressive process to model the dynamic evolution of the time factor matrix X, uses the W and X obtained by Bayesian inference to reconstruct the observation matrix and fill in the missing data, the expression method retains the original observation data, reflects the spatio-temporal correlation between data and the relationship between data indicators, completes the missing data of the highway traffic flow, enhances the reliability of the data and provides effective data support for data prediction. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 The flowchart of the missing data completion method of the present application. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0049] This embodiment takes highway traffic flow data as an example, and the data is obtained from a plurality of sensor networks with a sampling interval of 5 minutes and a total duration of 30 days.

[0050] As shown in Figure 1 , the missing data completion method for highway traffic flow based on spatio-temporal multi-scale matrix decomposition comprises the following steps:

[0051] (1) Step 1, obtaining highway multi-source heterogeneous traffic data and constructing an observation matrix wherein N represents the number of spatial positions and T represents the number of time steps.

[0052] Obtain highway traffic data, including toll station, gantry ETC, monitoring camera, GPS and other sensor data; organize the data into an observation matrix wherein N is the number of spatial positions and T is the number of time steps; define the set of observed elements For example, 214 sensors collect traffic data at 8784 time points, and the data format is

[0053] (2) Step 2, approximate the observation matrix Y as the product of two low-rank matrices using Bayesian time series matrix decomposition: Y ≈ W T X, wherein is a spatial factor matrix, is the time factor matrix, and R is the latent factor number.

[0054] In a specific embodiment, let the latent factor number be 10, the spatial factor matrix is decomposed into the road segment-level spatial feature matrix VAR coefficient matrix is decomposed into the road segment-level spatial feature matrix T X.

[0055] (Three) Step 3, decompose the spatial factor matrix into the road segment-level spatial feature matrix, the region-level spatial feature matrix, and the network-level spatial feature matrix, and set Gaussian priors for the decomposed matrices respectively.

[0056] Specifically, the observation matrix Y is expressed as the product of the multi-scale spatial factor matrix and the time factor matrix:

[0057] Y ≈ [W local ,W global ,W global ] T X,

[0058] wherein, is the road segment-level spatial feature, is the region-level spatial feature, is the network-level spatial feature, is the time factor matrix, and R is the latent factor number, R = R1+R2+R3.

[0059] Therefore, the observation elements follow the Gaussian distribution:

[0060] In a specific embodiment, the spatial factor matrix is decomposed into the road segment-level spatial feature matrix the region-level spatial feature matrix the network-level spatial feature matrix wherein R1=3, R2=3, R3=4, and R = R1+R2+R3=10.

[0061] (Four) Step 4, introduce the multivariate vector autoregressive (VAR) process to model the dynamic evolution of the time factor matrix X, and set the multivariate Gaussian prior for the spatial factor W.

[0062] The multivariate vector autoregressive process models the dynamic dependence relationship between different time factors through the VAR coefficient matrix A k and the lag number h k , effectively capturing the complex structure of time series data.

[0063] The time factor satisfies:

[0064]

[0065] where x t+1 is the traffic flow data (flow, speed, etc.) at time t+1, A k is the VAR coefficient matrix, h k is the lag number, ∈ t is the Gaussian white noise.

[0066] The spatial factor W is set with a multivariate Gaussian prior:

[0067]

[0068] where μ local , μ regional , μ global are the prior mean matrices of the spatial factors at different levels, Ψ local , Ψ regional , Ψ global are the prior covariance matrices of the spatial factors at different levels, and Ψ k is the inter-row covariance matrix of the spatial factors at different levels.

[0069] (5) Step 5, establish a Bayesian inference framework to set a prior distribution for the Bayesian inference model parameters.

[0070] The Bayesian inference framework includes a multivariate Gaussian prior for the spatial factor, a matrix normal-inverse Wishart joint prior for the VAR parameters, and a Gamma prior for the observation error precision.

[0071] VAR parameter prior: (A k , B k ) ~ MN(M k , V k , S k ), ∑ ε ~ IW(v, Ψ);

[0072] Observation error precision prior: τ i ~ Gamma(a τ , b τ );

[0073] where MN represents the matrix normal distribution, IW represents the inverse Wishart distribution, M k , V k , S k are the prior hyperparameters of the VAR parameters, v and Ψ are the degree of freedom and scale matrix parameters of the inverse Wishart distribution, and a τ , b τ are the shape and scale parameters of the Gamma distribution.

[0074] (6) Step 6, Gibbs sampling algorithm is used for Bayesian inference model inference and parameter learning.

[0075] The spatial factor W, the temporal factor X, the VAR parameters (A, S) and the error precision τ are updated in turn.

[0076] (1) Spatial factor update: for each scale of the spatial factor, the conditional posterior distribution is:

[0077] p(W scale |X,τ,Y)∝p(Y|W scale scale ,

[0078] where, The update is obtained by solving the normal equation; W scale is the spatial factor matrix of a specific scale, τ is the observation precision parameter vector, is the updated spatial factor prior mean matrix, is the updated spatial factor prior covariance matrix, is the updated spatial factor inter-row covariance matrix.

[0079] (2) Temporal factor update: the conditional posterior distribution is:

[0080] p(X|W,A,Σ ε ,τ,Y)∝p(Y|W,X,τ)·p(X|A,∑ ε ),

[0081] The update is efficiently calculated by Kalman filtering and smoothing algorithm:

[0082]

[0083] where, ∑ ε is the VAR model noise covariance matrix, is the updated temporal factor prior mean matrix, is the updated temporal factor prior covariance matrix, is the updated temporal factor inter-row covariance matrix.

[0084] (3) VAR parameter update:

[0085] The conditional posterior of the coefficient matrix:

[0086]

[0087] The conditional posterior of the covariance matrix:

[0088]

[0089] where,​ is the prior mean of the updated VAR coefficient matrix, is the prior row covariance of the updated VAR coefficient matrix, is the prior column covariance of the updated VAR coefficient matrix, ε is the noise covariance matrix of the VAR model, is the updated degree of freedom parameter of the inverse Wishart distribution, is the updated scale matrix parameter of the inverse Wishart distribution.

[0090] (4) Precision parameter update:

[0091]

[0092] where τ i is the observation precision parameter of the i-th spatial location, is the updated shape parameter of the Gamma distribution, is the updated scale parameter of the Gamma distribution.

[0093] In one embodiment, Gibbs sampling is used with 1000 iterations, the first 500 iterations are used as burn-in, and the last 500 iterations are used for parameter estimation. The results are saved every 10 iterations.

[0094] (7) Step 7, reconstruct the observation matrix using the inferred W and X, fill in the missing data, and output the completed data matrix.

[0095] Update and by Bayesian inference local , W regional , W global are concatenated into the spatial factor matrix Reconstruct the observation matrix using and , and then compare the missing data part to output the completed complete data matrix

[0096] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for completing missing traffic flow data on highways based on spatiotemporal multi-scale matrix decomposition, characterized in that, Includes the following steps: Step 1: Acquire multi-source heterogeneous traffic data of highways and construct an observation matrix. Where N represents the number of spatial locations and T represents the number of time steps; Step 2: Approximate the observation matrix Y as the product of two low-rank matrices using Bayesian time series matrix decomposition: Y≈W T X, where, For the space factor matrix, Let R be the time factor matrix, and R be the number of latent factors. Step 3: Decompose the spatial factor matrix into segment-level, region-level, and network-level spatial feature matrices; Step 4: Introduce a multi-order vector autoregression process to model the dynamic evolution of the time factor matrix; Step 5: Establish a Bayesian inference framework and set the prior distribution for the parameters of the Bayesian inference model; Step 6: Use the Gibbs sampling algorithm to perform Bayesian inference model inference and parameter learning; Step 7: Reconstruct the observation matrix using the inferred W and X, fill in the missing data, and output the completed data matrix.

2. The method for completing missing highway traffic flow data based on spatiotemporal multi-scale matrix decomposition according to claim 1, characterized in that, In step 3, the observation matrix Y is represented as: Y≈[W local ,W regional ,W global ] T X, in, For road segment-level spatial features, As a regional spatial feature, For network-level spatial features, Let R be the time factor matrix, where R is the number of potential factors, and R = R1 + R2 + R3; The observed elements follow a Gaussian distribution:

3. The method for completing missing highway traffic flow data based on spatiotemporal multi-scale matrix decomposition according to claim 1, characterized in that, In step 4, the time factor satisfies: Where, x t+1 For traffic flow data at time t+1, A k Let h be the VAR coefficient matrix. k The number of lag steps, ∈ t It is Gaussian white noise; Spatial factor W sets the multivariate Gaussian prior: Where, μ local μ regional μ global Let be the prior mean matrix of each level of spatial factors. Let Ψ be the prior covariance matrix of each spatial factor. local Ψ regional Ψ global Let be the inter-row covariance matrix of each level of spatial factors.

4. The method for completing missing highway traffic flow data based on spatiotemporal multi-scale matrix decomposition according to claim 1, characterized in that, In step 5, the Bayesian inference framework includes the multivariate Gaussian prior of the spatial factor, the matrix normal-inverse Wishart joint prior of the VAR parameters, and the Gamma prior of the observation error accuracy. VAR parameter priors: (A k B k )~MN(M k V k ,S k ),∑ ε ~IW(ν,Ψ); Prior knowledge of observation error accuracy: τ i ~Gamma(a τ ,b τ ); Where MN represents the matrix normal distribution, IW represents the inverse Wishart distribution, and M k V k S k Let ν and Ψ be the prior hyperparameters of the VAR parameters, and let a be the degrees of freedom and scaling matrix parameters of the inverse Wishart distribution. τ b τ For the shape and scale parameters of the Gamma distribution.

5. The method for completing missing highway traffic flow data based on spatiotemporal multi-scale matrix decomposition according to claim 1, characterized in that, In step 6, the Gibbs sampling algorithm is used to update the spatial factor W, time factor X, VAR parameters and error precision τ sequentially, and the inference efficiency is improved by using a sliding window method.

6. The method for completing missing highway traffic flow data based on spatiotemporal multi-scale matrix decomposition according to claim 5, characterized in that, For each spatial factor at each scale, the conditional posterior distribution is: p(W scale |X,τ,Y)∝p(Y|W scale ,X,τ)·p(W scale ), in, The update is obtained by solving the system of normal equations; W scale Let τ be the spatial factor matrix at a specific scale, and τ be the observation accuracy parameter vector. This is the updated spatial factor prior mean matrix. This is the updated spatial factor prior covariance matrix. This is the updated spatial factor inter-row covariance matrix.

7. The method for completing missing highway traffic flow data based on spatiotemporal multi-scale matrix decomposition according to claim 5, characterized in that, The time factor update uses a conditional posterior distribution, and the formula is: p(X|W,A,Σ ε ,τ,Y)∝p(Y|W,X,τ)·p(X|A,∑ ε ), Efficient computation using Kalman filtering and smoothing algorithms: Where, ∑ ε The noise covariance matrix of the VAR model. This is the updated prior mean matrix of the time factors. This is the updated time factor prior covariance matrix. This is the updated time factor inter-row covariance matrix.

8. The method for completing missing highway traffic flow data based on spatiotemporal multi-scale matrix decomposition according to claim 5, characterized in that, The VAR parameters have been updated as follows: Conditional posterior of the coefficient matrix: Conditional posterior of the covariance matrix: in, The prior mean of the updated VAR coefficient matrix. The prior row covariance of the updated VAR coefficient matrix, For the prior column covariance of the updated VAR coefficient matrix, ∑ ε The noise covariance matrix of the VAR model. The updated inverse Wishart distribution degrees of freedom parameters. These are the parameters of the updated inverse Wishart distribution scaling matrix.

9. The method for completing missing highway traffic flow data based on spatiotemporal multi-scale matrix decomposition according to claim 5, characterized in that, The accuracy parameter τ is updated as follows: Where, τ i Let be the observation accuracy parameter for the i-th spatial location. For the updated Gamma distribution shape parameters, This is the updated scale parameter for the Gamma distribution.

Images