Traffic data interpolation method based on third-order Tucker decomposition

Abstract

The invention discloses a traffic data interpolation method based on third-order Tucker decomposition, and the method comprises the steps: firstly constructing a three-dimensional traffic flow tensor according to the features of original traffic spatio-temporal data, so as to comprehensively represent the distribution conditions of traffic flows in space and time dimensions; and then, by constructing a third-order Tucker decomposition model, the third-order kernel tensor is decomposed into three third-order cubic tensors, so that the calculation complexity is effectively reduced, and the spatial-temporal characteristics in the traffic data are accurately extracted. On this basis, an optimization objective function based on third-order Tucker decomposition is established, an efficient solving algorithm is designed based on a least square method, and an approximate tensor is obtained to realize interpolation of a traffic flow missing value. According to the method, the accuracy of traffic flow data interpolation is improved on the whole, the interpolation performance is still excellent under the condition that the data missing rate is high, and the method has good application and popularization values.

Description

Technical Field

[0001] This invention belongs to the field of intelligent transportation, and in particular relates to a traffic data interpolation method based on third-order Tucker decomposition. Background Technology

[0002] With the rapid development of communication technology, artificial intelligence, and data science, a large number of intelligent data acquisition devices have been applied in recent years to capture the spatiotemporal traffic conditions of transportation networks. Various spatiotemporal traffic data form the foundation of intelligent transportation systems research; however, due to limitations in real-world acquisition equipment and recording technologies, the problem of missing traffic data is unavoidable. This missing traffic data not only seriously affects the intelligent development of transportation systems but also hinders practical applications such as traffic state estimation and traffic signal control. Therefore, achieving completeness of traffic network characteristic data is of great significance for urban traffic state prediction, traffic system management, and traffic strategy optimization.

[0003] Traffic spatiotemporal data can be constructed as two-dimensional matrices or multi-dimensional tensors, and missing values ​​in the traffic tensor can be estimated through tensor decomposition. Tensor decomposition methods integrate the temporal and spatial dimensions of traffic data into a multi-order tensor and leverage the low-rank characteristic of traffic data to extract potential patterns of multilinear factors from partially valid data, thereby achieving imputation of missing traffic values. Currently, common tensor decomposition methods are CP decomposition and Tucker decomposition, which can be seen as higher-order generalizations of singular value decomposition. However, current classic tensor decomposition algorithms still have some problems: on the one hand, the accuracy of CP decomposition and Tucker decomposition in traffic data imputation needs improvement, and their computational efficiency is relatively low for large datasets; on the other hand, related research focuses on traffic data imputation for single detectors, ignoring the spatial distribution characteristics of regional detectors. Furthermore, most current traffic data imputation problems focus on scenarios with low missing data rates, and the performance of decomposition models needs further improvement for scenarios with large amounts of missing data. Summary of the Invention

[0004] The purpose of this invention is to reduce the error caused by missing traffic data in urban traffic state estimation, traffic signal control and other analyses, and to provide a traffic data interpolation method based on third-order Tucker decomposition.

[0005] To achieve the above objectives, the present invention employs a traffic data interpolation method based on third-order Tucker decomposition, comprising the following steps:

[0006] Step 1: Preprocess the raw traffic data to obtain a road traffic flow dataset;

[0007] Step 2: Construct a 3D traffic feature tensor based on the road traffic flow dataset;

[0008] Step 3: Construct a third-order Tucker decomposition model;

[0009] Step 4: Determine the optimization objective function based on the third-order Tucker decomposition;

[0010] Step 5: Solve the objective function using the third-order Tucker decomposition optimization algorithm based on the least squares method;

[0011] Step 6: Verify the performance of the interpolation model.

[0012] Further, step 1 preprocesses the raw traffic data, including:

[0013] Obtain raw traffic flow data from multiple nodes in the road traffic network, remove outliers, and standardize the data.

[0014] Data from each node is randomly deleted at a predetermined missing data rate D, and the true values ​​of all missing node data are recorded as validation values ​​for the data imputation model.

[0015] In this process, a random number is generated for each data node, and the random numbers are sorted from smallest to largest. The data of the nodes in the top D groups are then deleted.

[0016] Further, step 2 constructs a three-dimensional traffic feature tensor, including:

[0017] In the road traffic flow dataset obtained from the preprocessing in step 1, aggregation analysis is performed at k-minute intervals to construct a three-dimensional traffic feature tensor.

[0018] Where n1 is the number of days the dataset covers, n2 is the number of traffic flows effectively monitored each day, calculated by n2 = 1440 / k, and n3 is the number of traffic flow detectors on the road segment.

[0019] Further, step 3 constructs a third-order Tucker decomposition model, including:

[0020] Based on the 3D traffic feature tensor constructed in step 2, the Tucker decomposition framework is used to process the feature tensor. Decompose and Decomposed into kernel tensors The form of multiplication with the outer product of the factor matrix:

[0021]

[0022] in, Let r1, r2, and r3 represent the row factor matrix, column factor matrix, and layer factor matrix, respectively, and let r1, r2, and r3 represent the three dimensions of the kernel tensor, namely the row dimension, column dimension, and layer dimension.n n = 1, 2, 3 represents the modal product between the tensor and the matrix; based on the Tucker decomposition framework, the original kernel tensor... Decomposed into third-order cubic tensors The triple product form, decomposed into the following form:

[0023]

[0024] Where r is the rank corresponding to the third-order Tucker decomposition, and ^ represents the estimated tensor value.

[0025] Furthermore, the rank r corresponding to the third-order Tucker decomposition in step 3 is no greater than the original kernel tensor. The rank of the original Tucker decomposition is no greater than the median value of its rank.

[0026] r≤mid{n1,n2,n3} (3)

[0027] Here, mid{n1,n2,n3} represents the second largest value among the three values.

[0028] Furthermore, step 4, based on the Tucker decomposition framework, optimizes the objective function as follows:

[0029]

[0030] in, Representing the Frobenius norm; substituting the third-order Tucker decomposition model constructed in step 3 into this objective function, we obtain:

[0031]

[0032] in, This represents the optimization objective function based on the third-order Tucker decomposition.

[0033] Further, step 5 solves the objective function using a third-order Tucker decomposition optimization algorithm based on the least squares method, including:

[0034] Based on the least squares algorithm structure, the parameters in the optimization objective function constructed in step 4 are respectively... U, V, and W are solved iteratively. After i iterations, the optimal parameters of the third-order Tucker decomposition are obtained, and the interpolated traffic flow tensor is obtained.

[0035] The optimization process for each parameter of the optimization algorithm is as follows:

[0036]

[0037]

[0038] Where λ and γ represent the parameters to be calibrated, γ∈[0,1]; i represents the number of iterations in the algorithm. U i V i and W i Let represent the estimated values ​​obtained from the i-th iteration of the calculation of the three-dimensional traffic feature tensor, three third-order cubic tensors, row factor matrix, column factor matrix, and layer factor matrix, respectively. U i+1 V i +1 and W i+1 These represent the estimated values ​​obtained from the (i+1)th iteration of the calculation of the three-dimensional traffic feature tensor, three third-order cubic tensors, row factor matrix, column factor matrix, and layer factor matrix, respectively. and These represent the approximate estimates of the three-dimensional traffic feature tensor, three third-order cubic tensors, row factor matrix, column factor matrix, and layer factor matrix calculated using the error function in the i-th iteration.

[0039] Further, step 6 verifies the performance of the interpolation model, including:

[0040] By substituting the traffic flow data of road segments in the traffic network into the third-order Tucker decomposition model in step 3, and continuously iterating through the optimization algorithm in step 5, the kernel tensors and factor matrices are updated.

[0041] After each update, the tensor is reconstructed, and the missing values ​​of the original tensor in the final traffic flow tensor are used as imputation values. The experimental verification values ​​set in step 1 are compared to verify the performance of the imputation model.

[0042] Further, in step 6, the error between the tensor constructed after each iteration and the original tensor is calculated using the objective function. When the error is less than the convergence value or the maximum number of iterations is reached, the update and iteration are stopped. At this time, the tensor constructed is the final traffic flow tensor.

[0043] The root mean square error (RMSE) is used as a metric to evaluate training performance, and is calculated as follows:

[0044]

[0045] Where, x m This represents the data validation value, x′ m M represents the estimated value of the data. a Indicates the total number of data.

[0046] Beneficial effects: Compared with the prior art, the technical solution of the present invention has the following beneficial technical effects:

[0047] This invention proposes a novel traffic data interpolation method. By constructing a three-dimensional spatiotemporal traffic tensor, it fully integrates information from historical data and spatially correlated data, extracting the distribution patterns and trends of relevant spatiotemporal data information, thereby improving the accuracy of traffic data interpolation. The kernel tensor is decomposed into three third-order factor tensors using a third-order Tucker decomposition algorithm, effectively reducing computational complexity and improving algorithm efficiency. Furthermore, this invention can achieve traffic data completion under various data missing rates. Compared with state-of-the-art methods, this invention effectively improves the accuracy of road traffic flow data interpolation, especially exhibiting higher interpolation precision under higher data missing rates. Attached Figure Description

[0048] Figure 1 This is a flowchart of the traffic data interpolation method based on third-order Tucker decomposition provided by the present invention.

[0049] Figure 2 This is a schematic diagram of the third-order Tucker decomposition model structure involved in the present invention. Detailed Implementation

[0050] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.

[0051] This invention proposes a traffic data interpolation method based on third-order Tucker decomposition, the process of which is as follows: Figure 1 As shown, it includes the following steps:

[0052] Step 1: Preprocess the raw traffic data to obtain a road traffic flow dataset.

[0053] The process involves acquiring raw traffic flow data from multiple nodes in a road network, removing outliers, and standardizing the data. Data from each node is then randomly deleted at a predetermined missing data rate D. The true values ​​of all missing node data are recorded as validation values ​​for the data imputation model.

[0054] In this step, a random number is generated for each data node, and the random numbers are sorted from smallest to largest. The data of the nodes in the first D groups are then deleted.

[0055] Step 2: Construct a three-dimensional traffic feature tensor based on the road traffic flow dataset.

[0056] In the road traffic flow dataset obtained from the preprocessing in step 1, aggregation analysis is performed at k-minute intervals to construct a three-dimensional traffic feature tensor. Where n1 is the number of days the dataset covers, n2 is the number of traffic flows effectively monitored each day, calculated by n2 = 1440 / k, and n3 is the number of traffic flow detectors on the road segment.

[0057] Step 3: Construct a third-order Tucker decomposition model.

[0058] Based on the 3D traffic feature tensor constructed in step 2, the Tucker decomposition framework is used to process the feature tensor. Decompose and Decomposed into kernel tensors The form of multiplication with the outer product of the factor matrix:

[0059]

[0060] in, Let r1, r2, and r3 represent the row factor matrix, column factor matrix, and layer factor matrix, respectively, and let r1, r2, and r3 represent the three dimensions of the kernel tensor, namely the row dimension, column dimension, and layer dimension. n n = 1, 2, 3 represents the modal product between the tensor and the matrix; based on the Tucker decomposition framework, the original kernel tensor... Decomposed into third-order cubic tensors The triple product form, decomposed structure as follows Figure 2 As shown, the decomposition form is:

[0061]

[0062] Where r is the rank corresponding to the third-order Tucker decomposition, and ^ represents the estimated tensor value.

[0063] In this step, the rank r corresponding to the third-order Tucker decomposition should not be greater than the original kernel tensor. The rank of the original Tucker decomposition is no greater than the median value of its rank.

[0064] r≤mid{n1,n2,n3} (3)

[0065] Here, mid{n1,n2,n3} represents the second largest value among the three values.

[0066] Step 4: Determine the optimization objective function based on the third-order Tucker decomposition.

[0067] Based on the Tucker decomposition framework, the objective function to be optimized is:

[0068]

[0069] in, Let Frobenius norm be represented. Substituting the third-order Tucker decomposition model constructed in step 3 into this objective function, we obtain the optimization objective function based on the third-order Tucker decomposition.

[0070]

[0071] Step 5: Solve the objective function using the third-order Tucker decomposition optimization algorithm based on the least squares method.

[0072] Based on the least squares algorithm structure, the parameters in the optimization objective function constructed in step 4 are respectively... The parameters U, V, and W are solved iteratively. After i iterations, the optimal parameters of the third-order Tucker decomposition are obtained, and the interpolated traffic flow tensor is obtained.

[0073] The specific process for optimizing the parameters of the optimization algorithm is as follows:

[0074]

[0075] Where λ and γ represent the parameters to be calibrated, γ∈[0,1]; i represents the number of iterations in the algorithm. U i V i and W i Let represent the estimated values ​​obtained from the i-th iteration of the calculation of the three-dimensional traffic feature tensor, three third-order cubic tensors, row factor matrix, column factor matrix, and layer factor matrix, respectively. U i+1 V i+1 W i+1 These represent the estimated values ​​obtained from the (i+1)th iteration of the calculation of the three-dimensional traffic feature tensor, three third-order cubic tensors, row factor matrix, column factor matrix, and layer factor matrix, respectively. and These represent the approximate estimates of the three-dimensional traffic feature tensor, three third-order cubic tensors, row factor matrix, column factor matrix, and layer factor matrix calculated using the error function in the i-th iteration.

[0076] Step 6: Verify the performance of the interpolation model.

[0077] By substituting the traffic flow data of road segments in the traffic network into the third-order Tucker decomposition model in step 3, and continuously iterating through the optimization algorithm in step 5 to update each kernel tensor and factor matrix, and reconstructing the tensor after each update, the missing values ​​of the original tensor in the traffic flow tensor are finally calculated as imputation values, and the performance of the imputation model is verified by comparing it with the experimental verification values ​​set in step 1.

[0078] In this step, the error between the tensor constructed after each iteration and the original tensor is calculated using the objective function. When the error is less than the convergence value or the maximum number of iterations is reached, the update and iteration stop. At this point, the constructed tensor is the final traffic flow tensor.

[0079] In this step, the root mean square error (RMSE) is used as a metric to evaluate training performance. The specific calculation method is as follows:

[0080]

[0081] Where, x m This represents the data validation value, x′ m M represents the estimated value of the data. a Indicates the total number of data.

[0082] The invention will be further illustrated in detail below with a specific example. This example uses the traffic flow dataset of Chongqing City in February 2023 as an example. This dataset includes 7 days of data, with a 15-minute interval set for the aggregation analysis. The dataset contains data from 10 sensors, resulting in a third-order tensor dimension of (7, 96, 10) for the traffic flow data in this example. This example sets the missing data rates to 20%, 50%, and 80%, respectively. The error of the objective function is calculated using the root mean square error, and the maximum number of iterations in the model is 100.

[0083] When the data missing rate is 20%, the root mean square error (RMSE) between the third-order Tucker decomposition imputation result and the actual flow verification value is 18.40; when the data missing rate is 50%, the RMSE is 25.03; and when the data missing rate is 80%, the RMSE is 29.32. Compared with existing tensor decomposition methods such as the original Tucker decomposition model, the RMSE of the third-order Tucker decomposition model is reduced by 7.58%–33.31%.

[0084] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, any improvements, modifications, changes or substitutions made within the scope of the technology disclosed in the present invention without departing from the principle of the present invention are all within the protection scope of the present invention.

Claims

1. A traffic data interpolation method based on third-order Tucker decomposition, characterized in that, Includes the following steps: Step 1: Preprocess the raw traffic data to obtain a road traffic flow dataset; Step 2: Construct a 3D traffic feature tensor based on the road traffic flow dataset; Step 3: Construct a third-order Tucker decomposition model; Step 4: Determine the optimization objective function based on the third-order Tucker decomposition; Step 5: Solve the objective function using the third-order Tucker decomposition optimization algorithm based on the least squares method; Step 6: Verify the performance of the interpolation model.

2. The traffic data interpolation method based on third-order Tucker decomposition according to claim 1, characterized in that, Step 1 involves preprocessing the raw traffic data, including: Obtain raw traffic flow data from multiple nodes in the road traffic network, remove outliers, and standardize the data. Data from each node is randomly deleted at a predetermined missing data rate D, and the true values ​​of all missing node data are recorded as validation values ​​for the data imputation model. In this process, a random number is generated for each data node, and the random numbers are sorted from smallest to largest. The data of the nodes in the top D groups are then deleted.

3. The traffic data interpolation method based on third-order Tucker decomposition according to claim 1, characterized in that, Step 2 involves constructing a 3D traffic feature tensor, including: In the road traffic flow dataset obtained from the preprocessing in step 1, aggregation analysis is performed at k-minute intervals to construct a three-dimensional traffic feature tensor. Where n1 is the number of days the dataset covers, n2 is the number of traffic flows effectively monitored each day, calculated by n2 = 1440 / k, and n3 is the number of traffic flow detectors on the road segment.

4. The traffic data interpolation method based on third-order Tucker decomposition according to claim 3, characterized in that, Step 3 involves constructing a third-order Tucker decomposition model, including: Based on the 3D traffic feature tensor constructed in step 2, the Tucker decomposition framework is used to process the feature tensor. Decompose and Decomposed into kernel tensors The form of multiplication with the outer product of the factor matrix: in, Let r1, r2, and r3 represent the row factor matrix, column factor matrix, and layer factor matrix, respectively, and let r1, r2, and r3 represent the three dimensions of the kernel tensor, namely the row dimension, column dimension, and layer dimension. n n = 1, 2, 3 represents the modal product between the tensor and the matrix; based on the Tucker decomposition framework, the original kernel tensor... Decomposed into third-order cubic tensors The triple product form, decomposed into the following form: Where r is the rank corresponding to the third-order Tucker decomposition, and ^ represents the estimated tensor value.

5. The traffic data interpolation method based on third-order Tucker decomposition according to claim 4, characterized in that, The rank r corresponding to the third-order Tucker decomposition in step 3 is no greater than the original kernel tensor. The rank of the original Tucker decomposition is no greater than the median value of its rank. r≤mid{n1,n2,n3} (3) where mid{n1,n2,n3} represents the second largest value among the three values.

6. The traffic data interpolation method based on third-order Tucker decomposition according to claim 5, characterized in that, Step 4: Based on the Tucker decomposition framework, optimize the objective function as follows: in, Representing the Frobenius norm; substituting the third-order Tucker decomposition model constructed in step 3 into this objective function, we obtain: in, This represents the optimization objective function based on the third-order Tucker decomposition.

7. A traffic data interpolation method based on third-order Tucker decomposition according to claim 6, characterized in that, Step 5 involves solving the objective function using a third-order Tucker decomposition optimization algorithm based on the least squares method, including: Based on the least squares algorithm structure, the parameters in the optimization objective function constructed in step 4 are respectively... U, V, and W are solved iteratively. After i iterations, the optimal parameters of the third-order Tucker decomposition are obtained, and the interpolated traffic flow tensor is obtained. The optimization process for each parameter of the optimization algorithm is as follows: Where λ and γ represent the parameters to be calibrated, γ∈[0,1]; i represents the number of iterations in the algorithm. U i V i and W i Let represent the estimated values ​​obtained from the i-th iteration of the calculation of the three-dimensional traffic feature tensor, three third-order cubic tensors, row factor matrix, column factor matrix, and layer factor matrix, respectively. U i+1 V i+1 and W i+1 These represent the estimated values ​​obtained from the (i+1)th iteration of the calculation of the three-dimensional traffic feature tensor, three third-order cubic tensors, row factor matrix, column factor matrix, and layer factor matrix, respectively. and These represent the approximate estimates of the three-dimensional traffic feature tensor, three third-order cubic tensors, row factor matrix, column factor matrix, and layer factor matrix calculated using the error function in the i-th iteration.

8. A traffic data interpolation method based on third-order Tucker decomposition according to any one of claims 1-7, characterized in that, Step 6 verifies the performance of the interpolation model, including: By substituting the traffic flow data of road segments in the traffic network into the third-order Tucker decomposition model in step 3, and continuously iterating through the optimization algorithm in step 5, the kernel tensors and factor matrices are updated. After each update, the tensor is reconstructed, and the missing values ​​of the original tensor in the final traffic flow tensor are used as imputation values. The experimental verification values ​​set in step 1 are compared to verify the performance of the imputation model.

9. A traffic data interpolation method based on third-order Tucker decomposition according to claim 8, characterized in that, Step 6: Calculate the error between the tensor constructed after each iteration and the original tensor using the objective function. When the error is less than the convergence value or the maximum number of iterations is reached, stop updating and iterating. At this point, the constructed tensor is the final traffic flow tensor. The root mean square error (RMSE) is used as a metric to evaluate training performance, and is calculated as follows: Where, x m This represents the data validation value, x′ m M represents the estimated value of the data. a Indicates the total number of data.

Images