A traffic flow data space-time completion method and system under a big data environment

Abstract

The application relates to a traffic flow data space-time completion method and system in a big data environment, which comprises the following steps: collecting road vector data, point of interest and surface of interest information and real-time multi-source sensing data, and performing data preprocessing; converting the road vector data into a weighted directed graph; establishing a spatial correlation relationship between the point of interest and surface of interest information and road sections; matching and fusing the weighted directed graph and the multi-source sensing data; extracting features by using an attention mechanism to form a preliminary feature vector; further analyzing to obtain a spatial feature vector and a time feature vector representing the global spatial correlation of the road network, and inputting the spatial feature vector and the time feature vector into a fusion prediction model to obtain a preliminary traffic flow prediction value; and based on the preliminary traffic flow prediction value, dynamically calculating the traffic distribution proportion of each intersection, and iteratively calculating the movement and accumulation of vehicles among road sections to output a final traffic flow prediction value. Compared with the prior art, the application has the advantages of adapting to special scenes and being strong in model robustness.

Description

Technical Field

[0001] This invention relates to the field of traffic data processing, and in particular to a method and system for spatiotemporal completion of traffic flow data in a big data environment. It integrates road vector data, POI (Point of Interest), AOI (Area of ​​Interest) information and real-time multi-source perception data, and combines the principle of spatiotemporal conservation of traffic flow with a deep learning adaptive mechanism. Through data increment-driven model dynamic optimization, it achieves high-precision and complete PCU (Standard Traffic Unit) completion for all road segments in the region under different date types, time periods (e.g., 5 minutes), and special traffic scenarios. Background Technology

[0002] With the acceleration of urbanization and the popularization of intelligent transportation systems, traffic flow data has become the core foundational data for urban traffic management, planning decisions, and road network optimization. However, in the actual data collection process, traffic flow data often suffers from complex missing data, such as random missing data, continuous missing data, and regional missing data, due to various factors such as insufficient sensor deployment density, equipment failure, extreme weather interference, network transmission delay, and temporary traffic control.

[0003] Traffic flow data exhibits strong spatiotemporal coupling characteristics: spatially, there is a dynamic transmission relationship between traffic flow in adjacent road segments and upstream / downstream road networks; temporally, it displays multiple periodicities, including intraday peak fluctuations, differences between weekdays and holidays, and seasonal variations; furthermore, the functional attributes of POI / AOI (such as shopping demand in commercial areas and commuting demand in residential areas) and their real-time status (such as mall foot traffic and tourist volume in scenic spots) have a significant dynamic impact on the generation and absorption of traffic flow. Adding new date data reveals even more complex temporal periodic patterns, and adding new road segment data alters the road network topology. Existing technologies have failed to achieve real-time fusion and adaptive model adjustment of this incremental information. Therefore, a comprehensive completion method that considers multi-source data fusion, spatiotemporal dynamic characteristics, adaptation to special scenarios, and incremental data-driven optimization is urgently needed to improve the accuracy and long-term usability of the completed data.

[0004] The invention disclosed in CN113724504B presents a system and method for predicting urban traffic flow based on big data of vehicle trajectories. First, it constructs a historical information dataset by collecting vehicle trajectory data and environmental information data, building an overall historical inflow-outflow matrix and an urban traffic transfer map. Then, it constructs a spatiotemporal convolutional attention network deep learning model for traffic prediction based on joint features. Next, it extracts spatiotemporal features of traffic transfer and inter-regional traffic transfer, embedding external features. Finally, the model performs joint prediction using global spatiotemporal features of traffic flow and inter-regional traffic transfer features, embedding external features such as environmental information, to obtain the predicted traffic flow for the next time moment. However, this scheme exhibits weak predictive ability and poor robustness under different date types, different traffic scenarios, and data incremental update scenarios. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies by providing a method and system for spatiotemporal completion of traffic flow data in a big data environment. This is achieved by constructing a regional traffic flow intelligent network model that integrates multi-source heterogeneous data, road vector data, POI / AOI information, and real-time multi-source sensing data, establishing a data framework that includes topological relationships, spatiotemporal characteristics, and dynamic influencing factors. Based on the principle of spatiotemporal conservation of traffic flow, and combining the hybrid modeling ideas of deep learning and traditional traffic models, an adaptive traffic flow estimation algorithm is designed. Through dynamic hierarchical masking and multi-dimensional deviation correction mechanisms, the model parameters and structure are continuously optimized. The core innovation lies in introducing a data increment-driven adaptive optimization mechanism. When new date data or new road segment data is added, it automatically triggers local model updates or adaptive parameter adjustments without requiring full retraining. Ultimately, it achieves high-precision completion of PCUs for all road segments within the region at various time periods (e.g., 5 minutes) under different missing rates, different date types, different traffic scenarios, and data increment update scenarios.

[0006] The objective of this invention can be achieved through the following technical solutions: A method for spatiotemporal completion of traffic flow data in a big data environment, the method includes the following steps: Step 1: Collect road vector data, points of interest and areas of interest information to characterize the distribution of urban functions and activity intensity, and real-time multi-source sensing data, and perform data preprocessing; Step 2: Convert the road vector data into a weighted directed graph; establish the spatial association between the points of interest and interest surfaces and road segments based on the KNN algorithm; match and fuse the weighted directed graph and multi-source sensing data based on the spatial association. Step 3: Use an attention mechanism to extract features from the fused data in both time and space dimensions to form preliminary feature vectors; based on the preliminary feature vectors, analyze the weighted directed graph and time series data respectively to obtain spatial feature vectors and time feature vectors representing the global spatial correlation of the road network, and then feed them into the pre-built fusion prediction model to obtain preliminary traffic flow prediction values. Step 4: Based on the preliminary traffic flow forecast, dynamically calculate the traffic flow distribution ratio at each intersection, and according to the law of conservation of traffic flow, iteratively calculate the movement and accumulation of vehicles between each road segment based on the traffic flow distribution ratio until the traffic flow of the entire road network reaches dynamic equilibrium, and output the final traffic flow forecast.

[0007] Furthermore, the formula for dynamically calculating the traffic flow distribution ratio at each intersection is as follows: in, For road section to section of road The traffic distribution weight, For road section to section of road The number of lanes, For road section to section of road Speed ​​limit, For road section to section of road The congestion attenuation coefficient, Let i be the index of the adjacent road segment. For road section to adjacent road section The number of lanes, For road section to adjacent road section Speed ​​limit, For road section to adjacent road section The congestion attenuation coefficient, For road section to section of road Historical average flow For road section to section of road Traffic capacity, For road section to section of road Real-time congestion level.

[0008] Furthermore, the iterative calculation process specifically includes: based on the law of conservation of traffic flow and the traffic flow allocation ratio, dividing each road segment into several cells, and adaptively adjusting the cell size according to the road segment length and real-time traffic density. The cell length calculation formula is as follows: in, The length of the cell. For road section Length, For road section Real-time traffic density, For road section Traffic capacity, It is a rounding function; Within each time step, calculate the inflow and outflow traffic flow of each cell based on the traffic flow allocation ratio, and update the number of vehicles in each cell. The updated formula is: in, for Time Cell The number of vehicles, for Time Cell The number of vehicles, for Inflow of time into cells Traffic flow for Cells constantly flowing out Traffic flow; Repeat the above update process until the traffic flow status of all road segments in the entire road network reaches a dynamic equilibrium, that is, the traffic flow change between adjacent iteration steps is less than the preset convergence threshold, and then output the final traffic flow prediction value.

[0009] Furthermore, the final traffic flow prediction is also constrained by the real-time activity intensity of interest areas and special scenarios, by... The flow update formula, incorporated into the cellular transport model, dynamically constrains the real-time activity intensity of the interest surface on flow generation or attraction. The calculation formula is as follows: in, For road section The basic traffic generation / attraction coefficient, The influence weighting coefficient of AOI activity intensity For road section Surrounding POI density, For the road section The associated real-time activity intensity of AOI, This is a time correction factor; The constraints of the special scenario are determined by the interference coefficient. Characterization, The initial traffic flow prediction is corrected by multiplication, and the result has different values ​​depending on the weather or scenario.

[0010] Furthermore, the fusion prediction model consists of a pre-built graph neural network model and a temporal convolutional network model; the pre-built fusion prediction model, dynamic diversion and merging model, and cell transmission model are fused together to form a regional traffic flow intelligent network model; and a dynamic hierarchical masking mechanism and a multi-dimensional deviation correction mechanism are used to train the regional traffic flow intelligent network model. The graph neural network model is used to analyze the weighted directed graph to obtain spatial feature vectors representing the global spatial correlation of the road network; the temporal convolutional network model is used to analyze the time series data to obtain temporal feature vectors; the dynamic diversion and merging model is used to dynamically calculate the traffic flow allocation ratio at each intersection based on the preliminary traffic flow prediction value; the cellular transmission model is used to iteratively calculate the movement and accumulation of vehicles between each road segment based on the traffic flow allocation ratio, and output the final traffic flow prediction value. The dynamic layered masking mechanism specifically includes: Masking is used to represent missing data on roads; roads are randomly or continuously masked based on their importance scores, with higher importance scores resulting in lower masking probabilities; all roads within a random specific area are masked according to a pre-set missing rate. The importance score of the road is calculated using a weighted summation formula: Where S represents the road importance score, This is a weighting coefficient for road grade. The weighting coefficient is the average daily traffic volume. The weighting coefficients for regional functional dependence. To normalize the road grade score, The normalized score for average daily traffic. The score represents the regional functional dependence; the constraints are as follows: .

[0011] Furthermore, the multi-dimensional bias correction mechanism specifically includes: The prediction bias generated during the training of the regional traffic flow intelligent network model is subjected to residual fitting using a pre-built radial basis function neural network model, and a bias compensation term is generated. The calculation formula is as follows: in, In a radial basis function neural network, the first... The output value of the activation function of each hidden layer neuron. For indexing hidden layer neurons, for The input feature vector at time t, For the first The center point of each radial basis function For the Euclidean norm, for Time of the first The width parameter of each radial basis function. For the first The initial width of the radial basis functions, This represents the number of iterations. Furthermore, based on the aforementioned deviation compensation term, the parameters of the regional traffic flow intelligent network model are updated using the improved Gauss-Seidel iterative method, with the update formula as follows: in, For the first After the second iteration, the road section and The corresponding model weight parameters, For the first After the second iteration, the road section and The corresponding model weight parameters, for Adaptive learning rate at any given time. The model loss function, For the loss function with respect to the weight parameters The partial derivatives, This is the deviation compensation coefficient. for Time and Section Deviation compensation item, The regularization coefficient is . Let be the mean of all weight parameters. for Norm, for Adaptive learning rate at any given time. This is the initial learning rate.

[0012] Furthermore, the regional traffic flow intelligent network model triggers an adaptive optimization process by receiving incremental data in real time. This adaptive optimization process specifically includes: When the incremental data is date data: extract the time features of the newly added date data and update the time feature library; update the weight parameters related to the newly added time features in the Long Short-Term Memory network model; adjust the time correction coefficient in the regional traffic flow intelligent network model. Adjustment coefficient for activity intensity related to interests Furthermore, a parameter-weighted fusion method is used to adjust the key parameters in the regional traffic flow intelligent network model, retaining historical parameters according to a pre-set weight ratio; the specific calculation formula in the parameter-weighted fusion method is as follows: in, For the new parameters after fusion, These are the weighting coefficients for historical parameters. These are the historically optimal parameters for the model. The weight coefficients for training parameters on newly added data. The parameters are obtained by training based on the newly added data, and the constraints are as follows: ; When the incremental data is road segment data: the new road segment is automatically added to the weighted directed graph and the edge weight matrix is ​​updated; the node feature vectors, diversion weight parameters, and interest surface association mapping relationships of adjacent road segments are reused through transfer learning as the initial parameters of the new road segment; Local training is performed using data from newly added road segments, optimizing only the graph convolution kernel parameters of the graph neural network model directly related to the newly added road segments and the cell partitioning coefficients of the cell transfer model; the diversion weights of the newly added road segments and surrounding road segments in the dynamic diversion and merging model are updated.

[0013] Furthermore, the adaptive optimization process is triggered by a triggering condition, which specifically involves: calculating the information gain between the incremental data and existing data using the mutual information formula, and calculating the distribution similarity between the incremental data and existing data using KL divergence. The calculation formula is: in, for divergence, The probability distribution of the existing data. For the probability distribution of incremental data, To assign feature values ​​to the data samples, The feature values ​​in the existing data are... The probability, For the feature values ​​in the incremental data The probability of; When the information gain is greater than a preset value or the distribution similarity is greater than a preset value, the adaptive optimization process is triggered. The mutual information formula is: in, The information gain between incremental data and existing data. For existing data Information entropy Given incremental data Data already available conditional entropy, For an existing dataset, For incremental datasets, for Values and Values The joint probability, for Values hour Values The conditional probability.

[0014] Furthermore, the formula for calculating the edge weights of a weighted directed graph is: in, For road section and road section Edge weights between them For road section and road section common similarity, For road section and road section The degree of topological association, , These are the weight coefficients for common similarity and topological affinity, respectively, and satisfy the following conditions: ; The real-time multi-source sensing data specifically includes video surveillance traffic, floating car data, meteorological data, and traffic incident information.

[0015] The present invention also provides a spatiotemporal completion system for traffic flow data in a big data environment, including a memory and a processor. The memory stores a computer program, and the processor calls the computer program to execute the steps of any of the methods described above.

[0016] Compared with the prior art, the present invention has the following advantages: (1) This invention constructs a multi-source heterogeneous dataset based on road vector data, points of interest and areas of interest, and real-time multi-source perception data, and converts the road vector data into a weighted directed graph to establish a dynamic influencing factor library; it breaks through the limitation of traditional methods that rely on only a single data and improves the model's ability to perceive dynamic changes in traffic flow; by integrating graph neural network model, temporal convolutional network model, cell transmission model and dynamic diversion and merging model, and combining the traffic flow conservation principle, a regional traffic flow intelligent network model is constructed. It ensures that the traffic flow predictions output by the model are constrained by the physical laws of traffic flow, while also capturing complex spatiotemporal nonlinear correlations, making it suitable for high missing rates and complex traffic scenarios.

[0017] (2) This invention trains and optimizes the parameters of the regional traffic flow intelligent network model by adopting a dynamic hierarchical masking mechanism and a multi-dimensional deviation correction mechanism. The masking mode is dynamically adjusted according to different missing types and road segment importance. Combined with the improved RBF neural network and Gauss-Seidel iterative method, the multi-dimensional deviation of the regional traffic flow intelligent network model is accurately corrected, and the robustness of the model is improved. Combined with Bayesian optimization and transfer learning, the hyperparameters are efficiently tuned and adapted quickly across regions and scenarios, reducing the model deployment cost and improving the engineering application value of the regional traffic flow intelligent network model.

[0018] (3) This invention establishes an adaptive optimization mechanism for newly added date / segment data for the first time, and through data value assessment, parameter incremental fine-tuning and transfer learning adaptation, it realizes that when encountering newly added date or segment data, the model is locally updated to adapt to the change of environment instead of retraining the model. This greatly improves the data adaptation efficiency (the update time is reduced by more than 80%), and avoids the model accuracy decay caused by long-term use. It adapts to the actual needs of road network expansion and data accumulation, and has good adaptability and industrial applicability. Attached Figure Description

[0019] Figure 1 This is a flowchart of a method for spatiotemporal completion of traffic flow data in a big data environment, provided in an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0021] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0022] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0023] Definitions: The Gauss-Seidel iteration method is a classic iterative algorithm in numerical linear algebra for solving large systems of linear equations Ax=b. It is a significant improvement on the Jacobi iteration method: its core idea lies in instantaneous updates. In each iteration, when a new approximate value for an unknown quantity is calculated, this latest value immediately replaces the old value and is used to calculate subsequent unknown quantities in the same iteration. This strategy of "using the latest information" makes the Gauss-Seidel iteration method generally converge faster than the Jacobi iteration method because the newly generated improvement information can be utilized immediately.

[0024] RBF neural network: A typical three-layer feedforward neural network, its name comes from the radial basis functions used by its hidden layer neurons. Its core idea is to map the input vector to a high-dimensional feature space through a nonlinear transformation, thereby transforming a linearly inseparable problem in the original space into a linearly separable problem in the high-dimensional space, and finally performing a linear combination in the output layer.

[0025] KNN algorithm: A basic and intuitive supervised machine learning algorithm that can be used for both classification and regression. Its core principle is "like attracts like," meaning that the category or value of a sample is determined by its K nearest known samples (neighbors) through "majority voting" or "average value."

[0026] The LWR model is a classic macroscopic continuous model in traffic flow theory. Its core idea is to analogize traffic flow to a compressible fluid, using three macroscopic variables—traffic density, traffic flow, and average speed—to describe the overall state of traffic flow across a road cross-section. Its foundation is the assumption that flow and density satisfy a definite functional relationship, i.e., a basic graph, and based on this, a partial differential equation (continuity equation) describing the conservation properties of traffic flow is constructed.

[0027] Transfer learning is an important paradigm in machine learning. Its core idea is to transfer knowledge (model parameters, feature representations, etc.) learned from one task (source domain) and apply it to another related but different task (target domain). Bayesian optimization is a sequential global optimization strategy for efficiently optimizing black-box functions, particularly suitable for scenarios where the objective function is computationally expensive, cannot be differentiated, or lacks a clear expression. Its core idea is to use a probabilistic model to proxy the unknown objective function, balancing exploration and exploitation through continuous iteration.

[0028] GIS spatial analysis tools are a core functional module of Geographic Information Systems (GIS). They are a collection of specialized methods and software functions used to process, interpret, and mine patterns, relationships, and trends behind geospatial data. These tools generate new information and insights based on the location, attributes, and spatial relationships (such as distance, direction, connectivity, and overlay) of geographic objects through specific spatial operations. Core types include: overlay analysis (such as intersection and union), buffer analysis, network analysis (such as optimal path finding), terrain analysis (such as slope, aspect, and watershed extraction), and spatial statistical analysis (such as hotspot analysis and spatial interpolation).

[0029] Spatiotemporal interpolation models are a class of data analysis methods used to estimate the values ​​of target variables at unsampled spatiotemporal locations. They consider both the spatial correlation and temporal dynamics of the data. Unlike simple spatial interpolation, these models treat time as a dimension equal to spatial coordinates, or as a dynamic factor influencing the evolution of spatial processes, aiming to reconstruct a continuous and complete spatiotemporal field. The core challenge lies in how to reasonably model spatial autocorrelation, temporal autocorrelation, and the spatiotemporal covariance structure of their interaction.

[0030] Statistical forecasting models are a class of mathematical models that use historical data and statistical theory to make probabilistic inferences about future or unknown states by quantifying the relationships and patterns between variables. Their core lies in identifying and utilizing trend, seasonal, and random components from the data to construct a mathematical structure (such as regression equations or time series models) that can summarize past behavior. This structure is then assumed to remain stable over a future period, and extrapolated to obtain predicted values ​​and their uncertainty intervals (such as confidence intervals).

[0031] Example 1 like Figure 1 As shown in the figure, this embodiment provides a method for spatiotemporal completion of traffic flow data in a big data environment. The method includes the following steps: S1: Collect road vector data, points of interest and areas of interest information used to characterize the distribution of urban functions and activity intensity, and real-time multi-source sensing data, and perform data preprocessing; S2: Convert road vector data into a weighted directed graph; establish spatial relationships between interest points and interest surfaces and road segments based on the KNN algorithm; match and fuse the weighted directed graph and multi-source sensing data based on the spatial relationships. A regional traffic flow intelligent network model is constructed based on a pre-built graph neural network model, a temporal convolutional network model, a cellular transport model, and a dynamic diversion and merging model. The regional traffic flow intelligent network model is trained based on a multi-source heterogeneous dataset through a dynamic hierarchical masking mechanism and a multi-dimensional deviation correction mechanism. S201: Construct a regional traffic flow intelligent network model that integrates multi-source heterogeneous data, integrates road vector data, POI / AOI information, and real-time multi-source sensing data, and establishes an integrated data framework that includes topological relationships, spatiotemporal characteristics, and dynamic influencing factors; Specifically, Refining the processing of road vector data: Converting road vector data into a weighted directed graph model, with road segments as nodes and connections between road segments as edges, and edge weights incorporating traffic similarity and road network topology correlation; node feature vectors contain static attributes such as road grade, speed limit, number of lanes, and median type, as well as dynamic attributes such as real-time congestion status and traffic efficiency; Deeply link multi-source data: Achieve accurate matching between POI and road checkpoints based on the improved KNN algorithm (k value adaptive adjustment), and establish the association between AOI and road segments through spatial topology mapping; Integrate real-time multi-source sensing data (video surveillance traffic, floating car data, meteorological data, traffic event information) to establish a dynamic influencing factor database; Constructing a spatiotemporal data integration framework: Adopting a three-layer architecture of isomorphic data fusion, spatiotemporal classifier, and model integration. Isomorphic data fusion achieves multi-source data alignment through difference analysis and correlation mining; the spatiotemporal classifier identifies the spatiotemporal evolution patterns of traffic flow based on an attention mechanism; and model integration uses a weighted voting strategy to aggregate the inference results of multiple sub-models. Design an incremental data compatibility framework: The framework reserves an incremental data interface to support the time dimension expansion of newly added date data and the spatial dimension expansion of newly added road segment data, and automatically updates the road network topology and spatiotemporal feature library.

[0032] S202: Design an adaptive traffic flow estimation algorithm based on the principle of spatiotemporal conservation and deep learning hybrid modeling. It integrates cellular transmission model and graph neural network, considers the dynamic law of diversion and merging, the real-time activity intensity of AOI, and interference factors in special scenarios, and establishes a multi-dimensional traffic flow calculation model. Specifically, Integrating the flow conservation equation with deep learning: The flow conservation equation based on the LWR model is combined with a graph neural network (GNN) to construct a spatiotemporal feature extraction module to capture the global spatiotemporal correlation of the road network; Introducing a cell transmission model: A dynamic cell partitioning strategy is introduced, which adaptively adjusts the cell size according to the road segment length and real-time traffic density, discretizes the time step, and updates the cell state synchronously. Dynamic diversion and merging modeling: Based on the static attributes of road segments and real-time traffic conditions (flow rate, density, congestion level), establish a dynamic calculation model for diversion weights and introduce a congestion attenuation coefficient to optimize the weight formula; Multi-factor impact quantification: The real-time activity intensity of AOI (calculated based on real-time passenger flow of POI and regional population flow data) is transformed into the generation / absorption flow offset, and special scenario interference coefficients (such as traffic accidents, extreme weather, large-scale events) are introduced to correct the flow calculation results; The incremental data adaptation module is designed with a built-in parameter memory unit and feature update channel. When new data is added, the parameters of the spatiotemporal feature extraction module and the basic data of the split weight calculation model are automatically updated.

[0033] S203: Develop a dynamic hierarchical masking and multi-dimensional deviation correction mechanism. Hide data through an adaptive masking strategy and combine an RBF neural network with an improved Gauss-Seidel iterative method to achieve nonlinear deviation capture and parameter stability optimization. Specifically, Design an adaptive spatiotemporal hierarchical masking strategy: dynamically adjust the mask ratio and range based on road segment importance, traffic flow stability, and missing type. The higher the road level and the stronger the traffic flow stability, the lower the mask ratio. Design masking modes for random missing, continuous missing, and regional missing. Constructing a multi-dimensional bias correction model: Using an RBF neural network to fit the nonlinear residuals and optimizing the hidden layer activation function parameters; improving the Gauss-Seidel iterative method by introducing an adaptive learning rate and bias penalty term to dynamically adjust the model parameters; Perform bias diagnosis and structure optimization: Through cross-validation and error decomposition, distinguish between model underfitting, overfitting and scenario adaptation bias, and adjust model complexity, training data volume or feature dimension accordingly. Incremental bias correction: When new data introduces new bias, the local fine-tuning of the RBF neural network is automatically triggered to update the residual fitting model without retraining the entire correction module.

[0034] Constructing a spatiotemporal data integration framework: Adopting a three-layer architecture of isomorphic data fusion, spatiotemporal classifier, and model integration. Isomorphic data fusion achieves multi-source data alignment through difference analysis and correlation mining; the spatiotemporal classifier identifies the spatiotemporal evolution patterns of traffic flow based on an attention mechanism; and model integration uses a weighted voting strategy to aggregate the inference results of multiple sub-models. S204: Incremental Data Compatibility Design: The framework reserves an incremental data interface to support the time dimension expansion of newly added date data and the spatial dimension expansion of newly added road segment data, and automatically updates the road network topology and spatiotemporal feature library.

[0035] S3: An attention mechanism is used to extract features from the fused data in both time and space dimensions to form preliminary feature vectors. Based on these preliminary feature vectors, the weighted directed graph and time series data are analyzed to obtain spatial and temporal feature vectors that characterize the global spatial correlation of the road network. These vectors are then fed into a pre-built fusion prediction model to obtain preliminary traffic flow prediction values. Specifically, The fusion prediction model consists of a pre-built graph neural network model and a temporal convolutional network model; the regional traffic flow intelligent network model is formed by fusing the pre-built fusion prediction model, dynamic diversion and merging model and cell transmission model; and the regional traffic flow intelligent network model is trained using a dynamic hierarchical masking mechanism and a multi-dimensional deviation correction mechanism. Specifically, The performance of the trained regional traffic flow intelligent network model is verified using multi-dimensional evaluation indicators, and the verified regional traffic flow intelligent network model is deployed. Multi-source sensing data is received in real time, and missing traffic flow data is supplemented through inference. Incremental data is also received to trigger the adaptive optimization process of the regional traffic flow intelligent network model.

[0036] S4: Based on the preliminary traffic flow forecast, dynamically calculate the traffic flow distribution ratio at each intersection, and based on the law of conservation of traffic flow, iteratively calculate the movement and accumulation of vehicles between each road segment based on the traffic flow distribution ratio until the traffic flow of the entire road network reaches a dynamic balance, and output the final traffic flow forecast.

[0037] Specifically, The performance of the trained regional traffic flow intelligent network model is verified using multi-dimensional evaluation indicators, and the verified regional traffic flow intelligent network model is deployed. Multi-source sensing data is received in real time, and missing traffic flow data is supplemented through inference. Incremental data is also received to trigger the adaptive optimization process of the regional traffic flow intelligent network model.

[0038] Specifically, The missing traffic flow data is supplemented through reasoning. The specific process includes: A difference analysis method based on mutual information is adopted to match and fuse weighted directed graphs and multi-source sensing data based on spatial correlation. A long short-term memory network model based on an attention mechanism is used to extract features from the fused data in both temporal and spatial dimensions to form preliminary feature vectors. Based on these preliminary feature vectors, weighted directed graphs and time-series data are input into pre-built graph neural network models and temporal convolutional network models, respectively, to obtain spatial and temporal feature vectors representing the global spatial correlation of the road network. The spatial and temporal feature vectors are then fused to obtain basic feature vectors, which are then fed into a pre-built prediction model to obtain preliminary traffic flow predictions. The preliminary traffic flow forecast is input into the dynamic diversion and merging model and the cellular transmission model, and the final traffic flow forecast is obtained through collaborative computation.

[0039] Specifically, Incremental data identification and evaluation: Automatically identify new data types (date data / road segment data), and determine the importance of incremental data through data value evaluation indicators (information gain, data distribution similarity). When information gain is greater than or equal to the gain threshold (e.g., 0.3, exemplary parameter, non-limiting) or distribution similarity is less than or equal to the similarity threshold (e.g., 0.7, exemplary parameter, non-limiting), the optimization process is triggered. Adaptive optimization for newly added date data: When new date data is added, its time features (such as seasonality and holiday patterns) are automatically extracted, and the spatiotemporal feature library and related parameters of the LSTM model are updated; Adaptive optimization of newly added road segment data: When new road segment data is accessed, the weighted directed graph model of the road network is automatically updated. By reusing the feature parameters and weight configurations of adjacent road segments through transfer learning, the traffic characteristics of the new road segments are quickly adapted, and the diversion and merging relationships and AOI association mapping are updated synchronously. Verify the optimization effect: Use an incremental validation set to evaluate the performance of the optimized model. If the MAE decreases by ≥ the expected mean absolute error (e.g., 10%, exemplary parameter, non-limiting) or the MAPE decreases by ≥ the expected mean absolute percentage error (e.g., 8%, exemplary parameter, non-limiting), the optimization result is retained; otherwise, roll back to the previous version of the parameters and only update the data storage layer.

[0040] Specifically, The full-scenario verification and adaptive optimization steps include: Design a comprehensive test scenario: Set test combinations for missing data types (random missing, consecutive missing, regional missing), missing rates (e.g., 30%, 50%, 70%, 90%, exemplary parameters, not limited), date types (weekdays, holidays, special holidays), special traffic scenarios (traffic accidents, extreme weather, large-scale events), and data increment scenarios (e.g., adding 1 month / 3 months / 6 months of date data, adding 5 / 10 / 15 road segment data, exemplary parameters, not limited); The design incorporates a multi-dimensional evaluation index system: basic indicators include MAE, RMSE, MAPE, and WMAPE; scenario-based indicators include peak-hour volatility, anomaly response accuracy, continuous missing data completion accuracy, and special scenario adaptability; incremental optimization indicators include parameter update efficiency (update time / full training time) and accuracy retention rate (optimized MAPE / unoptimized MAPE). Design an adaptive optimization strategy: Use Bayesian optimization algorithm to tune hyperparameters, and combine it with transfer learning to achieve rapid model adaptation in different regions and scenarios; establish a model update mechanism to incrementally train and optimize the model based on real-time data.

[0041] Example 2 This embodiment provides specific implementation steps for a spatiotemporal completion method for traffic flow data in a big data environment, as described in Embodiment 1, including: 1. Construct a regional traffic flow intelligent network model that integrates multi-source heterogeneous data. (1) Refined processing of road vector data Collect road vector data containing information such as directed road segments, start and end points, road grade, speed limit, number of motor vehicle lanes, type and width of median strip, type and width of motor vehicle and non-motor vehicle strip, and non-motor vehicle lanes, and convert it into a weighted directed graph model: Node definition: Each directed road segment is treated as an independent node, and the node ID is associated with the unique identifier of the road segment; Edge definition: The connectivity between road segments is defined as an edge, and the edge weight is calculated as follows: in, For road section and road section The edge weights between road segments (unitless, used to characterize the degree of connection between road segments). For road section and road section Traffic similarity (unitless, value range [0,1], calculated by normalization of static attributes such as number of lanes and speed limit); For road section and road section The topological correlation degree (unitless, value range [0,1], calculated based on the connectivity level of road segments and the proportion of shortest paths in the road network topology); , These are the weight coefficients for common similarity (unitless, ranging from [0,1]) and topological affinity (unitless, ranging from [0,1]), respectively, and they satisfy the following conditions: (The optimal value is determined through cross-validation, with an example value as follows) =0.6, =0.4).

[0042] Node feature vector: An 8-dimensional feature vector is used, which includes static features (collecting road vector data containing information such as directed road segments, start and end points, road grade, speed limit, number of motor vehicle lanes, type of central median, and type of non-motorized vehicle median) and dynamic features (real-time congestion coefficient, real-time traffic efficiency, and traffic flow fluctuation coefficient). The dynamic features are updated once every time period (e.g., every 5 minutes).

[0043] (2) Deep correlation of multi-source data POI and road matching: An adaptive k-value KNN algorithm is used, with the k value dynamically adjusted according to the POI density (k=30 in dense POI areas and k=60 in sparse areas). The spatial distance and accessibility between POI and each road checkpoint are calculated, and the checkpoint with the highest comprehensive score is selected as the matching object, improving the matching accuracy to over 95%. AOI and Road Association: Calculate the intersection of AOI boundary and road using GIS spatial analysis tools to establish topological association between AOI and adjacent road segments; extract static attributes such as AOI type (commercial area, residential area, scenic area, etc.), building area, and population density, and combine them with real-time collected POI visitor flow and regional mobile phone signaling data to calculate the real-time activity intensity of AOI. Real-time multi-source sensing data fusion: Integrate video surveillance traffic data (e.g., statistics every 5 minutes), floating car GPS data (calculate average vehicle speed and travel time on road sections), meteorological data (rainfall intensity and visibility), and traffic incident data (accident location, impact range, and processing time) to establish a dynamic influencing factor database. Through data cleaning (outlier removal and preliminary filling of missing values) and normalization processing, achieve time synchronization (e.g., unified to a 5-minute granularity, exemplary parameter, not restrictive) and spatial alignment of multi-source data.

[0044] (3) Construction of a spatiotemporal data integration framework The framework consists of a three-layer structure: Homogeneous data fusion layer: It uses a difference analysis method based on mutual information to identify the distribution differences of different source data, and establishes the mapping relationship between data through association rule mining to achieve accurate matching and fusion of road topology data, POI / AOI data and real-time perception data; Spatiotemporal classifier layer: Based on the attention mechanism, the LSTM network extracts features from the fused data in terms of time dimension (daily peak, weekly cycle) and spatial dimension (road network cluster, regional function) to identify the spatiotemporal evolution pattern of traffic flow. Model Integration Layer: Constructs three sub-models (spatiotemporal interpolation model, statistical prediction model, and deep learning model), and uses a weighted voting strategy to aggregate the inference results of the sub-models. The weights are dynamically adjusted based on the prediction accuracy of each sub-model on historical data. Incremental data compatibility design: The framework has a built-in incremental data interface that supports batch import of new date data by time dimension (such as automatically importing the previous month's data every month), adding new road segment data by spatial dimension (including road segment attributes and topological association information), and automatically updating the road network topology, spatiotemporal feature library and model input dimensions without the need to reconstruct the framework.

[0045] 2. Design an adaptive traffic flow estimation algorithm based on the principle of spatiotemporal conservation and deep learning hybrid modeling. (1) Fusion of flow conservation equation and deep learning Flow conservation equation based on LWR model A spatiotemporal feature extraction module is constructed by combining graph neural networks (GNNs): The weighted directed graph model is used as input to the GNN, and the global spatial correlation features of the road network are captured through graph convolutional layers; Temporal convolutional networks (TCNs) are introduced to extract time-series features of traffic flow, and spatiotemporal features are fused to form the basic feature vector for road segment traffic flow prediction; The constraints of the flow conservation equation are embedded into the loss function of the GNN to ensure that the prediction results satisfy the spatiotemporal consistency.

[0046] (2) Optimization of cell transport model Dynamic cell partitioning: based on road segment length Real-time traffic density Adaptive adjustment of cell size and cell length The calculation formula is: in: The cell length (unit: meter, a positive real number representing the physical length of a single cell in the cell transport model). For road section The length (unit: meters, the value is the actual measured length of the road); For road section Real-time traffic density (unit: PCU / km, calculated as the ratio of real-time vehicle count to road segment length); For road section Traffic capacity (unit: PCU / h, theoretical maximum traffic flow calculated based on road segment attributes); It is a rounding function (ensuring that the number of cells is a positive integer).

[0047] Number of vehicles per cell The updated formula is: in: for Time Cell The number of vehicles (unit: vehicles, value is a non-negative integer); for Time Cell The number of vehicles (unit: vehicles, value is a non-negative integer); for Inflow of time into cells Traffic flow (unit: vehicles / time period, value is a non-negative integer, time period is the time granularity set by the model, such as 5 minutes); for Cells constantly flowing out Traffic flow (unit: vehicles / time period, value is a non-negative integer).

[0048] (3) Dynamic diversion and merging modeling Based on the static attributes of road segments and real-time traffic conditions, a dynamic calculation model for diversion weights is established: in: For road section to section of road The diversion weight (unitless, value range [0,1], represents the traffic flow from road segment) to section of road (the proportion of diversion) For road section to section of road The number of lanes (unit: lanes, value is a positive integer); For road section to section of road Speed ​​limits (unit: km / h, the value is the actual effective value of the road design speed limit); For road section to section of road The congestion attenuation coefficient (unitless, value range [0,1], the higher the congestion level, the larger the value). The index of the adjacent road segments of road segment i (unitless); For road section to adjacent road section The number of lanes (unit: lanes, value is a positive integer); For road section to adjacent road section Speed ​​limits (unit: km / h, the value is the actual effective value of the road design speed limit); For road section to adjacent road section The congestion attenuation coefficient (unitless, value range [0,1]); For road section to section of road Historical average flow rate (unit: PCU / h, statistical period is the average of the corresponding time period of each day in the past 3 months); For road section to section of road Traffic capacity (unit: PCU / h, theoretical maximum traffic flow calculated based on road grade, number of lanes, intersection control method, etc.); For road section to section of road The real-time congestion level (unitless, value range [0,1], 0 indicates no congestion, 1 indicates severe congestion, obtained by normalizing the ratio of real-time traffic flow to traffic capacity).

[0049] (4) Quantification of the influence of multiple factors Impact of AOI real-time activity intensity: The generated traffic formula is optimized as follows: in: For road section The basic traffic generation / attraction coefficient (unitless, range [0, +∞) characterizes the road segment. (Intensity of traffic generated or attracted by surrounding POI / AOI activities). The influence weighting coefficient of AOI activity intensity (unitless, value range [0,2], calibrated through historical data, exemplary values: commercial area 1.6, residential area 1.3, scenic area 1.8, office area 1.4). For road section Surrounding POI density (unit: points / km, statistical road sections) The ratio of the number of POIs within 500 meters on both sides to the length of the road segment. For the road section The real-time activity intensity of the associated AOI (unitless, value range [0,1], calculated based on normalized calculation of real-time passenger flow, mobile signaling data, etc. within the AOI); This is a time correction factor (unitless, value range [0.5, 1.5], example values: 1.3 for peak hours, 0.9 for off-peak hours, and 0.6 for nighttime hours).

[0050] Constraints in special scenarios are determined by interference coefficients. The interference coefficient is characterized by having different values ​​under different weather conditions or scenarios.

[0051] in: This is a special scene interference coefficient (unitless, value range [0.5, 1.5]). For example, the value selection rule is: Normal weather / scene: =1.0; Light rain / Mild traffic congestion: =0.9; Moderate rain / moderate traffic congestion: =0.8; Heavy rain / dense fog / severe traffic congestion: =0.7; Small events (less than 1000 participants): =1.1; Medium-sized events (1000 people ≤ number of participants < 5000 people): =1.2; Large-scale events (≥5000 participants): =1.3-1.5 (increasing with the number of people); Traffic accident (duration < 30 minutes): =0.9; Traffic accident (30 minutes ≤ impact time < 60 minutes): =0.8; Traffic accidents (duration ≥ 60 minutes): =0.7.

[0052] in, For road section Basic flow generation / attraction coefficient (unitless, range of values) (This characterizes the intensity of AOI activity's contribution to traffic flow in a road segment). The influence weighting coefficient of AOI activity intensity (unitless, range of values) Calibration through historical data: Business District Residential area Scenic Area Office area ), For road section Surrounding POI density (unit: points / km, statistical road sections) The ratio of the number of POIs within 500 meters on both sides to the length of the road segment. For the road section Related AOI real-time activity intensity (unitless, range of values) Based on real-time passenger flow and normalized mobile signaling data within the AOI, 0 indicates no activity and 1 indicates full-capacity activity. Time correction factor (unitless, range of values) ,peak hours Off-peak hours Nighttime ); Constraint Logic: It is positively correlated with the final traffic flow forecast, by... Traffic update formula incorporated into cellular transport model ( This enables dynamic constraints on the generation / attraction of traffic by the intensity of AOI activity.

[0053] Constraints in special scenarios are determined by interference coefficients. Characterization, By applying a multiplicative correction to the initial traffic flow forecast, the final traffic flow forecast = initial traffic flow forecast × The specific rules are as follows: 1. Interference coefficient Physical meaning: Unitless, value range [0.5, 1.5], used to correct abnormal fluctuations in traffic flow in special scenarios (δ<1 indicates flow attenuation, δ>1 indicates flow surge). 2. Different scenarios Value selection criteria: Weather scenario: Normal weather =1.0; Light rain / light haze =0.9; Moderate rain / moderate haze =0.8; Heavy rain / dense fog / blizzard =0.7; Traffic incident scenario: No incident =1.0; Minor incident (duration <30 minutes) =0.9; Medium-sized incident (30 minutes ≤ impact duration < 60 minutes) =0.8; Major incident (duration ≥ 60 minutes) =0.7; Large-scale event scenario: No event =1.0; Small-scale events (less than 1000 participants) =1.1; Medium-sized event (1000 people ≤ number of participants < 5000 people) =1.2; Large-scale events (≥5000 participants) =1.3~1.5 (increasing with the number of people); 3. Multi-scene overlay rule: When multiple special scenes exist simultaneously, take... The minimum value (e.g., heavy rain + major accident), =min(0.7,0.7)=0.7).

[0054] Note: Real-time activity intensity constraints on interest areas and special scenario constraints act independently but are cumulatively applied. The final traffic flow forecast value = (preliminary traffic flow forecast value × ...) )× This ensures that the dual impact of AOI dynamic activities and special scenarios is fully considered.

[0055] (5) Incremental data adaptation module Parameter memory unit: Stores key parameters during model training (such as graph convolution kernel parameters of GNN, temporal convolution parameters of TCN, and basic coefficients for calculating the split weights of the model), avoiding parameter reset when new data is added; Feature update channel: When new date data is added, the time characteristics of the new data are automatically calculated (such as the seasonal characteristics of the new month and the traffic fluctuation characteristics of special holidays), and the input feature set of the spatiotemporal feature extraction module is updated; when new road segment data is added, the relationship between adjacent road segments and basic data in the diversion weight calculation model are updated synchronously to ensure the accuracy of diversion and merging calculation.

[0056] 3. Develop dynamic layered masking and multi-dimensional bias correction mechanism (1) Adaptive spatiotemporal layered masking strategy Road segment importance assessment: An importance scoring system is constructed based on road grade, average daily traffic volume, and regional functional dependence. The importance score is ≥0.8 for expressways and arterial roads, ≥0.6 for secondary arterial roads, and ≥0.4 for local roads. These are exemplary parameters and are not restrictive. Masking strategy design: Random missing data: Data is randomly hidden according to the missing rate. The higher the importance score, the lower the masking probability (masking probability of expressway = missing rate × 0.5, branch = missing rate × 1.0, exemplary parameters, not restrictive). Continuous missing data: Simulate equipment failure scenarios, randomly select continuous time periods (1-4 hours) to hide data according to the importance of road segments. The lower the importance, the higher the probability of continuous masking duration. Regional missing data: Simulate regional network failure scenarios, select all road segments in a specific area (such as a commercial area or transportation hub), and hide the data according to the missing data rate; Mask generation process: First, determine the mask mode based on the missing type, then assign mask probabilities based on the importance of road segments, and finally randomly select road segments and time periods to execute the mask to ensure coverage of different scenarios.

[0057] (2) Multidimensional deviation correction model RBF Neural Network Residual Fitting: Optimizing Hidden Layer Activation Functions and Introducing Adaptive Width Parameters : in: In a radial basis function neural network, the first... The activation function output values ​​of each hidden layer neuron (unitless, ranging from (0,1]). Index of hidden layer neurons (unitless, values ​​are positive integers); for The input feature vector at time step (unitless, composed of features fused from multiple sources, with dimensions consistent with the number of neurons in the input layer). For the first The center points of the radial basis functions (unitless, with dimensions consistent with the input feature vectors, determined from the training data using a clustering algorithm); is the Euclidean norm (unitless, representing the distance between vectors); for Time of the first The width parameter of each radial basis function (unitless, takes the value of a positive real number, and is adaptively adjusted with the number of iterations); For the first The initial width of each radial basis function (unitless, positive real number, example value range [0.1, 1.0], determined by cross-validation); The number of iterations (unitless, takes the value of a non-negative integer). (Indicates the initial iteration); It is the natural logarithm function (unitless).

[0058] The improved Gauss-Seidel iterative method is then used to update the model parameters, with the update formula as follows: in: ( For the first After the second iteration, the road section and The corresponding model weight parameters (unitless, real numbers). For the first After the second iteration, the road section and The corresponding model weight parameters (unitless, real numbers). This is the index for the number of iterations (unitless, and a non-negative integer). for The adaptive learning rate at each time step (unitless, taking the value of a positive real number, controlling the step size of parameter updates); The model loss function (unitless, taking the value of a non-negative real number, used to measure the deviation between the predicted value and the true value). For the loss function with respect to the weight parameters The partial derivatives (unitless, representing the degree of influence of the weight parameters on the loss). This is the deviation compensation coefficient (unitless, ranging from [0, 0.1], used to balance the magnitude of deviation correction). for Time and Section The deviation compensation item (unitless, real number, calculated from historical residual data); This is the regularization coefficient (unitless, ranging from [0, 0.01], used to prevent model overfitting). This is the mean of all weight parameters (unitless, real numbers, used to constrain the distribution of weight parameters); for Norm (unitless, representing the degree of deviation of the weight parameter from the mean); for The adaptive learning rate at time step (unitless, with a positive real number as the value); The initial learning rate (unitless, ranging from [0.001, 0.01], with an example value of 0.005, determined through cross-validation). The number of iterations (unitless, a non-negative integer); It is a natural constant (unitless, approximately equal to 2.71828).

[0059] Bias diagnosis: The error is decomposed into bias error, variance error and noise error through cross-validation. When the bias error accounts for ≥40%, the model complexity is increased (e.g., the number of GNN layers); when the variance error accounts for ≥40%, the regularization strength or the amount of training data is increased; when the noise error accounts for ≥30%, the data preprocessing process is optimized.

[0060] (3) Incremental deviation correction When new data is added, the deviation detection module calculates the extrapolated residuals corresponding to the new data. If the mean residual changes by ≥15% compared to the historical mean residual, the local fine-tuning of the RBF neural network is automatically triggered. Only update the hidden layer node parameters related to the new data, without changing the network structure and other node parameters; The number of fine-tuning iterations was set to 1 / 5 of the number of full training iterations, and the learning rate was 0.3 times that of full training iterations, to ensure fast convergence without destroying the historical fitting effect. After fine-tuning, the residual fitting accuracy is verified using the validation set. If the residual decreases by ≥20%, the updated parameters are saved; otherwise, the original parameters are maintained.

[0061] 4. Establish an incremental data-driven adaptive optimization mechanism (1) Incremental data identification and evaluation Data type identification: Automatically distinguish new data as date data (including timestamp and corresponding road segment traffic) or road segment data (including road segment attributes, topology association and traffic data) through data tags. Data value assessment: Information gain calculation: The information gain of new data and existing data is calculated using the mutual information formula. , .

[0062] Information gain between new and existing data (unitless, range of values) The larger the value, the greater the supplementary value of the new data to the existing data. For existing data Information entropy (unitless, range of values) Characterizing the uncertainty of existing data. ); Given new data Data already available Conditional entropy (unitless, range of values) This characterizes the remaining uncertainty of existing data after the addition of new data. ); The dataset is an existing dataset (unitless, consisting of historical multi-source sensing data). For newly added datasets (unitless, consisting of data from the date of addition or road segment data); for Values and Values The joint probability (unitless, ranging from [0,1]); for Values hour Values The conditional probability (unitless, range [0,1]).

[0063] Distribution similarity calculation: KL divergence is used to calculate the distribution similarity between new data and existing data. , .

[0064] in: for Divergence (unitless, range [0, +∞)) represents the distribution of existing data. Distribution of New Data The degree of difference (the larger the value, the greater the difference). The probability distribution of the existing data (unitless, satisfying...) , ); The probability distribution of the newly added data (unitless, satisfying...) ); The feature values ​​for the data sample are unitless and can be continuous or discrete depending on the data type. The feature values ​​in the existing data are... The probability (unitless, range [0,1]); The feature values ​​in the newly added data are The probability (unitless, range [0,1]); It is the natural logarithm function (unitless).

[0065] Triggering condition: When or If the incremental data is deemed to be of high value, an adaptive optimization process is triggered; otherwise, only the data storage layer is updated, and the model parameters are not adjusted.

[0066] (2) Adaptive optimization of newly added date data Time feature update: Automatically extract time features of newly added date data, including intraday peak period shifts, intraweek traffic fluctuation patterns, seasonal trends (such as adding winter data to supplement winter traffic features), and special holiday-specific features (such as Spring Festival travel rush traffic features), and update the time feature library; Incremental parameter tuning: For the LSTM layer of the spatiotemporal classifier, only the weight parameters related to the newly added time features are updated, and the number of iterations is 1 / 4 of that of the full training. Adjusting the time correction factor of the flow estimation algorithm Adjustment factor for AOI activity intensity The optimal value is recalculated based on the new data; Historical parameter retention: A parameter weighted fusion strategy is adopted, with the weight of the new parameter = 0.3 × new training parameter + 0.7 × historical best parameter, to avoid parameter oscillation due to a small amount of new data.

[0067] (3) Adaptive optimization of newly added road segment data Road network structure update: Automatically add new road segments to the weighted directed graph model and calculate the topological correlation between the new road segments and adjacent road segments. Update the edge weight matrix; Feature parameter transfer: Reusing adjacent road segments (topological correlation) through transfer learning The node feature vectors, diversion weight parameters, and AOI association mapping relationships are used as the initial parameters for the new road segments. Rapid adaptation training: Use a small amount of data (≥3 days) of newly added road segments for local training, only optimize the GNN graph convolution kernel parameters and cell partitioning coefficients of the cell transfer model that are directly related to the newly added road segments, and control the training time to within 1 / 10 of the full training. Topology synchronization: Update the set of adjacent road segments in the diversion and merging model, and recalculate the diversion weights of newly added road segments and surrounding road segments. This ensures the consistency of traffic flow transmission across the road network.

[0068] (4) Optimization effect verification Incremental validation set construction: Randomly select 30% of the new data as the incremental validation set, and the remaining 70% as the incremental training set; Evaluation metrics calculation: Calculate the MAPE, MAE, and incremental optimization metrics (parameter update efficiency, accuracy retention rate) of the incremental validation set. Result determination: If MAPE decreases by ≥8% compared to before optimization and parameter update efficiency is ≤20% (update time / full training time ≤0.2), then the optimization result is retained; otherwise, the model parameters before optimization are rolled back, and only the newly added data is added to the data storage layer, and optimization is triggered again after more incremental data is accumulated in the future.

[0069] 5. Conduct full-scenario verification and adaptive optimization (1) Full coverage design of test scenarios Missing data types: random missing data, consecutive missing data (1 hour, 2 hours, 4 hours), and regional missing data (commercial areas, transportation hubs, suburbs). Missing rates: 30%, 50%, 70%, 90%; Date type: Weekdays (Monday to Friday), Holidays (Weekends, Statutory Holidays, Special Holidays such as Spring Festival and National Day); Special traffic scenarios: traffic accidents (with varying durations and ranges of impact), extreme weather (heavy rain, dense fog, high temperatures), and large-scale events (concerts, sporting events, exhibitions). Data increment scenarios: adding date data for 1 month / 3 months / 6 months, adding data for 5 / 10 / 15 road segments; Data source: Based on real data of 60 main and secondary roads in the Shanghai World Expo area, a test set was constructed by combining simulated data, including 1 year of 5-minute granular PCU data, POI / AOI data, and real-time sensing data.

[0070] (2) Multi-dimensional evaluation index system Basic indicators: MAE (Mean Absolute Error): RMSE (Root Mean Square Error): MAPE (Mean Absolute Percentage Error): WMAPE (Weighted Average Absolute Percentage Error): Contextualized metrics: Peak volatility: (Morning and evening rush hours are calculated separately) Anomaly response accuracy: Percentage of time periods with prediction errors ≤10% in special scenarios Successive missing value completion accuracy: The percentage of consecutive missing value periods in which the predicted value and the actual value have a similarity of ≥85%. Special scenario adaptability: MAPE increase of ≤20% in special scenarios compared to regular scenarios. Incremental optimization metrics: Parameter update efficiency: (Target ≤ 20%) Accuracy retention rate: (Target ≤ 90%) Data adaptation efficiency: (Target ≤ 0.5 hours / 10,000 records) (3) Adaptive optimization strategy Hyperparameter optimization: A Bayesian optimization algorithm is used to optimize the weighted score of the scenario-based indicator and the incremental optimization indicator (basic indicator weight 0.4, scenario-based indicator weight 0.3, incremental optimization indicator weight 0.3). The optimized hyperparameters include the number of GNN layers, the number of hidden layer nodes of the RBF neural network, the initial learning rate, the regularization coefficient, the incremental trigger threshold, etc. The number of iterations is set to 100, and the optimal combination of hyperparameters is output. Transfer learning adaptation: For new regions or new scenes, the transfer learning method is used to use the pre-trained model parameters as initial values ​​and fine-tune them using a small amount of data from the new region / new scene, thereby shortening the model training time and improving generalization ability. Model update mechanism: A real-time data monitoring module is established. When the MAPE exceeds the threshold (20%) for three consecutive time periods, incremental training of the model is triggered to optimize the model parameters based on the latest data, ensuring the long-term stability of the model.

[0071] Example 3 This embodiment provides a specific implementation case of a spatiotemporal completion method for traffic flow data in a big data environment, as described in Embodiments 1 and 2, specifically including: The study focused on the Shanghai World Expo area, which includes more than 60 main and secondary roads and covers various functional areas such as commercial areas, residential areas, transportation hubs, and scenic spots. The data missing rate of traffic flow data for some road sections is as high as 70%, and there are special traffic scenarios such as traffic accidents and large-scale exhibitions. Subsequently, data for 10 branch roads were added due to the expansion of the road network, and data for 6 months of dates (including special holidays such as Spring Festival and National Day) were added due to data accumulation.

[0072] The implementation steps include: 1. Data collection and preprocessing: Collect the initial one-year road vector data, POI / AOI data (including 3000+ POIs and 50+ AOIs), video surveillance traffic data, floating car data, meteorological data, and traffic event data for the region. Perform data cleaning, normalization, and spatiotemporal alignment to construct an initial multi-source dataset. 2. Initial model construction and training: A regional traffic flow intelligent network model is constructed according to the method of this invention. An adaptive traffic flow estimation algorithm and a multi-dimensional deviation correction mechanism are designed. The model is trained using the initial dataset, and the hyperparameters are optimized through Bayesian optimization. 3. Initial Model Validation: Test scenarios with different missing types, missing rates, date types, and special scenarios were set up. Multi-dimensional evaluation metrics were used to validate the performance of the initial model. The MAPE of the initial model was 13.5% at a 70% missing rate. 4. Incremental data access and optimization: Adding 6 months of new date data: automatically identified as date data, calculated information gain = 0.42 (≥0.3), triggering the optimization process, updating the time feature library (supplementing Spring Festival travel traffic features), fine-tuning the spatiotemporal classifier parameters, the optimization time is only 15% of the full training time; Ten new branch road data entries were added: they were automatically identified as road segment data, and the distribution similarity was calculated to be 0.78 (≥0.7). This triggered the optimization process, updated the road network topology, and reused parameters from adjacent road segments through transfer learning, resulting in rapid adaptation. The training time was reduced to 8% of the time required for full training. 5. Validation after incremental optimization: Conduct full-scenario testing on the optimized model and evaluate the incremental optimization metrics. 6. Actual Deployment and Application: The optimized model is deployed to the regional traffic data processing platform to receive multi-source sensing data and incremental data in real time, automatically trigger the optimization process, complete missing traffic flow data, and output complete PCU data with a 5-minute granularity.

[0073] The implementation results are as follows: 1. Significantly improved completion accuracy: With a missing rate of 70%, the MAPE decreased to 11.2% after incremental optimization, a 17% improvement compared to the initial model; in scenarios with continuous missing data for 4 hours, the completion accuracy reached 88%. 2. Excellent incremental optimization results: After optimizing the data for new dates, the volatility during peak holiday periods decreased from 8.5% to 6.2%, parameter update efficiency reached 15%, and accuracy retention rate reached 83%. After optimizing the data for the newly added road sections, the MAPE of the newly added branch roads is 12.8% (consistent with the accuracy of the original road sections), and the data adaptation efficiency is 0.3 hours / 10,000 records, which meets the needs of engineering applications. 3. Excellent adaptability to special scenarios: The anomaly response accuracy reaches 92% in traffic accident scenarios, the volatility during peak periods of large-scale exhibitions is only 6.8%, and the MAPE increase is ≤15% in rainy weather; 4. Significant Application Results: Based on the supplemented traffic flow data, after optimizing the regional traffic signal timing, the average travel time on main and secondary roads decreased by 18.5%, and the number of stops decreased by 27.3%; after the newly added road sections were included in the management, the traffic efficiency of the surrounding road network improved by 15%; and it provided accurate data support for regional road network planning, improving the traffic efficiency of the planned scheme by 22%. 5. Outstanding engineering value: After deployment, the model can automatically adapt to missing data and incremental data access in different road segments. When deployed across regions, it can complete the adaptation in just 3 days through transfer learning, which is 70% shorter than the deployment cycle of traditional models. Incremental data optimization does not require manual intervention, which greatly reduces operation and maintenance costs.

[0074] Implementation Conclusion: This invention achieves high-precision spatiotemporal completion of traffic flow data in scenarios with high missing rates, complex traffic conditions, and incremental data updates through deep fusion of multi-source data, a hybrid modeling architecture, a dynamic correction mechanism, and incremental data-driven optimization. The core innovative incremental optimization mechanism effectively solves the model adaptation problem caused by road network expansion and data accumulation, significantly improving the long-term practicality and engineering value of the model. Implementation results in the Shanghai World Expo area demonstrate that this method offers high completion accuracy, strong robustness, and fast adaptation efficiency, providing strong data support for urban traffic management, road network planning, and the construction of intelligent transportation systems.

[0075] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method for spatiotemporal completion of traffic flow data in a big data environment, characterized in that, include: Step 1: Collect road vector data, points of interest and areas of interest information to characterize the distribution of urban functions and activity intensity, and real-time multi-source sensing data, and perform data preprocessing; Step 2: Convert the road vector data into a weighted directed graph; establish the spatial association between the points of interest and interest surfaces and road segments based on the KNN algorithm; match and fuse the weighted directed graph and multi-source sensing data based on the spatial association. Step 3: Use an attention mechanism to extract features from the fused data in both time and space dimensions to form preliminary feature vectors; based on the preliminary feature vectors, analyze the weighted directed graph and time series data respectively to obtain spatial feature vectors and time feature vectors representing the global spatial correlation of the road network, and then feed them into the pre-built fusion prediction model to obtain preliminary traffic flow prediction values. Step 4: Based on the preliminary traffic flow forecast, dynamically calculate the traffic flow distribution ratio at each intersection, and according to the law of conservation of traffic flow, iteratively calculate the movement and accumulation of vehicles between each road segment based on the traffic flow distribution ratio until the traffic flow of the entire road network reaches dynamic equilibrium, and output the final traffic flow forecast.

2. The method for spatiotemporal completion of traffic flow data in a big data environment according to claim 1, characterized in that, The formula for dynamically calculating the traffic flow distribution ratio at each intersection is as follows: in, For road section to section of road The traffic distribution weight, For road section to section of road The number of lanes, For road section to section of road Speed ​​limit, For road section to section of road The congestion attenuation coefficient, Let i be the index of the adjacent road segment. For road section to adjacent road section The number of lanes, For road section to adjacent road section Speed ​​limit, For road section to adjacent road section The congestion attenuation coefficient, For road section to section of road Historical average flow For road section to section of road Traffic capacity, For road section to section of road Real-time congestion level.

3. The method for spatiotemporal completion of traffic flow data in a big data environment according to claim 1, characterized in that, The iterative calculation process specifically includes: based on the law of conservation of traffic flow and the traffic flow allocation ratio, dividing each road segment into several cells, and adaptively adjusting the cell size according to the road segment length and real-time traffic density. The cell length calculation formula is as follows: in, The length of the cell. For road section Length, For road section Real-time traffic density, For road section Traffic capacity, It is a rounding function; Within each time step, calculate the inflow and outflow traffic flow of each cell based on the traffic flow allocation ratio, and update the number of vehicles in each cell. The updated formula is: in, for Time Cell The number of vehicles, for Time Cell The number of vehicles, for Inflow of time into cells Traffic flow for Cells constantly flowing out Traffic flow; Repeat the above update process until the traffic flow status of all road segments in the entire road network reaches a dynamic equilibrium, that is, the traffic flow change between adjacent iteration steps is less than the preset convergence threshold, and then output the final traffic flow prediction value.

4. The method for spatiotemporal completion of traffic flow data in a big data environment according to claim 3, characterized in that, The final traffic flow prediction is also constrained by the real-time activity intensity of the interest surface and special scenarios, by... The flow update formula, incorporated into the cellular transport model, dynamically constrains the real-time activity intensity of the interest surface on flow generation or attraction. The calculation formula is as follows: in, For road section The basic traffic generation / attraction coefficient, The influence weighting coefficient of AOI activity intensity For road section Surrounding POI density, For the road section The associated real-time activity intensity of AOI, This is a time correction factor; The constraints of the special scenario are determined by the interference coefficient. Characterization, The initial traffic flow prediction is corrected by multiplication, and the result has different values ​​under different weather or scenarios.

5. The method for spatiotemporal completion of traffic flow data in a big data environment according to claim 1, characterized in that, The fusion prediction model consists of a pre-built graph neural network model and a temporal convolutional network model; the regional traffic flow intelligent network model is formed by fusing the pre-built fusion prediction model, dynamic diversion and merging model and cell transmission model; and the regional traffic flow intelligent network model is trained using a dynamic hierarchical masking mechanism and a multi-dimensional deviation correction mechanism. The graph neural network model is used to analyze the weighted directed graph to obtain spatial feature vectors representing the global spatial correlation of the road network; the temporal convolutional network model is used to analyze the time series data to obtain temporal feature vectors. The dynamic diversion and merging model is used to dynamically calculate the traffic flow distribution ratio at each intersection based on the preliminary traffic flow prediction value. The cell transport model is used to iteratively calculate the movement and accumulation of vehicles between road segments based on the traffic flow allocation ratio, and output the final traffic flow prediction value. The dynamic layered masking mechanism specifically includes: Masking is used to represent missing data on roads; roads are randomly or continuously masked based on their importance scores, with higher importance scores resulting in lower masking probabilities; all roads within a random specific area are masked according to a pre-set missing rate. The importance score of the road is calculated using a weighted summation formula: Where S represents the road importance score, This is a weighting coefficient for road grade. The weighting coefficient is the average daily traffic volume. The weighting coefficients for regional functional dependence. To normalize the road grade score, The normalized score for average daily traffic. The score represents the regional functional dependence; the constraints are as follows: .

6. The method for spatiotemporal completion of traffic flow data in a big data environment according to claim 5, characterized in that, The multi-dimensional deviation correction mechanism specifically includes: The prediction bias generated during the training of the regional traffic flow intelligent network model is subjected to residual fitting using a pre-built radial basis function neural network model, and a bias compensation term is generated. The calculation formula is as follows: in, In a radial basis function neural network, the first... The output value of the activation function of each hidden layer neuron. For indexing hidden layer neurons, for The input feature vector at time t, For the first The center point of each radial basis function For the Euclidean norm, for Time of the first The width parameter of each radial basis function. For the first The initial width of the radial basis functions, This represents the number of iterations. Furthermore, based on the aforementioned deviation compensation term, the parameters of the regional traffic flow intelligent network model are updated using the improved Gauss-Seidel iterative method, with the update formula as follows: in, For the first After the second iteration, the road section and The corresponding model weight parameters, For the first After the second iteration, the road section and The corresponding model weight parameters, for Adaptive learning rate at any given time. The model loss function, For the loss function with respect to the weight parameters The partial derivatives, This is the deviation compensation coefficient. for Time and Section Deviation compensation item, The regularization coefficient is . Let be the mean of all weight parameters. for Norm, for Adaptive learning rate at any given time. This is the initial learning rate.

7. The method for spatiotemporal completion of traffic flow data in a big data environment according to claim 5, characterized in that, The regional traffic flow intelligent network model receives incremental data in real time and triggers an adaptive optimization process, which specifically includes: When the incremental data is date data: extract the time features of the newly added date data and update the time feature library; update the weight parameters related to the newly added time features in the Long Short-Term Memory network model; adjust the time correction coefficient in the regional traffic flow intelligent network model. Adjustment coefficient for activity intensity related to interests Furthermore, a parameter-weighted fusion method is used to adjust the key parameters in the regional traffic flow intelligent network model, retaining historical parameters according to a pre-set weight ratio; the specific calculation formula in the parameter-weighted fusion method is as follows: in, For the new parameters after fusion, These are the weighting coefficients for historical parameters. These are the model's historically optimal parameters. The weight coefficients for training parameters on newly added data. The parameters are obtained by training based on the newly added data, and the constraints are as follows: ; When the incremental data is road segment data: the new road segment is automatically added to the weighted directed graph and the edge weight matrix is ​​updated; the node feature vectors, diversion weight parameters, and interest surface association mapping relationships of adjacent road segments are reused through transfer learning as the initial parameters of the new road segment; Local training is performed using data from newly added road segments, optimizing only the graph convolution kernel parameters of the graph neural network model directly related to the newly added road segments and the cell partitioning coefficients of the cell transfer model; the diversion weights of the newly added road segments and surrounding road segments in the dynamic diversion and merging model are updated.

8. The method for spatiotemporal completion of traffic flow data in a big data environment according to claim 7, characterized in that, The adaptive optimization process is triggered by triggering conditions, which specifically include: calculating the information gain between incremental data and existing data using the mutual information formula, and calculating the distribution similarity between incremental data and existing data using KL divergence. The calculation formula is: in, for divergence, The probability distribution of the existing data. For the probability distribution of incremental data, To assign feature values ​​to the data samples, The feature values ​​in the existing data are... The probability, For the feature values ​​in the incremental data The probability of; When the information gain is greater than a preset value or the distribution similarity is greater than a preset value, the adaptive optimization process is triggered. The mutual information formula is: in, The information gain between incremental data and existing data. For existing data Information entropy Given incremental data Data already available conditional entropy, For an existing dataset, For incremental datasets, for Values and Values The joint probability, for Values hour Values The conditional probability.

9. The method for spatiotemporal completion of traffic flow data in a big data environment according to claim 1, characterized in that, The formula for calculating the edge weights of the weighted directed graph is as follows: in, For road section and road section Edge weights between them For road section and road section common similarity, For road section and road section The degree of topological association, , These are the weight coefficients for common similarity and topological affinity, respectively, and satisfy the following conditions: ; The real-time multi-source sensing data specifically includes video surveillance traffic, floating car data, meteorological data, and traffic incident information.

10. A spatiotemporal completion system for traffic flow data in a big data environment, characterized in that, It includes a memory and a processor, the memory storing a computer program, the processor invoking the computer program to perform the steps of the method as described in any one of claims 1 to 9.

Images