Highway traffic space-time service platform and method based on multi-source heterogeneous data fusion

Abstract

The invention provides a road traffic space-time service platform and method based on multi-source heterogeneous data fusion. The platform comprises a data acquisition module; a data fusion management module; the base map making module is used for generating a road traffic space-time service base map taking a road intersection as a key node; the traffic situation analysis module is used for forming a traffic situation prediction map; and the decision support and personalized service module is used for providing real-time and visual traffic situation display, traffic guidance strategies and personalized travel services for traffic managers and traffic participating users based on the traffic situation prediction map. According to the invention, real-time and visual decision support is provided for traffic managers, and personalized travel schemes can be customized for traffic participating users, so that comprehensive perception and accurate regulation and control of road traffic operation states are realized, and the intelligent level of traffic operation management is greatly improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent transportation, and particularly relates to a highway traffic spatio-temporal service platform and method based on multi-source heterogeneous data fusion. Background Art

[0002] With the continuous improvement of the highway network, the high-efficiency and intelligence of highway traffic operation management have become the key to the development of the transportation field. However, with the continuous and rapid growth of traffic demand, the traditional management mode is facing increasingly severe challenges: road congestion, frequent accidents, frequent extreme weather and emergencies, resulting in an urgent need to improve the road network management ability. The existing highway traffic spatio-temporal service management mode has problems such as single data source, insufficient information collection, limited analysis dimension and lagging decision support, and it is difficult to accurately, real-time and personalized reflect the traffic operation state.

[0003] Specifically, there are still obvious gaps in the fusion research of geographic information data and spatio-temporal information services in the existing system. The data at all levels of highway traffic are not fully combined, and there is a lack of multi-source and dynamic integration of road, vehicle and roadside information; in the two dimensions of time and space, the data processing and comprehensive analysis are insufficient, and it is impossible to finely depict indicators such as traffic flow, vehicle speed, and congestion index, resulting in managers being difficult to make timely and accurate decisions. In addition, traditional platforms often operate separately, lacking a unified data sharing and management mechanism, and it is impossible to achieve information interconnection and personalized service customization between traffic managers and users. Summary of the Invention

[0004] The purpose of the present invention is to solve the deficiencies existing in the above background art, and provide a highway traffic spatio-temporal service platform and method based on multi-source heterogeneous data fusion, which not only provides real-time and visual decision support for traffic managers, but also can customize personalized travel plans for traffic participating users, so as to achieve a comprehensive perception and precise regulation of the highway traffic operation state, and greatly improve the intelligent level of traffic operation management.

[0005] The technical solution adopted by the present invention is: a highway traffic spatio-temporal service platform based on multi-source heterogeneous data fusion, including:

[0006] A data acquisition module, configured to acquire highway traffic data including satellite remote sensing data, UAV image data, vehicle-mounted Beidou GNSS trajectory data, and roadside equipment.

[0007] A data fusion management module, configured to construct a hierarchical highway traffic spatio-temporal service data warehouse by preprocessing, data matching, integration and representation of the acquired data, and realize the unified management of multi-source heterogeneous data.

[0008] The base map production module is used to extract road network information based on a data warehouse by using deep learning, edge detection, and trajectory analysis methods, and perform weighted fusion based on the weights corresponding to each extraction result to generate a road traffic spatio-temporal service base map with highway intersections as key nodes;

[0009] The traffic situation analysis module is used to perform time series hierarchical analysis and convolutional neural network prediction on traffic flow, vehicle speed, and congestion index in the data warehouse. Based on the road traffic spatio-temporal service base map, clustering analysis is performed in combination with the radiation area with highway intersections as the reference points to detect abnormal traffic events, and a traffic situation prediction map is formed;

[0010] The decision-making support and personalized service module is used to provide real-time and visual traffic situation display, traffic guidance strategies, and personalized travel services to traffic managers and traffic participating users based on the traffic situation prediction map.

[0011] In the above technical solution, the base map production module uses remote sensing satellite image data and a trained deep learning model to extract the road network; uses unmanned aerial vehicle aerial photography image data and an image edge detection algorithm model to extract the road network; uses vehicle Beidou GNSS trajectory data and extracts the road network structure through a trajectory clustering and rasterization model.

[0012] In the above technical solution, the calculation process of the weights corresponding to the extraction results of each road network extraction model includes:

[0013] Select a certain number of road network true samples to form a sample data set;

[0014] Divide the sample data set into K mutually exclusive sub-sample sets; and further divide the sample set into a training sample set and a validation sample set;

[0015] Based on the mutually exclusive sub-sample sets corresponding to each model, accuracy data sets including user accuracy and producer accuracy are respectively obtained under K-fold cross-validation;

[0016] Calculate the coefficient of variation of each model's accuracy data set respectively;

[0017] Perform summation normalization calculation on the coefficient of variation of each model's accuracy data set to obtain the weights corresponding to each model.

[0018] In the above technical solution, the base map production module extracts highway intersections based on the road network data of the road network generated by weighted fusion of each extraction result; matches and adds radar, cameras, and traffic sign elements to the base map according to their positional relationships with the highway intersections and the road network, and realizes the construction and visual display of the road traffic spatio-temporal service base map.

[0019] In the above technical solution, the process of extracting highway intersections includes: extracting all road centerlines from the fused road network and obtaining the intersection points of the centerlines as candidate intersections; using the on-vehicle Beidou GNSS trajectory data to perform trajectory concentration analysis on the candidate intersection points to filter out pseudo-intersection points caused only by positioning errors or data drift; verifying the positions of the remaining intersection points by combining the spatial features of satellite remote sensing data and UAV image data, and finally identifying the set of real highway intersection nodes.

[0020] In the above technical solution, the traffic situation analysis module obtains the traffic situation prediction values at each coordinate position on the highway traffic spatio-temporal service base map through weighted calculation based on the predicted traffic flow, vehicle speed, and congestion index.

[0021] In the above technical solution, the process of detecting abnormal traffic events includes:

[0022] For each highway intersection, it spreads outwards with different set radii to form different radiation areas corresponding to each highway intersection;

[0023] For each radiation area, based on the corresponding traffic situation prediction value, perform kernel density analysis on different time scales to obtain the time and location where the abnormal occurrence probability of the radiation area is greater than the set threshold.

[0024] In the above technical solution, the decision support and personalized service module visualizes the traffic situation base map and prediction results for traffic managers; the intelligent decision-making engine automatically generates traffic guidance and control strategies based on the real-time traffic information and traffic situation prediction results, combined with vehicle speed changes, traffic flow fluctuations, and accident alerts, and sends the strategies to road participants and relevant management terminals in the forms of broadcast prompts, variable message sign information release, toll station control, traffic flow diversion, etc., to implement highway traffic operation regulation decisions.

[0025] In the above technical solution, the decision support and personalized service module shares traffic management decision information to the user mobile terminal through the Internet information publishing middleware of the platform for traffic participating users, and provides personalized travel route planning suggestions and traffic information push services based on the travel preference model established by combining the user's historical travel data; at the same time, it receives the real-time road condition information or personalized demand requests fed back by the user terminal, and uploads them back to the data warehouse to update the highway traffic spatio-temporal service base map and the prediction model in the traffic situation analysis module.

[0026] The present invention provides a highway traffic spatio-temporal service method based on multi-source heterogeneous data fusion, including the following steps:

[0027] Collect highway traffic data including satellite remote sensing data, UAV image data, on-vehicle Beidou GNSS trajectory data, and roadside equipment.

[0028] By preprocessing, data matching, integrating, and representing the collected data, a hierarchical highway traffic spatio-temporal service data warehouse is constructed to achieve unified management of multi-source heterogeneous data;

[0029] Based on the data warehouse, deep learning, edge detection, and trajectory analysis methods are used to extract road network information, and weighted fusion is performed based on the weights corresponding to each extraction result to generate a highway traffic spatio-temporal service base map with highway intersections as key nodes;

[0030] Perform time-series hierarchical analysis and convolutional neural network prediction on traffic flow, vehicle speed, and congestion index in the data warehouse. Based on the highway traffic spatio-temporal service base map, clustering analysis is carried out in combination with the radiation area with highway intersections as the reference points to detect abnormal traffic events, forming a traffic situation prediction map;

[0031] Based on the traffic situation prediction map, provide real-time and visual traffic situation display, traffic guidance strategies, and personalized travel services to traffic managers and traffic participating users

[0032] The beneficial effects of the present invention are as follows: The present invention gives full play to the advantageous effects of the "space-air-ground" perception technology in intelligent transportation, and combines data fusion management technologies and methods to create a hierarchical, safe, and efficient data warehouse; Based on multi-source highway traffic heterogeneous data and machine learning technologies, a highway traffic spatio-temporal service platform and a special highway base map of the system with intersections as key nodes are constructed, which have advantages such as high accuracy and complete data structure; A dynamic highway traffic time and space situation clustering analysis system is constructed to comprehensively consider traffic conditions in terms of time and space dimensions, and information visualization in the base model of the highway traffic spatio-temporal service platform is realized. This platform and method are fully oriented to traffic managers and individual traffic users, realizing the sharing of information and the timeliness of data acquisition between the two.

[0033] Furthermore, the base map production module of the present invention extracts road network information by using remote sensing satellite images (using deep learning models), UAV images (using edge detection algorithms), and vehicle Beidou GNSS trajectory data (using trajectory clustering and rasterization methods) respectively, thereby improving the accuracy of road network extraction and the integrity of the data structure. The multiple extraction methods complement each other, making full use of their respective advantages to generate high-precision and topologically coherent road network data, solving the problem of insufficient extraction accuracy in the prior art; Using different data sources to verify and supplement each other to ensure that the road network information is both comprehensive and accurate, thus providing a solid foundation for subsequent spatio-temporal situation analysis.

[0034] Furthermore, in the base map production module of the present invention, the weight calculation of the results of each road network extraction model adopts the following method: select real road network samples, divide them into K mutually exclusive sub-sample sets, use K-fold cross-validation to obtain the user accuracy and producer accuracy of each model, calculate the coefficient of variation and then normalize it, and finally determine the weights of each model; through the adaptive weighted fusion method, the prediction results of different models are automatically assigned appropriate weights according to the accuracy, so as to improve the accuracy and consistency of the overall road network fusion; use K-fold cross-validation and statistical indicators (such as the coefficient of variation) to objectively assign weights to the model weights, making the base map production process have sufficient publicity and feasibility, and meeting the requirements of conventional technologies in this field.

[0035] Furthermore, the base map production module of the present invention extracts road intersections from the weighted fusion road network data, and matches and adds elements such as radars, cameras, and traffic signs to the base map according to their spatial positions to realize the construction and visual display of the base map; by extracting this key node of the intersection and superimposing traffic facility information, the formed base map not only has a fine road structure, but also can intuitively reflect the important facilities in the road network, meeting the needs of traffic management and services; it realizes the organic combination of the static road network and the dynamic traffic element information, providing an intuitive and reliable spatial basis for subsequent traffic situation analysis and decision support.

[0036] Furthermore, the present invention extracts the intersections of all road centerlines from the fused road network as candidate intersections, then uses in-vehicle Beidou GNSS trajectory data for trajectory aggregation analysis to filter out false intersections, and combines the spatial characteristics verification of satellite and UAV images to finally obtain a set of real intersection nodes; through multi-data source verification and clustering analysis to filter out false intersections caused by positioning errors, ensuring that the extracted intersections truly reflect the key nodes of the road network and further improving the accuracy of subsequent traffic situation prediction; the real intersection nodes as the benchmark for spatial analysis help to more accurately construct the radiation area during anomaly detection and form a scientific spatial clustering analysis system.

[0037] Furthermore, the traffic situation analysis module of the present invention uses the predicted traffic flow, vehicle speed, and congestion index to obtain the traffic situation prediction value of each coordinate position on the highway traffic spatio-temporal service base map through weighted calculation; perform weighted calculation on each coordinate position, making the numerical distribution of the traffic situation more finely and truly reflect the local traffic state; the fine-grained traffic situation prediction value can provide more accurate traffic condition distribution information for managers, helping to identify possible congestion or accident risk areas in advance.

[0038] Furthermore, the present invention sets different radii for diffusion centered on each highway intersection to form different radiation regions; for each radiation region, kernel density analysis is performed at different time scales to obtain the time and location where the probability of anomaly occurrence is greater than the set threshold; by constructing the radiation region with the intersection as the reference point, the diffusion effect of the traffic flow around the intersection can be captured, and traffic anomalies can be detected in a timely manner through multi-scale kernel density analysis, significantly improving the accuracy of accident or congestion warning; identifying high-risk regions and time periods in advance provides a basis for traffic managers to implement preventive measures, thereby effectively reducing the probability of accidents and alleviating traffic congestion.

[0039] Furthermore, the decision support and personalized service module of the present invention is targeted at traffic managers. Through intuitive visual display and automated decision generation, traffic managers can quickly grasp the changes in road conditions, formulate emergency measures in a timely manner, and effectively respond to traffic emergencies; issuing traffic guidance information through multiple channels ensures that the information is quickly transmitted to each intersection and traffic participants, optimizing resource allocation and improving the overall traffic management efficiency.

[0040] Furthermore, the decision support and personalized service module of the present invention is targeted at traffic participating users, realizing information sharing between traffic managers and users: users receive traffic decisions and road condition information in real time through mobile terminals, improving the timeliness and coverage of information transmission; combined with the user's travel history and real-time feedback, the system can continuously optimize route planning and traffic information push to achieve user-centered personalized travel services; user feedback data is used to update the traffic situation analysis model and base map data, continuously improving the prediction accuracy and response ability of the platform, thereby realizing intelligent dynamic traffic management. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is the overall architecture diagram of the platform and system of the present invention.

[0042] Figure 2 is the schematic diagram of the integration and management of multi-source heterogeneous data in highway traffic.

[0043] Figure 3 is the schematic diagram of the extraction of road network intersections in the highway traffic basic base.

[0044] Figure 4 is the schematic diagram of highway traffic spatio-temporal situation data. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0045] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, which are convenient for clearly understanding the present invention, but they do not constitute a limitation to the present invention.

[0046] Embodiment 1

[0047] As Figure 1As shown in the figure, the present invention provides a road traffic spatio-temporal service platform based on multi-source heterogeneous data fusion, including:

[0048] A data acquisition module, configured to acquire road traffic data including satellite remote sensing data, UAV image data, vehicle-mounted Beidou GNSS trajectory data, and road-side equipment-acquired road traffic data;

[0049] A data fusion management module, configured to construct a hierarchical road traffic spatio-temporal service data warehouse by preprocessing, data matching, integration, and representation of the acquired data, so as to realize unified management of multi-source heterogeneous data;

[0050] A base map production module, configured to extract road network information based on the data warehouse by using deep learning, edge detection, and trajectory analysis methods, and perform weighted fusion based on the weights corresponding to the extraction results, so as to generate a road traffic spatio-temporal service base map with road intersections as key nodes;

[0051] A traffic situation analysis module, configured to perform time-series hierarchical analysis and convolutional neural network prediction on traffic flow, vehicle speed, and congestion index in the data warehouse, and perform clustering analysis based on the road traffic spatio-temporal service base map in combination with the radiation area with the road intersection as the reference point to detect abnormal traffic events, so as to form a traffic situation prediction map;

[0052] A decision-making support and personalized service module, configured to provide real-time and visual traffic situation display, traffic guidance strategies, and personalized travel services to traffic managers and traffic participating users based on the traffic situation prediction map.

[0053] Preferably, the generated data types of the multi-source data acquisition module are original road traffic data from different sources, including high-resolution remote sensing satellite images, UAV aerial images, trajectory data generated by vehicle Beidou GNSS terminals, and dynamic monitoring data such as vehicle speeds, traffic flow counts, and video images acquired by road-side cameras and radars. After preprocessing such as format conversion and noise filtering, the data from these high-altitude, aerial, and ground sources are normalized and output. Subsequent use: The multi-source data output by the acquisition module will be transmitted to the data fusion management module as the input basis for fusion processing. For example, remote sensing images and UAV images will be used for road network extraction in subsequent base map production, and GNSS trajectory data and camera traffic data will be used for traffic flow feature calculation in situation analysis.

[0054] The generated data type of the data fusion management module is the integrated highway traffic comprehensive database after cleaning, alignment, and integration. This module unifies the formats, aligns the time and space, and eliminates redundancy for heterogeneous data from satellites, drones, vehicle terminals, and roadside devices, outputs a structured fusion data set, and stores it in the cloud data warehouse (divided into different levels, such as the raw data layer, processed data layer, and analyzed data layer). Subsequent use: The fused high-quality data serves as the unified data source for subsequent modules. On the one hand, the base map production module obtains the fused remote sensing images, aerial images, and trajectory data from the data warehouse for fine road network extraction and feature overlay; on the other hand, the situation analysis module extracts the fused real-time and historical traffic index data (such as flow, speed) from the warehouse for calculating the current traffic state and training prediction models. Therefore, the data fusion management module transforms the originally scattered data into high-quality inputs that can be directly utilized by the base map production and situation analysis, improving the efficiency and accuracy of subsequent processing.

[0055] The generated data type of the base map production module is the high-precision highway traffic spatio-temporal service base map data. This base map includes the detailed highway road network structure (the road centerlines and connection relationships represented in vector form), the key nodes of the extracted road intersections, and the information of traffic sensing devices and traffic signs overlaid on it. The base map data contains both static road geometry and topological information and is also associated with the locations of relevant traffic infrastructure. The base map serves as the basis for situation analysis and decision-making display. In the situation analysis module, the base map provides a spatial reference framework - the analyzed traffic flow, congestion, and other indicators are mapped to the corresponding road segments or nodes on the base map, thus generating an intuitive traffic situation distribution map (such as rasterized display of congestion levels). In particular, the intersection nodes are used as the benchmark units for spatial aggregation and anomaly detection in situation analysis, and abnormal high-incidence areas are identified by comparing the traffic conditions in different intersection areas. Additionally, in the decision support module, the base map is presented as the visualization background on the monitoring platform to assist traffic managers in intuitively viewing the real-time road conditions and prediction results and more accurately deploying control measures based on the locations of the sensing devices on the base map. The high precision and element integrity of the base map ensure the accurate correspondence between the subsequent situation analysis results and the actual road environment.

[0056] The data types generated by the situation analysis module are the dynamic situation information and prediction results of road traffic, including the currently calculated traffic indicators (such as the real-time traffic flow, average vehicle speed, and congestion index of each road section), the predicted traffic status data for future time periods, and the detection and warning information for abnormal events (accidents, abnormal congestion). Formally, these data may be embodied as a set of numerical indicators, a traffic status matrix or raster map output by a prediction model, and a list of events marking abnormal locations. The results generated by the situation analysis will be directly used for decision support and user services. In the decision support module, the analyzed traffic situation data will be presented to traffic managers in the form of a chart and map overlay, helping them understand the operation status of the road network in the next few hours or days; at the same time, the location and severity information of abnormal events provide a basis for managers to formulate emergency plans or dispatch resources. In addition, the situation prediction results are also provided to ordinary traffic participants through the personalized service module. For example, the predicted road conditions are displayed in navigation applications, and users are reminded of accidents ahead. It can be said that the situation analysis module serves as a link between the upper and lower levels: it uses the basic data provided by the data warehouse and the base map for calculations, and feeds the obtained high-level information to the decision-making and service layers, driving the formulation of corresponding traffic control strategies and the adjustment of travel plans.

[0057] The data types generated by the decision support module are decision-making information and control instructions for road traffic management. Specifically, it includes a visual traffic situation panel (a monitoring screen generated by integrating the base map with real-time / predicted traffic data), a traffic control strategy plan given by an intelligent decision-making engine, and the finally issued decision commands or notifications (such as diversion instructions for congested road sections, warning broadcast content for accident road sections, etc.). These output data include both the display content of the human-computer interaction interface and the control parameters for the system to execute or broadcast. On the one hand, the output of the decision support module directly affects traffic managers through the platform interface - managers can confirm or adjust control strategies based on this; on the other hand, it is sent to road users and related devices through the communication network: for example, control instructions are issued to highway variable message signs, or pushed to navigation software to guide vehicles to detour. These decision-making information are also shared with the personalized service module, enabling ordinary users to timely learn about the regulatory measures of the management department (such as the closure of a toll station or traffic control on a certain road section), so as to adjust their travel plans. In addition, the feedback during the decision execution process (such as data on whether traffic congestion has eased after the implementation of control measures) will be returned to the situation analysis module as new input, further enriching the data in the data warehouse regarding the effects of management interventions and providing reference for the next decision, forming a closed-loop decision optimization process.

[0058] The generated data types of the personalized service module are customized service information for individual traffic participants and user feedback data. The former includes personalized travel route suggestions, real-time traffic reminders (such as accident warnings ahead and detour tips for congested sections), estimated arrival time updates, etc. generated based on the user's current location, destination orientation, and preferences, which are usually presented in the form of messages or notifications; the latter refers to the travel feedback data uploaded by the user terminal, such as road condition information reported by the user, route preferences, destination change requests, etc. The personalized service information is directly provided to the end user to help optimize their travel decisions and is the ultimate manifestation of the entire platform's external services. After the user feedback data is sent back to the platform, it will enter the front end of the data process: on the one hand, the real-time road conditions in the feedback will be supplemented to the multi-source data collection layer, enriching the perception of the current traffic state together with sensor data such as cameras and improving the accuracy of the situation analysis; on the other hand, the long-term feedback data of the user (such as travel habits and route selection) can be stored in the data warehouse for updating the user preference model and improving the personalization degree of future route recommendations. This feedback mechanism enables the platform to continuously calibrate the analysis model and service content, further improve the real-time update of the base map and situation analysis, and provide more accurate information support for managers and other users, forming a cyclic improvement data chain from data collection, analysis and decision-making to user application.

[0059] Specifically, the data collection module gives full play to the advantages of the "space-air-ground" perception technology in intelligent transportation. By obtaining high-spatial-resolution remote sensing image data, such as using satellite images (such as WorldView, QuickBird, and domestic high-resolution series, etc.), and performing preprocessing operations including radiometric calibration, atmospheric correction, geometric correction, cloud (snow) removal processing, fusion, mosaicking, and cropping, as well as image enhancement operations such as histogram equalization and contrast stretching, to obtain a high-quality image set.

[0060] Considering the limitation of the satellite image spatial resolution, its extraction effect for the main road network is good, but the extraction quality for the non-main road network is poor, so the UAV image information is further obtained. This data uses a UAV equipped with a high-resolution camera to conduct aerial photography according to the set flight path and overlap degree (usually 60-80% forward overlap and 20-40% side overlap), and further adopts preprocessing operations such as image stitching, orthorectification, and image enhancement.

[0061] In order to obtain highway traffic dynamic information, on the basis of obtaining the highway road network information, vehicle trajectories are further obtained by using in-vehicle Beidou GNSS terminals. The main hardware required for this process includes: in-vehicle Beidou GNSS terminals for receiving Beidou satellite signals and positioning and solving; antennas for receiving satellite signals, usually integrated with the terminal or externally connected; power supplies for powering the terminal; communication devices for transmitting the data collected by the terminal to the server; storage devices for locally storing trajectory data.

[0062] For a large amount of vehicle trajectory data obtained using in-vehicle Beidou GNSS terminals, it is first necessary to screen the vehicle trajectories, that is, to screen the vehicle trajectory data of vehicles that have been driving in a state close to the traffic flow; then, discrete trajectory points are removed, that is, trajectory points without a similar set of trajectory points are removed; finally, the trajectory points are encrypted, that is, linear encryption is used to improve the trajectory aggregation level to obtain the overall vehicle trajectory information.

[0063] Specifically, the data fusion management module is the management module of the entire platform and system. It uses satellites, drones, in-vehicle Beidou GNSS terminals, cameras, and radar highway traffic equipment to obtain spatio-temporal data and information on high-spatial-resolution images, trajectories, speeds, pictures, and videos. For the obtained multi-source heterogeneous highway traffic data, in order to solve problems such as information redundancy, data heterogeneity, time asynchrony, and position misalignment existing between the data, further data fusion and management are carried out.

[0064] Data fusion technology generally includes data preprocessing, data matching, data integration, and data representation. Data preprocessing includes cleaning data and converting data formats so that they can be further analyzed. Data matching involves identifying the relationships between data items from different sources. Data integration is to fuse the matched data together to provide a unified view. Data representation is to ensure that the fused data can be understood and operated by end-users in an intuitive way. In addition, the development of data fusion technology also depends on machine learning and artificial intelligence algorithms, which can help automate complex tasks in the data fusion process, such as pattern recognition and predictive analysis. Using these advanced technologies, the highway traffic management platform can provide more accurate and timely information, thus supporting more effective management decisions and operations. The main framework is as Figure 2 shown.

[0065] Specifically, the base map production module is the basis for building the highway traffic spatio-temporal service platform and system, mainly including three parts: obtaining road network information, extracting key intersection nodes, and adding and visualizing typical highway elements. The main content process is shown as Figure 3 shown, specifically including the following steps:

[0066] (1) Obtaining road network information

[0067] For obtaining road network information using satellite image data sources, a supervised learning method is adopted, combined with the existing road network sample set and a deep learning model for highway network extraction to achieve the planar extraction of the road network. Its main process and content are shown in Table 1.

[0068] Table 1 Process and content of satellite extraction of road network information

[0069]

[0070] By using a drone device to conduct aerial photography on a specific regional road network to obtain drone images and process them, and then using an edge detection algorithm to extract the road network. The road network extraction process and content are shown in Table 2.

[0071] Table 2 Drone extraction road network information process and content

[0072]

[0073]

[0074] In addition, the main processes and content for obtaining road network data using the trajectory information obtained by a vehicle-mounted Beidou GNSS terminal are as follows:

[0075] ① Calculate the driving direction angles of all trajectory points, then obtain a set of similar trajectory points considering the position and driving direction, and calculate the offset distance generated by each similar trajectory point;

[0076] ② Sum and average the offset distances in the offset direction to obtain the offset distance of the trajectory points to be offset. When the average offset distance of all trajectory points is greater than the threshold, recalculate the driving direction angle of the trajectory points with the offset coordinates until the average offset distance is less than the threshold;

[0077] ③ Based on the results of trajectory clustering, after removing the unclustered trajectory points, obtain the trajectory data that can reflect the road structure, and then use the method of raster digitization to extract the road network.

[0078] There are certain differences in the road network results extracted based on remote sensing satellite images, drone images, and Beidou GNSS terminals. In order to integrate different road network extraction methods to reduce the differences and improve the road network accuracy, an adaptive weighted fusion strategy is adopted to fuse the three road network extraction results. The calculation of the adaptive weighted fusion is as follows:

[0079] P = W1 × P s + W2 × P t + W3 × P r (1)

[0080] In the formula, P is the fusion prediction value; P s , P t and P r are the three road network extraction models (deep learning model, image edge detection algorithm model, trajectory clustering and rasterization model) respectively; W1, W2, and W3 are the weights of the three road network extraction models, and the sum of W1, W2, and W3 is 1.

[0081] To automatically determine the weight values of three road network extraction models, first, a certain number of real road network samples and the classification results corresponding to the positions of the road network extraction models are selected by combining high-resolution images and manual screening.

[0082] Generally speaking, the user accuracy and producer accuracy of the classification model are obtained using the confusion matrix method; the calculation formulas for the user accuracy and producer accuracy are as follows:

[0083] UA = TP / (TP + FP) (2)

[0084] PA = TP / (TP + FN) (3)

[0085] Among them, UA and PA refer to the user accuracy (measuring the proportion of correctly predicted categories in the classification results among the total number of predictions for that category) and the producer accuracy (measuring the proportion of samples correctly classified among the actual categories); TP refers to the number of samples in which the road network is correctly classified, FP refers to the number of samples misclassified as the road network, and FN refers to the number of samples misclassified as other categories.

[0086] In this embodiment, in order to make the calculated accuracy indicators more representative and reduce the influence of sample imbalance on them, the K-fold cross-validation method is further used to obtain the user accuracy and producer accuracy of the classification model. Its main steps are as follows:

[0087] ① Sample data set division, divide the sample set D into K mutually exclusive sub-sample sets:

[0088] D = D1 ∪ D2 ∪ … ∪ D K (4)

[0089] ② Divide the sample set D divided into K mutually exclusive sub-sample sets into K classification model training sample sets and validation sample sets in order, where the training sample set is K - 1 sub-sample sets, and the validation sample set is 1 sub-sample set outside the training sample set.

[0090] For each classification model training sample set and validation sample set, calculate the user accuracy (UA) and producer accuracy (PA) corresponding to the three road network extraction models according to formulas (2) and (3) respectively;

[0091] ③ Calculate the K-fold cross-validation results. Under the K mutually exclusive sub-sample sets, calculate the user accuracy and producer accuracy of the three road network extraction models corresponding to each mutually exclusive sub-sample set:

[0092]

[0093] The combined datasets of user accuracy and producer accuracy for calculating three road network extraction models (deep learning model, image edge detection algorithm model, trajectory clustering and rasterization model) can be obtained as above, which are X s :[X s1 , X s2 , …, X sm , X r :[X r1 , X r2 , …, X rm , X t :[X t1 , X t2 , …, X tm , X ni contains the user accuracy and producer accuracy of the road network extraction model n for the i-th mutually exclusive sub-sample set.

[0094] Use this dataset and the information entropy weight method to calculate the weights of the three classification models. The information entropy weight method is an objective weight assignment method. Its idea is to use the coefficient of variation of the data for weight assignment. If the coefficient of variation is larger, it means the information it carries is greater, and thus the weight will also be larger. The calculation process is as follows:

[0095] ① Calculate the mean and standard deviation:

[0096]

[0097] where m refers to the number of combined datasets of user accuracy and producer accuracy, and n = s, r, t respectively represent 3 datasets, represents the mean, and SD n represents the standard deviation.

[0098] ② Calculate the coefficient of variation (CV coefficient), which is the ratio of the standard deviation to the mean:

[0099]

[0100] ③ Calculate the weight, which is obtained by summing and normalizing the coefficient of variation:

[0101]

[0102] where k = 3.

[0103] (2) Extraction of key intersection nodes

[0104] Considering that the accident rate at road intersections is generally higher than that at non-road intersection locations, in order to better serve highway traffic operation, it is necessary to extract the intersection locations and spatial ranges. On the basis of the above processing, high-quality road network data can be obtained. In order to extract typical road network nodes (intersections), the extraction work of further road network intersections is realized by combining trajectory features. The specific content is as follows:

[0105] Extract the road centerlines according to the road network plane and edge information, extract the intersection positions of the road centerlines and mark them as candidate road intersections. The specific process mainly includes:

[0106] ① Screen the tangent points P1 and P2 of two parallel tangents in the near neighborhood of the road network edge, and obtain the center point (Xi, Yi) of the connection line of the two tangent points;

[0107] ② After connecting all the center points (Xi, Yi), generate the road centerline CL;

[0108] ③ According to all the road centerlines CL, obtain all the intersection points CP of the roads.

[0109] Use trajectory data clustering to remove false intersection points, and screen the intersection positions with a high degree of trajectory point aggregation near the candidate road intersections. The specific process mainly includes:

[0110] ① It is necessary to screen the vehicle trajectories, that is, screen the vehicle trajectory data that has been driving in a state close to the traffic flow;

[0111] ② Eliminate discrete trajectory points, that is, eliminate the trajectory points that do not have a similar trajectory point set;

[0112] ③ Encrypt the trajectory points, that is, adopt a linear encryption method to improve the trajectory aggregation level;

[0113] ④ Considering that there are usually multiple trajectories converging and diverging at real intersections, while false intersection points are often caused by trajectory intersections due to positioning errors, data drift, etc. Therefore, by clustering and analyzing the trajectory density near the intersection points, and combining the spectral, texture and other characteristics obtained from remote sensing satellite images and UAV images, these trajectory intersection points caused by accidental factors are identified and removed.

[0114] (3) Visualization of typical highway elements

[0115] On the basis of the above extraction results of the road network and its key nodes, taking the intersection as the base point of the highway traffic spatio-temporal service base map, match and add typical elements of the highway traffic spatio-temporal service, such as radars, cameras, traffic signs, etc. to the base map, and realize the visualization of the base of the highway traffic spatio-temporal service platform, so as to facilitate the update of real-time base map information and the efficient analysis and management of highway traffic spatio-temporal services.

[0116] Combined with WebGL technology, that is, a 3D visualization technology based on JavaScript and OpenGLES2.0, with the help of its rich API interfaces, use the OpenGL interface for 3D rendering, and draw graphics based on the Canvas tag built into HTML5, so as to achieve smooth loading of the base model of the highway traffic spatio-temporal service platform.

[0117] Based on the fusion and management of heterogeneous highway traffic data, the situation analysis module comprehensively monitors and predicts the highway traffic time and space situation to achieve an overall grasp of the regional highway traffic conditions.

[0118] The analysis steps of the highway traffic spatio-temporal situation by the situation analysis module include:

[0119] ① First, screen the required main data in the data warehouse, collect data from various sensors, monitoring devices, and vehicle Beidou GNSS terminals, and calculate key indicators including traffic flow, speed, and congestion index;

[0120] ② For the highway traffic time situation, first divide the data according to different time levels of day, month, and quarter, and combine time series analysis and convolutional neural network models to model traffic flow, speed, and congestion conditions, and predict the traffic conditions in a future period. The specific formulas included are as follows:

[0121] The input data is expressed as:

[0122] X t =[X1(t),X2(t),…,X n (t)] (11)

[0123] Among them, X t represents the input feature vector at time t, including n features, that is, the key indicators of traffic flow, speed, and congestion index in ①; X i (t) represents the observed value of the i-th feature at time t.

[0124] The convolution operation extracts the local time series feature representation:

[0125] Z t =X t *W+b (12)

[0126] Among them, W is the convolution kernel and b is the bias term.

[0127] The activation function is expressed as:

[0128] A t =ReLU(Z t ) (13)

[0129] Among them, At It is the output after activation, and the ReLU function is the activation function.

[0130] The pooling layer (taking max pooling as an example) is used to reduce the data dimension while retaining the main features, which is expressed as:

[0131] P t = MaxPool(A t ) (14)

[0132] where P t is the output after pooling.

[0133] The fully connected layer is expressed as:

[0134] Y t = W fc * P t + b fc (15)

[0135] where Y t refers to the prediction of the traffic condition at a future moment, W fc is the weight of the fully connected layer, and b fc is the bias term.

[0136] ③ Based on the key indicators such as the predicted traffic flow, speed, and congestion index, further construct a comprehensive traffic situation evaluation model, and express the predicted results in the form of different values on the road network to construct a traffic situation prediction grid map:

[0137] S = W1 * A + W2 * (1 - B) + W3 * C (16)

[0138] where S ∈ [0, 1], and the larger the value, the worse the traffic situation; A, B, and C refer to the normalized predicted data of traffic flow, speed, and congestion index; W1, W2, and W3 refer to the corresponding weight values of the predicted data of traffic flow, speed, and congestion index.

[0139] R = S i , i ∈ (x i , y i ) (17)

[0140] where R refers to the final traffic situation prediction grid map of the road network; S i refers to the predicted value of the traffic situation at position i; x i , y i refer to the coordinates of position i.

[0141] ④ For the spatial situation of highway traffic, taking highway intersections as reference points, spreading outward with different radii, and determining the corresponding radiation areas for each highway intersection respectively. Its formula can be:

[0142]

[0143] Among them, r refers to any coordinate position within the radiation area, which is the distance from the intersection; F refers to the radiation area centered on the intersection. When the distance of any coordinate position from the intersection exceeds 2 km, it is considered that this position is not within the radiation area of this intersection. The 0.5 km, 1 km, and 2 km in Formula 18 are all artificially set thresholds and can be adjusted according to different application scenarios. At the same time, the radiation area of the intersection can also be divided into several according to actual needs, not limited to the 3 shown in Formula 18, or Figure 4 the 2 shown in

[0144] As Figure 4 shown, two non-overlapping radiation areas are determined for each intersection according to r1 and r2 respectively. The coordinate positions with a straight-line distance from the specified intersection exceeding r1 + r2 are determined to be outside the radiation range of this intersection.

[0145] ⑤ Conduct comprehensive cluster analysis on the traffic situation prediction values S within different time (day, month, season) and different spatial ranges (different radiation areas), and locate the areas with higher kernel density to obtain the time and location where abnormal traffic situations are likely to occur in the future for each radiation area. The specific formula is as follows:

[0146] Z = {K d,F , K m,F , K p,F} (19)

[0147] Among them, Z refers to the result of comprehensive cluster analysis; K d,F , K m,F , K p,F respectively refer to the results of kernel density analysis of different intersection radiation areas on the time scales of day, month, and season. And the calculation formula of K in the formula is as follows:

[0148]

[0149] Among them, n is the number of traffic observation points (positions) used for analysis within the radiation area to be calculated; h is the bandwidth parameter; e represents the traffic situation prediction value of a certain position, and e i represents the traffic situation prediction values of all other positions except this position within the radiation area to be calculated; K(e) represents the density estimation value of this position. It should be noted that the traffic situation prediction value is time series data, including the traffic situation prediction results for different future moments.

[0150] The decision support and personalized service module correlates the abnormal intensity of traffic conditions in different radiation areas with the distance to intersections and adds it to the basic model of the highway traffic spatio-temporal service platform, realizing support decision-making for highway traffic managers and personalized service customization for participating users. Among them, the decision support for highway traffic spatio-temporal services mainly includes monitoring and management consoles, intelligent decision-making engines, decision deployment, etc.

[0151] The monitoring and management console is presented in the form of a web page and a large screen. It is the control interface for highway traffic managers to design dispatching operations, conduct dispatching monitoring and resource monitoring, and includes two roles: administrator and operator. The administrator has all the operation permissions of the platform, and the operator can only monitor the system operation status and has limited management functions when there are job abnormalities.

[0152] The intelligent decision-making engine is based on the basic model of the highway traffic spatio-temporal service platform with intersections as reference nodes. Based on the comprehensive spatio-temporal situation analysis results in a certain time domain and region, it uses advanced cloud computing and artificial intelligence algorithms to conduct a comprehensive evaluation according to factors such as vehicle driving speed, traffic volume changes, and traffic accidents. According to real-time road condition information, real-time traffic volume information, predicted road condition information, and estimated travel time information in different scenarios, it conducts intelligent research and judgment analysis, formulates or generates traffic guidance strategies in various traffic environments (including emergencies), and completes induction control such as broadcasting, induction release, toll station access management, and traffic diversion management.

[0153] Decision deployment is an effective way to effectively implement highway traffic operation decision deployment. It is mainly sent to main road participants and traffic management personnel in the form of broadcasts to achieve the normal and efficient operation of highway traffic.

[0154] For the personalized service customization of highway traffic participating users, on the one hand, it is based on the management of Internet information publishing middleware, connects to third-party services or mobile applications, timely shares the support decisions of highway traffic managers to the mobile side, constructs a cloud user travel preference model based on the user's historical travel data, and combines the obtained highway traffic decision data to provide personalized travel routes and traffic information push; on the other hand, it can also provide real-time feedback on highway traffic conditions to the management level to realize real-time update of the information of the basic model of the highway traffic spatio-temporal service platform.

[0155] Embodiment 2

[0156] The present invention provides a highway traffic spatio-temporal service method based on multi-source heterogeneous data fusion, including the following steps:

[0157] Collect highway traffic data including satellite remote sensing data, UAV image data, vehicle-mounted Beidou GNSS trajectory data, and road-side equipment collection.

[0158] By preprocessing, data matching, integrating, and representing the collected data, a hierarchical highway traffic spatio-temporal service data warehouse is constructed to achieve unified management of multi-source heterogeneous data;

[0159] Based on the data warehouse, deep learning, edge detection, and trajectory analysis methods are used to extract road network information, and weighted fusion is performed based on the weights corresponding to each extraction result to generate a highway traffic spatio-temporal service base map with highway intersections as key nodes;

[0160] Time series hierarchical analysis and convolutional neural network prediction are performed on traffic flow, vehicle speed, and congestion index in the data warehouse. Based on the highway traffic spatio-temporal service base map, clustering analysis is combined with the radiation area with highway intersections as the reference points to detect abnormal traffic events, forming a traffic situation prediction map;

[0161] Based on the traffic situation prediction map, real-time and visual traffic situation display, traffic guidance strategies, and personalized travel services are provided to traffic managers and traffic participating users.

[0162] Embodiment 3

[0163] The present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the highway traffic spatio-temporal service method based on multi-source heterogeneous data fusion described in the above technical solution is implemented.

[0164] Embodiment 4

[0165] The present invention provides an electronic device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to execute the highway traffic spatio-temporal service method based on multi-source heterogeneous data fusion described in the above technical solution.

[0166] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0167] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate a means for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0168] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction means that implements the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0169] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0170] The embodiments of the present invention have been described above in conjunction with the accompanying drawings. However, the present invention is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Under the inspiration of the present invention, those of ordinary skill in the art can also make many forms without departing from the spirit and scope protected by the present invention and the claims. These all fall within the protection scope of the present invention.

[0171] The content not described in detail in this specification belongs to the known prior art of those skilled in the art.

Claims

1. A highway traffic spatiotemporal service platform based on multi-source heterogeneous data fusion, characterized by: include: Data collection module, used to collect satellite remote sensing data, drone image data, vehicle-mounted Beidou GNSS trajectory data, and highway traffic data collected by roadside equipment; The data fusion management module is used to build a hierarchical highway traffic spatiotemporal service data warehouse by preprocessing, matching, integrating and representing the collected data, so as to achieve unified management of multi-source heterogeneous data; The base map production module is used to extract road network information based on the data warehouse using deep learning, edge detection and trajectory analysis methods, and perform weighted fusion based on the weights corresponding to each extraction result to generate a highway traffic spatiotemporal service base map with highway intersections as key nodes; Traffic situation analysis module, which is used to perform time series hierarchical analysis and convolutional neural network prediction on traffic flow, vehicle speed and congestion index in the data warehouse. Based on the highway traffic spatiotemporal service base map, cluster analysis is performed on the radiation area with highway intersections as the reference point to detect abnormal traffic events and form a traffic situation prediction map; The decision support and personalized service module is used to provide real-time, visual traffic situation display, traffic induction strategies and personalized travel services to traffic managers and traffic participants based on traffic situation prediction maps.

2. The highway traffic spatiotemporal service platform based on multi-source heterogeneous data fusion according to claim 1 is characterized by: The base map production module uses remote sensing satellite image data and a trained deep learning model to extract the road network; it uses drone aerial image data and an image edge detection algorithm model to extract the road network; Using vehicle Beidou GNSS trajectory data, the road network structure is extracted through trajectory clustering and rasterization model.

3. The highway traffic spatiotemporal service platform based on multi-source heterogeneous data fusion according to claim 2 is characterized by: The calculation process of the weights corresponding to the extraction results of each road network extraction model includes: Select a certain number of real road network samples to form a sample data set; Divide the sample data set into K mutually exclusive sub-sample sets; and further divide the sample set into a training sample set and a validation sample set; Based on the mutually exclusive subsample sets corresponding to each model, the accuracy data sets containing user accuracy and producer accuracy of each model are obtained under K-fold cross validation. Calculate the coefficient of variation of each model accuracy data set respectively; The coefficient of variation of each model accuracy data set is summed and normalized to obtain the corresponding weight of each model.

4. The highway traffic spatiotemporal service platform based on multi-source heterogeneous data fusion according to claim 1 is characterized by: The base map production module extracts highway intersections based on highway network data generated by weighted fusion of various extraction results; radar, camera, and traffic sign elements are added to the base map according to their positional relationship with the highway intersections and the road network, thereby realizing the construction and visualization of the highway traffic spatiotemporal service base map.

5. The highway traffic spatiotemporal service platform based on multi-source heterogeneous data fusion according to claim 4 is characterized by: The process of extracting highway intersections includes: extracting all road center lines according to the fused road network and obtaining the intersection points of the center lines as candidate intersections; using the vehicle-mounted Beidou GNSS trajectory data to perform trajectory aggregation analysis on the candidate intersections, filtering out pseudo intersections generated only due to positioning errors or data drift; combining the spatial characteristics of satellite remote sensing data and drone image data to verify the positions of the remaining intersections, and finally identifying the real highway intersection node set.

6. The highway traffic spatiotemporal service platform based on multi-source heterogeneous data fusion according to claim 1 is characterized by: The traffic situation analysis module uses the predicted traffic flow, vehicle speed and congestion index to obtain the traffic situation prediction value of each coordinate position on the highway traffic spatiotemporal service base map through weighted calculation.

7. The highway traffic spatiotemporal service platform based on multi-source heterogeneous data fusion according to claim 6 is characterized by: The process of detecting abnormal traffic events includes: For each highway intersection, spread outward with different set radii to form different radiation areas corresponding to each highway intersection; For each radiation area, based on its corresponding traffic situation prediction value, kernel density analysis is performed on different time scales to obtain the time and location where the probability of anomaly occurrence corresponding to the radiation area is greater than the set threshold.

8. The highway traffic spatiotemporal service platform based on multi-source heterogeneous data fusion according to claim 1 is characterized by: The decision support and personalized service module is aimed at traffic managers, visualizing the traffic situation base map and prediction results; The intelligent decision-making engine automatically generates a traffic induction control strategy based on the real-time traffic information and traffic situation prediction results, combined with changes in vehicle speed, traffic flow fluctuations and accident alarms, and sends the strategy to road participants and related management terminals in the form of broadcast prompts, variable information board information release, toll station control, traffic diversion, etc., to implement highway traffic operation control decisions.

9. The highway traffic spatiotemporal service platform based on multi-source heterogeneous data fusion according to claim 1 is characterized by: The decision support and personalized service module is aimed at traffic participants. It shares traffic management decision information to the user's mobile terminal through the platform's Internet information publishing middleware, and provides users with personalized travel route planning suggestions and traffic information push services based on the travel preference model established in combination with the user's historical travel data. At the same time, it receives real-time road condition information or personalized demand requests fed back by the user, and uploads them back to the data warehouse to update the highway traffic spatiotemporal service base map and the prediction model in the traffic situation analysis module.

10. A highway traffic spatiotemporal service method based on multi-source heterogeneous data fusion, characterized by: The following steps are involved: The data collected include satellite remote sensing data, drone image data, vehicle-mounted Beidou GNSS trajectory data, and highway traffic data collected by roadside equipment; By preprocessing, matching, integrating and representing the collected data, a hierarchical highway traffic spatiotemporal service data warehouse is constructed to achieve unified management of multi-source heterogeneous data; Based on the data warehouse, deep learning, edge detection and trajectory analysis methods are used to extract road network information, and weighted fusion is performed based on the weights corresponding to each extraction result to generate a highway traffic spatiotemporal service base map with highway intersections as key nodes; Conduct time series hierarchical analysis and convolutional neural network prediction on traffic flow, vehicle speed and congestion index in the data warehouse. Based on the highway traffic spatiotemporal service base map, cluster analysis is performed on the radiation area with highway intersections as the reference point to detect abnormal traffic events and form a traffic situation prediction map. Based on the traffic situation prediction map, it provides real-time, visual traffic situation display, traffic induction strategy and personalized travel services to traffic managers and traffic participants.

Images