Multi-source data fusion method and system based on digital twinning

Abstract

The invention relates to the technical field of multi-source data fusion calculation, and discloses a multi-source data fusion method and system based on digital twinning, and the method comprises the steps: S1, collecting multi-source data; s2, data integration and preliminary proofreading; s3, time-space reference unification; s4, feature extraction and logic association; s5, establishing a dynamic model; s6, behavior prediction and risk assessment; and S7, deployment strategy adjustment and early warning. Through hierarchical processing of a multi-source acquisition library, a GIS basic library, a history library, a fusion library and a thematic library, repeated storage and redundant calculation of original data are avoided, a federal learning technology realizes cross-department collaboration on the premise of ensuring data privacy, original data does not need to be concentrated, transmission and processing amount are greatly reduced, and the data processing efficiency is improved. And meanwhile, abnormal data are actively identified and cleaned by utilizing threshold limitation, state logic verification and Bayesian network reverse calculation, invalid information is eliminated in advance, and the calculation complexity during model updating is reduced, so that the response efficiency and the information processing efficiency of the whole system are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of multi-source data fusion computing technology, specifically to a multi-source data fusion method and system based on digital twins. Background Technology

[0002] Multi-source data fusion based on digital twins refers to integrating multiple data sources from different data sources into a virtual digital twin model that corresponds to the physical world, in order to achieve real-time monitoring and analysis of the physical world. For example, in the field of intelligent transportation, by integrating different information from different data sources, more comprehensive and accurate real-time status and dynamic information of the target area can be obtained to achieve real-time monitoring of traffic conditions, helping traffic management departments and drivers to understand road conditions and make corresponding adjustments.

[0003] However, due to the different data sources, integrating and verifying the acquired multi-source data is quite troublesome, which makes the data fusion process more prone to errors, putting more computational and verification pressure on the system. This leads to errors in the judgments derived from the analysis of this data, reducing the system's ability to grasp traffic conditions in real time. Moreover, since the data from multiple data sources are different and the related information has a strong logical connection, these data further increase the probability of errors. In view of this, we propose a multi-source data fusion method and system based on digital twins. Summary of the Invention

[0004] The purpose of this invention is to provide a multi-source data fusion method and system based on digital twins to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a multi-source data fusion method based on digital twins, comprising the following steps:

[0006] S1. Multi-source data acquisition: Acquire various data from the transportation system. The above data comes from the data uploaded by various fixed monitoring points and dynamic monitoring points in the transportation system.

[0007] S2. Data Integration and Preliminary Verification: Establish a "five-database linkage mechanism" to collaboratively verify the multi-source data collected in S1, ensuring the accuracy of data units and collection.

[0008] S3, Unified Spatiotemporal Reference: Employs millimeter-level spatiotemporal reference services from the BeiDou system, unifies the spatiotemporal coordinates of data obtained in S1 through satellite navigation signals, and achieves lane-level positioning compensation through collaboration of IoT devices in scenarios where satellite signals are missing.

[0009] S4, feature extraction and logical association: a "feature pyramid" is constructed using a spatio-temporal knowledge graph, heterogeneous data such as satellite images and BIM models are mapped to a unified spatio-temporal framework, and each item of data in the framework is associated, the logical relationship between each item of data is analyzed through big data analysis, and the related data is calculated in reverse through the logical relationship, and the accuracy of the data is verified;

[0010] S5, dynamic model establishment: a dynamic model of the traffic system is established based on the data obtained in the traffic system, and the subsequent traffic system data obtained is imported into the dynamic model for recording;

[0011] S6, behavior prediction and risk assessment: a ST-RetNet spatio-temporal prediction model is used to process long-time traffic flow prediction, the problem of prediction accuracy decay with time is solved by retaining the network structure, and the risk value is quantified by the data in the traffic system in the dynamic model, and the calculation formula is as follows:

[0012] R=V*P*U;

[0013] Wherein, R is the risk quantization value, V is the possibility of risk occurrence, P is the probability of risk occurrence, and U is the consequence of risk occurrence;

[0014] S7, deployment strategy adjustment and early warning: a "early warning-disposal-feedback" closed loop mechanism is established, information is released through VMS, traffic broadcast and other channels, and devices such as signal lights and contour warning lights are linked to guide.

[0015] Preferably, in the S1, the target obtained during multi-source data collection includes geomagnetic sensor, microwave radar, high-definition camera, floating car GPS, and weather sensor equipment, real-time acquisition of traffic flow, vehicle speed, lane occupancy, and weather condition data, and transmission to the command and control center through 4G / 5G network or optical fiber, the multi-source data collection also includes unmanned aerial vehicle group aerial photography, which can supplement the trajectory data collection of complex road sections, improve the accuracy and checkable dimension of data collection, and ensure the accuracy of data.

[0016] Preferably, the "five-knowledge linkage mechanism" in S2 includes multi-source collection library, GIS basic library, history library, fusion library and special library, the multi-source collection library converges raw data, the GIS basic library performs spatial modeling, the history library realizes spatio-temporal tracing, the fusion library completes feature extraction, and the special library customizes data according to scene, uses federal learning technology to solve the problem of data island, realizes cross-departmental data collaboration under the premise of having considerable security strength, and ensures the accuracy and uniformity of data output.

[0017] Preferably, the data integration in S2 uses threshold limits and state logic verification to clean up abnormal data obtained in the traffic system, remove abnormal data, and integrate and correct the remaining data after removal, thereby effectively ensuring data accuracy and reducing the computational burden of abnormal data on data acquisition and verification systems.

[0018] Preferably, the establishment of the "feature pyramid" in S4 uses a four-level geographic feature hierarchy, automatically activates matching levels through dynamic semantic mask technology, integrates building geometric features of BIM models and spectral features of satellite images, constructs a cross-modal spatiotemporal indexing network, and supports coordinate unification and attribute association of heterogeneous data.

[0019] Preferably, S4 uses a "process-relation" knowledge representation method to quantify the spatiotemporal coupling relationship between traffic flow and road topology, and uses a Bayesian network to inversely calculate data anomaly probability, thereby further integrating and inversely calibrating the data obtained in the traffic system to filter out abnormal data while performing reverse self-checking to ensure that the relevant monitoring equipment at the monitoring point in the traffic system is in a normal working state.

[0020] Preferably, S5 integrates multi-source real-time data streams when establishing a dynamic model, uses the Gaussian-Laplacian pyramid principle to achieve time series alignment of different resolution data, and ensures that the model update frequency is minute-level.

[0021] Preferably, when using the ST-RetNet model for processing in S6, the spatiotemporal gating unit is introduced to retain long-time dependency features to solve the prediction decay problem.

[0022] Preferably, the disposition decision in the "warning-disposition-feedback" mechanism in S7 uses a big data model to judge and select based on the quantitative risk value obtained in S6, and the warning levels are set to three levels, including yellow warning (risk value > 0.6), orange warning (risk value > 0.8), and red warning (risk value > 0.9). When the yellow warning is activated, VMS information release and signal light cycle fine-tuning are activated, and when the orange warning is activated, profile warning light flashing + broadcast system voice prompt are activated, and when the red warning is activated, the emergency lane opening protocol is forcibly activated.

[0023] The multi-source data fusion system based on digital twinning includes a data acquisition module, a data integration module, a data calibration and checking module, a risk calculation and evaluation module, a decision module, and a central scheduling module, wherein the data acquisition module includes a millimeter wave radar, a video monitoring, a laser radar, an Internet of Things sensor, and a GPS floating car arranged in a traffic system, the data integration module is used for unifying and standardizing data, the data calibration includes logical calibration and algorithm correlation calibration, the central scheduling module is associated with a signal lamp and an information board arranged in the traffic system, and is used for scheduling related equipment in the traffic system for guidance and command in decision-making.

[0024] Compared with the prior art, the multi-source data fusion method and system based on digital twinning have the following beneficial effects:

[0025] 1. The multi-source data fusion method and system based on digital twinning avoid repeated storage and redundant calculation of original data through hierarchical processing of a multi-source acquisition library, a GIS basic library, a history library, a fusion library, and a special library, realize cross-department collaboration under the premise of ensuring data privacy without the need for centralized original data, greatly reduce transmission and processing volume, actively identify and clean abnormal data by using threshold limitation, state logic verification, and Bayesian network back calculation, eliminate invalid information in advance, reduce the calculation load of subsequent fusion and modeling, and ensure efficient time sequence alignment of multi-resolution data by using the Gaussian-Laplacian pyramid principle, thereby reducing the calculation complexity when updating the model, and effectively improving the response efficiency and information processing efficiency of the entire system.

[0026] 2. The multi-source data fusion method and system based on digital twinning ensures the consistency of dynamic / static data in the time-space dimension by providing a unified space-time coordinate through the Beidou system and combining with lane-level positioning compensation of Internet of Things equipment, forms a global unified digital twinning framework by constructing a feature pyramid through a space-time knowledge graph and fusing multi-dimensional heterogeneous data such as BIM geometric features and satellite spectral features, and improves the perception accuracy by quantifying the coupling relationship of traffic elements based on the “process-relation” knowledge representation method and verifying the data reliability through Bayesian network back calculation.

[0027] 3. The multi-source data fusion method and system based on digital twinning solves the problem of prediction accuracy decay over time of traditional models by introducing a space-time gating attention mechanism through an ST-RetNet model, dynamically quantifies the possibility, probability, and consequences by using a risk quantification calculation formula, provides an objective basis for decision-making, forms a “warning-disposal-feedback” closed loop by linking VMS, signal lights, outline warning lights, a broadcast system, and an emergency lane protocol through three-level early warning (yellow / orange / red), and improves the emergency disposal efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0028] Fig. 1 The multi-source data fusion method steps schematic diagram of the application;

[0029] Fig. 2 The multi-source data fusion module composition schematic diagram of the application. DETAILED DESCRIPTION

[0030] As Figs. 1-2 shown, the application provides a technical solution: a multi-source data fusion method and system based on digital twinning, including the following steps:

[0031] S1, multi-source data acquisition: acquiring various data in the traffic system, the above-mentioned data being derived from the data uploaded by each fixed monitoring point and dynamic monitoring point in the traffic system;

[0032] S2, data integration and preliminary correction: establishing a "five-bank linkage mechanism", and collaboratively checking the multi-source data taken in S1 to ensure the accuracy of the data units and the collection;

[0033] S3, time and space reference unification: adopting the millimeter-level time and space reference service of the Beidou system to unify the time and space coordinates of the data acquired in S1 through satellite navigation signals, and for the satellite signal missing scene, realizing lane-level positioning compensation through the collaboration of Internet of Things devices;

[0034] S4, feature extraction and logical association: constructing a "feature pyramid" using a time and space knowledge graph, mapping heterogeneous data such as satellite images and BIM models to a unified time and space framework, and associating each item of data in the framework, analyzing the logical relationship between each item of data through big data, and through the logical relationship, performing reverse calculation on the relevant data to verify the accuracy of the data;

[0035] S5, establishing a dynamic model: establishing a traffic system dynamic model based on the acquired various data in the traffic system, and importing the subsequently acquired traffic system data into the dynamic model for recording;

[0036] S6, behavior prediction and risk assessment: using a ST-RetNet and other time and space prediction models to process long-time traffic flow prediction, solving the problem of prediction accuracy decay over time through a reserved network structure, and quantifying the risk value through each item of data in the dynamic model in the traffic system, the calculation formula being as follows:

[0037] R=V*P*U;

[0038] wherein R is the risk quantification value, V is the possibility of risk occurrence, P is the probability of risk occurrence, and U is the consequence of risk occurrence;

[0039] S7, deployment strategy adjustment and early warning: Establish a "early warning-treatment-feedback" closed loop mechanism, through VMS, traffic broadcast and other channels to release information, linkage signal lights, profile warning lights and other equipment for induction.

[0040] In an embodiment of the application, in S1, when multi-source data acquisition is performed, the target acquisition includes geomagnetic sensors, microwave radars, high-definition cameras, floating car GPS, weather sensor devices, real-time acquisition of traffic volume, vehicle speed, lane occupancy, and weather condition data, which are transmitted to the command and control center through 4G / 5G network or optical fiber. Multi-source data acquisition also includes UAV group aerial photography, which can supplement the trajectory data acquisition of complex road sections, improve the accuracy and checkable dimension of data acquisition, and ensure the accuracy of data. At the same time, the "five-knowledge linkage mechanism" in S2 includes multi-source acquisition library, GIS basic library, history library, fusion library and special library. The multi-source acquisition library converges raw data, the GIS basic library performs spatial modeling, the history library realizes time-space tracing, the fusion library completes feature extraction, and the special library customizes data according to scenes. The federal learning technology is used to solve the data island problem, realize cross-department data collaboration under the premise of considerable security strength, and ensure the accuracy and uniformity of data output.

[0041] Further, in S2, threshold limitation and state logic verification are used to clean up abnormal data obtained in the traffic system, remove abnormal data, and integrate and correct the remaining data after removal, thereby effectively ensuring the accuracy of the data, reducing the calculation pressure of abnormal data on the data acquisition and testing system. Specifically, in S4, a four-level geographic feature hierarchy is established, the matching level is automatically activated by dynamic semantic mask technology, the building geometric features of the BIM model are fused with the spectral features of the satellite image, a cross-modal spatio-temporal index network is constructed, and the coordinate unification and attribute association of heterogeneous data are supported.

[0042] In addition, in S4, the spatio-temporal coupling relationship between traffic flow and road topology is quantified based on the "process-relation" knowledge representation method, and the data anomaly probability is calculated in reverse through the Bayesian network, so as to further integrate and calibrate the data obtained in the traffic system, and then screen out abnormal data. At the same time, reverse self-checking is performed during screening to ensure that the related monitoring equipment of the monitoring point in the traffic system is in a normal working state.

[0043] In an embodiment of the application, in S5, multi-source real-time data streams are integrated when establishing a dynamic model, and the Gaussian-Laplacian pyramid principle is used to realize time sequence alignment of different resolution data, ensuring that the model update frequency reaches minutes. Specifically, in S6, when the ST-RetNet model is used for processing, a spatio-temporal gating unit with attention mechanism is introduced to retain long-time dependent features to solve the prediction decay problem.

[0044] It is worth noting that the treatment decision in the "early warning-treatment-feedback" mechanism in S7 adopts a big data model to judge and select according to the quantitative risk value obtained in S6, and the early warning levels are set to three levels, including yellow early warning (risk value > 0.6), orange early warning (risk value > 0.8) and red early warning (risk value > 0.9), wherein the yellow early warning activates the VMS information release and the signal lamp cycle fine-tuning, the orange early warning starts the contour warning light flashing + broadcast system voice prompt, and the red early warning starts the emergency lane opening protocol.

[0045] In the present application, a multi-source data fusion system based on digital twinning is provided, wherein the multi-source data fusion system comprises a data acquisition module, a data integration module, a data calibration and checking module, a risk calculation and evaluation module, a decision module and a central dispatching module, wherein the data acquisition module comprises a millimeter wave radar, a video monitoring, a laser radar, an Internet of Things sensor and a GPS floating car arranged in the traffic system, the data integration module is used for unifying and standardizing the data, the data calibration comprises logical calibration and algorithm correlation calibration, the central dispatching module is associated with the signal lamp and the information board arranged in the traffic system, and is used for dispatching the related equipment in the traffic system for guidance and command in the decision.

[0046] The above has made a detailed description of the present application in general, but some modifications or improvements can be made on the basis of the present application, which is obvious to those skilled in the art. Therefore, the modifications or improvements without departing from the spirit of the present application are within the protection scope of the present application.

Claims

1. A multi-source data fusion method based on digital twins, characterized in that: Includes the following steps: S1. Multi-source data acquisition: Acquire various data from the transportation system. The above data comes from the data uploaded by various fixed monitoring points and dynamic monitoring points in the transportation system. S2. Data Integration and Preliminary Verification: Establish a "five-database linkage mechanism" to collaboratively verify the multi-source data collected in S1, ensuring the accuracy of data units and collection. S3, Unified Spatiotemporal Reference: Employs millimeter-level spatiotemporal reference services from the BeiDou system, unifies the spatiotemporal coordinates of data obtained in S1 through satellite navigation signals, and achieves lane-level positioning compensation through collaboration of IoT devices in scenarios where satellite signals are missing. S4. Feature Extraction and Logical Association: Construct a "feature pyramid" using a spatiotemporal knowledge graph to map heterogeneous data such as satellite imagery and BIM models to a unified spatiotemporal framework, and associate the various data within the framework. Analyze the logical relationships between the various data through big data analysis, and use these logical relationships to reverse-engineer related data to verify data accuracy. S5. Establish a dynamic model: Use the data from the acquired traffic system to establish a dynamic model of the traffic system, and import the subsequently acquired traffic system data into the dynamic model for recording. S6. Behavior Prediction and Risk Assessment: Spatiotemporal prediction models such as ST-RetNet are used to handle long-term traffic flow prediction. The problem of prediction accuracy decaying over time is solved by preserving the network structure. Furthermore, the risk value is quantified by various data in the traffic system of the dynamic model. The calculation formula is as follows: R = V * P * U; Where R is the risk quantification value, V is the probability of the risk occurring, P is the probability of the risk occurring, and U is the consequence of the risk occurring. S7. Deployment Strategy Adjustment and Early Warning: Establish a closed-loop mechanism of "early warning-handling-feedback", release information through channels such as VMS and traffic radio, and link traffic lights, contour warning lights and other equipment for guidance.

2. The multi-source data fusion method based on digital twins according to claim 1, characterized in that: When performing multi-source data acquisition in S1, the targets include geomagnetic sensors, microwave radar, high-definition cameras, floating car GPS, and meteorological sensor equipment. Real-time data on traffic flow, vehicle speed, lane occupancy, and weather conditions are acquired and transmitted to the command and control center via 4G / 5G networks or optical fibers. The multi-source data acquisition also includes aerial photography by drone swarms.

3. The multi-source data fusion method based on digital twins according to claim 1, characterized in that: The "five-database linkage mechanism" in S2 includes a multi-source collection database, a GIS basic database, a historical database, a fusion database, and a thematic database. The multi-source collection database gathers raw data, the GIS basic database performs spatial modeling, the historical database enables spatiotemporal tracing, the fusion database completes feature extraction, and the thematic database customizes data according to scenarios. Federated learning technology is used to solve the problem of data silos, and cross-departmental data collaboration is achieved under the premise of a considerable degree of confidentiality, ensuring the accuracy and consistency of data output.

4. The multi-source data fusion method based on digital twins according to claim 1, characterized in that: In the S2 process, when integrating data, threshold limits and state logic verification are used to clean and remove abnormal data obtained from the traffic system. After removal, the remaining data is integrated and verified, thereby effectively ensuring the accuracy of the data and reducing the computational pressure of abnormal data on the data acquisition and verification system.

5. The multi-source data fusion method based on digital twins according to claim 1, characterized in that: When establishing the "feature pyramid" in S4, a four-level geographic feature layering is adopted. The matching level is automatically activated through dynamic semantic masking technology. The building geometric features of the BIM model and the spectral features of satellite imagery are integrated to construct a cross-modal spatiotemporal index network, which supports the coordinate unification and attribute association of heterogeneous data.

6. The multi-source data fusion method based on digital twins according to claim 1, characterized in that: In S4, the spatiotemporal coupling relationship between traffic flow and road topology is quantified by using a process-relationship knowledge representation method. The probability of data anomalies is calculated in reverse using a Bayesian network, which enables further integration and reverse calibration of the data obtained in the traffic system, thereby filtering out abnormal data. At the same time, a reverse self-check is performed during the filtering process to ensure that the relevant monitoring equipment at the monitoring point in the traffic system is in normal working condition.

7. The multi-source data fusion method based on digital twins according to claim 1, characterized in that: When establishing a dynamic model in S5, multiple real-time data streams are integrated, and the Gauss-Laplace pyramid principle is used to achieve temporal alignment of data with different resolutions, ensuring that the model update frequency reaches the minute level.

8. The multi-source data fusion method based on digital twins according to claim 1, characterized in that: The S6 uses the ST-RetNet model for processing and introduces a spatiotemporal gating unit with an attention mechanism to retain long-term dependent features in order to solve the prediction decay problem.

9. The multi-source data fusion method based on digital twins according to claim 1, characterized in that: The "early warning-response-feedback" mechanism in S7 uses a big data model to judge and select based on the quantitative risk value obtained in S6. The early warning level is set to three levels, including yellow warning (risk value > 0.6), orange warning (risk value > 0.8), and red warning (risk value > 0.9). When a yellow warning is triggered, VMS information release and traffic light cycle fine-tuning are activated. When an orange warning is triggered, the outline warning light flashes and the broadcast system provides voice prompts. When a red warning is triggered, the emergency lane opening protocol is forcibly activated.

10. A multi-source data fusion system based on digital twins, based on the multi-source data fusion method based on digital twins as described in claims 1-9, characterized in that: The multi-source data fusion system includes a data acquisition module, a data integration module, a data calibration and verification module, a risk calculation and assessment module, a decision-making module, and a central dispatch module. The data acquisition module includes millimeter-wave radar, video surveillance, lidar, IoT sensors, and GPS floating cars installed in the transportation system. The data integration module is used to unify and standardize the data. The data calibration includes logical calibration and algorithm correlation calibration. The central dispatch module is associated with traffic lights and information boards installed in the transportation system and is used to dispatch relevant equipment in the transportation system for guidance and command during decision-making.

Images