Multi-source data fusion method and application system based on digital twinning

Abstract

The invention relates to a multi-source data fusion method and application system based on digital twinning, and the method comprises the steps: collecting real-time and historical data from various sensors, devices, networks and other information sources, and carrying out the cleaning and format standardization preprocessing of the collected data; the data set fusion uses a Kalman filter algorithm to perform data fusion, and the prediction step comprises: predicting a state equation at a current moment based on a state equation and a covariance matrix at a previous moment, and predicting the covariance matrix and an observation vector; the updating step comprises the steps of calculating Kalman gain and updating state estimation and a covariance matrix. The feature set fusion uses deep learning line feature data fusion, features are extracted from different data sources by using CNN, and different feature vectors are subjected to weighted fusion to form a single feature vector; decision set fusion adopts a deep learning method for decision fusion, so that the accuracy, stability and reliability of decisions are improved; the method improves the efficiency and precision of data fusion, and has a good market application prospect.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and more specifically, to a multi-source data fusion method and application system based on digital twins. Background Art

[0002] In application scenarios such as smart cities and smart transportation, it is necessary to collect and integrate a large amount of heterogeneous data from different data sources to facilitate real-time monitoring and decision support.

[0003] The problems with existing technologies mainly include:

[0004] Challenges of multi-source data fusion:

[0005] Data collection may come from different regions, systems, platforms, devices and applications. These data often have different formats, logical structures, semantics and different timestamps, making it difficult to effectively integrate them into a complete and reliable data set.

[0006] Traditional data fusion models usually can only rely on historical data for processing, lack real-time performance, and cannot meet the needs of real-time data fusion.

[0007] Lack of real-time performance:

[0008] Most existing data fusion methods cannot achieve real-time data processing and analysis, which is a major limitation for application scenarios that require rapid response.

[0009] Data Quality and Security:

[0010] Data quality varies and may contain errors or omissions, posing a threat to the accuracy and reliability of digital twin models.

[0011] Data security is also an important issue, especially when it comes to sensitive data, which needs to be ensured to be properly protected throughout the entire processing process.

[0012] Accuracy issues:

[0013] Existing data fusion technologies often fail to achieve the required high precision, especially when building digital twin models, which affects the effectiveness and practicality of the models.

[0014] Flexibility and scalability:

[0015] As new data sources increase and technology advances, existing systems may find it difficult to adapt to such changes, thus limiting their long-term development and scope of application.

[0016] In summary, the existing technology has obvious limitations when facing multi-source data fusion, especially with respect to problems in real-time, accuracy, data quality and security, which are the key issues to be solved by the present invention. Summary of the invention

[0017] The purpose of the present invention is to provide a multi-source data fusion method and application system based on digital twins to solve the problems in the prior art.

[0018] In order to solve the above problems, the first aspect adopts the following technical solutions:

[0019] The multi-source data fusion method based on digital twins includes the following steps:

[0020] Collect data from sensors, monitoring devices and external data sources of different sources and types; perform pre-processing work such as cleaning and format standardization of the collected data;

[0021] Multi-source data fusion, using Kalman filter for data fusion:

[0022] The Kalman filter data fusion includes a prediction step: predicting the state equation at the current moment, predicting the covariance matrix and the observation vector based on the state equation and covariance matrix at the previous moment;

[0023] The Kalman filter for data fusion includes the following update steps: calculating the Kalman gain, updating the state estimate and covariance matrix; and using a convolutional neural network for feature layer data fusion:

[0024] Feature extraction: Use CNN to extract features from different data sources;

[0025] Feature fusion: different feature vectors are weighted and fused to form a single feature vector;

[0026] The weighted average method is used for decision fusion at the decision layer;

[0027] Build a digital twin model based on the fused data;

[0028] The digital twin model is updated in real time to reflect the immediate status of the physical entity.

[0029] The specific steps for preprocessing the collected data by cleaning and format standardization include:

[0030] Check and remove duplicate records to avoid analytical bias caused by data redundancy; handle missing data by interpolating, deleting records with missing values, or using machine learning methods to predict missing values; unify timestamps from different data sources to the same time standard; ensure that all numerical data use the same unit; and perform encoding conversion for non-numerical data, such as categorical variables.

[0031] The specific steps of using CNN to extract features from different data sources include:

[0032] Preprocessing: Before inputting into CNN, the data needs to be preprocessed;

[0033] Data format conversion: convert different types of data into a form suitable for CNN processing;

[0034] Model design:

[0035] Input layer: Design the input layer to accept data from different data sources;

[0036] Convolutional layer: local features are captured through convolutional layers; each convolutional layer has a set of learnable filters for detecting different features;

[0037] Fully connected layer: The extracted features are integrated through the fully connected layer;

[0038] Feature-level fusion: After feature extraction is completed, feature vectors from different data sources can be fused.

[0039] The specific steps of forming a single feature vector by weighted fusion of different feature vectors include: assigning a weight to each feature vector extracted from each data source; the weight can be pre-set or obtained through training, reflecting the importance of each feature vector; the weight can be based on the relevance and effectiveness of the feature or automatically determined through a machine learning algorithm; before weighted fusion, each feature vector needs to be normalized to ensure that they are on the same scale.

[0040] Building a digital twin model based on fused data includes:

[0041] First, the fused data is mapped to the corresponding digital twin model components;

[0042] Initialize the digital twin model to ensure that all components are set up correctly and are able to receive and process data; Model calibration: Use the fused initial data set to calibrate the model to ensure that the model's output is consistent with the behavior of the actual physical entity;

[0043] Model validation is performed by comparing the results predicted by the model with the performance of the actual physical entity; if deviations are found, further adjustments to the model parameters are required until satisfactory accuracy is achieved.

[0044] Update the digital twin model in real time to reflect the immediate status of the physical entity. Specifically: based on new data, the digital twin model updates its status in real time to reflect the latest changes of the physical entity.

[0045] The second aspect adopts the following technical solution: an application system of a multi-source data fusion method based on digital twins, including:

[0046] Data acquisition module, which is used to collect information from different sensors, monitoring devices and external data sources;

[0047] Data preprocessing module, used to clean and standardize the collected data to ensure data quality and consistency;

[0048] Data fusion module, used to fuse multi-source data using Kalman filter and convolutional neural network; including three submodules: Kalman filter fusion, feature layer data fusion and decision layer data fusion;

[0049] A digital twin model building module, used to build a digital twin model based on the fused data;

[0050] Decision support module, which is used to provide real-time monitoring and decision support using digital twin models.

[0051] The beneficial effects of the present invention are as follows: The core technical effects of the present application can be summarized as follows:

[0052] Efficient data fusion capability:

[0053] Through multi-source data fusion technology, data from different sources, formats and types can be effectively integrated, solving the problem of data fusion difficulties.

[0054] The Kalman filter is used for data prediction and updating, which improves the accuracy and real-time performance of data fusion.

[0055] Feature layer data fusion technology based on deep learning can extract and fuse features from different data sources, improving the processing speed and real-time performance of the system.

[0056] Real-time data processing and updating:

[0057] It realizes real-time transmission and integration of data and can adapt to dynamic changes in data.

[0058] Digital twin models can be updated dynamically and in real time to respond to changes in physical entities, thereby better reflecting their real-time status. Enhanced decision support capabilities:

[0059] Decision-level data fusion technology enables the system to integrate the prediction results of different models to obtain more accurate decisions.

[0060] Through precise data synchronization and analysis, the scientific nature and accuracy of decision-making are improved.

[0061] Scalability and flexibility:

[0062] The digital twin architecture allows new data sources and models to be easily added, facilitating the expansion and upgrade of the system.

[0063] It supports a variety of application scenarios, such as digital government affairs, intelligent manufacturing, smart cities, smart transportation, etc., and has broad application prospects.

[0064] Reduce costs and risks:

[0065] By improving the accuracy of data fusion and reducing data processing costs, unnecessary waste of resources is reduced.

[0066] Potential risks are reduced through real-time monitoring and prediction of physical entities. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the following briefly introduces the drawings required for use in the embodiment:

[0068] Figure 1 This is a block diagram of the system composition of this application;

[0069] Figure 2 This is the flow chart of data fusion;

[0070] Figure 3 This is the principle diagram of data fusion;

[0071] Figure 4 This is a schematic diagram showing the principle of the CNN model. DETAILED DESCRIPTION

[0072] In order to make the technical purpose, technical solution and beneficial effects of the present invention clearer, the technical solution of the present invention is further described below in conjunction with the drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention, that is, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0073] The present application discloses a multi-source data fusion method based on digital twins, comprising the steps of:

[0074] 1. Data collection and preprocessing

[0075] Step 1.1: Collect data from different sources and types of sensors, monitoring devices, and external data sources.

[0076] Physical entity data: Obtain real-time traffic flow, speed, and density data from vehicles, pedestrian detectors, road condition monitors, connected vehicles, radars, and cameras.

[0077] Geospatial basic data: map data, terrain data, and building information are collected through the Geographic Information System (GIS).

[0078] Sensor device data: Collect environmental data from temperature sensors, humidity sensors, and air quality sensors.

[0079] Camera data: Obtain real-time video streams from video surveillance systems to identify license plates and pedestrian behavior.

[0080] IoT system data: collects facility status and usage from smart street lights and smart parking systems.

[0081] Official Statistics: Obtain demographic information, transportation planning data from government agencies.

[0082] Social Media Data: Collect public feedback and comments about traffic conditions from social media platforms.

[0083] 2. Data Preprocessing

[0084] Step 1.2: Perform preprocessing work on the collected data by cleaning and format standardization to ensure the quality and consistency of the data.

[0085] The specific steps for preprocessing the collected data by cleaning and format standardization are as follows:

[0086] Data cleaning

[0087] De-duplicate data: Check and remove duplicate records to avoid analysis bias caused by data redundancy.

[0088] Missing value processing: missing data can be processed by interpolation (such as mean and median filling), deleting records with missing values, or using machine learning methods to predict missing values.

[0089] Outlier detection and processing: Use statistical methods (such as Z-score, IQR) or clustering-based methods to identify and process outliers.

[0090] Format standardization

[0091] Timestamp normalization: Unify timestamps from different data sources to the same time standard, such as UTC (Coordinated Universal Time).

[0092] Unit consistency: Make sure all numerical data uses the same units, such as speed in kilometers per hour or meters per second.

[0093] Data encoding: For non-numerical data, such as categorical variables, encoding conversion (such as One-Hot encoding) is performed.

[0094] Data conversion:

[0095] Data mapping: Convert unstructured data into structured data format for easy processing.

[0096] Feature engineering: creating new features, such as extracting distance and direction information from geographic location data.

[0097] Data normalization: Scale the data to a specific range, such as [0,1], to facilitate model processing.

[0098] Data security and privacy protection

[0099] Data desensitization: Desensitize sensitive information, such as license plate numbers and personal identity information.

[0100] Data encryption: Encrypt data during transmission and storage to ensure data security.

[0101] Data classification and storage

[0102] Data classification: Classify data according to data type and purpose to facilitate management and retrieval.

[0103] Data storage: Choose appropriate data storage solutions (such as relational databases, NoSQL databases) to ensure efficient storage and access of data.

[0104] Data security channel interface

[0105] Establish a secure data channel: Establish a secure data transmission channel between data and data, and between partitioned data and the big data platform to ensure the security and integrity of data transmission.

[0106] Data stream service: Provides real-time, secure, accurate and reliable data stream services to support the efficient operation of digital twin systems.

[0107] 2. Multi-source data fusion

[0108] Step 2.1: Data fusion using Kalman filter:

[0109] Prediction step: Predict the state equation at the current moment, predict the covariance matrix and observation vector based on the state equation and covariance matrix at the previous moment.

[0110] Update step: Calculate the Kalman gain and update the state estimate and covariance matrix.

[0111] Step 2.2: Use convolutional neural network (CNN) to perform feature layer data fusion:

[0112] Feature extraction: Convolutional Neural Network (CNN) is used to extract features from different data sources.

[0113] Data preparation:

[0114] Preprocessing: Before inputting into CNN, the data needs to be preprocessed, such as normalization and noise reduction.

[0115] Data format conversion: Convert different types of data into a form suitable for CNN processing. For example, convert time series data into image form, or organize geospatial data into a specific grid structure.

[0116] Model design:

[0117] Input layer: Design an appropriate input layer to accept data from different data sources.

[0118] Convolutional layers: Capture local features through a series of convolutional layers. Each convolutional layer has a set of learnable filters to detect different features.

[0119] Pooling layer: Pooling layers are optionally added to reduce the size of feature maps, thereby reducing computational complexity and preventing overfitting.

[0120] Fully connected layer: Finally, the extracted features are integrated through the fully connected layer.

[0121] Feature Extraction:

[0122] Multimodal Input: CNN can process multimodal data, i.e., different types of input data. For example, image data captured from a camera can be extracted through visual features, while time series data collected from sensors can be converted into image form and then feature extracted.

[0123] Feature Maps: Each convolutional layer produces a feature map representing the different features detected by that layer. These feature maps will be used as input to the next layer.

[0124] Feature Visualization: Visualizing the feature maps of certain intermediate layers helps understand how CNN learns and extracts features.

[0125] Fusion method:

[0126] Feature-level fusion: After feature extraction is completed, feature vectors from different data sources can be fused. For example, these feature vectors can be merged by simple concatenation or weighted fusion.

[0127] Weighted fusion: If weighted fusion is used, each feature vector will be multiplied by a weight factor and then added to form a comprehensive feature vector. The weight factor can be pre-defined or automatically learned through the training process.

[0128] Feature fusion: Different feature vectors are weighted and fused to form a single feature vector.

[0129] For each feature vector extracted from a data source, we assign a weight to it. The weight can be pre-set or obtained through training, reflecting the importance of each feature vector. The weight can be based on the relevance and effectiveness of the feature or automatically determined by a machine learning algorithm.

[0130] The weights can be pre-defined through expert knowledge.

[0131] The weights can be determined automatically by training a supervised learning model (e.g., support vector machine, linear regression).

[0132] The weights can be used to estimate the importance of features through unsupervised learning methods such as principal component analysis (PCA).

[0133] The weights can also be adjusted dynamically via reinforcement learning methods.

[0134] Before weighted fusion, it is usually necessary to normalize each feature vector to ensure that they are on the same scale.

[0135] Normalization can use L2 normalization or other normalization methods.

[0136] Step 2.3: Use weighted average method to perform decision fusion at the decision layer.

[0137] By using the weighted average method for decision fusion at the decision layer, the accuracy and reliability of the entire system can be improved, providing stronger support for digital twin technology. This approach can not only adapt to the characteristics of multi-source data, but also make full use of the advantages of various models to achieve better decision support effects.

[0138] 3. Construction and update of digital twin models

[0139] Step 3.1: Build a digital twin model based on the fused data.

[0140] Data mapping: First, the fused data is mapped to the corresponding digital twin model components. This means associating the fused data with each part of the digital twin model to ensure that each component receives relevant real-time data.

[0141] Model initialization: Initialize the digital twin model to ensure that all components are set up correctly and can receive and process data. This step may involve configuring model parameters such as physical properties and behavioral rules.

[0142] Model calibration: The model is calibrated using the fused initial dataset to ensure that the model’s output is consistent with the behavior of the actual physical entity. This may involve adjusting model parameters until the model output matches the actual observed behavior.

[0143] Model validation: Model validation is performed by comparing the results predicted by the model with the performance of the actual physical entity. If deviations are found, further adjustments to the model parameters are required until satisfactory accuracy is achieved.

[0144] Step 3.2: Update the digital twin model in real time to reflect the current status of the physical entity.

[0145] Continuous data input: The model needs to continuously receive and process new data as it flows in. This includes real-time data streams such as traffic flow and environmental changes.

[0146] Dynamic update: Based on new data, the digital twin model needs to update its status in real time to reflect the latest changes in the physical entity. For example, in a smart transportation scenario, the model needs to update the road conditions based on the latest traffic flow data.

[0147] Anomaly detection: The model also needs to be able to detect unusual events, such as traffic jams, accidents, or other emergencies, and respond quickly.

[0148] The following is a detailed description of the multi-source data fusion method based on digital twins in this application:

[0149] The present invention aims to achieve comprehensive perception and intelligent decision support for urban transportation systems by integrating and analyzing information from different sensors, monitoring equipment and external data sources. This method can effectively integrate data from different sources, formats and types, solve the problems of difficult data fusion, poor real-time performance and insufficient precision, and form an accurate digital twin model.

[0150] Specific steps of the multi-source data fusion method based on digital twins:

[0151] Data collection phase: Multi-source data acquisition is a key step in digital twins and many other data analysis technologies. It involves collecting data from different sources and types. These include: physical entities, geospatial basic data, sensor equipment, cameras and other monitoring data. Management data such as IoT systems and official and social news media statistics. These data have the characteristics of multi-source, heterogeneous, private, professional and time-sensitive.

[0152] Data preprocessing stage: The use of digital twin technology can only ensure the accuracy and reliability of the digital twin system if the quality and security of the data are guaranteed. Therefore, the collected data must be processed, including: data cleaning, format conversion, data fusion, data security, data classification, etc., to improve data storage and management. At the same time, a data security channel interface is established between data and data, and between partitioned data and the big data platform to provide real-time, secure, accurate and reliable data flow services for digital twins.

[0153] Multi-source data fusion stage, such as Figure 2 , Figure 3 : The Kalman filter is used to perform multi-source data fusion on the processed data, the data features extracted by deep learning, and the final decision data.

[0154] Kalman filter multi-source data fusion specific:

[0155] (1) Prediction steps:

[0156] According to the state equation at k-1 time and the covariance matrix P k|k-1 To predict the state equation at time k Prediction covariance matrix P k|k-1 and the observation vector Z k :

[0157]

[0158] Z k =Hx k +V

[0159] P k|k-1 =AP k-1|k-1 A T +Q

[0160] (2) Update steps:

[0161] Calculate the Kalman gain:

[0162] K k =P k|k-1 H T (HP k|k-1 H T +R) -1

[0163] The new state estimate is obtained by the Kalman gain in the above formula And the new covariance matrix P k :

[0164]

[0165] P k =(1-K k H)P k|k-1

[0166] in is the state estimation, V is the observation noise matrix at time k, P k is the estimation error covariance, K k is the Kalman gain, Z kare the observations, A and B are the state transfer matrix and control matrix, Q and R are the covariance matrices of process noise and observation noise.

[0167] Feature layer data fusion:

[0168] When extracting feature values ​​based on deep learning, refer to Figure 4 , using convolutional neural network. Adding different classification feature sets and feature samples to the CNN model for training, verification and classification can transform the input features layer by layer, convert the original data features into new features, realize the hierarchical abstraction of features, and facilitate classification and feature visualization. The advantage of feature layer fusion is that it reduces the amount of raw data to be processed, improves the system processing speed and real-time performance. The key mathematical operation in CNN is the convolution operation, which is expressed as:

[0169]

[0170] h (i,j) ) is the output at position (i, j), ω(m, n) is the multi-source data weight, x(i+m, j+m) is the input at position (i+m, j+m), b is the bias term, σ is the activation function, and the hyperbolic tangent function is selected as the activation function of the convolutional neural network. CNN gradually extracts data features through the combination of multiple convolutional layers and pooling layers.

[0171] Feature fusion: The feature vectors of different data are fused to form a single feature vector. The feature fusion method can be simple feature vector concatenation or weighted fusion. The formula for weighted fusion can be expressed as:

[0172]

[0173] Among them, f i The eigenvectors representing different modes, α i is the corresponding weight.

[0174] Decision-making data fusion

[0175] Decision fusion: In multimodal data fusion, decision fusion is usually performed at the output layer of the model, and different strategies can be used. The purpose of decision fusion is to integrate the prediction results of different models to obtain a more accurate final decision. The weighted average method is used for decision fusion.

[0176] d=f(d a , d b , d c )

[0177]

[0178] Among them, d a , d b , d c are the decision results of different modal models, f is the decision fusion function, ω i is the corresponding weight d i The eigenvectors representing different modes.

[0179] Digital twin model construction phase: Data of physical entities are collected through various sensors, cameras, Internet of Vehicles, social statistics, etc., and the fused data is mapped to the virtual space. The physical entities can be updated dynamically and in real time, and the mapping model can be studied, analyzed, and predicted to make scientific decisions.

[0180] The system structure of this application refers to Figure 1 :The application system of the multi-source data fusion method based on digital twins in this application includes:

[0181] Data acquisition module

[0182] Responsible for collecting information from different sensors, monitoring devices and external data sources. Support access to multi-source data, including but not limited to physical entity data, geospatial basic data, sensor device data, camera data, IoT system data, official statistics and social media data.

[0183] Data preprocessing module

[0184] The collected data is cleaned, formatted and pre-processed to ensure data quality and consistency. This includes sub-modules such as data cleaning, formatting standardization, data conversion, data security and privacy protection, and data classification and storage.

[0185] Data fusion module

[0186] The multi-source data fusion is carried out by using technologies such as Kalman filter and convolutional neural network (CNN). It includes three submodules: Kalman filter fusion, feature layer data fusion and decision layer data fusion.

[0187] Digital Twin Model Building Blocks

[0188] The digital twin model is built based on the fused data, including four sub-modules: data mapping, model initialization, model calibration and model verification.

[0189] Decision support module

[0190] Use digital twin models to provide real-time monitoring and decision support, including sub-modules such as real-time monitoring, anomaly detection and response, trend analysis and prediction, etc.

[0191] System functions:

[0192] Real-time data processing and integration

[0193] It supports the integration of real-time data streams and can adapt to the dynamic changes of data. It improves the accuracy of data fusion and the real-time response capability of the system through precise data synchronization.

[0194] Multi-source data fusion

[0195] The Kalman filter and CNN technology can effectively integrate data from different sources, formats and types, solving problems such as difficult data fusion, poor real-time performance and insufficient accuracy.

[0196] Digital twin model construction and update

[0197] Build and maintain a digital twin model that can be dynamically updated and respond to changes in physical entities in real time, so that the digital twin can reflect the real-time status of the physical entity.

[0198] Decision support

[0199] Use digital twin models for real-time monitoring and provide scientific decision support. Perform anomaly detection and response, trend analysis and prediction based on model output.

[0200] Application Cases

[0201] Monitor traffic flow, speed, density and other data in real time and predict traffic congestion points.

[0202] Respond quickly to traffic accidents and automatically dispatch emergency resources.

[0203] Adjust traffic light settings to optimize traffic flow.

[0204] Monitor air quality and take appropriate action.

[0205] Real-time monitoring of environmental parameters inside and outside the building, such as temperature, humidity, etc.

[0206] Automatically adjust your HVAC system to improve energy efficiency.

[0207] Predict energy demand and optimize energy distribution.

[0208] Monitor crowd density in real time to prevent safety issues caused by crowd gathering.

[0209] Quickly respond to emergencies such as fire, heavy rain, earthquake, etc.

[0210] Use social media data to conduct public opinion analysis.

[0211] System Advantages Efficient data fusion capability enables real-time data transmission and integration. Improves the accuracy and real-time performance of data fusion. Enhanced decision support capabilities improve the scientificity and accuracy of decision-making through precise data synchronization and analysis. Scalability and flexibility The digital twin architecture allows new data sources and models to be easily added, facilitating system expansion and upgrades. Reduced costs and risks Reduced unnecessary waste of resources. Potential risks are reduced through real-time monitoring and prediction of physical entities.

[0212] The present invention can integrate data from different sources and types and at different stages to provide a more comprehensive system. It can also transmit data in real time, support the integration of real-time data streams, and adapt to the dynamic changes of data, so that the digital twin can reflect the real-time status of the physical entity. Through precise data synchronization, the accuracy of data fusion is improved, the cost of data processing is reduced, and the real-time response capability of the system is improved. The digital twin architecture allows the addition and update of data sources and models to expand system functions. Digital twins, combined with multi-source data fusion, solve the pain points of data fusion difficulties in digital government affairs, intelligent manufacturing, smart cities, smart transportation and other fields, improve efficiency, reduce risks, and achieve intelligent, real-time and accurate data quantity and fusion.

[0213] The core technical effects of this application can be summarized as follows:

[0214] Efficient data fusion capability:

[0215] Through multi-source data fusion technology, data from different sources, formats and types can be effectively integrated, solving the problem of data fusion difficulties.

[0216] The Kalman filter is used for data prediction and updating, which improves the accuracy and real-time performance of data fusion.

[0217] Feature layer data fusion technology based on deep learning can extract and fuse features from different data sources, improving the processing speed and real-time performance of the system.

[0218] Real-time data processing and updating:

[0219] It realizes real-time transmission and integration of data and can adapt to dynamic changes in data.

[0220] The digital twin model can be updated dynamically and timely, responding to changes in physical entities in real time, thereby better reflecting their real-time status.

[0221] Enhanced decision support capabilities:

[0222] The decision-making layer data fusion technology enables the system to integrate the prediction results of different models to obtain more accurate decisions. Through precise data synchronization and analysis, the scientificity and accuracy of decision-making are improved.

[0223] Scalability and flexibility:

[0224] The digital twin architecture allows new data sources and models to be easily added, facilitating system expansion and upgrades. It supports a variety of application scenarios, such as digital government affairs, intelligent manufacturing, smart cities, smart transportation, etc., and has broad application prospects.

[0225] Reduce costs and risks:

[0226] By improving the accuracy of data fusion and reducing data processing costs, unnecessary waste of resources is reduced. Through real-time monitoring and prediction of physical entities, potential risks are reduced.

[0227] In summary, the most important core technical effect of this application is to provide a system and method that can integrate multi-source data in real time and accurately to support the application of digital twin technology in different fields, especially to improve the decision support capabilities and the overall performance of the system. Finally, it should be noted that the above embodiments are only used to illustrate but not limit the technical solutions of the present invention. Any equivalent substitution of the present invention and any modification or partial substitution that does not deviate from the spirit and scope of the present invention shall be covered within the scope of protection of the claims of the present invention.

Claims

1. A multi-source data fusion method based on digital twins, characterized in that: Includes steps: Collect data from sensors, monitoring devices and external data sources of different sources and types; perform pre-processing work such as cleaning and format standardization of the collected data; Multi-source data fusion, using Kalman filter for data fusion: The Kalman filter data fusion includes a prediction step: predicting the state equation at the current moment, predicting the covariance matrix and the observation vector based on the state equation and covariance matrix at the previous moment; The Kalman filter for data fusion includes the following update steps: calculating the Kalman gain, updating the state estimate and covariance matrix; and using a convolutional neural network for feature layer data fusion: Feature extraction: Use CNN to extract features from different data sources; Feature fusion: different feature vectors are weighted and fused to form a single feature vector; The weighted average method is used for decision fusion at the decision layer; Build a digital twin model based on the fused data; The digital twin model is updated in real time to reflect the immediate status of the physical entity.

2. The multi-source data fusion method based on digital twin according to claim 1 is characterized in that: The specific steps for preprocessing the collected data by cleaning and format standardization include: Check and remove duplicate records to avoid analysis bias caused by data redundancy; Handle missing data by interpolating, deleting records with missing values, or using machine learning methods to predict missing values; Unify timestamps from different data sources to the same time standard; Make sure all numerical data are in the same units; For non-numeric data, such as categorical variables, encoding conversion is performed.

3. The multi-source data fusion method based on digital twin according to claim 1 is characterized in that: The specific steps of using CNN to extract features from different data sources include: Preprocessing: Before inputting into CNN, the data needs to be preprocessed; Data format conversion: convert different types of data into a form suitable for CNN processing; Model design: Input layer: Design the input layer to accept data from different data sources; Convolutional layer: local features are captured through convolutional layers; each convolutional layer has a set of learnable filters for detecting different features; Fully connected layer: The extracted features are integrated through the fully connected layer; Feature-level fusion: After feature extraction is completed, feature vectors from different data sources can be fused.

4. The multi-source data fusion method based on digital twin according to claim 1 is characterized in that: The specific steps of forming a single feature vector by weighted fusion of different feature vectors include: assigning a weight to each feature vector extracted from each data source; the weight is pre-set or obtained through training, reflecting the importance of each feature vector; the weight can be based on the relevance and effectiveness of the feature or automatically determined through a machine learning algorithm; before weighted fusion, each feature vector needs to be normalized to ensure that they are on the same scale.

5. The multi-source data fusion method based on digital twin according to claim 1 is characterized in that: Building a digital twin model based on fused data includes: First, map the fused data to the corresponding digital twin model components; initialize the digital twin model to ensure that all components are set up correctly and can receive and process data; model calibration: secondly, calibrate the model using the fused initial data set to ensure that the output of the model is consistent with the behavior of the actual physical entity; finally, verify the model by comparing the results predicted by the model with the performance of the actual physical entity; if deviations are found, the model parameters need to be further adjusted until satisfactory accuracy is achieved.

6. The multi-source data fusion method based on digital twin according to claim 1 is characterized in that: Update the digital twin model in real time to reflect the immediate status of the physical entity. Specifically: based on new data, the digital twin model updates its status in real time to reflect the latest changes of the physical entity.

7. The application system of the multi-source data fusion method based on digital twins is characterized by: include: Data acquisition module, which is used to collect information from different sensors, monitoring devices and external data sources; Data preprocessing module, used to clean and standardize the collected data to ensure data quality and consistency; Data fusion module, used to fuse multi-source data using Kalman filter and convolutional neural network; including three submodules: Kalman filter fusion, feature layer data fusion and decision layer data fusion; A digital twin model building module, used to build a digital twin model based on the fused data; Decision support module, which is used to provide real-time monitoring and decision support using digital twin models.

Images