Intelligent data interaction processing method based on big data gateway in digital twin system

Abstract

The invention provides an intelligent data interaction processing method based on a big data gateway in a digital twinning system, and belongs to the technical field of data twinning and big data gateways, and the method comprises the steps: collecting data from multiple sources; processing the data; simulation and control are carried out through a digital twinning technology. According to the method, a real-time synchronization mechanism of the actual system and the virtual model in the digital twin system is optimized by using a big data gateway technology, so that the digital twin system can reflect the change of the actual system more quickly and more accurately. An intelligent data fusion algorithm of a big data gateway is introduced, data from a plurality of sensors and devices are intelligently fused, a more comprehensive and consistent digital twinborn view is provided, and the adaptability of the system to a complex environment is improved. The big data gateway provides richer data basis for the digital twinning system, so that an intelligent decision support system in the digital twinning system can perform prediction and decision more accurately, and the operation of the actual system is optimized.

Description

Technical Field

[0001] This invention relates to the fields of data twin and big data gateway technology, and in particular to an intelligent data interaction processing method based on a big data gateway in a digital twin system. Background Technology

[0002] A digital twin is a concept of a virtual model that corresponds one-to-one with a real-world system, reflecting the system's state and behavior in real time. The rise of digital twins can be traced back to the convergence of computer science, virtual reality, and the Internet of Things (IoT), and this technology plays a crucial role in modern industry, urban planning, and IoT applications.

[0003] Digital twin technology is based on a combination of technologies including physical systems, sensors, models, and simulations. The physical system is an actual existing device or system; sensors are used to capture data from the physical system in real time; the model is a virtual representation of the physical system; and simulation uses the model to model the behavior of the physical system. The key to digital twins lies in capturing the state of the actual system into the model through sensors, enabling the virtual model to reflect the state of the actual system in real time.

[0004] Big data gateway technology involves the collection, processing, analysis, and transmission of large-scale data to build intelligent and efficient data communication networks. It is a key technology that has emerged in the era of big data. Its main task is to achieve efficient collection, transmission, storage, and processing of large-scale data. As a bridge connecting the physical and digital layers, the big data gateway ensures the efficient and secure transmission of data in the Internet of Things (IoT) environment through various technical means.

[0005] Achieving real-time synchronization and high-fidelity simulation of digital twin systems is a significant technical challenge. Data synchronization, model updates, and precise correspondence with the actual system are issues that need to be addressed in the development of digital twin technology. Meanwhile, in the Internet of Things (IoT) environment, big data gateways need to handle diverse and distributed data sources while ensuring data security and privacy protection. Efficiently storing, transmitting, and analyzing massive amounts of data is also a technical challenge that needs to be solved.

[0006] Therefore, there is an urgent need in this field for an intelligent data interaction processing solution based on a big data gateway in a digital twin system that can efficiently store, transmit, and analyze data. Summary of the Invention

[0007] The purpose of this invention is to provide an intelligent data interaction processing technology solution based on a big data gateway in a digital twin system that can efficiently store, transmit, and analyze data.

[0008] To achieve the above objectives, the present invention provides the following solution:

[0009] A method for intelligent data interaction processing based on a big data gateway in a digital twin system includes:

[0010] Data is collected from multiple sources;

[0011] Data governance;

[0012] Simulation and control are achieved through digital twin technology.

[0013] Optionally, the collection of data from multiple sources specifically includes:

[0014] Data is collected by a data acquisition device and transmitted to a smart gateway device. The smart gateway device then transmits the data to Elasticsearch, MQTT, and Redis. The data is then processed and stored in a relational database.

[0015] Optionally, data governance specifically includes:

[0016] The data and GIS information transmitted by the smart gateway are transmitted to the big data system through the IoT real-time data transmission channel. After ETL, they are transmitted to the data governance platform and digital twin applications to form device dynamic data and GIS data.

[0017] Source data from other systems are sequentially processed through ETL, classification and cleaning, ETL, transformation and filtering, ETL, association modeling, and ETL before being transmitted to the data governance platform and digital twin applications, forming operational data and basic data.

[0018] Optionally, the simulation and control using digital twin technology includes: digital modeling, model display, model simulation, and model control.

[0019] Optionally, the digital modeling is divided into macro scenes and micro scenes; the macro scenes include two-dimensional map services and DEM elevation services; the micro scenes include oblique photogrammetry modeling and manual modeling.

[0020] Optionally, the model display may include displaying the model using WebGL technology.

[0021] Optionally, the model simulation includes integrating water resource data, video data, operation monitoring data, hydrological monitoring data, and engineering monitoring data from the IoT platform, as well as visualized data from the business system, and then performing video fusion, lighting rendering, weather rendering, monitoring signs, model animation, and flooding effects.

[0022] Optionally, the model control specifically involves: transmitting data from the physical twin to the digital twin, and the digital twin sending control commands to the physical twin based on the data.

[0023] Optionally, the data acquisition equipment includes: intelligent monitoring equipment, intelligent pan-tilt-zoom (PTZ) devices, intelligent fire-fighting equipment, intelligent lighting equipment, and environmental sensing equipment.

[0024] Optionally, the big data system includes: HBase, ElasticSearch Flink, and Kafka.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] The innovation of this invention lies in the introduction of big data gateway technology into the digital twin system. Through intelligent data interaction and processing methods, it enhances the real-time performance, comprehensiveness, and intelligence level of the digital twin system. Specific beneficial effects include:

[0027] Real-time synchronization optimization: Utilize big data gateway technology to optimize the real-time synchronization mechanism between the actual system and the virtual model in the digital twin system, ensuring that the digital twin system can reflect changes in the actual system more quickly and accurately.

[0028] Intelligent fusion of multi-source data: The intelligent data fusion algorithm of the big data gateway is introduced to intelligently fuse data from multiple sensors and devices, providing a more comprehensive and consistent digital twin view and increasing the system's adaptability to complex environments.

[0029] Upgrade of intelligent decision support system: The big data gateway provides a richer data foundation for the digital twin system, enabling the intelligent decision support system in the digital twin system to make more accurate predictions and decisions, and optimize the operation of the actual system.

[0030] Enhanced network communication and security: Leveraging big data gateway technology to strengthen network communication in digital twin systems, and using advanced encryption and authentication mechanisms to enhance the security of internal and external communication and protect the privacy of critical data. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a schematic diagram illustrating simulation and control using digital twin technology, provided as an embodiment of the present invention.

[0033] Figure 2 This is a schematic diagram of the data acquisition process provided in an embodiment of the present invention.

[0034] Figure 3This is a schematic diagram of the data governance process provided in an embodiment of the present invention.

[0035] Figure 4 This is a schematic diagram of the intelligent data interaction processing system provided in an embodiment of the present invention. Detailed Implementation

[0036] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] The purpose of this invention is to provide an intelligent data interaction processing scheme based on a big data gateway in a digital twin system that can efficiently store, transmit, and analyze data.

[0038] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0039] Example 1:

[0040] Digital twin technology has wide applications in manufacturing, industrial automation, urban planning, and healthcare. In manufacturing, digital twin technology can be used for equipment health monitoring and predictive maintenance; in urban planning, digital twin models can simulate urban infrastructure to improve urban operational efficiency; and in healthcare, digital twin technology can be used for surgical planning and medical training.

[0041] Big data gateway technology plays a crucial role in Internet of Things (IoT) applications. By collecting and integrating the massive amounts of data generated by IoT devices, big data gateways provide support for upper-layer applications. In the context of digital twin systems, the task of a big data gateway is to integrate real-time data from multiple sources into the system, ensuring the accuracy and real-time performance of the model.

[0042] This embodiment combines digital twin technology with big data gateway technology to provide a new method for real-time synchronization, processing, and optimization of digital twin models, including:

[0043] Data is collected from multiple sources;

[0044] Data governance;

[0045] Simulation and control are achieved through digital twin technology.

[0046] Furthermore, such as Figure 2 As shown, the specific process of collecting data from multiple sources is as follows:

[0047] Data is collected by a data acquisition device and transmitted to a smart gateway device. The smart gateway device then transmits the data to Elasticsearch, MQTT, and Redis. The data is then processed and stored in a relational database.

[0048] Furthermore, such as Figure 3 As shown, data governance specifically involves:

[0049] The data and GIS information transmitted by the smart gateway are transmitted to the big data system through the IoT real-time data transmission channel. After ETL, they are transmitted to the data governance platform and digital twin applications to form device dynamic data and GIS data.

[0050] Source data from other systems are sequentially processed through ETL, classification and cleaning, ETL, transformation and filtering, ETL, association modeling, and ETL before being transmitted to the data governance platform and digital twin applications, forming operational data and basic data.

[0051] Furthermore, such as Figure 1 As shown, the simulation and control using digital twin technology includes: digital modeling, model display, model simulation, and model control.

[0052] Furthermore, the digital modeling is divided into macro-scenes and micro-scenes; the macro-scenes include two-dimensional map services and DEM elevation services; the micro-scenes include oblique photogrammetry modeling and manual modeling.

[0053] Furthermore, the model display includes model display using WebGL technology.

[0054] Furthermore, the model simulation includes data access and model rendering of water resources data, video data, operation monitoring data, hydrological monitoring data, and engineering monitoring data from the IoT platform, as well as visualized data from the business system, followed by video fusion, lighting rendering, weather rendering, monitoring signs, model animation effects, and flooding effects.

[0055] Furthermore, the model control specifically involves: transmitting data from the physical twin to the digital twin, and the digital twin sending control commands to the physical twin based on the data.

[0056] Furthermore, the data acquisition equipment includes: intelligent monitoring equipment, intelligent pan-tilt-zoom (PTZ) units, intelligent fire-fighting equipment, intelligent lighting equipment, and environmental sensing equipment.

[0057] Furthermore, the big data system includes HBase, ElasticSearch Flink, and Kafka.

[0058] This embodiment also provides an intelligent data interaction processing system, such as Figure 4 As shown, it includes: a digital twin platform, a business middleware platform, and a data acquisition layer.

[0059] The data twin platform includes: application entry points, user types, and smart scenarios. The application entry points include PC and large-screen terminals. The user types include: smart buildings, smart parks, smart ports, smart airports, smart subways, smart transportation, smart cultural tourism, smart water conservancy, smart cities, and smart exhibition halls.

[0060] The business middle platform includes a data middle platform and an IoT middle platform. The data middle platform includes data capabilities and data sources. Data capabilities include data sharing, data storage, data integration, data acquisition, data governance, and databases. Data sources include MQTT data sources, relational databases, video data sources, NoSQL, Redis, and Elasticsearch. The IoT middle platform includes IoT capabilities. IoT capabilities include device access, data dashboards, online monitoring, centralized management, video streaming, and intelligent early warning.

[0061] The data acquisition layer includes a big data intelligent gateway and intelligent devices. The device gateway includes cameras, pan-tilt units, monitoring equipment, intelligent devices, environmental sensors, and lighting equipment.

[0062] This embodiment utilizes big data gateway technology to achieve intelligent fusion of multi-source data, improving the comprehensiveness and real-time performance of the digital twin system. Digital twins require real-time synchronization with the actual system's state, and the big data gateway ensures this real-time performance through intelligent data transmission and storage mechanisms. Through fusion, the digital twin system can more accurately reflect the complexity and changes of the actual system, thus enhancing its real-time performance.

[0063] The big data gateway integrates data from different sensors and devices through intelligent data fusion algorithms, providing a more consistent and comprehensive digital twin view. This helps improve the system's simulation accuracy and global perspective.

[0064] Big data gateways provide digital twin systems with more real-time and historical data, offering a stronger information foundation for intelligent decision support systems. Digital twin systems, through data analysis, machine learning, and other technologies, can make more intelligent decisions and optimize system operation.

[0065] Big data gateway technology ensures the efficiency and security of data transmission within the digital twin system, while enhancing network communication security by protecting critical data and communication processes through advanced encryption and authentication mechanisms.

[0066] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0067] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for intelligent data interaction processing based on a big data gateway in a digital twin system, characterized in that, include: Data is collected from multiple sources; Data governance; Simulation and control are achieved through digital twin technology.

2. The intelligent data interaction processing method based on a big data gateway in a digital twin system according to claim 1, characterized in that, The specific details of collecting data from multiple sources are as follows: Data is collected by a data acquisition device and transmitted to a smart gateway device. The smart gateway device then transmits the data to Elasticsearch, MQTT, and Redis. The data is then processed and stored in a relational database.

3. The intelligent data interaction processing method based on a big data gateway in the digital twin system according to claim 1, characterized in that, Data governance specifically includes: The data and GIS information transmitted by the smart gateway are transmitted to the big data system through the IoT real-time data transmission channel. After ETL, they are transmitted to the data governance platform and digital twin applications to form device dynamic data and GIS data. Source data from other systems are sequentially processed through ETL, classification and cleaning, ETL, transformation and filtering, ETL, association modeling, and ETL before being transmitted to the data governance platform and digital twin applications, forming operational data and basic data.

4. The intelligent data interaction processing method based on a big data gateway in the digital twin system according to claim 1, characterized in that, The simulation and control using digital twin technology includes: digital modeling, model display, model simulation, and model control.

5. The intelligent data interaction processing method based on a big data gateway in a digital twin system according to claim 4, characterized in that, The digital modeling is divided into macro-scenes and micro-scenes; macro-scenes include 2D map services and DEM elevation services; micro-scenes include oblique photogrammetry modeling and manual modeling.

6. The intelligent data interaction processing method based on a big data gateway in a digital twin system according to claim 4, characterized in that, The model display includes model display using WebGL technology.

7. The intelligent data interaction processing method based on a big data gateway in a digital twin system according to claim 4, characterized in that, The model simulation includes data access and model rendering of water resources data, video data, operation monitoring data, hydrological monitoring data, and engineering monitoring data from the Internet of Things platform, as well as visualized data from the business system. The simulation then performs video fusion, lighting rendering, weather rendering, monitoring signs, model animation, and flooding effects.

8. The intelligent data interaction processing method based on a big data gateway in a digital twin system according to claim 4, characterized in that, The model control specifically involves transmitting data from the physical twin to the digital twin, and then the digital twin sending control commands to the physical twin based on the data.

9. The intelligent data interaction processing method based on a big data gateway in a digital twin system according to claim 2, characterized in that, The data acquisition equipment includes: intelligent monitoring equipment, intelligent pan-tilt units, intelligent fire-fighting equipment, intelligent lighting equipment, and environmental sensing equipment.

10. The intelligent data interaction processing method based on a big data gateway in a digital twin system according to claim 3, characterized in that, The big data system includes HBase, ElasticSearch, Flink, and Kafka.

Images