Digital twin front-end and back-end data interaction method and system

Abstract

The application relates to the technical field of digital twinning, and discloses a digital twinning front-end and back-end data interaction method and system, which comprises the following steps: a digital twinning back-end regularly collects data from a data source, wherein the data source comprises system data of different application scenarios; the collected data is divided into time-sensitive data and report data; when the collected data is time-sensitive data, the digital twinning back-end filters the time-sensitive data, and then pushes the time-sensitive data to a digital twinning front-end in real time through a WebSocket long connection; when the collected data is report data, the digital twinning back-end stores the report data into a database after cleaning and analyzing the report data; when the digital twinning back-end receives a report data request initiated by the digital twinning front-end, the digital twinning back-end queries data related to the report data request from the database and sends the data to the digital twinning front-end; and the method can improve the interaction efficiency of the digital twinning front-end and back-end and reduce the load of the digital twinning front-end.

Description

Technical Field

[0001] The present invention relates to the field of digital twin technology, and in particular to a digital twin front-end and back-end data interaction method and system. Background Art

[0002] Digital twins, a core technology for the digital transformation of industries like manufacturing, energy, and transportation, enable real-time monitoring, prediction, and optimization of physical entities by constructing digital models in virtual space that mirror them. A typical digital twin system consists of a perception layer, a data layer (the digital twin backend), and an application layer (the digital twin frontend), relying on data interaction to form a closed loop. Existing technologies primarily use WebSockets to push high-quality sensor data from the digital twin backend to the digital twin frontend's 3D scene in real time, and utilize REST / APIs to enable command-result interaction between digital twin frontend operation requests and backend computational results.

[0003] However, the above solution has significant flaws: First, data categories are mixed, and time-sensitive data and cacheable data are not processed separately, resulting in low transmission efficiency. Second, synchronization blocking is a prominent problem. Report-type big data calculations and real-time data push share a common link, which easily causes transmission blockage and delays. Third, multi-source heterogeneous data processing is difficult. The data formats of systems such as MES, EAP, and overhead cranes are not unified. The digital twin front-end requires a large amount of adaptation code to be written, resulting in high maintenance costs. These problems directly lead to inefficient interaction and response delays between the front-end and back-end of the digital twin system, making it difficult to meet the high concurrency and low-latency real-time interaction requirements in complex scenarios.

[0004] Therefore, the existing technology still needs to be improved and developed. Summary of the Invention

[0005] The present invention provides a digital twin front-end and back-end data interaction method and system for improving the efficiency of digital twin front-end and back-end interaction and reducing the front-end load.

[0006] The first aspect of the present invention provides a digital twin front-end and back-end data interaction method, which includes: the digital twin back-end collects data from the data source at regular intervals, and the data source includes system data of different application scenarios; the collected data is divided into time-sensitive data and report data; when the collected data is time-sensitive data, the digital twin back-end filters the time-sensitive data, and then pushes it to the digital twin front-end in real time through a WebSocket long connection. When the collected data is report data, the digital twin back-end cleans and analyzes the report data, and then stores the report data in a database; when the digital twin back-end receives a report data request initiated by the digital twin front-end, the digital twin back-end queries the data related to the report data request from the database and sends it to the digital twin front-end.

[0007] Preferably, the data source includes an MES system, an overhead crane WebSocket server and an EAP NATS message queue, wherein the overhead crane WebSocket server is used to provide overhead crane position status data, the EAP NATS message queue is used to provide EAP machine status data, and the MES system is used to provide MES system data.

[0008] Preferably, the time-sensitive data includes overhead crane position status data and EAP machine status data, and the report data includes operation or production data in the MES system data.

[0009] Preferably, when the collected data is time-sensitive data, the digital twin backend filters the time-sensitive data and then pushes it to the digital twin front end in real time through the WebSocket long connection. When the collected data is report data, the digital twin backend cleans and analyzes the report data and stores the report data in the database, including: when the collected data is time-sensitive data, the digital twin backend performs field integrity verification, value legitimacy filtering and jitter deduplication processing on the time-sensitive data to obtain processed time-sensitive data; the digital twin backend pushes the processed time-sensitive data to the digital twin front end in real time through the WebSocket long connection; when the collected data is report data, the digital twin backend cleans and analyzes it and then stores it in the database.

[0010] Preferably, when the collected data is report data, the digital twin backend is cleaned and analyzed and then stored in the database, specifically including: when the collected data is report data, the digital twin backend processes the report data for null values ​​or outliers to obtain cleaned data; performs field conversion and statistical aggregation on the cleaned data to obtain analysis data; writes the analysis data into the TimescaleDB partition table of the database, and establishes a B-tree index with machine_id+ts as the primary key.

[0011] Preferably, when the collected data is time-sensitive data, the digital twin backend filters the time-sensitive data and then pushes it to the digital twin front end in real time through the WebSocket long connection. When the collected data is report data, the digital twin backend cleans and analyzes the report data and stores the report data in the database. It then includes: obtaining the key indicators and status codes in the report data, judging whether the change value of the key indicator is greater than the corresponding key indicator change threshold, and if so, marking the key indicator whose change value is greater than the corresponding key indicator change threshold as change data, and judging whether the status code is different from the status code collected last time, and if so, marking the current status code as change data; generating a push package based on the change data, and pushing the push package to the digital twin front end through WebSocket.

[0012] Preferably, the key indicator change thresholds include the machine yield change threshold, the equipment overall efficiency change threshold and the AGV coordinate displacement threshold. The machine yield change threshold is 2%, the equipment overall efficiency change threshold is -10%, and the AGV coordinate displacement threshold is 0.5m.

[0013] Preferably, when the digital twin backend receives the report data request initiated by the digital twin front end, the digital twin backend queries the data related to the report data request from the database and sends it to the digital twin front end, including: when the digital twin backend receives the report data request initiated by the digital twin front end, it parses the report data request to obtain request parameters; constructs a database query instruction according to the request parameters, queries the database to obtain the report data corresponding to the database query instruction, and pushes it to the digital twin front end.

[0014] The second aspect of the present invention provides a digital twin front-end and back-end data interaction system, which is used to execute the digital twin front-end and back-end data interaction method as described above. The digital twin front-end and back-end data interaction system includes a digital twin front-end and a digital twin back-end. The digital twin back-end includes a data distribution center, a database, a data transfer station and a data input and output unit. The data input and output unit communicates with the digital twin front-end, the data distribution center and the data transfer station communicate with the data input and output unit respectively, and the data distribution center communicates with the database. The data distribution center is used to receive report data, the data transfer station is used to receive time-sensitive data, the digital twin front-end is used to receive data pushed by the data input and output unit and convert it into visual interactive data, the digital twin front-end is also used to initiate a report data request to the data input and output unit, the data distribution center is used to clean and analyze the report data and store it in the database, and the data transfer station is used to filter the time-sensitive data and transmit it to the data input and output unit.

[0015] In the technical solution provided by the present invention, multi-source heterogeneous data is collected regularly by the digital twin backend and divided into time-sensitive data and report data. The time-sensitive data is filtered and pushed in real time through a WebSocket long connection to ensure the real-time dynamic interaction of the digital twin front end. The report data is cleaned and analyzed and stored in the database. When the digital twin front end initiates a report data request, the digital twin back end quickly queries and returns the processed data, realizing hierarchical processing and asynchronous caching of data, avoiding transmission blockage of real-time data and report data, and reducing the load of the digital twin front end through preprocessing, effectively solving the problems of low data interaction efficiency, response delay and high maintenance cost in the existing technology, and significantly improving the interaction efficiency and system performance of the digital twin front and back ends. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A flowchart of the digital twin front-end and back-end data interaction method provided by an embodiment of the present invention;

[0017] Figure 2 This is a structural block diagram of the digital twin front-end and back-end data interaction system provided by an embodiment of the present invention.

[0018] Description of Figure Numbers:

[0019] 100. Digital twin front-end; 200. Digital twin back-end; 201. Data input and output unit; 202. Data transfer station; 203. Data distribution center; 204. Database. DETAILED DESCRIPTION

[0020] The terms "first," "second," "third," "fourth," and so on (if any) in the description and claims of the present invention and in the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that shown or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus that includes a series of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, product, or apparatus.

[0021] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 As shown, a digital twin front-end and back-end data interaction method in an embodiment of the present invention includes:

[0022] S101. The digital twin backend regularly collects data from data sources, where the data sources include system data from different application scenarios.

[0023] S102, dividing the collected data into time-sensitive data and report data;

[0024] S103. When the collected data is time-sensitive data, the digital twin backend filters the time-sensitive data and then pushes it to the digital twin frontend in real time through a WebSocket long connection. When the collected data is report data, the digital twin backend cleans and analyzes the report data and stores the report data in the database.

[0025] S104. When the digital twin backend receives the report data request initiated by the digital twin front end, the digital twin backend queries the data related to the report data request from the database and sends it to the digital twin front end.

[0026] In this embodiment, in step S101, the data sources include the MES system, the overhead traveling crane WebSocket server, and the NATS message queue of the EAP.

[0027] In this embodiment, the overhead crane WebSocket server provides the overhead crane position status data. The overhead crane position status data comes from the automated handling equipment of the semiconductor production line, and includes coordinates (x / y / z), device ID (car_id), and timestamp (ts). The collection period is 100ms.

[0028] In this embodiment, the NATS message queue of EAP provides EAP machine status data. The EAP machine status data comes from the equipment automation system and includes the machine ID (machine_id), status code (status_code), and timestamp. The collection period is 500 ms, and the status code is preset as RUN / IDLE / DOWN / MAINT.

[0029] In this embodiment, the MES system provides MES system data. The MES system data comes from the manufacturing execution system and includes batch ID (lot_id), machine ID, yield (yield), overall equipment effectiveness (OEE), and production start and end time (start / end). The collection cycle is 5 minutes.

[0030] In this embodiment, the digital twin backend uses a timed polling method to pull data from the interfaces of each data source, supporting the parallel collection of multi-source heterogeneous data. For example, through the Flask backend application, it simultaneously connects to the overhead crane WebSocket server, EAP's NATS message queue, and the API interface of the MES system.

[0031] In this embodiment, in step S102, in this embodiment, the time-sensitive data includes overhead crane position status data and EAP machine status data. The time-sensitive data requires real-time response and high update frequency data, such as overhead crane position (100ms level) and EAP machine status (500ms level), which are used to drive the dynamic refresh of the digital twin front-end three-dimensional model.

[0032] Report data includes operating or production data in the MES system data. Report data is periodic statistics and allows asynchronous processing, such as MES yield, equipment overall efficiency and other production and operation data, which are used to generate historical reports and trend analysis.

[0033] In this embodiment, the digital twin backend identifies data types based on the data collection cycle and business scenario requirements. For example, data with a collection cycle ≤ 500ms is determined to be time-sensitive data; data with a collection cycle ≥ 5 minutes and requiring aggregation calculation is determined to be report data.

[0034] In this embodiment, in step S103, the digital twin backend uses independent transmission channels and processing threads for time-sensitive data and report data.

[0035] In this embodiment, when the collected data is time-sensitive data, the digital twin backend filters the time-sensitive data and then pushes it to the digital twin front end in real time through the WebSocket long connection. When the collected data is report data, the digital twin backend cleans and analyzes the report data and stores the report data in the database, including: when the collected data is time-sensitive data, the digital twin backend performs field integrity verification, value legitimacy filtering and jitter deduplication processing on the time-sensitive data to obtain processed time-sensitive data; the digital twin backend pushes the processed time-sensitive data to the digital twin front end in real time through the WebSocket long connection; when the collected data is report data, the digital twin backend cleans and analyzes it and then stores it in the database.

[0036] In this embodiment, field integrity checking includes discarding data when coordinate or timestamp fields are missing. Value validity filtering includes identifying vehicle coordinates outside the 0-100m range as abnormal and recording an alarm when the machine status code is not in the preset state. Jitter deduplication processing includes merging or not pushing data when the displacement or temperature change within a preset time window is less than a threshold.

[0037] For example, the digital twin backend acts as the overhead crane WebSocket client, establishes a long connection with the overhead crane WebSocket server, receives and performs field integrity verification, value legitimacy filtering and jitter deduplication processing on the overhead crane real-time data, and then pushes it to the digital twin front end; at the same time, the digital twin backend obtains the EAP machine status data by subscribing to the NATS message queue, performs field integrity verification, value legitimacy filtering and jitter deduplication processing, and then pushes it to the digital twin front end through WebSocket.

[0038] In this embodiment, when the collected data is report data, the digital twin backend cleans and analyzes it and then stores it in the database, specifically including: when the collected data is report data, the digital twin backend processes the report data for null values ​​or outliers to obtain cleaned data; performs field conversion and statistical aggregation on the cleaned data to obtain analysis data; writes the analysis data into the TimescaleDB partition table of the database, and establishes a B-tree index with machine_id+ts as the primary key.

[0039] Furthermore, when the collected data is time-sensitive data, the digital twin backend filters the time-sensitive data and then pushes it to the digital twin front end in real time through the WebSocket long connection. When the collected data is report data, the digital twin backend cleans and analyzes the report data and stores the report data in the database. It then includes: obtaining key indicators and status codes in the report data, judging whether the change value of the key indicator is greater than the corresponding key indicator change threshold, and if so, marking the key indicator whose change value is greater than the corresponding key indicator change threshold as change data, and judging whether the status code is different from the status code collected last time, and if so, marking the current status code as change data; generating a push package based on the change data, and pushing the push package to the digital twin front end through WebSocket.

[0040] In this embodiment, the key indicator change thresholds include the machine yield change threshold, the equipment overall efficiency change threshold, and the AGV coordinate displacement threshold. The machine yield change threshold is 2%, the equipment overall efficiency change threshold is -10%, and the AGV coordinate displacement threshold is 0.5m.

[0041] In this embodiment, in step S104, the digital twin backend receives the report data request initiated by the digital twin front end and parses the parameters therein (such as time range, data type, filtering conditions, etc.), and then generates corresponding database query conditions based on the parsed request parameters, retrieves qualified data from the database based on the database query conditions, and pushes it to the digital twin front end.

[0042] In this embodiment, the query results can be formatted (such as converted to JSON format) or aggregated (such as statistical indicators) to adapt to the digital twin front-end display requirements.

[0043] The report data query in this embodiment is completely based on the processed report data, does not block the real-time push link, the digital twin backend data processing is executed asynchronously, and the digital twin front-end performance is not affected.

[0044] This embodiment provides a digital twin front-end and back-end data interaction method, which regularly collects multi-source heterogeneous data through the digital twin back-end and divides it into time-sensitive data and report data. The time-sensitive data is filtered and pushed in real time through a WebSocket long connection to ensure the real-time dynamic interaction of the digital twin front-end. The report data is cleaned and analyzed and stored in the database. When the digital twin front-end initiates a report data request, the digital twin back-end quickly queries and returns the processed data, realizing hierarchical processing and asynchronous caching of data, avoiding transmission congestion of real-time data and report data, and reducing the load of the digital twin front-end through preprocessing, effectively solving the problems of low data interaction efficiency, response delay and high maintenance cost in the existing technology, and significantly improving the digital twin front-end and back-end interaction efficiency and system performance.

[0045] See also Figure 2 As shown, an embodiment of the present invention also provides a digital twin front-end and back-end data interaction system, including: a digital twin front-end 100 and a digital twin back-end 200, the digital twin back-end 200 includes a data distribution center 203, a database 204, a data transfer station 202 and a data input and output unit 201, the data input and output unit 201 communicates with the digital twin front-end 100, the data distribution center 203 and the data transfer station 202 communicate with the data input and output unit 201 respectively, the data distribution center 203 communicates with the database 204, the data distribution center 203 is used to receive report data, the data transfer station 202 is used to receive time-sensitive data, the digital twin front-end 100 is used to receive data pushed by the data input and output unit 201 and convert it into visual interactive data, the digital twin front-end 100 is also used to initiate a report data request to the data input and output unit 201, the data distribution center 203 is used to clean and analyze the report data and store it in the database 204, the data transfer station 202 is used to filter the time-sensitive data and transmit it to the data input and output unit 201.

[0046] In this embodiment, the time-sensitive data includes the crane position status data and the EAP machine status data. The report data includes the operation or production data in the MES system data.

[0047] In this embodiment, when the digital twin front end 100 initiates a report data request to the data input and output unit 201, the data distribution center 203 queries the data related to the report data request from the database 204 and transmits it to the data input and output unit 201, and the data input and output unit 201 sends it to the digital twin front end 100.

[0048] In this implementation, the digital twin backend 200 regularly collects multi-source heterogeneous data and divides it into time-sensitive data and report data. The time-sensitive data is filtered and pushed in real time through a WebSocket long connection to ensure the real-time dynamic interaction of the digital twin frontend 100. The report data is cleaned and analyzed and stored in the database 204. When the digital twin frontend 100 initiates a report data request, the digital twin backend 200 quickly queries and returns the processed data, realizing hierarchical processing and asynchronous caching of data, avoiding transmission congestion of real-time data and report data, and reducing the load of the digital twin frontend 100 through preprocessing, effectively solving the problems of low data interaction efficiency, response delay and high maintenance cost in the existing technology, and significantly improving the interaction efficiency and system performance of the digital twin frontend and backend.

[0049] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0050] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0051] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A digital twin front-end and back-end data interaction method, characterized in that: The digital twin front-end and back-end data interaction method includes: The digital twin backend regularly collects data from data sources, including system data from different application scenarios; The collected data is divided into time-sensitive data and report data. The data sources include the MES system, the crane WebSocket server, and the NATS message queue of the EAP. The crane WebSocket server is used to provide the crane position status data, the EAP NATS message queue is used to provide the EAP machine status data, and the MES system is used to provide the MES system data. The time-sensitive data includes the crane position status data and the EAP machine status data. The report data includes the operating or production data in the MES system data. The collection period of the crane position status data is 100ms, the collection period of the EAP machine status data is 500ms, and the collection period of the MES system data is 5 minutes. When the collected data is time-sensitive data, the digital twin backend filters the time-sensitive data and then pushes it to the digital twin frontend in real time through a WebSocket long connection. When the collected data is report data, the digital twin backend cleans and analyzes the report data and stores the report data in the database; Obtain the key indicators and status codes in the report data, determine whether the change value of the key indicator is greater than the corresponding key indicator change threshold, and if so, mark the key indicator whose change value is greater than the corresponding key indicator change threshold as change data, and at the same time determine whether the status code is different from the status code collected last time, and if so, mark the current status code as change data; generate a push package based on the change data, and push the push package to the digital twin front end through WebSocket; When the digital twin backend receives the report data request initiated by the digital twin frontend, the digital twin backend queries the data related to the report data request from the database and sends it to the digital twin frontend; When the collected data is time-sensitive data, the digital twin backend filters the time-sensitive data and then pushes it to the digital twin frontend in real time through the WebSocket long connection. When the collected data is report data, the digital twin backend cleans and analyzes the report data and stores the report data in the database, including: when the collected data is time-sensitive data, the digital twin backend performs field integrity verification, value legitimacy filtering and jitter deduplication processing on the time-sensitive data to obtain processed time-sensitive data; the digital twin backend pushes the processed time-sensitive data to the digital twin frontend in real time through the WebSocket long connection; When the collected data is report data, the digital twin backend cleans and analyzes it and then stores it in the database; when the collected data is report data, the digital twin backend cleans and analyzes it and then stores it in the database, specifically including: when the collected data is report data, the digital twin backend processes the report data for null values ​​or outliers to obtain cleaned data; performs field conversion and statistical aggregation on the cleaned data to obtain analysis data; writes the analysis data into the TimescaleDB partition table of the database, and establishes a B-tree index with machine_id+ts as the primary key.

2. The digital twin front-end and back-end data interaction method according to claim 1 is characterized in that: The key indicator change thresholds include the machine yield change threshold, the equipment comprehensive efficiency change threshold, and the AGV coordinate displacement threshold. The machine yield change threshold is 2%, the equipment comprehensive efficiency change threshold is -10%, and the AGV coordinate displacement threshold is 0.5m.

3. The digital twin front-end and back-end data interaction method according to claim 1 is characterized in that: When the digital twin backend receives the report data request initiated by the digital twin frontend, the digital twin backend queries the data related to the report data request from the database and sends it to the digital twin frontend, including: When the digital twin backend receives the report data request initiated by the digital twin front end, it parses the report data request to obtain request parameters; A database query instruction is constructed according to the request parameters, and report data corresponding to the database query instruction is obtained from the database and pushed to the digital twin front end.

4. A digital twin front-end and back-end data interaction system, characterized in that: The digital twin front-end and back-end data interaction system is used to execute the digital twin front-end and back-end data interaction method according to any one of claims 1 to 3. The digital twin front-end and back-end data interaction system includes a digital twin front-end and a digital twin back-end. The digital twin back-end includes a data distribution center, a database, a data transfer station and a data input and output unit. The data input and output unit communicates with the digital twin front-end, the data distribution center and the data transfer station communicate with the data input and output unit respectively, and the data distribution center communicates with the database. The data distribution center is used to receive report data, the data transfer station is used to receive time-sensitive data, the digital twin front-end is used to receive data pushed by the data input and output unit and convert it into visual interactive data. The digital twin front-end is also used to initiate a report data request to the data input and output unit. The data distribution center is used to clean and analyze the report data and then store it in the database. The data transfer station is used to filter the time-sensitive data and then transmit it to the data input and output unit.

Images