Digital twin front-end and back-end data interaction method and system

Abstract

The invention relates to the technical field of digital twinning, and discloses a digital twinning front-end and rear-end data interaction method and system, and the method comprises the steps: a digital twinning rear end collects data from a data source at regular time, and the data source comprises system data of different application scenes; dividing the collected data into aging data and report data; when the collected data is aging data, the digital twin rear end filters the aging data and pushes the aging data to the digital twin front end in real time through WebSocket long connection, and when the collected data is report data, the digital twin rear end cleans and analyzes the report data and stores the report data to a database; when the digital twin rear end receives the report data request initiated by the digital twin front end, the digital twin rear end queries the data related to the report data request from the database and sends the data to the digital twin front end, and the method can improve the interaction efficiency of the digital twin front end and the digital twin rear end and reduce the load of the digital twin front end.

Description

Technical Field

[0001] The present invention relates to the technical field of digital twins, and in particular to a digital twin front-end and back-end data interaction method and system. Background Art

[0002] As a core technology for digital transformation in industries such as manufacturing, energy, and transportation, digital twins realize real-time monitoring, prediction, and optimization of entity states by constructing digital models in the virtual space that map to physical entities. A typical digital twin system consists of a sensing layer, a data layer (digital twin back-end), and an application layer (digital twin front-end), and relies on data interaction to form a closed loop. Existing technologies mainly use WebSocket to achieve real-time push of high-timeliness sensor data from the digital twin back-end to the 3D scene of the digital twin front-end, and use REST / API to achieve instruction-result interaction between the operation requests of the digital twin front-end and the calculation results of the digital twin back-end.

[0003] However, the above solutions have significant defects: firstly, the data categories are mixed, and high-timeliness data and cacheable data are not classified and processed, resulting in low transmission efficiency; secondly, the synchronous blocking problem is prominent, and big data calculations for reports and real-time data pushes share the same link, easily causing transmission blocking and delays; thirdly, it is difficult to process multi-source heterogeneous data. The data formats of systems such as MES, EAP, and overhead cranes are not unified, and the digital twin front-end needs to write a large amount of adaptation code, resulting in high maintenance costs. These problems directly lead to low interaction efficiency and response delays between the digital twin front-end and the digital twin back-end, and it is difficult to meet the real-time interaction requirements of high concurrency and low latency in complex scenarios.

[0004] Therefore, the existing technology still needs to be improved and developed. Summary of the Invention

[0005] The present invention provides a digital twin front-end and back-end data interaction method and system for improving the interaction efficiency between the digital twin front-end and back-end and reducing the front-end load.

[0006] In the first aspect of the present invention, a digital twin front-end and back-end data interaction method is provided. The digital twin front-end and back-end data interaction method includes: the digital twin back-end regularly collects data from a data source, and the data source includes system data of different application scenarios; divides the collected data into timeliness data and report data; when the collected data is timeliness data, the digital twin back-end filters the timeliness data and then pushes it to the digital twin front-end in real time through a WebSocket long connection. When the collected data is report data, the digital twin back-end cleans and analyzes the report data and then stores the report data in a database; when the digital twin back-end receives a report data request initiated by the digital twin front-end, the digital twin back-end queries the data related to the report data request from the database and sends it to the digital twin front-end.

[0007] Preferably, the data source includes an MES system, an overhead crane WebSocket server, and an NATS message queue of an EAP. The overhead crane WebSocket server is used to provide overhead crane position status data, the NATS message queue of the EAP is used to provide EAP machine status data, and the MES system is used to provide MES system data.

[0008] Preferably, the aging data includes overhead crane position status data and EAP machine status data, and the report data includes operation or production data in the MES system data.

[0009] Preferably, when the collected data is aging data, the digital twin backend filters the aging data and then pushes it to the digital twin frontend in real time through a WebSocket long connection. When the collected data is report data, the digital twin backend cleans and analyzes the report data and then stores the report data in a database, including: when the collected data is aging data, the digital twin backend performs field integrity verification, numerical legality filtering, and jitter deduplication processing on the aging data to obtain processed aging data; the digital twin backend pushes the processed aging data to the digital twin frontend in real time through a WebSocket long connection; when the collected data is report data, the digital twin backend stores it in the database after cleaning and analysis.

[0010] Preferably, when the collected data is report data, the digital twin backend stores it in the database after cleaning and analysis, specifically including: when the collected data is report data, the digital twin backend processes null values or outliers in the report data to obtain cleaned data; performs field conversion and statistical aggregation on the cleaned data to obtain analyzed data; writes the analyzed data into the TimescaleDB partition table of the database and creates a B-tree index with machine_id + ts as the primary key.

[0011] Preferably, when the collected data is time-sensitive data, the digital twin backend filters the time-sensitive data and then pushes it to the digital twin frontend in real time through a WebSocket long connection. When the collected data is report data, the digital twin backend cleans and analyzes the report data, stores the report data in the database, and then further includes: obtaining the key indicators and status codes in the report data, determining whether the change value of the key indicator is greater than the corresponding key indicator change threshold. If so, the key indicator with a change value greater than the corresponding key indicator change threshold is marked as changed data. At the same time, it is determined whether the status code is different from the status code collected last time. If so, the current status code is marked as changed data; a push packet is generated according to the changed data and pushed to the digital twin frontend through WebSocket.

[0012] Preferably, the key indicator change thresholds include the machine good rate change threshold, the overall equipment effectiveness change threshold, and the AGV coordinate displacement threshold. The machine good rate change threshold is 2%, the overall equipment effectiveness change threshold is -10%, and the AGV coordinate displacement threshold is 0.5 m.

[0013] Preferably, when the digital twin backend receives a report data request initiated by the digital twin frontend, the digital twin backend queries the data related to the report data request from the database and sends it to the digital twin frontend, including: when the digital twin backend receives a report data request initiated by the digital twin frontend, it parses the report data request to obtain the request parameters; constructs a database query instruction according to the request parameters, queries the report data corresponding to the database query instruction from the database, and pushes it to the digital twin frontend.

[0014] In a second aspect of the present invention, a digital twin front-end and back-end data interaction system is provided. The digital twin front-end and back-end data interaction system is used to execute the digital twin front-end and back-end data interaction method as described above. The digital twin front-end and back-end data interaction system includes a digital twin frontend and a digital twin backend. The digital twin backend includes a data dispersion center, a database, a data transfer station, and a data access unit. The data access unit communicates with the digital twin frontend. The data dispersion center and the data transfer station communicate with the data access unit respectively. The data dispersion center communicates with the database. The data dispersion center is used to receive report data. The data transfer station is used to receive time-sensitive data. The digital twin frontend is used to receive the data pushed by the data access unit and convert it into visual interactive data. The digital twin frontend is also used to initiate a report data request to the data access unit. The data dispersion center is used to clean and analyze the report data and then store it in the database. The data transfer station is used to filter the time-sensitive data and then transmit it to the data access unit.

[0015] In the technical solution provided by the present invention, the digital twin backend periodically collects multi-source heterogeneous data and divides it into timeliness data and report data. The timeliness data is filtered and then pushed in real time through a WebSocket long connection to ensure the real-time nature of the dynamic interaction of the digital twin front end. The report data is cleaned and analyzed and then stored in a database. When the digital twin front end initiates a request for report data, the digital twin backend quickly queries and returns the processed data, realizing the hierarchical processing and asynchronous caching of data. This not only avoids the transmission blockage of real-time data and report data, but also reduces the load of the digital twin front end through preprocessing, effectively solving the problems of low data interaction efficiency, response delay, and high maintenance cost in the prior art, and significantly improving the interaction efficiency between the digital twin front and back ends and the system performance. Description of the Drawings

[0016] Figure 1 It is a flowchart of the digital twin front and back end data interaction method provided by an embodiment of the present invention; Figure 2 It is a structural block diagram of the digital twin front and back end data interaction system provided by an embodiment of the present invention.

[0017] Description of the reference numerals in the drawings: 100, digital twin front end; 200, digital twin backend; 201, data input / output unit; 202, data transfer station; 203, data distribution center; 204, database. Detailed Embodiment

[0018] The terms "first", "second", "third", "fourth", etc. (if any) in the specification, claims, and above-mentioned drawings of the present invention are used to distinguish similar objects and do not necessarily need to describe a specific order or sequence. It should be understood that such used data can be interchanged under appropriate circumstances so that the embodiments described herein can be implemented in an order different from that illustrated or described herein. In addition, the terms "comprising" or "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0019] For ease of understanding, the specific process of an embodiment of the present invention is described below. Please refer to Figure 1 As shown, a digital twin front and back end data interaction method in an embodiment of the present invention includes: S101. The digital twin backend periodically collects data from a data source, and the data source includes system data of different application scenarios; S102. Divide the collected data into timeliness data and report data; S103. When the collected data is real-time data, the digital twin backend filters the real-time data and then pushes it to the digital twin front end in real time through a WebSocket long connection. When the collected data is report data, the digital twin backend cleans and analyzes the report data and then stores the report data in the database. S104. When the digital twin backend receives a report data request initiated by the digital twin front end, the digital twin backend queries the data related to the report data request from the database and sends it to the digital twin front end.

[0020] In this embodiment, in step S101, the data sources include the MES system, the overhead crane WebSocket server, and the NATS message queue of the EAP.

[0021] In this embodiment, the overhead crane WebSocket server provides overhead crane position status data. The overhead crane position status data comes from the automated handling equipment on the semiconductor production line and includes coordinates (x / y / z), equipment ID (car_id), and timestamp (ts). The collection period is 100 ms.

[0022] In this embodiment, the NATS message queue of the EAP provides EAP machine status data. The EAP machine status data comes from the equipment automation system and includes machine ID (machine_id), status code (status_code), and timestamp. The collection period is 500 ms, and the status code is preset to RUN / IDLE / DOWN / MAINT.

[0023] In this embodiment, the MES system provides MES system data. The data of the MES system comes from the manufacturing execution system and includes batch ID (lot_id), machine ID, yield, overall equipment efficiency (OEE), and production start and end times (start / end). The collection period is 5 minutes.

[0024] In this embodiment, the digital twin backend uses a timed polling method to pull data from the interfaces of each data source, supporting parallel collection of multi-source heterogeneous data. For example, the Flask background application is used to connect to the overhead crane WebSocket server, the NATS message queue of the EAP, and the API interface of the MES system simultaneously.

[0025] In this embodiment, in step S102, in this embodiment, the real-time data includes overhead crane position status data and EAP machine status data. Real-time data are data that require real-time response and a high update frequency, such as overhead crane position (at the 100 ms level) and EAP machine status (at the 500 ms level), which are used to drive the dynamic refresh of the 3D model on the digital twin front end.

[0026] The report data includes the operation or production data in the MES system data. The report data is periodically statistical data that allows asynchronous processing, such as production operation data like the yield rate and overall equipment efficiency of MES, which is used to generate historical reports and trend analysis.

[0027] In this embodiment, the digital twin backend identifies the data type based on the data collection period and business scenario requirements. For example, data with a collection period ≤ 500 ms is determined to be time-sensitive data; data with a collection period ≥ 5 min and requiring aggregation calculation is determined to be report data.

[0028] In this embodiment, in step S103, the digital twin backend uses independent transmission channels and processing threads for time-sensitive data and report data.

[0029] In this embodiment, when the collected data is time-sensitive data, the digital twin backend filters the time-sensitive data and then pushes it to the digital twin front end in real time through a WebSocket long connection. When the collected data is report data, the digital twin backend cleans and analyzes the report data and then stores the report data in the database, including: when the collected data is time-sensitive data, the digital twin backend performs field integrity verification, numerical legality filtering, and jitter deduplication processing on the time-sensitive data to obtain the processed time-sensitive data; the digital twin backend pushes the processed time-sensitive data to the digital twin front end in real time through a WebSocket long connection; when the collected data is report data, the digital twin backend stores it in the database after cleaning and analysis.

[0030] In this embodiment, field integrity verification includes discarding data when the coordinate field or timestamp field is missing. Numerical legality filtering includes treating it as an anomaly when the crane coordinate exceeds the range of 0 - 100 m, and recording an alarm when the machine status code is not the preset status. Jitter deduplication processing includes merging or not pushing data when the displacement or temperature change is less than the threshold within a preset time window.

[0031] Exemplarily, the digital twin backend acts as a crane WebSocket client, establishes a long connection with the crane WebSocket server, receives and performs field integrity verification, numerical legality filtering, and jitter deduplication processing on the real-time crane data, and then pushes it to the digital twin front end; at the same time, the digital twin backend obtains the EAP machine status data by subscribing to the NATS message queue, performs field integrity verification, numerical legality filtering, and jitter deduplication processing, and then pushes it to the digital twin front end through WebSocket.

[0032] In this embodiment, when the collected data is report data, the digital twin backend performs cleaning and analysis and then stores it in the database. Specifically, when the collected data is report data, the digital twin backend processes the null values or outliers in the report data to obtain cleaned data; performs field conversion and statistical aggregation on the cleaned data to obtain analysis data; writes the analysis data into the TimescaleDB partition table of the database, and creates a B-tree index with machine_id + ts as the primary key.

[0033] Further, when the collected data is timeliness data, the digital twin backend filters the timeliness data and then pushes it to the digital twin front end in real time through a WebSocket long connection. When the collected data is report data, after the digital twin backend performs cleaning and analysis on the report data and stores the report data in the database, it further includes: obtaining the key indicators and status codes in the report data, determining whether the change value of the key indicator is greater than the corresponding key indicator change threshold. If so, mark the key indicator with a change value greater than the corresponding key indicator change threshold as changed data. At the same time, determine whether the status code is different from the status code collected last time. If so, mark the current status code as changed data; generate a push package based on the changed data and push the push package to the digital twin front end through WebSocket.

[0034] In this embodiment, the key indicator change thresholds include the machine yield change threshold, the overall equipment efficiency change threshold, and the AGV coordinate displacement threshold. The machine yield change threshold is 2%, the overall equipment efficiency change threshold is -10%, and the AGV coordinate displacement threshold is 0.5 m.

[0035] In this embodiment, in step S104, the digital twin backend receives a report data request initiated by the digital twin front end, parses the parameters therein (such as time range, data type, filtering conditions, etc.), then generates corresponding database query conditions according to the parsed request parameters, retrieves the data that meets the conditions from the database according to the database query conditions, and pushes it to the digital twin front end.

[0036] In this embodiment, the query result can be formatted (such as converted to JSON format) or aggregated and calculated (such as statistical indicators) to adapt to the display requirements of the digital twin front end.

[0037] The report data query in this embodiment is completely based on the processed report data and does not block the real-time push link. The digital twin backend data processing is executed asynchronously, and the performance of the digital twin front end is not affected.

[0038] This embodiment provides a digital twin front-end and back-end data interaction method. It regularly collects multi-source heterogeneous data through the digital twin back-end and divides it into time-sensitive data and report data. The time-sensitive data is filtered and then pushed in real time through a WebSocket long connection to ensure the real-time nature of the dynamic interaction of the digital twin front-end. The report data is cleaned, analyzed, and stored in the database. When the digital twin front-end initiates a report data request, the digital twin back-end quickly queries and returns the processed data, realizing the hierarchical processing and asynchronous caching of data. This not only avoids the transmission blockage of real-time data and report data but also reduces the load on the digital twin front-end through preprocessing, effectively solving the problems of low data interaction efficiency, response delay, and high maintenance cost in the prior art, and significantly improving the interaction efficiency between the digital twin front-end and back-end and the system performance.

[0039] Please refer to Figure 2 As shown, the embodiment of the present invention also provides a digital twin front-end and back-end data interaction system, including: a digital twin front-end 100 and a digital twin back-end 200. The digital twin back-end 200 includes a data aggregation center 203, a database 204, a data transfer station 202, and a data input / output unit 201. The data input / output unit 201 communicates with the digital twin front-end 100. The data aggregation center 203 and the data transfer station 202 respectively communicate with the data input / output unit 201. The data aggregation center 203 communicates with the database 204. The data aggregation center 203 is used to receive report data. The data transfer station 202 is used to receive time-sensitive data. The digital twin front-end 100 is used to receive the data pushed by the data input / output unit 201 and convert it into visible interactive data. The digital twin front-end 100 is also used to initiate a report data request to the data input / output unit 201. The data aggregation center 203 is used to clean and analyze the report data and then store it in the database 204. The data transfer station 202 is used to filter the time-sensitive data and then transmit it to the data input / output unit 201.

[0040] In this embodiment, the time-sensitive data includes overhead crane position status data and EAP machine status data. The report data includes operation or production data in the MES system data.

[0041] In this embodiment, when the digital twin front-end 100 initiates a report data request to the data input / output unit 201, the data aggregation center 203 queries the data related to the report data request from the database 204, transmits it to the data input / output unit 201, and the data input / output unit 201 sends it to the digital twin front-end 100.

[0042] In this embodiment, the digital twin backend 200 periodically collects multi-source heterogeneous data and divides it into real-time data and report data. The real-time data is filtered and then pushed in real time through a WebSocket long connection to ensure the real-time nature of the dynamic interaction of the digital twin frontend 100. The report data is cleaned, analyzed, and then stored in the database 204. When the digital twin frontend 100 initiates a request for report data, the digital twin backend 200 quickly queries and returns the processed data, realizing the hierarchical processing and asynchronous caching of data. This not only avoids the transmission blockage of real-time data and report data but also reduces the load on the digital twin frontend 100 through preprocessing, effectively solving the problems of low data interaction efficiency, response delay, and high maintenance cost in the prior art, and significantly improving the interaction efficiency between the digital twin front and back ends and the system performance.

[0043] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the above-described system, device, or unit can refer to the corresponding processes in the foregoing method embodiments and will not be described herein again.

[0044] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0045] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments or perform equivalent replacements for some of the technical features. These modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for digital twin front-end and back-end data interaction, characterized in that, The digital twin front-end and back-end data interaction method includes: The digital twin back-end regularly collects data from data sources, and the data sources include system data of different application scenarios; Divide the collected data into real-time data and report data; When the collected data is real-time data, the digital twin back-end filters the real-time data and then pushes it to the digital twin front-end in real-time through a WebSocket long connection. When the collected data is report data, the digital twin back-end cleans and analyzes the report data and then stores the report data in the database; When the digital twin back-end receives a report data request initiated by the digital twin front-end, the digital twin back-end queries the data related to the report data request from the database and sends it to the digital twin front-end.

2. The digital twin front-end and back-end data interaction method according to claim 1, wherein The data sources include the MES system, the overhead crane WebSocket server, and the NATS message queue of EAP. The overhead crane WebSocket server is used to provide overhead crane position status data, and the NATS message queue of EAP is used to provide EAP machine status data. The MES system is used to provide MES system data.

3. The digital twin front-end and back-end data interaction method according to claim 2, wherein, The real-time data includes overhead crane position status data and EAP machine status data, and the report data includes operation or production data in the MES system data.

4. The digital twin front-end and back-end data interaction method according to claim 1, wherein When the collected data is real-time data, the digital twin back-end filters the real-time data and then pushes it to the digital twin front-end in real-time through a WebSocket long connection. When the collected data is report data, the digital twin back-end cleans and analyzes the report data and then stores the report data in the database, including: When the collected data is real-time data, the digital twin back-end performs field integrity verification, numerical legitimacy filtering, and jitter deduplication processing on the real-time data to obtain processed real-time data; The digital twin back-end pushes the processed real-time data to the digital twin front-end in real-time through a WebSocket long connection; When the collected data is report data, the digital twin back-end stores it in the database after cleaning and analyzing.

5. The digital twin front-end and back-end data interaction method according to claim 4, wherein When the collected data is report data, the digital twin back-end stores it in the database after cleaning and analyzing, specifically including: When the collected data is report data, the digital twin back-end performs null value or outlier processing on the report data to obtain cleaned data; Perform field conversion and statistical aggregation on the cleaned data to obtain analysis data; Write the analysis data into the TimescaleDB partition table of the database and establish a B-tree index with machine_id + ts as the primary key.

6. The digital twin front-end and back-end data interaction method according to claim 1, wherein When the collected data is real-time data, the digital twin back-end filters the real-time data and then pushes it to the digital twin front-end in real-time through a WebSocket long connection. When the collected data is report data, the digital twin back-end cleans and analyzes the report data and then stores the report data in the database. After that, it also includes: Obtain the key indicators and status codes in the report data, and determine whether the change value of the key indicator is greater than the corresponding key indicator change threshold. If so, mark the key indicator with a change value greater than the corresponding key indicator change threshold as changed data. At the same time, determine whether the status code is different from the status code collected last time. If so, mark the current status code as changed data; Generate a push package according to the changed data, and push the push package to the digital twin front end through WebSocket.

7. The digital twin front-end and back-end data interaction method according to claim 6, characterized in that, The key indicator change thresholds include the machine good rate change threshold, the overall equipment efficiency change threshold, and the AGV coordinate displacement threshold. The machine good rate change threshold is 2%, the overall equipment efficiency change threshold is -10%, and the AGV coordinate displacement threshold is 0.5m.

8. The digital twin front-end and back-end data interaction method according to claim 1, characterized in that, When the digital twin backend receives a report data request initiated by the digital twin front end, the digital twin backend queries the data related to the report data request from the database and sends it to the digital twin front end, including: When the digital twin backend receives a report data request initiated by the digital twin front end, parse the report data request to obtain the request parameters; Construct a database query instruction according to the request parameters, query the report data corresponding to the database query instruction from the database, and push it to the digital twin front end.

9. A digital twin front-end and back-end data interaction system, characterized in that, The digital twin front and backend data interaction system is used to execute the digital twin front and backend data interaction method according to any one of claims 1-8. The digital twin front and backend data interaction system includes a digital twin front end and a digital twin backend. The digital twin backend includes a data aggregation center, a database, a data transfer station, and a data input / output unit. The data input / output unit communicates with the digital twin front end. The data aggregation center and the data transfer station communicate with the data input / output unit respectively. The data aggregation center communicates with the database. The data aggregation center is used to receive report data. The data transfer station is used to receive timeliness data. The digital twin front end is used to receive the data pushed by the data input / output unit and convert it into visible interaction data. The digital twin front end is also used to initiate a report data request to the data input / output unit. The data aggregation center is used to clean and analyze the report data and then store it in the database. The data transfer station is used to filter the timeliness data and then transmit it to the data input / output unit.

Images