Digital twin-driven three-dimensional visual immersive interaction technology

Abstract

The invention provides a digital twinning driven three-dimensional visual immersive interaction technology, and relates to the field of digital twinning three-dimensional visual. The digital twin-driven three-dimensional visual immersive interaction technology comprises a simulation system debugging function module, a digital twin communication interface development function module, a three-dimensional visual interface development function module and a visual drive communication interface development function module, and the simulation system debugging function module specifically comprises the steps of building a simulation model according to an actual topological structure of a test system; and the digital twin communication interface development function module specifically comprises a set of general physical communication module based on a Modbus protocol, and is used for communicating with hardware in a simulation process. According to the method, the simulation model is driven by the test data through digital twinning, and the simulation data and the test data are displayed on the three-dimensional visual interface, so that engineers can visually know the running states of the current test system and the simulation system in real time.

Description

Technical Field

[0001] The present invention relates to the field of digital twin three-dimensional vision, and specifically to a three-dimensional vision immersive interaction technology driven by digital twin. Background Art

[0002] Informatization and digital transformation are the current development directions of many industries in my country. With the rapid development of new-generation information technology, the integration of digital twin technology with various industries is deepening. At this stage, digital twins have the technical characteristics of digital modeling, multi-source data fusion, real-time data update, visualization, simulation analysis and intelligent decision-making. Among them, real-time data update means that the digital twin model can be connected to the real object in real time to achieve synchronous update of the digital twin model and the real object. At the same time, it can perform real-time detection and analysis of physical entities to keep abreast of their status and changes. Visualization refers to the visual presentation of the digital twin model to facilitate users to better understand and master the characteristics and behaviors of the digital twin model.

[0003] At present, the research objects of real-time data updating and visual display in China are mainly physical data, supplemented by simulation data. However, in order to facilitate operators to compare and analyze simulation data and physical data for better management and maintenance, it is very necessary to display simulation data and physical data in real time and synchronously. Summary of the Invention

[0004] (1) Technical problems solved

[0005] In response to the shortcomings of the existing technology, the present invention provides a three-dimensional visual immersive interactive technology driven by digital twins. Through digital twins, the test data drives the simulation model, and the simulation data and test data are displayed in a three-dimensional visual interface, which helps engineering personnel to understand the current operating status of the test system and simulation system in real time and intuitively.

[0006] (2) Technical solution

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions: three-dimensional visual immersive interaction technology driven by digital twins, which includes four functional modules: simulation system debugging, digital twin communication interface development, three-dimensional visual interface development, and visual driver communication interface development. The simulation system debugging functional module specifically includes building a simulation model based on the actual topological structure of the test system and calibrating and debugging the simulation model. The digital twin communication interface development functional module specifically includes a set of general physical communication modules based on the Modbus protocol, which is used for communication with hardware during the simulation process, that is, communication between the simulation model and the test bench, so as to realize The test data is now injected into the simulation model to drive the operation calculation of the simulation model. The three-dimensional visual interface development function module includes visual model development based on 3DMAX, specifically importing the three-dimensional CAD model into 3DMAX software, retopologically reconstructing the three-dimensional model, and constructing it into a visualization model, namely, the visual model. The visual driver communication interface development function module includes a set of general physical communication modules based on the TCP / IP protocol. The data source for driving visualization is MWORKS.Sysplorer. MWORKS.Sysplorer transmits data with the visualization display platform, and transmits the data to the visualization platform in real time, and the visualization interface performs real-time display.

[0008] Preferably, the simulation system debugging function module is based on MWORKS.Sysplorer software and uses Modelica language to model the system simulation model. The modeling process steps are:

[0009] S1: Disassemble the system model of the actual topology structure step by step until it is disassembled into device-level components;

[0010] S2: Build device-level models based on the physical mechanisms of each device and conduct device testing to ensure that the device can perform simulation calculations based on actual physical phenomena;

[0011] S3: Based on the test data, verify each equipment model, i.e. parameter calibration;

[0012] S4: Assemble the simulation models of each device into a system-level simulation model, ensuring that the system-level simulation model is consistent with the actual physical topology;

[0013] S5: Select typical test conditions as benchmark data for the system-level simulation model and calibrate the system-level simulation model.

[0014] Preferably, the hardware communication module within the digital twin communication interface development functional module primarily comprises three components: a Modbus protocol module, a logging module, and a configuration reading module. The entire module provides two external interfaces: a variable reading interface and a variable writing interface. Test data injected into the simulation model undergoes data screening, data processing, and other related data flows before being output to the digital twin model construction software and model simulation runtime system. This, through data interaction, closely connects the real physical world and the virtual digital world, achieving virtual-reality integration, virtual-reality mapping, and virtual-reality interaction.

[0015] Preferably, after the visual model is developed, the three-dimensional visual interface development function module can develop a visual interface and a visual display function based on Unreal Engine 4. The display function development includes multiple forms such as secondary page expansion and dynamic display.

[0016] Preferably, the specific implementation process of the visual driver communication interface development function module is that MWORKS.Sysplorer transmits the simulation software data into JSON data format by configuring the xml file, UE4 uses the TCPSocket module to receive the data, parses the JSON data and displays it in the visual interface, edits and sets the variable names of all physical quantities that need to be transmitted in the xml file, and configures one-to-one variable names in UE4 to correspond, and finally displays the corresponding variables of the passed variable values ​​on the UI interface.

[0017] (3) Beneficial effects

[0018] The present invention provides a three-dimensional visual immersive interactive technology driven by digital twins. It has the following beneficial effects:

[0019] The present invention provides a three-dimensional visual immersive interactive technology driven by digital twins, which can be applied in engineering fields that need to monitor the operating status of test systems / simulation systems. Without hindering the normal operation of the test system, the test data can be used to drive the simulation model through the digital twin, and the simulation data and test data can be displayed in a three-dimensional visual interface, which helps engineering personnel to understand the current operating status of the test system and simulation system in real time and intuitively; and when there is a large deviation between the simulation data and the test data, that is, the error value of the two is large, it can remind engineering personnel to pay special attention to the current operating status, and provide data support for engineering personnel to determine the next test operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a flowchart of the digital twin driving three-dimensional visual display of the present invention;

[0021] Figure 2This is a schematic diagram of the digital twin communication interface architecture of the present invention;

[0022] Figure 3 This is a topological diagram of the lubricating oil system module in a specific embodiment of the present invention;

[0023] Figure 4 A visual model diagram of the lubricating oil system in a specific embodiment of the present invention;

[0024] Figure 5 This is a visual interface diagram of the lubricating oil system in a specific embodiment of the present invention;

[0025] Figure 6 Flowchart of the experimental data driven 3D visual scene in a specific embodiment of the present invention

[0026] Figure 7 Encoding page diagram for a visual communication XML file in a specific embodiment of the present invention. DETAILED DESCRIPTION

[0027] The following will provide a clear and complete description of the technical solutions in the embodiments of the present invention, in conjunction with the accompanying drawings. It should be noted that the described embodiments are only some of the embodiments of the present invention, and are not intended to be exhaustive. Based on the embodiments of the present invention, all other embodiments derived by persons of ordinary skill in the art without inventive effort are intended to fall within the scope of protection of the present invention. It should be noted that the terms used herein are intended solely to describe specific embodiments and are not intended to limit the exemplary embodiments of the present invention. For ease of description, the dimensions of the various parts shown in the drawings are not drawn to scale. Technologies, methods, and devices known to persons of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, such technologies, methods, and devices should be considered part of the specification. In all examples shown and discussed herein, any specific values ​​should be interpreted as merely illustrative and not limiting. Therefore, other examples of the exemplary embodiments may have different values. It should be noted that similar reference numerals and letters denote similar items in the following figures. Therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0028] Example 1:

[0029] like Figure 1-7 As shown, an embodiment of the present invention provides a three-dimensional visual immersive interaction technology driven by a digital twin. The three-dimensional visual immersive interaction technology driven by a digital twin includes four functional modules: simulation system debugging, digital twin communication interface development, three-dimensional visual interface development, and visual drive communication interface development.

[0030] The simulation system debugging function module specifically includes building a simulation model based on the actual topology of the test system and calibrating and debugging the simulation model. Based on MWORKS.Sysplorer software, the system simulation model is modeled using the Modelica language. The modeling process steps are as follows:

[0031] S1: Disassemble the system model of the actual topology structure step by step until it is disassembled into device-level components;

[0032] S2: Build device-level models based on the physical mechanisms of each device and conduct device testing to ensure that the device can perform simulation calculations based on actual physical phenomena;

[0033] S3: Based on the test data, verify each equipment model, i.e. parameter calibration;

[0034] S4: Assemble the simulation models of each device into a system-level simulation model, ensuring that the system-level simulation model is consistent with the actual physical topology;

[0035] S5: Select typical test conditions as benchmark data for the system-level simulation model and calibrate the system-level simulation model;

[0036] The digital twin communication interface development function module specifically includes a set of general physical communication modules based on the Modbus protocol, which are used for communication with hardware during the simulation process, that is, communication between the simulation model and the test bench, so as to inject test data into the simulation model to drive the operation calculation of the simulation model. The hardware communication module mainly includes three components: Modbus protocol module, log module and configuration reading module. The entire module provides two external interfaces, variable reading interface and variable writing interface. The test data injected into the simulation model needs to go through data screening, data processing and other corresponding data processes, and finally output to the digital twin model construction software and model simulation operation system, thereby closely linking the real physical world and the virtual digital world through data interaction. Achieve the purpose of combining virtual and real, virtual and real mapping and virtual and real interaction;

[0037] The 3D visual interface development function module includes developing a visual model based on 3DMAX, specifically importing a 3D CAD model into 3DMAX software, retopologically reconstructing the 3D model into a visual model, i.e., a visual model. After the visual model is developed, the visual interface and visual display function development can be carried out based on Unreal Engine 4. The display function development includes various forms such as secondary page expansion and dynamic display.

[0038] The visual driver communication interface development function module includes a set of universal physical communication modules based on TCP / IP protocol;

[0039] The data source driving the visualization is MWORKS.Sysplorer. MWORKS.Sysplorer transmits data to the visualization display platform, passing the data to the visualization platform in real time, and the visualization interface displays the data in real time. The specific implementation process is as follows:

[0040] MWORKS.Sysplorer transmits simulation software data in JSON data format by configuring XML files. UE4 uses the TCPSocket module to receive data, parses the JSON data and displays it in a visual interface. The variable names of all physical quantities that need to be transmitted are edited and set in the XML file, and one-to-one variable names are configured in UE4 to correspond. Finally, the corresponding variables of the incoming variable values ​​are displayed on the UI interface.

[0041] Example 2:

[0042] like Figure 1-7 As shown, an embodiment of the present invention provides a three-dimensional visual immersive interactive technology driven by digital twins. This technology can drive the simulation model with test data through digital twins without hindering the normal operation of the test system. Simulation data and test data can be displayed on a three-dimensional visual interface, helping engineers to understand the current operating status of the lubricating oil test system and simulation system in real time and intuitively. It also provides data support for engineers to determine the next test operation. The data transmission and operation process between the five major modules of lubricating oil system simulation system debugging, digital twin communication interface development, three-dimensional visual interface development, and visual-driven communication interface development is shown in ERROR! REFERENCE SOURCE NOT FOUND. Its specific implementation steps are as follows:

[0043] Step 1: Debugging of lubricating oil system simulation system;

[0044] First, the Modelica language is used to build and test simulation models of various devices in the oil system, such as the oil pump, oil filter, and oil tank. After ensuring that each device-level simulation model conforms to physical mechanisms, a system-level simulation model is constructed according to the actual oil test bench topology. The oil system simulation model is calibrated and debugged to ensure normal and reliable operation. This paper uses MWORKS.Sysplorer software for simulation model building and testing.

[0045] Step 2: Development of the digital twin communication interface for the lubricating oil system;

[0046] Due to the needs of semi-physical simulation, a set of universal physical communication modules based on the Modbus protocol is needed to realize the communication with the hardware during the simulation process, that is, the communication between the lubricating oil system simulation model and the lubricating oil test bench, so as to transmit the test data such as valve opening, pump speed, lubricating oil temperature, and lubricating oil pressure to the simulation model. Among them, the control test data such as valve opening and pump speed will drive the simulation model to run the calculation, such as Figure 2 As shown, the hardware communication module mainly consists of three parts: Modbus protocol module, log module and configuration reading module. The entire module provides two external interfaces, variable reading interface and variable writing interface, among which:

[0047] Modbus protocol module: Based on the open source library libmodbus, secondary development is done to add bit reading and writing functions to adapt to the large number of bit reading requirements in the project.

[0048] Configuration reading module: The project uses XML files as a bridge to correspond the variables in the model to the PLC registers in the test bench. The configuration reading module reads and saves the variable information as the basis for the Modbus protocol module to read and write register addresses.

[0049] Log module: records the log information of calling the hardware communication module, including time, read and write variable names, read and write values, and error information.

[0050] It is necessary to configure the relevant XML file to define the data transmission content between the simulation platform and the test bench and the address of each variable. The XML file contains:

[0051] Slaver: console device node, including IP, port, and frequency of sending read and write requests.

[0052] name: variable name in the model, unique in the entire XML;

[0053] addr: register address corresponding to the variable;

[0054] bit: the bit of the register corresponding to the variable;

[0055] bRead: whether it is a read-only variable;

[0056] Unit: dimension, used for unit control of instrument display;

[0057] description: The Chinese name of the variable, which should be consistent with the variable description on the control panel.

[0058] Step 3: Development of the 3D visual interface of the lubricating oil system;

[0059] First, import the 3D CAD model of the lubricating oil system into 3DMAX software, such as Figure 3 As shown, the 3D model is retopologically constructed as follows Figure 4 The visualization model shown in the figure is then imported into the visualization display platform UnrealEngine 4 to develop the visualization interface and visualization display function. The developed visualization interface is as follows Figure 5 As shown, the center of the interface is a visual model, and the key parameters of key equipment are displayed on both sides of the interface. These parameters include simulation data, test data and error values.

[0060] Step 4: Development of visual driver communication interface;

[0061] In order to meet the data transmission between the simulation software MWORKS.Sysplorer and the visualization display platform, it is necessary to open up the data communication between the two based on the TCP / IP protocol, transmit the data to the visualization platform in real time, and drive the visualization interface for real-time display, such as Figure 6 As shown, it should be noted that the related actions of the 3D visualization model are driven by test data, but the 3D visualization interface will simultaneously display test data, simulation data and the error values ​​of the two. The communication between the two requires MWORKS.Sysplorer to transmit the simulation software data in JSON data format by configuring the XML file. UE4 uses the TCPSocket module to receive the data, parse the JSON data and display it in the visualization interface. The variable names of all physical quantities that need to be transmitted are edited and set in the XML file, as shown in the following example: Figure 7 Show.

[0062] The digital twin-driven three-dimensional visual immersive interactive technology of the present invention constructs a lubricating oil simulation system with the same topology as the lubricating oil test system through the MWORKS.Sysplorer platform based on the Modelica language, and uses the Modbus protocol to configure the digital twin communication interface to realize the communication between the test system and the simulation system, so that the test data can be injected into the simulation model in real time to drive the simulation model to perform simulation calculations. At the same time, the communication between MWORKS.Sysplorer and the visualization platform is opened up through the TCP / IP protocol, so that the test data and simulation data can be transmitted to the visualization platform in real time to drive the operation of the visualization interface, helping engineering operators to intuitively understand the operating status of the experimental system and the simulation system; and when the error value between the simulation data and the test data is large, it can remind engineering personnel to pay special attention to the current operating status, and provide data support for engineering personnel to determine the next test operation.

[0063] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. Digital twin-driven 3D visual immersive interactive technology, characterized by: The digital twin-driven 3D visual immersive interactive technology includes four functional modules: simulation system debugging, digital twin communication interface development, 3D visual interface development, and visual driver communication interface development. The simulation system debugging functional module specifically includes building a simulation model based on the actual topological structure of the test system, and calibrating and debugging the simulation model. The digital twin communication interface development functional module specifically includes a set of general physical communication modules based on the Modbus protocol, which are used for communication with hardware during the simulation process, that is, communication between the simulation model and the test bench, so as to inject test data into the simulation model to drive the operation calculation of the simulation model. The 3D visual interface development functional module includes visual model development based on 3DMAX, specifically importing the 3D CAD model into the 3DMAX software, retopologically reconstructing the 3D model, and constructing it into a visualization model, that is, a visual model. The visual driver communication interface development functional module includes a set of general physical communication modules based on the TCP / IP protocol. The data source driving the visualization is MWORKS.Sysplorer. MWORKS.Sysplorer transmits data with the visualization display platform, and transmits the data to the visualization platform in real time, and the visualization interface is displayed in real time.

2. The digital twin-driven three-dimensional visual immersive interactive technology according to claim 1, characterized in that: The simulation system debugging function module is based on MWORKS.Sysplorer software and uses Modelica language to model the system simulation model. The modeling process steps are as follows: S1: The system model of the actual topology structure is disassembled step by step until it is disassembled into device-level components; S2: Build device-level models based on the physical mechanisms of each device and perform device testing to ensure that the device can perform simulation calculations based on actual physical phenomena; S3: Verify each equipment model based on the test data, i.e. parameter calibration; S4: Assemble the simulation models of each device into a system-level simulation model to ensure that the system-level simulation model is consistent with the actual physical topology structure; S5: Select typical test conditions as benchmark data for the system-level simulation model and calibrate the system-level simulation model.

3. The digital twin-driven three-dimensional visual immersive interactive technology according to claim 1, characterized in that: The hardware communication module in the digital twin communication interface development function module mainly includes three components: Modbus protocol module, including log module and configuration reading module. The entire module provides two interfaces to the outside, variable reading interface and variable writing interface. The test data injected into the simulation model needs to go through data screening, data processing and other corresponding data processes, and finally output to the digital twin model construction software and model simulation operation system, so as to closely link the real physical world and the virtual digital world through data interaction. Achieve the purpose of virtual-real combination, virtual-real mapping and virtual-real interaction.

4. The digital twin-driven three-dimensional visual immersive interactive technology according to claim 1, characterized in that: After the visual model is developed, the three-dimensional visual interface development function module can be used to develop a visual interface and a visual display function based on Unreal Engine 4. The display function development includes multiple forms such as secondary page expansion and dynamic display.

5. The digital twin-driven three-dimensional visual immersive interactive technology according to claim 1, characterized in that: The specific implementation process of the visual driver communication interface development function module is as follows: MWORKS.Sysplorer transmits the simulation software data in JSON data format by configuring the xml file, UE4 uses the TCPSocket module to receive the data, parses the JSON data and displays it in the visual interface, edits and sets the variable names of all physical quantities that need to be transmitted in the xml file, and configures one-to-one variable names in UE4 to correspond, and finally displays the corresponding variables of the passed variable values ​​in the UI interface.

Images