Digital twin scene and network page interactive communication method and device and storage medium
By setting interactive communication files and instructions in the digital twin scenario, two-way communication between 3D twin scenarios and network pages is achieved, solving the problem of lack of UI development framework in Unity, improving development efficiency and reducing development cycles.
Patent Information
- Application Number
- CN202411029477.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-30
- Publication Date
- 2024-11-15
AI Technical Summary
The lack of mature UI development framework in Unity's three-dimensional software has caused twin scenarios to be unable to communicate directly with network pages, and the development process is time-consuming and labor-intensive.
By setting interactive communication files and instructions in a digital twin scenario, obtaining network pages, and sending interactive communication instructions to achieve two-way communication, including obtaining and storing interactive communication files, writing interactive communication scripts, creating embedded frameworks and listening instructions, realizing 3D Interaction between twin scenes and network pages.
It improves development efficiency, reduces development cycle, combines Unity's 3D twin scene production capabilities and web page UI development capabilities, and realizes two-way communication between 3D twin scenes and web pages.
Smart Images

Figure CN118963752A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of digital twin technology, and in particular to a method, device and storage medium for interactive communication between a digital twin scene and a network page. Background Art
[0002] Digital twin technology originated from aerospace engineering of the National Aeronautics and Space Administration (NASA) in 2010. It is used to perform process simulation of aircraft or integrated multi-physical, multi-probability, and multi-scale systems. The purpose is to create a dynamic and updateable virtual model to simulate the behavior and performance of physical entities, thereby supporting predictive maintenance, optimized operations, and product development.
[0003] With the rapid development of new-generation information technologies such as the Internet of Things (IOT), big data, cloud computing, artificial intelligence (AI) and mixed reality (MR) in recent years, digital twin technology has expanded to more fields, such as manufacturing, medical monitoring, construction, urban planning, etc., and has found application scenarios in various fields.
[0004] In the field of intelligent manufacturing, digital twins are seen as an important means to achieve interactive integration between the information world and the physical world. It allows companies to simulate, predict and optimize based on virtual models during the entire product life cycle, including design, production, operation and maintenance, thereby improving efficiency and reducing costs and risks. With the continuous advancement of technology and the reduction of costs, digital twins are expected to play a key role in more industries and become an important force in promoting digital transformation and innovation.
[0005] Unity is a professional 3D real-time development engine used for 3D scene creation, physical simulation and real-time rendering. It can create and publish twin scenes on the WebGL platform. However, the twin scene cannot communicate directly with the Vue project. There is no mature framework for UI development in 3D software such as Unity. Developers need to stack pictures and text elements to form lists, pop-ups and other content, which is time-consuming and labor-intensive, so it needs to be improved. Summary of the invention
[0006] In view of this, the present invention provides a method, device and storage medium for interactive communication between a digital twin scene and a network page.
[0007] Specifically, the present invention is achieved through the following technical solutions:
[0008] According to a first aspect of the present invention, a method for interactive communication between a digital twin scene and a web page is provided, the method comprising the steps of:
[0009] Get digital twin scenarios;
[0010] Setting an interactive communication file in the digital twin scenario;
[0011] Setting interactive communication instructions for the interactive communication file;
[0012] Get the web page;
[0013] sending an interactive communication instruction to the network page;
[0014] Obtain the interactive event fed back by the interactive communication instruction.
[0015] Optionally, setting the interactive communication file in the digital twin scene includes the steps of:
[0016] Search the engineering directory of the digital twin scenario;
[0017] Search for plugin folders in the project directory;
[0018] Obtain interactive communication files;
[0019] The interactive communication file is stored in the plug-in folder.
[0020] Optionally, the step of obtaining the interactive communication file comprises the following steps:
[0021] Write initial interactive communications documentation;
[0022] The interactive communication file is adjusted according to the interactive communication grammar requirement and an interactive communication file is obtained.
[0023] Optionally, the step of setting the interactive communication instruction for the interactive communication file comprises the steps of:
[0024] Obtaining a calling script for the digital twin scene to call the interactive communication file;
[0025] Storing the calling script in the digital twin scene;
[0026] Creating a communication script for interactive communication between the network page and the interactive communication file;
[0027] The communication script is stored in the interactive communication file.
[0028] Optionally, the creating of a communication script for interactive communication between the network page and the interactive communication file comprises the steps of:
[0029] Obtain the web 3D drawing protocol in the digital twin scene;
[0030] Obtaining a markup language file in the webpage three-dimensional drawing protocol;
[0031] An interactive communication event is added to the markup language file.
[0032] Optionally, storing the communication script in the interactive communication file comprises the steps of:
[0033] creating an embedded frame in the interactive communication document;
[0034] The web page text is introduced into the embedded frame.
[0035] Optionally, the step of setting the interactive communication instruction for the interactive communication file comprises the steps of:
[0036] Setting a network webpage monitoring instruction in the interactive communication file;
[0037] The digital twin scene calling instruction is set in the interactive communication file.
[0038] According to a second aspect of the present invention, there is provided a device for interactive communication between a digital twin scene and a web page, comprising:
[0039] The scene acquisition module is used to obtain the digital twin scene;
[0040] A file setting module, used to set interactive communication files in the digital twin scene;
[0041] An instruction setting module, used for setting interactive communication instructions for the interactive communication file;
[0042] Page acquisition module, used to acquire network pages;
[0043] An instruction sending module, used for sending interactive communication instructions to the network page;
[0044] The event acquisition module is used to acquire the interactive event fed back by the interactive communication instruction.
[0045] According to a third aspect of the present invention, there is provided an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the aforementioned methods when executing the program.
[0046] According to a fourth aspect of the present invention, there is provided a storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the steps of any one of the aforementioned methods.
[0047] The technical solution provided by the present invention brings at least the following beneficial effects:
[0048] The digital twin scene and network page interactive communication method, device and storage medium provided in the present application can combine Unity's 3D twin scene production capabilities and web page UI development capabilities to achieve two-way communication between 3D twin scenes and web pages, thereby improving development efficiency and reducing development cycles. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.
[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0051] Figure 1 A schematic diagram of a flow chart of a method for interactive communication between a digital twin scene and a web page provided in an embodiment of the present invention;
[0052] Figure 2 A schematic diagram of the structure of a digital twin scene and web page interactive communication device provided in an embodiment of the present invention;
[0053] Figure 3 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention;
[0054] Figure 4 A schematic diagram of the structure of a storage medium provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0055] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0056] Figure 1 A flow chart of a method for interactive communication between a digital twin scene and a web page applicable to an embodiment of the present invention is schematically shown.
[0057] See also Figure 1 , an embodiment of the present invention provides a method for interactive communication between a digital twin scene and a network page, which can be applied to electronic devices, such as PCs, servers, terminals, etc. The method may include the following steps:
[0058] S1: Obtain digital twin scene;
[0059] In an embodiment of the present application, Unity3D software is used to create a digital twin scene, and the digital twin scene is stored in the Unity3D software.
[0060] S2: setting an interactive communication file in the digital twin scenario;
[0061] Exemplarily, the step of setting an interactive communication file in the digital twin scene includes the following steps:
[0062] Search the engineering directory of the digital twin scenario;
[0063] Search for plugin folders in the project directory;
[0064] Obtain interactive communication files;
[0065] The interactive communication file is stored in the plug-in folder.
[0066] In the embodiment of the present application, a project directory Unity Assets is provided in the Unity3D software, a plug-in folder Plugins subfolder is provided in the project directory Unity Assets, and the interactive communication file is placed in the plug-in folder Plugins subfolder.
[0067] Exemplarily, the step of obtaining the interactive communication file includes the following steps:
[0068] Write initial interactive communications documents;
[0069] The interactive communication file is adjusted according to the interactive communication grammar requirement and an interactive communication file is obtained.
[0070] In the embodiment of the present application, JavaScript code is used to write the initial interactive communication file. Since the interactive communication file is placed in the Plugins subfolder of the plug-in folder with a .jslib extension, the jslib file needs to be adjusted to conform to the interactive communication grammar Js grammar. The jslib script format is as follows:
[0071]
[0072]
[0073] Exemplarily, the step of setting the interactive communication instruction for the interactive communication file comprises the following steps:
[0074] Obtaining a calling script for the digital twin scene to call the interactive communication file;
[0075] Storing the calling script in the digital twin scene;
[0076] Creating a communication script for interactive communication between the network page and the interactive communication file;
[0077] The communication script is stored in the interactive communication file.
[0078] In an embodiment of the present application, C# provides a DllImport method under the namespace System.Runtime.InteropServices, which is a calling script that allows the introduction of unmanaged code assemblies. The calling script DllImport method is stored in the Unity software, and the external assembly can be called through the DllImport method in the digital twin scene of the Unity software. The method of Unity calling jslib (written in a class inherited from MonoBehaviour, and both the class and the script are named GameRoot, as a script specifically used to handle interactive communication between Unity and the web, that is, a communication script) is added to the object of the same name in the Unity scene, that is, the communication script is stored in the interactive communication file.
[0079] The Unity calling method is as follows:
[0080] [DllImport("__Internal")]
[0081] private static extern void SendToHTML(string unityData); / / The method name is consistent with jslib
[0082] Exemplarily, the creation of a communication script for interactive communication between the network page and the interactive communication file comprises the following steps:
[0083] Obtain the web 3D drawing protocol in the digital twin scene;
[0084] Obtaining a markup language file in the webpage three-dimensional drawing protocol;
[0085] An interactive communication event is added to the markup language file.
[0086] In an embodiment of the present application, the html file (i.e., markup language file) in the Webgl (web 3D graphics protocol) project is obtained in the Unity software, and then interactive communication events are added to the markup language file. The notification front-end event can be called to obtain the corresponding data.
[0087] The interactive communication event content is as follows:
[0088]
[0089]
[0090] Exemplarily, the step of storing the communication script in the interactive communication file includes the following steps:
[0091] creating an embedded frame in the interactive communication document;
[0092] The web page text is introduced into the embedded frame.
[0093] In the embodiment of the present application, the unity software package content is placed under the public class of the vue (3D rendering solution product) project, an iframe tag (embedded frame) is created in the project and the index.html file (web page text) is introduced. The specific format content is as follows:
[0094] <iframe id="iframe" ref="iframe" src=" / unity / index.html" width="100%" height="100%" frameborder="0" scrolling="auto" / >.
[0095] S3: setting interactive communication instructions for the interactive communication file;
[0096] Exemplarily, the step of setting the interactive communication instruction for the interactive communication file comprises the following steps:
[0097] Setting a network webpage monitoring instruction in the interactive communication file;
[0098] The digital twin scene calling instruction is set in the interactive communication file.
[0099] In the embodiment of the present application, a network webpage monitoring instruction is set in the vue file (i.e., the interactive communication file) introduced into unity. The specific code is as follows:
[0100]
[0101]
[0102] At this point, Unity has completed the communication, calling and parameter passing operations with the front-end Vue. Then, the front-end Vue has completed the communication, calling and parameter passing operations with Unity through the following steps. After that, Vue calls the object of the iframe page to pass parameters to Unity (that is, the digital twin scene call instruction). The specific code is as follows:
[0103]
[0104] In the HTML file of the Webgl project packaged by Unity, add a custom event, call to notify Unity of the event, and then get the corresponding data on the Unity side. The content of the custom event is as follows:
[0105] function locateDevice(obj){
[0106] unityInstance.SendMessage("GameRoot","GetHTMLEvent",obj);
[0107] };
[0108] Unity processes the obj parameter obtained by the GetHTMLEvent method and writes to implement 3D functions.
[0109] S4: Get the network page;
[0110] S5: Sending an interactive communication instruction to the network page;
[0111] S6: Obtaining the interactive event fed back by the interactive communication instruction.
[0112] In the embodiment of the present application, an interactive communication instruction is sent to a network page, and the network page feeds back an interactive event to the interactive communication instruction.
[0113] like Figure 2 , the present application provides a digital twin scene and network page interactive communication device, including:
[0114] A scene acquisition module 10 is used to acquire a digital twin scene;
[0115] A file setting module 20, used to set an interactive communication file in the digital twin scene;
[0116] An instruction setting module 30, used for setting interactive communication instructions for the interactive communication file;
[0117] The page acquisition module 40 is used to acquire a network page;
[0118] An instruction sending module 50, used for sending interactive communication instructions to the network page;
[0119] The event acquisition module 60 is used to acquire the interactive event fed back by the interactive communication instruction.
[0120] A digital twin scene and web page interactive communication device provided in the present application can execute a digital twin scene and web page interactive communication method provided in the above steps.
[0121] It should be understood that the above specific embodiments of the present invention are only used to illustrate or explain the principles of the present invention, and do not constitute a limitation of the present invention. Therefore, any modifications, equivalent substitutions, improvements, etc. made without departing from the spirit and scope of the present invention should be included in the protection scope of the present invention. In addition, the appended claims of the present invention are intended to cover all changes and modifications that fall within the scope and boundaries of the appended claims, or the equivalent forms of such scope and boundaries.
[0122] Reference below Figure 3 , which shows a schematic diagram of the structure of an electronic device 100 suitable for implementing the embodiment of the present disclosure. The electronic device in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0123] like Figure 3 As shown, the electronic device 100 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 101, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 102 or a program loaded from a storage device 108 to a random access memory (RAM) 103. In the RAM 103, various programs and data required for the operation of the electronic device 100 are also stored. The processing device 101, the ROM 102, and the RAM 103 are connected to each other via a bus 104. An input / output (I / O) interface 105 is also connected to the bus 104.
[0124] Typically, the following devices may be connected to the I / O interface 105: an input device 106 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 107 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 108 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 109. The communication device 109 may allow the electronic device 100 to communicate with other devices wirelessly or by wire to exchange data. Although the electronic device 100 having various devices is shown in the figure, it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have alternatively.
[0125] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 109, or installed from the storage device 108, or installed from the ROM 102. When the computer program is executed by the processing device 101, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.
[0126] Reference below Figure 4 , which shows a structural schematic diagram of a computer-readable storage medium suitable for implementing an embodiment of the present disclosure, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it can implement any of the digital twin scene and network page interactive communication methods described above.
[0127] The present application also provides a computer program product, including a computer program / computer executable instructions, which, when executed by a processor of an electronic device, implements the steps of any of the aforementioned methods for interactive communication between a digital twin scene and a web page.
[0128] The digital twin scene and network page interactive communication method, device and storage medium provided in the present application can combine Unity's 3D twin scene production capabilities and web page UI development capabilities to achieve two-way communication between 3D twin scenes and web pages, thereby improving development efficiency and reducing development cycles.
[0129] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0130] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.
Claims
1. A method for interactive communication between a digital twin scene and a web page, characterized in that: The method comprises the steps of: Get digital twin scenarios; Setting an interactive communication file in the digital twin scenario; Setting interactive communication instructions for the interactive communication file; Get the web page; sending an interactive communication instruction to the network page; Obtain the interactive event fed back by the interactive communication instruction.
2. The method for interactive communication between a digital twin scene and a web page according to claim 1, characterized in that: The step of setting the interactive communication file in the digital twin scene comprises the following steps: Search the engineering directory of the digital twin scenario; Search for plugin folders in the project directory; Obtain interactive communication files; The interactive communication file is stored in the plug-in folder.
3. The method for interactive communication between a digital twin scene and a web page according to claim 2, characterized in that: The obtaining of the interactive communication file comprises the steps of: Write initial interactive communications documentation; The interactive communication file is adjusted according to the interactive communication grammar requirement and an interactive communication file is obtained.
4. The method for interactive communication between a digital twin scene and a web page according to claim 1, characterized in that: The step of setting the interactive communication instruction for the interactive communication file comprises the following steps: Obtaining a calling script for the digital twin scene to call the interactive communication file; Storing the calling script in the digital twin scene; Creating a communication script for interactive communication between the network page and the interactive communication file; The communication script is stored in the interactive communication file.
5. The method for interactive communication between a digital twin scene and a web page according to claim 4, characterized in that: The communication script for creating the network page to interact with the interactive communication file comprises the following steps: Obtain the web 3D drawing protocol in the digital twin scene; Obtaining a markup language file in the webpage three-dimensional drawing protocol; An interactive communication event is added to the markup language file.
6. The method for interactive communication between a digital twin scene and a web page according to claim 4, characterized in that: The step of storing the communication script in the interactive communication file comprises the following steps: creating an embedded frame in the interactive communication document; The web page text is introduced into the embedded frame.
7. The method for interactive communication between a digital twin scene and a web page according to claim 1, characterized in that: The step of setting the interactive communication instruction for the interactive communication file comprises the following steps: Setting a network webpage monitoring instruction in the interactive communication file; The digital twin scene calling instruction is set in the interactive communication file.
8. A digital twin scene and web page interactive communication device, characterized in that: include: The scene acquisition module is used to obtain the digital twin scene; A file setting module, used to set interactive communication files in the digital twin scene; An instruction setting module, used for setting interactive communication instructions for the interactive communication file; Page acquisition module, used to acquire network pages; An instruction sending module, used for sending interactive communication instructions to the network page; The event acquisition module is used to acquire the interactive event fed back by the interactive communication instruction.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 7 are implemented.
10. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.
