Control interaction system based on live-action twin virtual digital scene graph

Abstract

The utility model discloses a control interaction system based on a live-action twin virtual digital scene graph, which comprises a cloud server arranged at a cloud end, and further comprises a sensor, a control unit and an interaction terminal which are connected with the cloud server in a real scene, according to the utility model, a real-scene twinning virtual digital scene graph database of a real scene in which a sensor model and a control unit model are implanted is stored in the cloud server, and the cloud server is used for calling a corresponding scene graph database for comparison, loading and coverage according to the visual angle of the interactive terminal in the real scene. A user is connected with a cloud server through a mobile terminal and VR or AR equipment in a real scene, a corresponding prompt label is triggered in the terminal to send out a corresponding control instruction, deep fusion of a virtual scene generated in the real scene and the real scene is achieved, and interaction control is carried out in an immersive highly-simulated scene.

Description

Technical Field

[0001] The utility model relates to the technical field of digital twin simulation, in particular to a control interaction system based on a real-scene twin virtual digital scene graph. Background Art

[0002] At present, with the continuous development of digital technology and intelligent control, people's demand for the integration of virtual and reality is increasing. At present, in the existing technical landscape, the interactive control of virtual and reality often presents certain limitations. Although the existing interactive system can realize remote interaction, it has deficiencies in terms of poor flexibility, inconsistency with people's actual operating habits, unnatural human-computer interaction, poor visual experience of the interactive interface, and adaptability to diverse scenarios. It cannot effectively meet users' expectations for high-quality, convenient and innovative interactive experience. These existing problems not only limit the development of virtual and reality integration applications, but also hinder the widespread promotion and in-depth application of related technologies in fields such as smart homes, smart cities, and smart multi-functional scenarios to a certain extent. Summary of the invention

[0003] In view of the deficiencies of the above prior art, the utility model provides a control interaction system based on a real-scene twin virtual digital scene graph.

[0004] The technical solution provided by the utility model is: a control interaction system based on a real-scene twin virtual digital scene graph, comprising a cloud server arranged in the cloud, and also comprising a sensor, a control unit and an interactive terminal connected to the cloud service area in the real scene;

[0005] The cloud server stores a scene graph database based on the real scene twin virtualization with sensor models and control unit models implanted therein, and is used to call the corresponding scene graph database for comparison, loading, and overwriting according to the perspective of the interactive terminal in the real scene, and can interactively control the control unit in the real scene according to the interactive instructions issued by the interactive terminal;

[0006] The interactive terminal is used to connect to the cloud server in the real scene to transmit the on-site perspective image, receive the corresponding scene graph data in the perspective loaded by the cloud server, and send interactive control instructions to the cloud server.

[0007] Furthermore, the interactive terminal includes but is not limited to: mobile phones, tablet computers, VR devices, and AR devices.

[0008] Furthermore, the control unit is connected to a control actuator and a remote communication device, the control actuator is used to receive execution control instructions, and the remote communication device is used to connect to a cloud server.

[0009] Furthermore, the control unit includes but is not limited to: light switches, valves, motors, electronic locks, door magnetic switches, and can flexibly customize sensors and control units according to different application scenarios and user needs. It is suitable for interactive control fields in different scenarios.

[0010] Furthermore, the sensors include but are not limited to: position sensors, displacement sensors, temperature sensors, humidity sensors, radar sensors, pressure sensors, infrared sensors, and variable frequency power sensors, and sensors and control units can be flexibly customized according to different application scenarios and user needs.

[0011] Furthermore, the communication methods between the cloud server and the sensors, control units and interactive terminals include but are not limited to: Wi-Fi, Bluetooth, radar, Zigbee, to achieve data transmission between various parts of the system and ensure the real-time and accuracy of control instructions.

[0012] The beneficial effects of the utility model are as follows: a cloud server stores a scene graph database based on a real-life twin virtual scene implanted with a sensor model and a control unit model, which is used to call the corresponding scene graph database for comparison, loading, and overwriting according to the perspective of the interactive terminal in the real scene, and can interactively control the control unit in the real scene according to the interactive instructions issued by the interactive terminal; the user connects to the cloud server through the interactive terminal in the real scene, and issues corresponding control instructions by triggering corresponding prompt annotations in the perspective of the terminal. The control instructions are loaded into the control unit of the real scene through the cloud server and the corresponding control instructions are executed, thereby realizing the deep integration of the virtual scene generated in the real scene with the real scene, and interactive control in an immersive and highly simulated scene.

[0013] The utility model realizes a high degree of simulation of the virtual reality of the interactive control interface. The interactive control method conforms to the operating habits in reality and provides a real-time feedback mechanism to ensure the accurate execution and timely response of the control instructions. The system is highly scalable and can flexibly customize sensors and control units according to different application scenarios and user needs. The virtual scene map covers a variety of real scenes such as real scenes, animations, and hand-painted scenes. The visual experience is strong, and a deep and efficient control interaction between virtual and reality is realized. It can be applied to common lighting control, scene interaction, waterscape management, music playback, and can also be extended to the field of precise control technology of various electronic devices. It can perform interactive control of switch quantities and analog quantities, and can be applied to tourist attractions, urban landscapes, buildings, industrial production line control, education and other related fields.

[0014] The utility model can integrate various types of sensors and control units to achieve multi-element collaborative control in complex scenarios, improve the overall performance and stability of interactive control, and can be applied to bring innovative solutions and significant value enhancement to various fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a schematic block diagram of the structure of the utility model. DETAILED DESCRIPTION

[0016] The technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0017] See also Figure 1 , a control interaction system based on a real-scene twin virtual digital scene graph, comprising a cloud server arranged in the cloud, and also comprising a sensor, a control unit and an interactive terminal connected to the cloud server in a real scene;

[0018] The cloud server stores a scene graph database based on the real scene twin virtualization with sensor models and control unit models implanted therein, and is used to call the corresponding scene graph database for comparison, loading, and overwriting according to the perspective of the interactive terminal in the real scene, and can interactively control the control unit in the real scene according to the interactive instructions issued by the interactive terminal;

[0019] The interactive terminal is used to connect to the cloud server in the real scene to transmit the on-site perspective image, receive the corresponding scene graph data in the perspective loaded by the cloud server, and send interactive control instructions to the cloud server.

[0020] The interactive terminals include but are not limited to mobile phones, tablet computers, VR devices, and AR devices. The system can support the use of any terminal device or multiple terminals at the same time.

[0021] The control unit is connected to a control actuator and a remote communication device, the control actuator is used to receive and execute control instructions, and the remote communication device is used to connect to a cloud server.

[0022] The control unit includes but is not limited to: light switches, valves, motors, electronic locks, door magnetic switches, and can flexibly customize sensors and control units according to different application scenarios and user needs. It is suitable for different scenarios and control fields.

[0023] The sensors include but are not limited to: position sensors, displacement sensors, temperature sensors, humidity sensors, radar sensors, pressure sensors, infrared sensors, and variable frequency power sensors, and sensors and control units can be flexibly customized according to different application scenarios and user needs.

[0024] The communication methods between the cloud server and the sensors, control units and interactive terminals include but are not limited to: Wi-Fi, Bluetooth, radar, Zigbee, so that multiple data transmission methods can be used between systems to form system redundancy and ensure the real-time, reliability and accuracy of control instructions.

[0025] The specific implementation process is as follows:

[0026] Step S1: Based on the accurate mapping and modeling of the real scene, a digital copy that completely corresponds to the real scene is constructed, the spatial layout and object positions are simulated with high precision, a 3D or 2D virtual scene graph of the real scene is generated, and the geographic location coordinates are embedded in the virtual scene graph;

[0027] Step S2: creating sensor models and control unit models for the corresponding positions of the sensors and control units in the real scene in the virtual scene graph, and implanting triggerable prompt annotations for interactive control into the control unit model; establishing an association and mapping relationship between the geographic coordinates and the interactive mechanism between the control unit model and the prompt annotation, calibrating and matching the coordinates of the virtual scene graph with the coordinates of the sensor model and the control unit model, and constructing a scene graph database integrating the sensor model and the control unit model;

[0028] Step S3, uploading the scene graph database of step S2 to the cloud server, and connecting the sensors and control units in the real scene to the cloud server through a secure network;

[0029] Step S4: The user connects to the cloud server through the interactive terminal in the real scene. The cloud server calls the corresponding scene graph database for comparison according to the perspective of the user's interactive terminal, and loads the corresponding data of the scene graph database to cover the corresponding real scene in the perspective of the interactive terminal in a transparent or semi-transparent manner. The control unit in the real scene displays the corresponding triggerable prompt annotation in the perspective of the interactive terminal, and the sensor displays the real-time parameters of the sensor; the user triggers the corresponding prompt annotation through the interactive terminal to issue the corresponding control instruction, and the control instruction is loaded to the control unit of the real scene through the cloud server and executes the corresponding control instruction. The sensor and the control unit upload the real-time status information to the cloud server, so as to realize interactive control of the control unit in the real scene through the interactive terminal in the real scene.

[0030] In this embodiment, the step S1, constructing a virtual scene graph method includes but is not limited to: digital twin map, AR map, animation map, hand-drawn map, and establishing associations and mapping relationships between them.

[0031] In step S2, the triggerable prompt annotation is integrated with the geographic location coordinates and is placed at the corresponding position of the virtual scene graph for display. To ensure that the prompt annotation matches the coordinate system and interaction mechanism of the virtual real scene graph, the triggerable prompt annotation uses a 3D rendering library or a 3D engine to load the prompt annotation model into the scene graph and ensure that it is displayed at the correct position.

[0032] In step S2, the triggerable prompt annotation is integrated with a TCP / UDP link, and a control instruction can be sent in a triggering form, so that interactive control can be performed naturally and quickly.

[0033] In step S2, the data in the scene graph database is divided into different data display levels, and the data display levels are from high to low: triggerable prompt annotations, sensor models and control unit models, scene graph data and geographic location coordinates. Hierarchical display is achieved from the user's perspective, and the triggerable prompt annotations and control unit models are displayed in front of the scene graph to achieve accurate and reliable interactive control.

[0034] In step S2, the sensor model and the control unit model are established in the virtual scene graph, and the model is simplified, the data is compressed, and the cache mechanism is optimized to improve the response speed and reliability of the interactive control.

[0035] In the step S4, the status information of the sensor model and the control unit model includes switch quantity status information and analog quantity status information, the control instructions include switch quantity control instructions and analog quantity control instructions, and the control instructions have instruction retransmission and error correction mechanisms.

[0036] In step S4, the methods of triggering the prompt labeling include but are not limited to: touch screen, voice command, gesture, which conforms to people's common operating habits and realizes a natural, realistic and friendly interactive interface.

[0037] In step S4, the user connects to the cloud server through the interactive terminal in the real scene, and the cloud server calls the corresponding scene graph database for comparison according to the user's perspective, and loads the corresponding data of the scene graph database in a transparent or semi-transparent manner to cover the corresponding real scene in the perspective of the interactive terminal; the control unit in the real scene displays the corresponding triggerable prompt mark in the perspective of the interactive terminal, and the sensor displays the real-time parameters of the sensor; the user issues the corresponding control instruction by triggering the corresponding prompt mark, and the control instruction is loaded to the control actuator of the control unit of the real scene through the cloud server to execute the corresponding control instruction.

[0038] In step S4, after receiving the control instruction from the interactive terminal, the cloud server parses, classifies and retrieves the control instruction, and after retrieving the corresponding instruction, distributes the instruction to the corresponding control object through data communication, and the distribution method includes but is not limited to one-to-one or one-to-many methods; after receiving the distribution instruction from the cloud server, the execution device terminal of the control unit compares the terminal device database of the control unit, calls the effect stored in the terminal device to perform corresponding control output. The redundancy, real-time and accuracy of the control instruction distribution are improved.

[0039] Users can also remotely connect to the cloud server through a PC terminal, access the sensor model in the virtual scene graph, or trigger the corresponding prompt annotation to issue corresponding control instructions to the control unit model. Through the authorized PC terminal, sensors can be remotely monitored and the control unit can be remotely interactively controlled.

Claims

1. A control and interaction system based on a real-scene twin virtual digital scene graph, comprising a cloud server arranged in the cloud, and also comprising a sensor, a control unit and an interactive terminal connected to the cloud server in a real scene, characterized in that: The cloud server stores a scene graph database based on the real scene twin virtualization with sensor models and control unit models implanted therein, and is used to call the corresponding scene graph database for comparison, loading, and overwriting according to the perspective of the interactive terminal in the real scene, and can interactively control the control unit in the real scene according to the interactive instructions issued by the interactive terminal; The interactive terminal is used to connect to the cloud server in the real scene to transmit the on-site perspective image, receive the corresponding scene graph data in the perspective loaded by the cloud server, and send interactive control instructions to the cloud server.

2. A control interaction system based on a real-scene twin virtual digital scene graph according to claim 1, characterized in that: The interactive terminals include but are not limited to: mobile phones, tablet computers, VR devices, and AR devices.

3. A control interaction system based on a real-scene twin virtual digital scene graph according to claim 1, characterized in that: The control unit is connected with a control actuator and a remote communication device.

4. A control interaction system based on a real-scene twin virtual digital scene graph according to claim 1, characterized in that: The control unit includes but is not limited to: a light switch, a valve, a motor, an electronic lock, and a door magnetic switch.

5. A control interaction system based on a real-scene twin virtual digital scene graph according to claim 1, characterized in that: The sensors include but are not limited to: position sensors, displacement sensors, temperature sensors, humidity sensors, radar sensors, pressure sensors, infrared sensors, and variable frequency power sensors.

6. A control interaction system based on a real-scene twin virtual digital scene graph according to claim 1, characterized in that: The communication methods between the cloud server and the sensor, control unit and interactive terminal include but are not limited to: Wi-Fi, Bluetooth, radar, and Zigbee.

Images