Smart home 3D virtual simulation method and system

Abstract

The application discloses a kind of based on smart home 3D virtual simulation method and system, it is related to virtual simulation technical field, including, collection physical home environment data, constructs smart home 3D virtual scene, the semantic binding of equipment model in scene is carried out, and is given space attribute and interactive parameter;Set the interactive rule of virtual equipment, establish the mapping relationship of virtual equipment and real smart home equipment, access real smart home equipment data and transmit control instruction;In equipment abstract layer, each equipment is defined uniform equipment object model, and control logic layer is called uniform API when interacting;Through the practical training mode manager of control logic layer, practical training mode switching is carried out, and preset pure virtual simulation mode, virtual-real combination control mode and full physical deployment mode.The method disclosed in the application provides zero-cost, zero-risk experimental environment;Virtual-real combination mode realizes the synchronous response of virtual picture and physical action by mirror distribution technology.

Description

Technical Field

[0001] This invention relates to the field of virtual simulation technology, and in particular to a 3D virtual simulation method and system based on smart homes. Background Technology

[0002] To reduce the costs and physical wear and tear of developing, debugging, and training smart home systems, 3D virtual simulation technology has gradually become a hot research topic in the industry. Simulation methods primarily focus on visual rendering based on graphics engines, using computer graphics technology to geometrically model the home environment in a virtual space. However, by mapping real-time data of physical entities to virtual models through digital twins, and by establishing a connection between 3D scenes and sensor data, visualized monitoring of home status and preliminary verification of interactive logic can be achieved.

[0003] However, existing virtual simulation technologies still face multi-dimensional technical bottlenecks and constraints when facing high-fidelity, cross-mode switching simulation training scenarios. Existing simulation systems generally suffer from the phenomenon of virtual-physical silos; the logical verification of the virtual environment and the physical deployment of equipment are often under two independent architectural systems. This makes it difficult to seamlessly migrate control algorithms successfully debugged in the virtual environment to the physical environment, and the training process lacks a smooth evolution path from purely virtual to fully physical. The lack of standardized definitions for spatial semantics and interaction parameters increases the complexity of system construction and limits the teaching and verification capabilities of training systems when dealing with complex engineering problems. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a smart home 3D virtual simulation method to solve the problems of insufficient virtual-real mapping depth, single training mode, poor compatibility of heterogeneous hardware, and high coupling between control logic and actuator in existing smart home simulation technologies.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, the present invention provides a method for 3D virtual simulation of smart homes, which includes collecting physical home environment data, constructing a 3D virtual scene of a smart home, semantically binding device models in the scene, and assigning spatial attributes and interaction parameters.

[0008] Set the interaction rules for virtual devices, establish the mapping relationship between virtual devices and real smart home devices, access data from real smart home devices and transmit control commands;

[0009] In the device abstraction layer, a unified device object model is defined for each device, and the control logic layer calls a unified API during interaction;

[0010] The training mode is switched through the training mode manager of the control logic layer, with preset pure virtual simulation mode, virtual-real combined control mode and full physical deployment mode; the device abstraction layer routes virtual device data to the device execution layer to perform simulation training according to the current training mode.

[0011] As a preferred embodiment of the smart home 3D virtual simulation method described in this invention, the setting of interaction rules for virtual devices includes collecting physical home environment data, performing preprocessing, and obtaining room device topology, node set, and device installation pose.

[0012] Based on the node set and topological relationship, the corresponding scene hierarchy skeleton is generated in the 3D virtual engine, and the room scene graph is constructed in the form of room node-device node;

[0013] The node set includes a room node set and a device node set;

[0014] For each room, a room node is created, and the geometric models of walls, doors, windows, and the floor are loaded; for each device, a device node is created, and the device node is attached as a child node to the room node to form an index for semantic retrieval of the room scene graph and the spatial context of the 3D virtual scene.

[0015] During the device node connection process, the output device installation pose is used to transform the device to match the physical installation point in the world coordinate system.

[0016] As a preferred embodiment of the smart home 3D virtual simulation method described in this invention, the step of defining a unified device object model for each device in the device abstraction layer includes binding device nodes to their functional semantics in the device abstraction layer; constructing a semantic tuple for each device, including a unique identifier, device type, capability set, room identifier, and access method identifier; writing the semantic tuple into the semantic attribute field of the device node, and establishing an index mapping from the device's unique identifier to the device node in the scene.

[0017] By defining the spatial attributes and interaction parameters of a device, the device node in the room scene graph will possess: geometric entity, semantic identity, spatial context, and interaction parameters.

[0018] As a preferred embodiment of the smart home 3D virtual simulation method described in this invention, the control logic layer calls the unified API during interaction, including traversing the room scene graph, completing object instantiation of the control logic layer by parsing semantic tuples in the device nodes; the control logic layer outputs standardized action vectors for the unified device nodes and receives device status updates and events; and configures the unified API called by the control logic layer to ensure that the output standardized action vectors are routable.

[0019] Establish a reversible, closed-loop bidirectional mapping and protocol adaptation mechanism. Using the protocol type and hardware identifier of each device as an index, construct a protocol adapter and form a mapping table. For each protocol type, define a pair of complementary mapping functions to perform downlink mapping from the abstract domain to the protocol domain and receive uplink mappings.

[0020] Downlink mapping is used to convert unified action requests into protocol-recognizable messages and send them to the device's physical gateway. Uplink mapping is used to parse the original messages returned by the physical gateway into unified state updates and events, forming a closed-loop link of instruction issuance, execution feedback, and state update.

[0021] The complementary mapping function encodes the action vector of a unified device object into a control message of a specific protocol, including the target device identifier, standardized action vector, and reliability verification parameters.

[0022] As a preferred embodiment of the smart home 3D virtual simulation method described in this invention, the training mode manager of the control logic layer includes maintaining a global mode vector and configuring the logical path of the dynamic routing scheduler in the control logic layer.

[0023] The virtual logic branch of path A associates the geometric entity attributes of the device nodes in the scene graph; when the mode involves virtuality, it activates the animation logic based on the physics engine and calls the installation pose parameters to update the 3D rendering effect.

[0024] The protocol adapter branch of Path B is associated with the protocol adapter of the device abstraction layer. When the training mode involves physical objects, the gateway communication link is activated.

[0025] As a preferred embodiment of the smart home 3D virtual simulation method described in this invention, the step of switching the training mode includes: when the user initiates interaction at the control logic layer through an automatic control algorithm, the system executes the operation through a unified API.

[0026] Based on the operation objective, retrieve the corresponding device nodes and their semantic attribute domains in the room scene graph; extract the unique identifier and functional semantics of the device nodes, and encapsulate the control intent into standardized action vectors;

[0027] The standardized action vector includes the target device ID, action command, parameter set, high-precision timestamp, and reliability verification parameters;

[0028] The dynamic routing scheduler intercepts action vectors output by the unified API in real time and requests the current running mode from the training mode manager; the scheduler executes path distribution based on the mode status and routing logic.

[0029] The routing and traffic splitting logic includes:

[0030] In pure virtual simulation mode, the dynamic routing scheduler pushes action vectors only to the virtual logic branches of the device execution layer.

[0031] In the virtual-real combined control mode, the dynamic routing scheduler copies the action vector into two mirrors and pushes them in parallel to the virtual logical branch and the protocol adaptation branch.

[0032] In the fully physical deployment mode, the dynamic routing scheduler shields the action vectors from the virtual logic and only pushes them to the protocol adaptation branch.

[0033] As a preferred embodiment of the smart home 3D virtual simulation method described in this invention, the execution simulation training includes performing low-level conversion based on the attributes of the device nodes after the instruction flow is directed to a specific execution target.

[0034] When the virtual logic branch is executed, the 3D virtual engine receives the motion vector and parses the corresponding device node's device installation pose in the world coordinate system; based on the parameters in the vector, it triggers shader changes or animation scripts for geometric entities to achieve visual feedback.

[0035] The protocol adaptation branch is executed, the action vector enters the device abstraction layer, and calls the predefined downlink mapping function; the downlink mapping function encodes the standardized instructions into a raw protocol message with a check bit that can be recognized by the physical gateway according to the access method identifier in the semantic tuple, and sends it to the real hardware through the physical communication bus;

[0036] After the instruction is executed, the dynamic routing scheduler is responsible for maintaining the consistency between the virtual and real states, and completing the closed-loop link of instruction issuance, execution feedback and state update.

[0037] The 3D virtual engine returns a confirmation receipt indicating successful virtual execution, and the physical gateway sends back the original status message after hardware execution.

[0038] The device abstraction layer uses the uplink mapping function in the protocol adapter to parse the original message into a unified format state update event; the dynamic routing scheduler uses the returned device identifier to reverse look up the device node in the room scene graph and update the real-time state in the semantic attribute domain of the device node.

[0039] If an alarm event occurs, not only will the semantic state of the current node be updated, but also the linkage action vectors of related nodes in the same room will be automatically triggered based on the spatial context index in the room scene graph, thus completing the simulation training loop across devices.

[0040] Secondly, the present invention provides a smart home 3D virtual simulation system, including a scene construction module, which collects data of the physical home environment, performs preprocessing, generates a virtual room scene, and constructs device models in the scene;

[0041] The interactive control module maps virtual devices to real smart home devices through a device abstraction layer and sets the interaction rules for the devices; the control commands of the virtual devices are mapped to the real devices through a protocol adapter, completing the interaction between virtual and physical devices.

[0042] The control logic layer module includes a training mode manager, which controls the device according to different training modes.

[0043] The status update module updates the device status in real time based on the current mode through a dynamic routing scheduler.

[0044] Thirdly, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, wherein when the computer program is executed by the processor, it implements any step of the smart home 3D virtual simulation method described in the first aspect of the present invention.

[0045] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the smart home 3D virtual simulation method described in the first aspect of the present invention.

[0046] The beneficial effects of this invention are as follows: This invention encapsulates complex physical hardware instructions and virtual animation logic into standardized action vectors through a device abstraction layer and a unified API. The control logic layer does not need to be aware of whether the underlying layer is a virtual engine or physical hardware during interaction, achieving complete decoupling between the control program and the execution environment. A hierarchical scene graph is constructed, binding semantic tuples containing capability sets and access identifiers. When an alarm event occurs (such as water leakage or fire), the system can automatically trigger linked actions of related devices in the same room or area (such as flashing lights in the area while closing the main valve), significantly enhancing the logical rigor and semantic retrieval efficiency of the simulation system when handling complex scene linkages. The pure virtual mode provides a zero-cost, zero-risk experimental environment for beginners; the virtual-real combined mode, through mirror distribution technology, achieves synchronous response between virtual images and physical actions, providing a strong sense of immersion and intuitive verification methods. Attached Figure Description

[0047] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a flowchart of a 3D virtual simulation method for smart homes.

[0049] Figure 2 This is a schematic diagram of a control algorithm based on a smart home 3D virtual simulation method.

[0050] Figure 3 This is a schematic diagram of a device based on a smart home 3D virtual simulation method. Detailed Implementation

[0051] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0052] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0053] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0054] Reference Figures 1-2 This is one embodiment of the present invention, which provides a 3D virtual simulation method for smart homes, including the following steps:

[0055] S1: Collect physical home environment data, construct a smart home 3D virtual scene, semantically bind the device models in the scene, and assign spatial attributes and interaction parameters.

[0056] The raw data is preprocessed to identify boundary elements such as walls, floors, and ceilings, and the geometric outlines of the rooms are extracted. Topological logic extraction, based on a spatial inclusion algorithm, automatically analyzes the dependency relationships between devices and rooms. If a device's coordinates are within the bounding box defined by a room, it is marked as a child node of that room, thus generating a room-device topology graph.

[0057] Installation pose calculation involves determining the precise position and rotation angle of each device relative to the room coordinate system, thus generating the device installation pose parameters.

[0058] The interaction rules for setting up virtual devices include collecting physical home environment data, preprocessing it, and obtaining the room device topology, node set, and device installation position.

[0059] Based on the node set and topological relationships, the corresponding scene hierarchy skeleton is generated in the 3D virtual engine, and the room scene graph is constructed in the form of room nodes-device nodes.

[0060] The node set includes a room node set and a device node set.

[0061] For each room, a room node is created, and the geometric models of walls, doors, windows, and the floor are loaded. For each device, a device node is created, and the device node is attached as a child node to the room node to form an index for semantic retrieval of the room scene graph and the spatial context of the 3D virtual scene.

[0062] During the device node connection process, the output device installation pose is used to transform the device to match the physical installation point in the world coordinate system.

[0063] Skeleton construction: With the root node (House) as the top level, create room nodes based on the preprocessed set of rooms.

[0064] Static geometry loading: Under each room node, a wall mesh is dynamically generated based on the scan data, and a standardized door and window model is loaded to complete the restoration of the hard decoration environment.

[0065] Devices are attached and aligned with coordinates. The collection of device nodes is traversed, and each device is instantiated as a child object of the room node.

[0066] Spatial transformation involves using the pose data output during the acquisition phase and performing transformation matrix operations from the world coordinate system to the local coordinate system to ensure that the virtual air conditioner's position on the wall and the virtual light fixture's height on the ceiling are completely consistent with the physical objects.

[0067] S2: Set the interaction rules for virtual devices, establish the mapping relationship between virtual devices and real smart home devices, access data from real smart home devices, and transmit control commands.

[0068] The step of defining a unified device object model for each device in the device abstraction layer includes binding device nodes with their functional semantics in the device abstraction layer; constructing a semantic tuple for each device, including a unique identifier, device type, capability set, room identifier, and access method identifier; writing the semantic tuple into the semantic attribute field of the device node, and establishing an index mapping from the device's unique identifier to the device node in the scene.

[0069] First, for each device The system uses a unique device identifier. As the primary key, the bound semantic information, spatial information, and interaction information are injected as metadata into the unified object to construct a standardized device object:

[0070]

[0071] in, This is the device state vector. A set of actions that the device can perform. This represents a set of device events. The three categories of elements mentioned above correspond to the minimum closed-loop requirements of the control logic layer for reading status, issuing actions, and subscribing to events. Simultaneously, the device object implicitly associates various interfaces to support 3D interaction and visualization, but the control logic layer only needs to interact with... A unified interface is sufficient.

[0072] To ensure that the state representations of different devices are comparable, routable, and verifiable within the same dimensional system, this step defines a standard state space and incorporates the original states of each protocol / vendor. Mapped to a unified state vector via a state encoding function :

[0073]

[0074] in, Organize device status into a unified key-value / vector representation. For example, map on / off status, brightness percentage, temperature setting, door lock closure status, etc., into status fields that can be processed uniformly.

[0075] For missing fields, predefined default values ​​or null values ​​are used as placeholders to ensure... It can be stably parsed at the logic layer. Through this unified processing, the control logic layer does not need to be aware of the encoding differences in the underlying protocol state, thereby enabling the cross-device reuse of control strategies.

[0076] By defining the spatial attributes and interaction parameters of a device, the device node in the room scene graph will possess: geometric entity, semantic identity, spatial context, and interaction parameters.

[0077] S3: Define a unified device object model for each device in the device abstraction layer, and call the unified API in the control logic layer during interaction.

[0078] Virtual control of real devices (downlink): When a user interacts with a device model in a 3D scene, such as clicking a virtual switch, the system obtains the node's UID and access method through its semantic attributes and encapsulates it into a control command. The command is then sent to the actual smart home device through the home gateway.

[0079] Realistic feedback to virtual (uplink): Changes in the status of real devices (such as sensors detecting a temperature increase or lights being manually turned off) are reported in real time. The system uses the UID index to find the corresponding virtual node in the scene and synchronously updates the model's visual state (such as changing light brightness or displaying dynamic values).

[0080] Spatial attributes are bound to interaction parameters, giving virtual nodes specific interaction logic.

[0081] Interaction parameter settings allow you to configure the interaction radius, click detection area, and animation trigger script for device nodes.

[0082] Enhanced semantic retrieval: By combining the constructed scene graph index, the system can handle complex spatial semantic commands. For example, when receiving the command "turn off all lights in the living room", the system can locate and operate all the lights in that room in batches through the membership relationship of "room node - device node".

[0083] Furthermore, the control logic layer's call to the unified API during interaction includes: traversing the room scene graph; completing object instantiation of the control logic layer by parsing semantic tuples in the device nodes; the control logic layer outputting standardized action vectors for the unified device nodes and receiving device status updates and events; and configuring the unified API called by the control logic layer to ensure that the output standardized action vectors are routable.

[0084] Establish a reversible, closed-loop bidirectional mapping and protocol adaptation mechanism. Using the protocol type and hardware identifier of each device as an index, construct a protocol adapter and form a mapping table. For each protocol type, define a pair of complementary mapping functions to perform downlink mapping from the abstract domain to the protocol domain and receive uplink mapping.

[0085] Downlink mapping is used to convert unified action requests into protocol-recognizable messages and send them to the device's physical gateway. Uplink mapping is used to parse the original messages returned by the physical gateway into unified state updates and events, forming a closed-loop link of instruction issuance, execution feedback, and state update.

[0086] The control logic layer only outputs a unified device object to the outside world. Standardized action request and receive status updates in a unified format. With the event However, real hardware devices typically use heterogeneous communication protocols (such as MQTT, Zigbee, BLE, HTTP, etc.) and different message structures and field encoding methods. Therefore, a reversible and closed-loop bidirectional mapping and protocol adaptation mechanism needs to be established between the "unified API and heterogeneous protocols" to ensure that control commands can be reliably executed, physical states can be stably transmitted back, and the 3D scene can be updated in real time. To this end, this step addresses this issue based on the protocol type of each device. Hardware Identifier For the index, construct the protocol adapter and form a mapping table M.

[0087] Downlink mapping is used to convert unified action requests into protocol-recognizable messages and send them to the physical gateway. Uplink mapping is used to parse the original messages returned by the physical gateway into unified state increments and events, thus forming a closed-loop link of "instruction issuance - execution feedback - state update - visualization synchronization".

[0088] Protocol adaptation and mapping function generation (abstract domain ↔ protocol domain): First, for each protocol type p, define a pair of complementary mapping functions:

[0089] Downlink mapping (unified action → protocol message):

[0090]

[0091] This function encodes the action vector u of the unified device object into a control message for a specific protocol, including the target address / topic, command word, parameter fields, and checksum fields. Uplink mapping (protocol return → unified state / event):

[0092]

[0093] This function decodes and extracts fields from the packets returned by the physical gateway, and outputs a unified state increment. With the event (Such as alarms, execution failures, timeouts, offline status, etc.), thus ensuring that the control logic layer always processes data in a uniform format. At the implementation level, and It can be generated by combining "field mapping rules + unit conversion, enumeration values, and comparison verification strategies", so that devices from different manufacturers can be connected simply by replacing the adaptation rules, without affecting the unified API of the control logic layer.

[0094] The complementary mapping function encodes the action vector of a unified device object into a control message of a specific protocol, including the target device identifier, standardized action vector, and reliability verification parameters.

[0095] To ensure that the same device can be stably identified and located on both the virtual and physical sides, this step uses a unified device identifier. As the primary key, a virtual-physical binding mapping table M is established to bind device objects, hardware entities, and protocol mapping functions:

[0096]

[0097] in, This represents the location information of routable entities on the hardware side (such as gateway address, device short address, etc.). Through the establishment of M, the system receives any unified action request. In any case, the corresponding protocol type and routing information can be obtained by looking up a table, and the correct mapping function can be selected to encode and send the message; similarly, when receiving any physical return message... In this case, the corresponding index of M can also be determined through the index relationship of M. This allows for writing back to the correct unified device object. .

[0098] The closed-loop execution of command issuance—feedback transmission—state update (forming a visual synchronization) occurs when the control logic layer generates control commands (i.e., action vectors) through a unified API. After that, the system first obtains the corresponding protocol adapter according to the mapping table M, and then performs downlink encoding and transmission:

[0099]

[0100] After the physical gateway sends the message and drives the hardware to execute it, it will generate a status return message. .

[0101] The system receives the message and obtains the unified state increment and event through uplink mapping, expressed by the formula:

[0102]

[0103] The state increment is then written back to the unified device object state vector to achieve closed-loop state update:

[0104]

[0105] It should be noted that, in order to ensure that the 3D scene is consistent with the real hardware, the system maps the updated unified state to the rendering state or animation / material parameters, and drives the virtual scene to refresh in real time.

[0106] S4: The training mode is switched through the training mode manager of the control logic layer, with preset pure virtual simulation mode, virtual-real combined control mode and full physical deployment mode; the device abstraction layer routes virtual device data to the device execution layer to perform simulation training according to the current training mode.

[0107] Furthermore, the training mode manager of the control logic layer includes maintaining a global mode vector and configuring the logical path of the dynamic routing scheduler in the control logic layer.

[0108] The virtual logic branch of path A associates the geometric entity attributes of the device nodes in the scene graph; when the mode involves virtuality, it activates the animation logic based on the physics engine and calls the installation pose parameters to update the 3D rendering effect.

[0109] The protocol adapter branch of Path B is associated with the protocol adapter of the device abstraction layer. When the training mode involves physical objects, the gateway communication link is activated.

[0110] The switching of training modes includes the system executing operations through a unified API when a user initiates interaction at the control logic layer via an automatic control algorithm:

[0111] Based on the operational objective, the corresponding device nodes and their semantic attribute domains are retrieved from the room scene graph; the unique identifier and functional semantics of the device nodes are extracted, and the control intent is encapsulated into standardized action vectors.

[0112] The standardized action vector includes the target device ID, action command, parameter set, high-precision timestamp, and reliability verification parameters.

[0113] The dynamic routing scheduler intercepts action vectors output by the unified API in real time and requests the current running mode from the training mode manager; the scheduler executes path distribution based on the mode status and routing logic.

[0114] The routing and traffic splitting logic includes:

[0115] In pure virtual simulation mode, the dynamic routing scheduler pushes action vectors only to the virtual logic branches of the device execution layer.

[0116] In the virtual-real combined control mode, the dynamic routing scheduler copies the action vector into two mirror images and pushes them in parallel to the virtual logical branch and the protocol adaptation branch.

[0117] In the fully physical deployment mode, the dynamic routing scheduler shields the action vectors from the virtual logic and only pushes them to the protocol adaptation branch.

[0118] After completing the construction of the virtual scene, the modeling of unified device objects, and the establishment of the virtual-real two-way mapping relationship, the control commands and sensor events output by the control logic layer have a unified data structure and semantic boundaries.

[0119] To enable the same set of control logic to run in different training scenarios, a training mode manager is set up at the control logic layer. This manager is used to uniformly configure, controllably switch, and maintain the status of three operating scenarios: "pure virtual simulation," "virtual-real hybrid," and "fully physical deployment." The formula for the training mode manager to maintain the set of modes is expressed as follows:

[0120]

[0121] in, Indicates a pure virtual simulation mode. This indicates a combined virtual and real control mode. This indicates a full physical deployment mode. During the initialization phase, the manager generates a set of mode parameters based on the course script or user configuration (e.g., whether to enable consistency verification, whether to allow fault injection, whether to enable the real gateway, whether to enable exam locking, etc.), and abstracts these parameters into configuration functions bound to the mode.

[0122]

[0123] in, For pattern The configuration parameter vector is used to control the behavioral boundaries of subsequent instruction routing strategies, synchronization strategies, and visualization strategies. This represents a set of configuration parameter vectors.

[0124] The mode state machine and switching triggers are maintained by the manager during system operation, ensuring the current mode state is active. When a mode switching event occurs (For example, when there is a forced switch on the teacher's end, a switch in the course flow node, a switch in student operations, or the system detects that the gateway is online / offline, the manager updates the running mode according to the mode transition function.)

[0125]

[0126] The execution simulation training includes performing low-level conversion based on the attributes of the device node after the instruction flow is directed to a specific execution target.

[0127] When the virtual logic branch is executed, the 3D virtual engine receives the motion vector and parses the device installation pose of the corresponding device node in the world coordinate system. Based on the parameters in the vector, it triggers the shader change of the geometric entity or the animation script to achieve visual feedback.

[0128] The protocol adaptation branch is executed, the action vector enters the device abstraction layer, and calls the predefined downlink mapping function; the downlink mapping function encodes the standardized instructions into a raw protocol message with a check bit that can be recognized by the physical gateway according to the access method identifier in the semantic tuple, and sends it to the real hardware through the physical communication bus.

[0129] After the instruction is executed, the dynamic routing scheduler is responsible for maintaining the consistency between the virtual and real states, and completing the closed-loop link of instruction issuance, execution feedback and state update.

[0130] The 3D virtual engine returns a confirmation message indicating successful virtual execution, while the physical gateway sends back the original status message after hardware execution.

[0131] The device abstraction layer uses the uplink mapping function in the protocol adapter to parse the original message into a unified format state update event; the dynamic routing scheduler uses the returned device identifier to reverse look up the device node in the room scene graph and update the real-time state in the semantic attribute domain of the device node.

[0132] To avoid inconsistencies in the control link caused by mode switching, σ includes not only the target mode identifier, but also the switching intent and constraint information, where P is the target mode.

[0133] To ensure the stability of the control link before and after the handover process, the manager performs "mode entry" and "mode exit" actions before completing the mode update. Its core objective is to enable subsequent dynamic routing scheduling to deterministically route control commands and sensor data based solely on the latest mode.

[0134] It should be noted that if an alarm event occurs, not only will the semantic state of the current node be updated, but also the linkage action vectors of related nodes in the same room will be automatically triggered based on the spatial context index in the room scene graph, thus completing the simulation training loop across devices.

[0135] This embodiment also provides a smart home 3D virtual simulation system, including:

[0136] The scene building module collects data from the physical home environment, preprocesses it, generates virtual room scenes, and builds device models within those scenes.

[0137] The interactive control module maps virtual devices to real smart home devices through a device abstraction layer and sets the interaction rules for the devices. The control commands of the virtual devices are mapped to the real devices through a protocol adapter, completing the interaction between the virtual and physical devices.

[0138] The control logic layer module includes a training mode manager, which controls the equipment according to different training modes.

[0139] The status update module updates the device status in real time based on the current mode through a dynamic routing scheduler.

[0140] This embodiment also provides a computer device, such as... Figure 3 As shown, the method applicable to smart home 3D virtual simulation includes: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to realize the smart home 3D virtual simulation method proposed in the above embodiments.

[0141] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0142] This embodiment also provides a storage medium storing a computer program, which, when executed by a processor, implements the smart home 3D virtual simulation method proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0143] In summary, this invention achieves the following: collecting physical home environment data to construct a smart home 3D virtual scene; semantically binding device models within the scene and assigning them spatial attributes and interaction parameters; setting interaction rules for virtual devices, establishing a mapping relationship between virtual devices and real smart home devices, accessing real smart home device data, and transmitting control commands; defining a unified device object model for each device in the device abstraction layer, and calling a unified API during interaction in the control logic layer; switching training modes through the training mode manager in the control logic layer, with preset pure virtual simulation mode, virtual-real combined control mode, and full physical deployment mode; and routing virtual device data to the device execution layer for simulation training based on the current training mode.

[0144] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A 3D virtual simulation method based on smart homes, characterized in that: This includes collecting physical home environment data, constructing a smart home 3D virtual scene, semantically binding device models within the scene, and assigning spatial attributes and interaction parameters; Set the interaction rules for virtual devices, establish the mapping relationship between virtual devices and real smart home devices, access data from real smart home devices and transmit control commands; In the device abstraction layer, a unified device object model is defined for each device, and the control logic layer calls a unified API during interaction; The training mode can be switched through the training mode manager of the control logic layer, with preset pure virtual simulation mode, virtual-real combined control mode and full physical deployment mode; The device abstraction layer routes virtual device data to the device execution layer to perform simulation training based on the current training mode.

2. The smart home-based 3D virtual simulation method as described in claim 1, characterized in that: The construction of the smart home 3D virtual scene includes collecting physical home environment data, performing preprocessing, and obtaining the room equipment topology, node set, and equipment installation pose; Based on the node set and topological relationship, the corresponding scene hierarchy skeleton is generated in the 3D virtual engine, and the room scene graph is constructed in the form of room node-device node; The node set includes a room node set and a device node set; For each room, a room node is created, and the geometric models of walls, doors, windows, and the floor are loaded; for each device, a device node is created, and the device node is attached as a child node to the room node to form an index for semantic retrieval of the room scene graph and the spatial context of the 3D virtual scene. During the device node connection process, the output device installation pose is used to transform the device to match the physical installation point in the world coordinate system.

3. The smart home-based 3D virtual simulation method as described in claim 2, characterized in that: The step of defining a unified device object model for each device in the device abstraction layer includes binding device nodes to their functional semantics in the device abstraction layer; constructing a semantic tuple for each device, including a unique identifier, device type, capability set, room identifier, and access method identifier; writing the semantic tuple into the semantic attribute field of the device node, and establishing an index mapping from the device's unique identifier to the device nodes in the scene. By defining the spatial attributes and interaction parameters of a device, the device node in the room scene graph will possess: geometric entity, semantic identity, spatial context, and interaction parameters.

4. The smart home-based 3D virtual simulation method as described in claim 3, characterized in that: The control logic layer calls the unified API during interaction, including traversing the room scene graph, instantiating the control logic layer object by parsing the semantic tuples in the device nodes; the control logic layer outputs standardized action vectors for the unified device nodes and receives device status updates and events; and configures the unified API called by the control logic layer to ensure that the output standardized action vectors are routable. Establish a reversible, closed-loop bidirectional mapping and protocol adaptation mechanism. Using the protocol type and hardware identifier of each device as an index, construct a protocol adapter and form a mapping table. For each protocol type, define a pair of complementary mapping functions to perform downlink mapping from the abstract domain to the protocol domain and receive uplink mappings. Downlink mapping is used to convert unified action requests into protocol-recognizable messages and send them to the device's physical gateway. Uplink mapping is used to parse the original messages returned by the physical gateway into unified state updates and events, forming a closed-loop link of instruction issuance, execution feedback, and state update. The complementary mapping function encodes the action vector of a unified device object into a control message of a specific protocol, including the target device identifier, standardized action vector, and reliability verification parameters.

5. The 3D virtual simulation method for smart homes as described in claim 4, characterized in that: The training mode manager of the control logic layer includes maintaining a global mode vector and configuring the logical path of the dynamic routing scheduler in the control logic layer. The virtual logic branch of path A associates the geometric entity attributes of device nodes in the scene graph; when the mode involves virtuality, it activates the animation logic based on the physics engine and calls the installation pose parameters to update the 3D rendering effect. The protocol adapter branch of Path B is associated with the protocol adapter of the device abstraction layer. When the training mode involves physical objects, the gateway communication link is activated.

6. The 3D virtual simulation method for smart homes as described in claim 5, characterized in that: The switching of training modes includes the system performing operations through a unified API when the user initiates an interaction at the control logic layer through an automatic control algorithm. Based on the operation objective, retrieve the corresponding device nodes and their semantic attribute domains in the room scene graph; extract the unique identifier and functional semantics of the device nodes, and encapsulate the control intent into standardized action vectors; The standardized action vector includes the target device ID, action command, parameter set, high-precision timestamp, and reliability verification parameters; The dynamic routing scheduler intercepts action vectors output by the unified API in real time and requests the current running mode from the training mode manager. The scheduler performs path distribution based on the mode state and routing logic; The routing and traffic splitting logic includes: In pure virtual simulation mode, the dynamic routing scheduler pushes action vectors only to the virtual logic branches of the device execution layer. In the virtual-real combined control mode, the dynamic routing scheduler copies the action vector into two mirrors and pushes them in parallel to the virtual logical branch and the protocol adaptation branch. In the fully physical deployment mode, the dynamic routing scheduler shields the action vectors from the virtual logic and only pushes them to the protocol adaptation branch.

7. The 3D virtual simulation method for smart homes as described in claim 6, characterized in that: The execution simulation training includes performing low-level conversion based on the attributes of the device node after the instruction flow is directed to a specific execution target; When the virtual logic branch is executed, the 3D virtual engine receives the motion vector and parses the corresponding device node's device installation pose in the world coordinate system; based on the parameters in the vector, it triggers shader changes or animation scripts for geometric entities to achieve visual feedback. The protocol adaptation branch is executed, the action vector enters the device abstraction layer, and calls the predefined downlink mapping function; the downlink mapping function encodes the standardized instructions into a raw protocol message with a check bit that can be recognized by the physical gateway according to the access method identifier in the semantic tuple, and sends it to the real hardware through the physical communication bus; After the instruction is executed, the dynamic routing scheduler is responsible for maintaining the consistency between the virtual and real states, and completing the closed-loop link of instruction issuance, execution feedback and state update. The 3D virtual engine returns a confirmation receipt indicating successful virtual execution, and the physical gateway sends back the original status message after hardware execution. The device abstraction layer uses the uplink mapping function in the protocol adapter to parse the raw message into a uniformly formatted status update event; The dynamic routing scheduler uses the returned device identifier to reverse-search the device node in the room scene graph and updates the real-time status in the semantic attribute domain of the device node. If an alarm event occurs, not only will the semantic state of the current node be updated, but also the linkage action vectors of related nodes in the same room will be automatically triggered based on the spatial context index in the room scene graph, thus completing the simulation training loop across devices.

8. A smart home 3D virtual simulation system, based on the smart home 3D virtual simulation method according to any one of claims 1 to 7, characterized in that: This includes a scene building module, which collects data from the physical home environment, preprocesses it, generates virtual room scenes, and builds device models within those scenes; The interactive control module maps virtual devices to real smart home devices through a device abstraction layer and sets the interaction rules for the devices; the control commands of the virtual devices are mapped to the real devices through a protocol adapter, completing the interaction between virtual and physical devices. The control logic layer module includes a training mode manager, which controls the device according to different training modes. The status update module updates the device status in real time based on the current mode through a dynamic routing scheduler.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the smart home 3D virtual simulation method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the smart home 3D virtual simulation method according to any one of claims 1 to 7.

Images