A virtual-real interaction control method and system based on a virtual interaction carrier and a computer readable storage medium

Abstract

This invention discloses a virtual-real interaction control method, system, and storage medium based on a virtual interactive carrier, belonging to the field of spatial computing technology. The method includes: deploying a location-associated virtual interactive carrier in a virtual three-dimensional space; establishing a dynamic association between the carrier and digital content containing executable instructions; responding to interactive actions to determine the target instruction; sending the instruction to the execution interface via a secure path and receiving feedback; and synchronously updating the carrier's state based on the feedback, forming an interactive closed loop. This invention upgrades the virtual interactive carrier from a content display entry point to an executable operation endpoint, bridging the capability gap in spatial computing from "passive display" to "active operation," and is adaptable to various spatial computing scenarios such as smart homes, digital twins, mixed reality, and world models.

Description

Technical Field

[0001] This invention relates to the fields of spatial computing, virtual reality, augmented reality, mixed reality interaction, Internet of Things interface control, and computer software technology. Specifically, it relates to a virtual-real interaction control method, system, and computer-readable storage medium that triggers cross-domain execution based on a virtual interaction carrier in a virtual three-dimensional space and forms a closed loop of state synchronization and write-back.

[0002] The virtual three-dimensional space of this invention can be presented through virtual reality, augmented reality, mixed reality, holographic display, digital twin, world model, or other spatial computing environments with spatial location representation capabilities. The virtual interactive carrier can be triggered by touch, click, voice, gesture, eye tracking, posture recognition, neural electrical signal decoding results, or other interactive signals. The technical solution provided by this invention forms a closed-loop execution mechanism between the virtual interactive carrier, executable instructions, target execution interface, execution result feedback, and state synchronization write-back. Background Technology

[0003] With the development of 3D reconstruction, spatial computing, and mixed reality technologies, virtual 3D spaces have been widely applied in digital content display, spatial navigation, digital twins, smart home management, industrial simulation, remote collaboration, and immersive service scenarios. In these scenarios, users not only need to browse images, videos, models, or device status, but also hope to directly influence external physical devices, external services, or other target systems through interactive behaviors in the virtual space.

[0004] Existing solutions typically include the following types: The first category is virtual content display solutions. These solutions typically overlay virtual content carriers onto augmented reality visuals, 3D scenes, or virtual spaces. When a user clicks or selects on this virtual content carrier, the system displays images, videos, web pages, explanatory information, or content display pages. While this type of solution solves the problem of displaying content within a given space, its technical chain usually ends at content presentation, lacking a mechanism to convert virtual interactive actions into executable instructions that are then executed via external target interfaces.

[0005] The second category is device control or service invocation solutions. These solutions typically generate control commands through mobile applications, web interfaces, control panels, voice assistants, or automation platforms, and then control external devices or invoke external services via IoT platforms, service interfaces, or device protocols. While these solutions can solve the problem of device or service control, the control entry point is usually a flat interface or an abstract service entry point, lacking a stable connection with spatial location, virtual interaction carriers, and spatial state representation in a virtual three-dimensional space.

[0006] The third category is digital twin or 3D visualization status synchronization solutions. These solutions typically receive status data from external devices or physical systems and map it onto digital twin objects, 3D models, or visualization panels. While these solutions can solve status display or monitoring problems, their focus is usually on status acquisition and mapping, and they do not necessarily include a complete closed loop where external execution is triggered by a virtual interactive platform, the execution results are received, and the status is written back to the virtual interactive platform.

[0007] Therefore, the existing technology still has the following problems: the interactive carriers in virtual space often remain at the level of content display or internal scene response; external device control or service calls often deviate from the spatial context of the virtual three-dimensional space; and the external execution results are difficult to reliably write back to the virtual interactive carrier that triggered the execution or its associated state. Users find it difficult to form a continuous closed loop of "triggering execution, confirming execution, and perceiving results" in the virtual three-dimensional space.

[0008] The technical problem to be solved by this invention is: how to carry or associate executable instructions in a virtual three-dimensional space through a virtual interactive carrier, execute the executable instructions through the target execution interface after the interactive action is triggered, and write back the execution result to the running status of the virtual interactive carrier, thereby forming a closed loop of virtual space interaction, cross-domain execution and state synchronization write-back. Summary of the Invention

[0009] (1) Purpose of the invention The present invention aims to provide a virtual-real interaction control method, system and computer-readable storage medium based on a virtual interactive carrier, so as to solve the problems in the prior art of the separation between virtual content display and real execution control, the lack of external execution capability for virtual space interaction, and the difficulty in writing back the external execution results to the virtual interactive carrier state.

[0010] Through this invention, a virtual interactive carrier in a virtual three-dimensional space can serve as an executable operation endpoint. On the one hand, it can establish an association with digital content or executable instructions, and on the other hand, it can send the target executable instructions to the target execution interface through a secure transmission path and update its running status synchronously based on the execution result feedback.

[0011] (2) Technical solution According to one aspect of the present invention, a virtual-real interaction control method based on a virtual interactive carrier is provided, comprising the following steps.

[0012] First, at least one interactive virtual interactive carrier is deployed in a virtual three-dimensional space, and the virtual interactive carrier is associated with a spatial location in the virtual three-dimensional space.

[0013] The virtual 3D space can be a multi-view image 3D reconstruction space based on real physical space, a pre-made 3D space, an augmented reality space, a mixed reality space, a digital twin space, a world model space, or other spatial representations with spatial location expression capabilities. The virtual interactive carrier can be a virtual switch, virtual button, virtual icon, virtual frame, virtual panel, spatial anchor point, spatial hotspot, virtual area, scene node, or other interactive spatial object.

[0014] Second, an association structure is established between the virtual interactive carrier and the digital content, wherein the digital content includes at least executable instructions, and the association structure is dynamically determined and maintained based on the spatial position of the virtual interactive carrier in the virtual three-dimensional space.

[0015] The association structure is used to establish a direct or indirect correspondence between virtual interactive carriers and digital content. The association structure can be represented as a mapping table, configuration file, scene node attributes, script, rule, server-side strategy, object identifier, event identifier, process identifier, or runtime parsing result. Dynamic maintenance can include at least one of the following operations: establishing, updating, enabling, disabling, re-parsing, synchronizing, or maintaining the association structure.

[0016] Third, in response to an interactive action on the virtual interactive carrier, a target executable instruction corresponding to the interactive action is determined.

[0017] The interactive actions can be triggered by the user through clicks, touch, voice, gestures, eye tracking, posture, or other interactive methods. In some implementations, they can also be triggered by user-authorized automation rules, operating entities, service proxies, sensor events, or system policies. The target executable instruction can be determined based on at least one of the following: the association structure, user identity, current scene state, target object state, server-side rules, workflow configuration, or model inference results.

[0018] In this specification, interactive actions on a virtual interactive platform may include user interaction with the virtual interactive platform, or trigger signals, event messages, or system recognition results generated by such interaction.

[0019] Fourth, the target executable instruction is sent to the target execution interface through a secure transmission path, and the execution result feedback returned from the target execution interface is received.

[0020] The secure transmission path is a transmission channel used to transfer target executable instructions from the virtual interaction carrier side to the target execution interface. The secure transmission path may include at least one of the following: a relay server, an encrypted communication link, an edge node, an end-to-end communication channel, a LAN direct connection channel, a device gateway, a platform software development kit, or a message broker. In a preferred embodiment, the target executable instructions are encapsulated into a structured instruction package and sent to the IoT platform or service interface via an encrypted communication method through the relay server.

[0021] The target execution interface is an interface capable of receiving target executable instructions and triggering changes in the states of external devices, external services, business processes, automated tasks, simulation states, or other target systems. The target execution interface may include device interfaces, IoT platform interfaces, service interfaces, application programming interfaces, automated process interfaces, gateway proxy interfaces, state management interfaces, or combinations thereof.

[0022] The execution result feedback may include at least one of the following: synchronous receipt, asynchronous event, device status, service execution result, task processing status, control success indicator, control failure indicator, subscription message, polling result, log result, or target status change information. The execution result feedback may be returned directly by the target execution interface, or indirectly via a platform, gateway, relay server, message system, status manager, or other intermediate system.

[0023] Fifth, based on the feedback of the execution results, the running status of the virtual interactive carrier in the virtual three-dimensional space is updated synchronously to form a closed loop of virtual space interaction, cross-domain execution, and state synchronization write-back.

[0024] The operational status may include the visual attributes, text attributes, prompt attributes, animation attributes, interactive status, enabled status, status labels, spatial marker status, shared status nodes, or consistency status with the target device or service of the virtual interactive carrier or its associated display object in the virtual three-dimensional space. Preferred representations of the operational status include at least one of the following: material attributes, luminance, color saturation, transparency, on / off status, or visual prompt status.

[0025] The state synchronization write-back can be real-time write-back, near real-time write-back, asynchronous write-back, timed write-back, triggered write-back, temporary state write-back, or correction write-back. The state synchronization write-back can be applied to the virtual interactive carrier itself, or to prompt objects, status panels, device shadow objects, spatial markers, service panels, or shared status nodes associated with the virtual interactive carrier.

[0026] (3) System Scheme According to another aspect of the present invention, a virtual-real interaction control system based on a virtual interactive carrier is provided, including a carrier deployment module, an association management module, an interaction response module, an instruction execution module, and a state synchronization module.

[0027] The carrier deployment module is used to deploy at least one interactive virtual carrier in a virtual three-dimensional space, the virtual interactive carrier being associated with a spatial location in the virtual three-dimensional space.

[0028] The association management module is used to establish and dynamically maintain the association structure between the virtual interactive carrier and the digital content, wherein the digital content includes at least executable instructions.

[0029] The interaction response module is used to respond to the interaction action on the virtual interaction carrier and determine the target executable instruction corresponding to the interaction action.

[0030] The instruction execution module is used to send the target executable instruction to the target execution interface through a secure transmission path, and to receive the execution result feedback returned from the target execution interface.

[0031] The status synchronization module is used to synchronously update the running status of the virtual interactive carrier in the virtual three-dimensional space based on the feedback of the execution results.

[0032] The modules described above can be deployed in the same terminal device or distributed across terminal devices, servers, edge nodes, IoT platforms, gateways, target devices, service interfaces, or cloud systems. All or part of the business logic can be completed by the virtual 3D space terminal, local terminal, server, platform, or a combination thereof; the aforementioned execution loop can be formed when the virtual interaction carrier participates in at least one key link in the triggering, association, distribution, feedback reception, or status write-back chain.

[0033] (4) Beneficial effects Compared with the prior art, the present invention has at least the following beneficial effects.

[0034] First, this invention constructs a standardized closed loop between virtual space interaction and external target execution. Through virtual interaction carriers, digital content association structures, executable instructions, target execution interfaces, execution result feedback, and status synchronization write-back, the virtual three-dimensional space is expanded from a passive content display space into a spatial operation environment that can be actively manipulated and perceive external execution results.

[0035] Second, this invention improves the explicit executability of interactive behaviors in virtual space. After a user or other interaction initiator operates a virtual interactive carrier in a virtual three-dimensional space, they can trigger external device control, service interface calls, automated tasks, or other target executions, and perceive the execution results through changes in the carrier's operating state.

[0036] Third, this invention improves the state consistency between the virtual space and the external target system. Execution results are written back to the virtual interaction carrier or its associated state, reducing the need for users to repeatedly switch, query, or confirm between the virtual space and the external control system.

[0037] Fourth, the technical solution provided by this invention is adaptable to technological evolution. The secure transmission path, target execution interface, and operating state are not limited to a single protocol, a single interface form, or a single display method. Therefore, it can be adapted to direct end-to-end connection, edge computing, cloud platforms, mixed reality, digital twins, world models, brain-computer input, and other future spatial computing infrastructures. Key term definitions

[0038] To facilitate understanding of this invention, several terms used in the specification are explained below. Unless the context clearly defines otherwise, the following terms should not be construed as excluding their equivalents or alternative implementations.

[0039] (1) Virtual interactive carrier A virtual interactive carrier refers to a virtual object, virtual region, spatial anchor point, hotspot, control, marker, scene node, or combination thereof deployed in a virtual three-dimensional space, possessing a spatial location and capable of being interacted with. Virtual interactive carriers are used to carry digital content, executable capabilities, or state representations, and to respond to interactive actions from users or other interaction initiators.

[0040] The virtual interactive carrier is not limited to being displayed as a button, having a physical shape, or being an object that is always visible. Any virtual interactive carrier in this invention can be a locationable interactive entry point in a virtual three-dimensional space or spatial representation, and can participate in the determination, issuance, feedback reception, or status write-back link of target executable instructions.

[0041] (2) Digital content Digital content refers to content associated with a virtual interactive medium. The digital content includes at least executable instructions, and may also include media files, text, audio, video, 3D models, descriptive information, links, service entry points, task entry points, or other digital information.

[0042] In this invention, the triggering and execution of executable instructions and the feedback write-back loop constitute the main technical link of the closed-loop mechanism of this invention, while media files or content display can exist as parallel or optional branches.

[0043] (3) Association structure A relational structure refers to an explicit or implicit structure that establishes a correspondence between a virtual interactive medium and digital content. This relational structure can be a data table, configuration file, scene graph node attributes, object identifiers, event identifiers, scripts, rules, workflow configurations, server-side strategies, runtime parsing results, or any combination thereof.

[0044] Association structures can directly point to executable instructions, or indirectly to rules, processes, services, objects, or contexts used to generate or determine executable instructions. Association structures can be stored in terminal devices, servers, the cloud, scene model files, device models, gateways, or service orchestration systems.

[0045] (4) Executable instructions Executable instructions are operational information that can be parsed by the system and sent to the target execution interface for execution. Executable instructions can take the form of control commands, interface call parameters, event messages, function calls, workflow trigger information, task requests, message queue messages, natural language commands, model tool call results, smart contract call data, or other operational information that can trigger execution by an external target.

[0046] Executable instructions are not limited to data objects named "instructions"; any instruction that can cause changes in the state of a device, service, process, task, simulation state, or external system after being parsed by the target execution interface or intermediate system can be used as an executable instruction in this invention.

[0047] (5) Structured instruction package A structured instruction package is a preferred encapsulation form of executable instructions, which may include at least a target device identifier, control parameters, and authentication information. In some implementations, the structured instruction package may also include a target service identifier, task identifier, event identifier, timestamp, signature, session identifier, routing information, feedback address, priority, or extended fields.

[0048] A structured instruction packet can be a single physical data packet, or a logical structure formed by a message header, message body, path parameters, session context, server configuration, platform token, or associated structure. If the system can parse out the target, parameters, and authentication-related information, it can be considered to have structured instruction information.

[0049] In service interface, process interface, or automated task interface scenarios, the target device identifier can correspond to the identifier of the target service, target process, target execution object, or target interface.

[0050] (6) Secure transmission path A secure transmission path refers to the transmission channel through which executable instructions are transferred from the virtual interaction carrier to the target execution interface. This path can ensure the trustworthiness of the execution link through at least one of the following mechanisms: identity verification, permission verification, integrity protection, encryption protection, trusted channels, device binding, platform authentication, or secure domain isolation.

[0051] Secure transmission paths may include relay servers, encrypted communication links, edge nodes, end-to-end communication channels, direct LAN connections, device gateways, platform SDKs, message brokers, or combinations thereof. Relay servers and encrypted communication methods are preferred embodiments and do not constitute a limitation on the scope of protection of this invention.

[0052] (7) Target execution interface The target execution interface refers to an interface that can receive executable instructions and cause changes in the state of the target device, target service, business process, automated task, simulation object, world model state, or other external system.

[0053] The target execution interface can include device interfaces, IoT platform interfaces, service interfaces, application programming interfaces, automated process interfaces, third-party platform interfaces, gateway proxy interfaces, state management interfaces, simulation state interfaces, or world model execution interfaces. The target execution interface can be deployed in the cloud, at the edge, on local devices, at gateways, on the terminal side, or on a third-party platform.

[0054] (8) Feedback on execution results Execution result feedback refers to the feedback information returned from the target execution interface or related intermediate systems that relates to the execution behavior of executable instructions or changes in the target's state after execution. Execution result feedback can include synchronous receipts, asynchronous events, device status, service execution results, task status, control success flags, control failure flags, subscription messages, polling results, log results, state cache results, or predictive confirmation information.

[0055] The execution result feedback can come directly from the target execution interface, or it can be returned indirectly through an IoT platform, service platform, gateway, relay server, message queue, event system, state manager, or log system.

[0056] (9) Operating status Operational status refers to the state parameters of a virtual interactive carrier in a virtual three-dimensional space, or the state parameters associated with that virtual interactive carrier. Operational status can include visual attributes, text status, status labels, interactivity, enabled status, prompt status, animation status, sound prompts, haptic prompts, spatial marker status, shared status nodes, or their consistency with the target device or service.

[0057] Visual attributes are a preferred form of representation of operational status, and may include at least one of the following: material properties, luminance, color saturation, transparency, on / off status representation, or visual prompt status.

[0058] (10) Virtual three-dimensional space Virtual 3D space refers to a spatial representation that possesses spatial coordinates, spatial relationships, depth information, spatial anchor points, scene graph structure, or 3D rendering capabilities. Virtual 3D space can be an explicitly rendered 3D scene, or it can be an augmented reality overlay space, a mixed reality space, a digital twin space, a spatial representation within a world model, or other spatial environments that can support spatial positioning of virtual interactive carriers. Attached Figure Description

[0059] Figure 1 This is a schematic diagram of the overall process of the virtual-real interaction control method in an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the relationship between the virtual interactive carrier, digital content, executable instructions, and target execution interface in an embodiment of the present invention. Figure 3 This is a schematic diagram of the field structure of the structured instruction package in an embodiment of the present invention; Figure 4 This is a schematic diagram of the architecture of the secure transmission path in an embodiment of the present invention; Figure 5 This is a schematic diagram of the closed loop for state synchronization write-back in an embodiment of the present invention; Figure 6 This is a schematic diagram of the module structure of the virtual-real interaction control system in an embodiment of the present invention; Figure 7 This is a schematic diagram illustrating the synchronization of shared space status among multiple users in an embodiment of the present invention; Figure 8 This is a schematic diagram of a digital twin or world model bridging scenario in an embodiment of the present invention. Detailed Implementation

[0060] The present invention will be further described below with reference to embodiments. It should be understood that the following embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. Where there is no conflict, the technical features of the various embodiments can be combined with each other.

[0061] (1) Overall process of the method like Figure 1 As shown, the virtual-real interaction control method provided in this embodiment of the invention may include the following steps.

[0062] S101, Deploy at least one interactive virtual interactive carrier in a virtual three-dimensional space.

[0063] For example, in a virtual 3D space reconstructed from a residential space, virtual switches can be deployed on the kitchen wall; in a commercial exhibition space, virtual service panels can be deployed near the exhibits; and in an industrial digital twin space, operation panels, spatial anchors, or status markers can be deployed near the equipment model. The virtual interactive carrier is associated with its spatial location in the virtual 3D space, which can correspond to its position in the 3D scene model, its position in real physical space, the position of the digital twin object, or the position of a spatial node in the world model.

[0064] S102, Establish the connection structure between the virtual interactive carrier and digital content. For example... Figure 2 As shown, the virtual interactive carrier can establish a correspondence with digital content through an association structure. The digital content may include executable instructions and media files, and the executable instructions may further correspond to different types of target execution interfaces.

[0065] The digital content includes at least executable instructions. The associated structure can be dynamically determined based on the spatial location of the virtual interactive carrier. For example, a virtual switch on a kitchen wall can be associated with instructions to control the kitchen ceiling light; a virtual service panel next to an exhibit can be associated with a product ordering service interface; and a spatial anchor point next to an industrial equipment model can be associated with a device parameter adjustment task. This associated structure can be dynamically maintained during system operation, such as when equipment is replaced, service entry points are updated, user permissions change, scene states change, or server configurations are updated.

[0066] S103, in response to an interactive action on a virtual interactive carrier, determine the target executable instruction corresponding to the interactive action.

[0067] The interactive actions can be click operations, voice commands, gesture operations, gaze confirmation, brain-computer interface input, or other interactive signals. The target executable command can be determined locally by the terminal device, or it can be jointly determined by the server, rule engine, service orchestration system, IoT platform, or other backend systems based on carrier identifier, user identity, scene status, and association structure.

[0068] S104: Send the target executable instructions to the target execution interface through a secure transmission path and receive the execution result feedback.

[0069] In a preferred embodiment, such as Figure 3 As shown, the target executable instructions are encapsulated into a structured instruction package, which includes at least the target device identifier, control parameters, and authentication information. After the system parses the structured instruction package, as shown... Figure 4As shown, the command is sent to the corresponding IoT platform or service interface via a relay server using encrypted communication. After the target execution interface executes the command, it returns feedback on device status, service execution result, task status, or other execution results.

[0070] In other implementations, the target executable instruction may also manifest as an API request, event message, workflow trigger information, function call, task request, or message queue message. The transmission path may employ end-to-end encrypted channels, edge nodes, direct LAN connections, gateway proxies, or third-party platform forwarding methods.

[0071] S105, based on the execution result feedback, synchronously update the running status of the virtual interactive carrier in the virtual three-dimensional space.

[0072] For example, if the target device returns an "on" status, the virtual switch's brightness, color, position, or status label is updated; if the service interface returns "order successful," the virtual service panel displays a success status and changes its prompt attributes; if the task execution fails, the virtual interactive carrier or its associated prompt object enters a failure prompt state. Figure 5 As shown, the above methods can form a closed loop of virtual space interaction, cross-domain execution, and state synchronization write-back.

[0073] (2) System Architecture like Figure 6 As shown, the virtual-real interaction control system provided in this embodiment of the invention may include a terminal device, a relay server, a target execution platform, and a target object.

[0074] Terminal devices are used to present virtual 3D space, deploy virtual interactive carriers, capture interactive actions, and display status write-back results. Terminal devices can be VR devices, AR devices, MR devices, mobile terminals, desktop terminals, vehicle terminals, holographic display devices, or other spatial computing terminals.

[0075] Relay servers are used to receive or parse instruction information from terminal devices, and to authenticate, route, encrypt and forward, log, or relay the instruction information. In some implementations, relay servers can be replaced by edge nodes, device gateways, message brokers, cloud functions, or platform SDKs.

[0076] The target execution platform can be an IoT platform, service interface, automation platform, third-party platform, gateway service, workflow system, simulation state management system, or world model execution system. The target execution platform receives executable instructions and triggers the execution of the target object.

[0077] The target object can be a physical device, external service, automated process, data query task, order service, simulation object, world model state node, or other object that can be affected by the executable instructions.

[0078] The aforementioned components can be provided by the same entity or by different entities; the aforementioned modules can be deployed in the same device or in multiple devices, platforms, or services. This invention can also be implemented when multiple devices, platforms, or services jointly implement a closed loop that triggers the target execution interface of the virtual interactive carrier and feeds back the execution result to the virtual interactive carrier's running state.

[0079] (3) Example 1: Virtual switch controls physical device In a virtual 3D home space, the system generates a 3D scene model based on multi-view images of the user's residence or a pre-made floor plan model. A virtual switch is deployed on a corresponding location on the kitchen wall as a virtual interaction mechanism. This virtual switch is spatially linked to the kitchen ceiling light.

[0080] The system establishes a connection between the virtual switch and digital content. The digital content includes executable instructions for controlling the kitchen ceiling light switch, and may also include kitchen-related photos or explanatory information. The executable instructions are encapsulated into a structured instruction package, which includes a target device identifier, control parameters, and authentication information. For example, the target device identifier is the kitchen ceiling light's device ID, the control parameter is "on" or "off," and the authentication information is a user authorization token or platform session credential.

[0081] When a user clicks the virtual switch in the virtual 3D space, the system responds to the interaction and determines that the target executable instruction is to control the kitchen ceiling light switch. The terminal device or server then sends the target executable instruction to the IoT platform via a secure transmission path. In one implementation, the system sends the structured instruction packet to the IoT platform via a relay server using TLS-based HTTPS or MQTT over TLS protocols.

[0082] After verifying the authentication information, the IoT platform sends a control command to the kitchen ceiling light. Once the kitchen ceiling light executes the command, the IoT platform returns status feedback information, such as "on," "off," "execution failed," or the current brightness parameter. This status feedback information, as the execution result, is relayed back to the terminal device via a relay server.

[0083] The terminal device synchronously updates the operating status of the virtual switch based on the execution results. For example, when the kitchen ceiling light is on, the system increases the luminosity of the virtual switch material, adjusts the color to a brighter shade, and changes the switch status to "on" or displays an "on" status label. When the kitchen ceiling light is turned off by someone else via a physical wall switch, the IoT platform pushes a status change notification, and the system, upon receiving this change, synchronously adjusts the virtual switch status to "off".

[0084] Through this implementation method, users can trigger the control of real devices in a virtual three-dimensional space by using virtual switches, and perceive the execution results and current status of real devices by the changes in the state of the virtual switches.

[0085] This embodiment illustrates one implementation of the present invention in a physical device control scenario. Specifically, it involves triggering a target executable instruction through a virtual interactive carrier associated with a spatial location in a virtual three-dimensional space, and feeding back the execution result of the external device to the running status of the virtual interactive carrier.

[0086] Within the same virtual 3D space, the virtual switch or its adjacent virtual photo frame can also be associated with media files such as kitchen photos, equipment manuals, or maintenance records. When a user performs a preset viewing operation on the virtual switch or virtual photo frame, the system presents the associated media file; when the user performs a control operation, the system executes the associated executable command. Through this implementation, the same virtual interactive platform or adjacent virtual interactive platforms can be used simultaneously for media content presentation and executable command triggering.

[0087] (4) Example 2: Virtual service panel calls external service interface In a virtual commercial exhibition space, the system deploys virtual service panels near the exhibits as virtual interactive platforms. These virtual service panels are linked to services such as ordering, making appointments, or querying information.

[0088] The system establishes a connection structure between the virtual service panel and executable instructions. These executable instructions can be used to call the order service interface. Each executable instruction can be encapsulated into a structured instruction package, which includes the target service identifier or target device identifier, control parameters, and authentication information. The control parameters may include the product SKU, quantity, shipping address, appointment time, or user selections; the authentication information may include user session tokens, account credentials, or platform authorization information.

[0089] When a user clicks the "Place Order" area in the virtual service panel or issues a voice confirmation command, the system responds to the interaction and determines the target executable command. The system then sends the target executable command to the order service interface via a secure transmission path. After processing the request, the order service interface returns service execution result feedback, such as "Order Successful," "Insufficient Stock," "Payment Pending Confirmation," "Service Request Accepted," or task processing status.

[0090] The system updates the virtual service panel's running status based on the service execution results. For example, when an order is successfully placed, the virtual service panel displays a success status label, changes color saturation, displays the order number, or enters a completed state; when an order fails, the virtual service panel displays a failure message and remains in a retrievable state. This status feedback allows users to perceive the service call results within the original virtual space, thereby reducing the need for users to leave the virtual 3D space and enter a separate business page for confirmation.

[0091] In some implementations, the service execution result may first be returned to the state management system, message queue, or log system, and then retrieved by the terminal device through subscription or polling. When the result ultimately affects the virtual service panel or its associated state, a closed loop of execution result feedback and state synchronization write-back can be formed.

[0092] (5) Example 3: State synchronization in a multi-user shared space like Figure 7 As shown, in a shared virtual 3D space supporting multiple users online simultaneously, each user terminal shares the same 3D scene model, media layout, and virtual interactive carrier state. After any user interacts with the virtual interactive carrier, the system determines the target executable instruction and executes it through the target execution interface.

[0093] After the target execution interface returns the execution result feedback, the system not only updates the running status of the virtual interactive carrier in the initiating user's local terminal, but also synchronizes at least one of the following to other online user terminals through relay servers, status servers, shared databases, message queues, publish-subscribe systems or collaboration protocols: operation instructions, execution result feedback, device status, carrier running status or shared space status.

[0094] For example, when multiple users are viewing a virtual home space, if one user clicks the virtual kitchen switch and successfully turns on the real kitchen ceiling light, the virtual kitchen switch will appear as "on" on all online user terminals. Through this implementation, each user can synchronously perceive changes in the operational status of external target devices or services.

[0095] (6) Example 4: Virtual-Real Interaction Control in Mixed Reality Scenes In mixed reality scenarios, virtual interactive devices can be overlaid on real-world images or co-located with real physical objects. For example, when a user wears a mixed reality device to observe a real room, the system overlays a virtual control panel next to the real air conditioner. This virtual control panel is associated with the air conditioner control service interface.

[0096] When a user triggers the virtual control panel via gestures, voice, or gaze, the system identifies the target executable command and sends it to the air conditioner control interface via a secure transmission path. After the air conditioner executes the command, it returns status feedback, and the mixed reality device updates its current operating status, such as temperature, mode, on / off status, or indicator lights, in the virtual control panel next to the real air conditioner.

[0097] This implementation illustrates that the virtual interaction carrier is not limited to a fully immersive virtual reality space. The invention can also be implemented in augmented reality or mixed reality scenarios, where the virtual interaction carrier is associated with a spatial location and forms a closed loop of external execution and state write-back.

[0098] (7) Example 5: Brain-computer interface or new input method adaptation In some implementations, the interactive actions can be generated by a brain-computer interface (BCI) or other novel input devices. A neural signal decoding system can interpret the user's intention signals as selection, confirmation, or cancellation operations on the virtual interactive medium. This interpretation process can be performed solely by the BCI system, or it can be performed jointly by the BCI system and an eye-tracking system, gesture recognition system, or other assisted positioning system. This selection or confirmation operation triggers the determination of the target executable command.

[0099] For example, a user gazes at a virtual service panel in a virtual 3D space and triggers a service request associated with that virtual service panel through brain-computer interface (BCI) intent confirmation. The system still executes in a closed loop: virtual interaction medium, executable instructions, target execution interface, execution result feedback, and state synchronization write-back. Brain-computer input can serve as one way to generate interactive actions and does not alter the execution closed-loop mechanism of this invention.

[0100] (8) Example 6: Expansion of the Operating Subject and World Model like Figure 8 As shown, in some implementations, the running entity can act as the initiator of the interactive action. The running entity can be an Agent, service proxy, automated rule executor, delegated running entity, or a decision-making module in the world model. The triggering qualifications, authority scope, and governance rules of the running entity can be determined by other object governance schemes or system policies. After the operation is triggered, it can enter the target execution interface through a virtual interaction carrier and form a state write-back closed loop.

[0101] For example, in an industrial digital twin scenario, when the operating entity in the world model detects that the simulation parameters of a certain device exceed a safety threshold, it triggers a target executable command through the virtual interactive carrier corresponding to that device in the virtual space. This command is then transmitted to the control interface of the real factory equipment via a secure transmission path. After the real device executes the command, its status is transmitted back to the world model and the virtual 3D space via the status management system, and the system synchronously updates the operating status of the corresponding virtual interactive carrier.

[0102] In this implementation, the world model is used to perform observation, simulation, and decision-making, and the closed-loop mechanism of this invention is used to transform the decision results into executable cross-domain operations and confirm the execution results. Through this implementation, this invention can serve as an operational interface layer for a world model, digital twin system, or embodied intelligent system to act on a real physical system. Alternative implementation methods

[0103] (1) Implementation of replacement of virtual interactive carrier Virtual interactive carriers can be represented as visible objects, or as spatial hotspots, transparent interactive areas, spatial anchors, device outlines, scene nodes, gesture areas, gaze-triggered areas, or brain-computer interface areas. A virtual interactive carrier can be a single object or a collection of interactive carriers formed by multiple virtual objects, controls, areas, or anchors.

[0104] In one combined implementation, one virtual carrier is used to select the target object, another virtual carrier is used to select the action, and a third virtual carrier is used to confirm execution. Multiple interactive actions collectively determine the target executable instruction. This combined interaction still forms the execution loop of the present invention.

[0105] (2) Implementation of replacement of associated structures The association structure can be explicitly stored or implicitly generated. A virtual interaction vehicle can send only the object ID, event ID, rule ID, process ID, or service ID, and the backend rule engine will then generate the target executable instruction based on this identifier, user permissions, scenario status, or service configuration. In this case, a relationship indirectly determined through the association structure still exists between the virtual interaction vehicle and the executable instruction.

[0106] In some implementations, the same virtual interaction platform can be associated with different executable instructions under different users, permissions, times, scenarios, or device states. Dynamic maintenance of the association structure can be accomplished by terminals, servers, cloud platforms, rule engines, or service orchestration systems.

[0107] (3) Implementation of executable instruction replacement Executable instructions can manifest as traditional control commands, or as business events, task requests, service calls, workflow trigger information, function calls, model tool call results, natural language commands, or message queue messages. For example, the system can send a business event `room.light.toggle.requested`, and the event consumer can then convert this business event into a device control command.

[0108] If the event or message can be parsed by the target execution interface or intermediate system and trigger the target execution, it can be used as an implementation method of executable instructions.

[0109] (4) Implementation of replacement of transmission and execution links Secure transmission paths can be single-segment or multi-segment. Target executable commands can be transmitted cascaded through terminals, servers, relays, gateways, IoT platforms, third-party automation platforms, device SDKs, LAN channels, or edge nodes. Authentication, encryption, authorization verification, integrity verification, routing, or forwarding can be performed at any stage of the transmission path.

[0110] The system can directly control the target device, or it can indirectly execute the execution through third-party platforms, automation platforms, service orchestration platforms, gateways, proxy services, or state managers. All of these indirect execution methods can be considered implementations that trigger external execution through the target execution interface.

[0111] (5) Implementation of replacement of feedback and write-back links The execution result feedback can be returned synchronously or asynchronously; it can come directly from the target execution interface, or from the state management system, message system, log system, subscription system, polling interface, device shadow system, or third-party platform.

[0112] In some implementations, the system can first update the temporary running state based on local prediction or optimistic update mechanisms, and then correct the running state of the virtual interactive carrier after the actual execution results are received.

[0113] (6) Distributed implementation of business logic In some implementations, the virtual 3D space serves only as a front-end trigger, while the business logic is executed by the back-end control system, rule engine, IoT platform, service orchestration system, or third-party platform. When the interactive actions of the virtual interactive carrier are used to trigger the determination of the target executable instruction, and the execution result feedback ultimately affects the virtual interactive carrier or its associated state, the execution closed loop of this invention can be formed.

[0114] In some implementations, control parameters, authentication information, or target identifiers can be distributed across the message body, message header, path parameters, session context, platform permission table, server configuration, or associated structure, rather than being concentrated in a single data packet.

[0115] The present invention can also be implemented by a computer program. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, causes the processor to perform the steps of the method described in any of the above embodiments.

Claims

1. A virtual-real interaction control method based on a virtual interaction carrier, characterized in that, include: Deploy at least one interactive virtual interactive carrier in a virtual three-dimensional space, wherein the virtual interactive carrier is associated with a spatial location in the virtual three-dimensional space; Establish an association structure between the virtual interactive carrier and the digital content, wherein the digital content includes at least executable instructions, and the association structure is dynamically determined and maintained based on the spatial position of the virtual interactive carrier in the virtual three-dimensional space. In response to an interactive action on the virtual interactive carrier, a target executable instruction corresponding to the interactive action is determined; The target executable instruction is sent to the target execution interface through a secure transmission path, and the execution result feedback returned from the target execution interface is received; Based on the feedback of the execution results, the running status of the virtual interactive carrier in the virtual three-dimensional space is updated synchronously to form a closed loop of virtual space interaction, cross-domain execution and state synchronization write-back.

2. The method of claim 1, wherein, The virtual interactive carrier includes at least one of a virtual switch, a virtual button, a virtual icon, a virtual frame, and a virtual panel; the virtual three-dimensional space includes a three-dimensional scene model, and the spatial position of the virtual interactive carrier in the virtual three-dimensional space is associated with the corresponding spatial position in the three-dimensional scene model.

3. The method of claim 1, wherein, The virtual three-dimensional space includes a three-dimensional scene model. The association structure is dynamically determined based on the spatial position of the virtual interactive carrier in the three-dimensional scene model, and the association between the virtual interactive carrier and the digital content is dynamically maintained.

4. The method of claim 1, wherein, The step of sending the target executable instruction to the target execution interface through a secure transmission path and receiving execution result feedback includes: encapsulating the target executable instruction into a structured instruction package, the structured instruction package including at least the target device identifier, control parameters, and authentication information; parsing the structured instruction package and sending it to the corresponding IoT platform or service interface through a relay server in encrypted communication; the execution result feedback includes at least one of the following: target device status, service execution result, control success identifier, control failure identifier, and task processing status.

5. The method according to claim 1, characterized in that, The synchronous update of the operating status of the virtual interactive carrier includes: updating at least one visual attribute of the virtual interactive carrier in the virtual three-dimensional space based on the execution result feedback; the visual attribute includes at least one of material attributes, luminance, color saturation, transparency, on / off status expression or visual prompt status.

6. The method according to claim 1, characterized in that, The digital content also includes media files; in response to interactive actions on the virtual interactive carrier, the associated media files are presented or the associated executable instructions are executed.

7. The method of claim 1, wherein, The interactive action includes at least one of the user's click operation on the virtual interactive carrier or voice command.

8. The method of claim 1, wherein, The method supports collaborative spaces where multiple users are online simultaneously; each user terminal shares the same 3D scene model, media layout, and virtual interactive carrier status; when any user interacts with the virtual interactive carrier, their operation instructions and the resulting output are broadcast synchronously through a relay server, so that the virtual space views of other online user terminals are updated in real time and consistently.

9. A virtual-real interaction control system based on a virtual interaction carrier, characterized in that, include: A carrier deployment module is used to deploy at least one interactive virtual carrier in a virtual three-dimensional space, wherein the virtual interactive carrier is associated with a spatial location in the virtual three-dimensional space. The association management module is used to establish and dynamically maintain the association structure between the virtual interactive carrier and the digital content, wherein the digital content includes at least executable instructions. An interactive response module is used to respond to an interactive action on the virtual interactive carrier and determine a target executable instruction corresponding to the interactive action. The instruction execution module is used to send the target executable instruction to the target execution interface through a secure transmission path, and to receive the execution result feedback returned from the target execution interface; The status synchronization module is used to synchronously update the running status of the virtual interactive carrier in the virtual three-dimensional space based on the feedback of the execution results.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, When the program is executed by the processor, it implements the method of any one of claims 1 to 8.

Images