Method and device for remotely controlling smart home through vehicle-cloud combination

Abstract

The application provides a method and device for remotely controlling smart home through vehicle-cloud combination, a cloud server and a storage medium. The method comprises the following steps: receiving user input information collected by a vehicle end; calling a scene planning intelligent agent to perform home scene decomposition according to the user input information, so as to obtain at least one atomic operation, the atomic operation representing the smallest control action in the home scene; generating a first task instruction according to the at least one atomic operation, calling a semantic mapping intelligent agent to perform semantic mapping on the first task instruction to obtain a second task instruction, the first task instruction being used for regulating and controlling the smart home; the second task instruction being a native semantic instruction recognizable by the smart home; and forwarding the second task instruction to the smart home through a home gateway, so that the smart home responds. The method can realize accurate conversion from natural language to structured task, and provides active service for users.

Description

Technical Field

[0001] This application relates to the field of new energy technology, and in particular to a method, device, cloud server and storage medium for remotely controlling smart homes through vehicle-cloud collaboration. Background Technology

[0002] With the development of the automotive industry, smart cars are no longer just traditional means of transportation, but can serve as life assistants, providing greater convenience to people's lives. Meanwhile, the smart home ecosystem is becoming increasingly diversified, offering more possibilities for people's smart living. Among related technologies, users can remotely issue commands to their smart home devices via smart in-vehicle terminals based on vehicle-to-everything (V2X) technology. However, the implementation of this solution relies on the user issuing remote commands. Summary of the Invention

[0003] This application provides a method, device, cloud server, and storage medium for remotely controlling smart homes via vehicle-cloud collaboration. This method can achieve accurate conversion from natural language to structured tasks, providing proactive services to users.

[0004] The technical solution of this application embodiment is implemented as follows: In a first aspect, embodiments of this application provide a method for remotely controlling a smart home via vehicle-cloud collaboration. This method is applied to the cloud within a vehicle-cloud collaboration system, which also includes a vehicle, a home gateway, and a smart home device. The cloud is configured with a scene planning agent and a semantic mapping agent. The method includes: Receive user input information collected by the vehicle; The scene planning agent is invoked to deconstruct the home scene based on user input information, and obtain at least one atomic operation. The atomic operation represents the minimum control action in the home scene. A first task instruction is generated based on at least one atomic operation. A semantic mapping agent is invoked to perform semantic mapping on the first task instruction to obtain a second task instruction. The first task instruction is used to control the smart home. The second task instruction is a native semantic instruction that the smart home can recognize. The second task instruction is forwarded to the smart home system through the home gateway for the smart home system to respond to.

[0005] In one embodiment, generating a first task instruction based on at least one atomic operation includes: The equipment capability requirement is determined based on at least one atomic operation, and the equipment capability requirement indicates the equipment capability required by the atomic operation. Based on device capability requirements, obtain the device capability status of smart home devices through the home gateway; The first task instruction is generated based on at least one atomic operation and the device capability status of the smart home.

[0006] In one embodiment, there are multiple smart home devices. Based on device capability requirements, the device capability status of the smart home devices is obtained through a home gateway, including: Based on the device capability requirements, send a device capability request to the home gateway; Receive the device capability status of each smart home device corresponding to the home gateway, as reported by the home gateway.

[0007] In one embodiment, there are multiple smart home devices. Based on device capability requirements, the device capability status of the smart home devices is obtained through a home gateway, including: Based on the device capability requirements, a device capability request is sent to the home gateway, and the device capability request carries the target device identifier. Receive the device capability status of the smart home corresponding to the target device identifier reported by the home gateway.

[0008] In one embodiment, a first task instruction is generated based on at least one atomic operation and the device capability status of the smart home, including: Based on the device capability status of the smart home, determine at least one target atomic operation from at least one atomic operation, where the target atomic operation is the atomic operation to be executed; A structured task graph containing task dependencies is generated based on at least one target atomic operation. The structured task graph includes multiple subtasks. A first task instruction is generated for each subtask, and the first task instruction is used to control the smart home corresponding to at least one target atomic operation.

[0009] In one embodiment, the semantic mapping agent is invoked to perform semantic mapping on the first task instruction to obtain the second task instruction, including: Obtain semantic mapping relationships, which include the correspondence between proprietary protocols and standard protocols of smart home systems on different platforms; Invoke the semantic mapping agent to obtain the target smart home corresponding to the first task instruction; Based on semantic mapping, the first task instruction is converted into a second task instruction that conforms to the private protocol of the target smart home.

[0010] In one embodiment, the cloud is also configured with a protocol conversion agent to obtain semantic mapping relationships, including: Receive an access request sent by the home gateway. The access request carries access information of the home gateway and the smart home devices associated with the home gateway. The access information includes at least one of the following: device private property model parameters, attribute definitions, data value parameters, and device protocol specifications. The protocol conversion agent constructs a semantic mapping relationship based on the home gateway and the access information of the smart home devices associated with the home gateway. The semantic mapping relationship includes the correspondence between the private protocols and standard protocols of smart home devices on different platforms.

[0011] In one embodiment, the cloud is further configured with a conflict arbitration agent, and before generating a structured task graph containing task dependencies based on at least one target atomic operation, the method further includes: Invoke the conflict arbitration agent to perform conflict detection on at least one target atomic operation; For at least two conflicting target atomic operations, perform task orchestration or task elimination.

[0012] Secondly, embodiments of this application also provide a device for remotely controlling a smart home via vehicle-cloud collaboration, applied to the cloud within a vehicle-cloud collaboration system. The vehicle-cloud collaboration system further includes a vehicle terminal, a home gateway, and a smart home device. The cloud is configured with a scene planning agent and a semantic mapping agent. The device includes: The receiving module is used to receive user input information collected by the vehicle terminal; The processing module is used to invoke a scene planning agent to deconstruct the home scene based on user input information, obtaining at least one atomic operation, which represents the minimum control action in the home scene; generate a first task instruction based on at least one atomic operation; invoke a semantic mapping agent to perform semantic mapping on the first task instruction to obtain a second task instruction, which is used to control the smart home; the second task instruction is a native semantic instruction that the smart home can recognize; and forward the second task instruction to the smart home through the home gateway for the smart home to respond.

[0013] Thirdly, embodiments of this application also provide a cloud server, including a processor and a memory, wherein the memory stores a program or instructions that can run on the processor, and when the program or instructions are executed by the processor, they implement the steps of the method for remotely controlling a smart home via vehicle-cloud collaboration as described in any of the first aspects above.

[0014] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program or computer-executable instructions. When the computer program or computer-executable instructions are executed by a processor, they implement the steps of the method for remotely controlling a smart home via vehicle-cloud collaboration as described in any of the first aspects above.

[0015] Fifthly, embodiments of this application provide a computer program product, including a computer program or computer-executable instructions, which, when executed by a processor, implement the steps of the method for remotely controlling a smart home via vehicle-cloud collaboration as described in any of the above embodiments.

[0016] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of this application. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the architecture of a vehicle-cloud joint system provided in an embodiment of this application.

[0018] Figure 2 This is a flowchart illustrating a method for remotely controlling a smart home via vehicle-cloud collaboration, as provided in an embodiment of this application. Figure 1 .

[0019] Figure 3 This is a flowchart illustrating a method for remotely controlling a smart home via vehicle-cloud collaboration, as provided in an embodiment of this application. Figure 2 .

[0020] Figure 4 This is a flowchart illustrating a method for remotely controlling a smart home via vehicle-cloud collaboration, as provided in an embodiment of this application. Figure 3 .

[0021] Figure 5 This is a flowchart illustrating a method for remotely controlling a smart home via vehicle-cloud collaboration, as provided in an embodiment of this application. Figure 4 .

[0022] Figure 6 This is a logic block diagram of a device for remotely controlling a smart home via vehicle-cloud collaboration, as provided in an embodiment of this application.

[0023] Figure 7 This is a schematic diagram of the hardware structure of a cloud server provided in an embodiment of this application. Detailed Implementation

[0024] In order to gain a more detailed understanding of the features and technical content of the embodiments of this application, the implementation of the embodiments of this application will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for reference and illustration only and are not intended to limit the embodiments of this application.

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0026] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0027] It should also be noted that the terms "first, second, and third" used in the embodiments of this application are only used to distinguish similar objects and do not represent a specific order of objects. It is understood that "first, second, and third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0028] Furthermore, the reference to "embodiment" herein means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0029] With the development of IoT technology, the interconnection of heterogeneous devices such as vehicle terminals, home gateways, and smart home devices has become an important foundation for realizing smart living.

[0030] In existing technologies, users need to issue explicit semantic commands, such as turning on the air conditioner, opening the curtains, or turning on the welcome lights. The smart vehicle terminal then remotely transmits these commands to the smart home system via vehicle-to-everything (V2X) technology to control the smart home system's response. However, this solution relies on the user issuing remote commands, and these commands must be semantically explicit.

[0031] If the semantic information provided by the user is not a clear semantic command, then remote control of the smart home cannot be achieved.

[0032] In related technologies, after receiving explicit user commands, the intelligent vehicle terminal converts the commands into native commands recognizable by each smart home device based on preset rules or simple mapping logic, and then distributes them through the home gateway. However, if the user command is semantically ambiguous, such as "I'm home from get off work," the unclear meaning makes it impossible to determine which smart home device the command is targeting, thus preventing remote control of the smart home device.

[0033] The existing technologies suffer from the following main drawbacks: First, the instruction decomposition and semantic mapping process lacks dynamic adaptive capabilities, failing to effectively address the demands of multi-task parallelism in complex scenarios. Second, the semantic mapping accuracy is insufficient, easily leading to instruction misinterpretation or execution deviations, impacting user experience. Third, the system relies on a centralized architecture, which cannot guarantee basic device-level autonomous linkage in network outages or weak network environments, limiting reliability. Therefore, there is an urgent need for a method that enables intelligent scene decomposition and dynamic semantic mapping, allowing for remote control of smart homes via vehicle-cloud collaboration, to improve the accuracy and robustness of collaborative control of heterogeneous devices.

[0034] To address the aforementioned technical issues, this application provides a method for remotely controlling a smart home via vehicle-cloud collaboration. The method is characterized by its application in a cloud-based vehicle-cloud collaborative system, which also includes a vehicle, a home gateway, and a smart home device. The cloud is configured with a scene planning agent and a semantic mapping agent. The method includes: receiving user input information collected by the vehicle; invoking the scene planning agent to deconstruct a home scene based on the user input information, obtaining at least one atomic operation, where each atomic operation represents the minimum control action in the home scene; generating a first task instruction based on the at least one atomic operation; invoking the semantic mapping agent to perform semantic mapping on the first task instruction to obtain a second task instruction, where the first task instruction is used to control the smart home device; the second task instruction is a native semantic instruction recognizable by the smart home device; and forwarding the second task instruction to the smart home device through the home gateway for the smart home device to respond.

[0035] This solution achieves precise conversion from natural language to structured tasks by decomposing fuzzy or abstract instructions into multiple atomic operations of the smallest granularity. Then, based on these atomic operations, a first task instruction is generated and converted into native semantic instructions recognizable by smart homes through a semantic mapping agent. This solves the control failure problem caused by protocol incompatibility and semantic inconsistency between heterogeneous devices. Finally, a second task instruction is sent to the target smart home device through the home gateway, achieving seamless cross-ecosystem and cross-protocol linkage. This process proactively analyzes user input information and provides proactive services to users. It eliminates the need for users to send task instructions to each specific smart home device, significantly improving the efficiency and accuracy of interconnection between multi-brand and multi-protocol devices. Furthermore, through cloud-based intelligent agent collaborative processing, it effectively avoids the single point of failure risk inherent in traditional centralized architectures.

[0036] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0037] The various embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0038] In the embodiments of this application, the execution subject of the method for remotely controlling smart homes through vehicle-cloud collaboration provided in various embodiments of this application can be a computer device, such as a server, server cluster, cloud server, or other similar devices.

[0039] In this embodiment, the computer device may deploy various intelligent agents provided in this embodiment, such as scene planning intelligent agents, semantic mapping intelligent agents, protocol conversion intelligent agents, and conflict arbitration intelligent agents, etc. The computer device can invoke these intelligent agents to execute the method for remotely controlling smart homes via vehicle-cloud collaboration provided in this embodiment.

[0040] The intelligent agent can include AI (Artificial Intelligence) models such as large models or small models. The large model can be a general-purpose large model, such as a general-purpose large language model (LLM), a visual large model, a multimodal large model, etc., or it can be an expert large model obtained by fine-tuning a general-purpose large model based on requirements. This application does not limit this. The large model learns the features and patterns of the input information by training on a large amount of diverse data. It typically has hundreds of millions to hundreds of billions of model parameters (such as the weights corresponding to the model network layers, used to control model behavior), and can capture complex relationships and patterns in the input information for accurate and efficient reasoning / computation.

[0041] Please refer to Figure 1 , Figure 1 This is a schematic diagram of the architecture of a vehicle-cloud integrated system provided in an embodiment of this application. The vehicle-cloud integrated system 100 includes a vehicle terminal 101, a cloud terminal 102, a home gateway 103, and a smart home 104.

[0042] Among them, the vehicle terminal 101 allows users to send information to the vehicle terminal during the vehicle use process. The vehicle terminal 101 can collect user input information and report the user input information to the cloud.

[0043] Cloud 102 is used to receive user input information, decompose ambiguous or abstract instructions into multiple atomic operations of the smallest granularity to generate a first task instruction, perform semantic transformation on the first task instruction to obtain a second task instruction, and send the second task instruction to the home gateway. In this embodiment, the cloud refers to the cloud platform provided by the car manufacturer. The cloud is not directly associated with the smart home, but establishes a communication connection with the home gateway.

[0044] In this embodiment of the application, the home gateway can be pre-registered with the cloud, and the cloud can associate the vehicle terminal with the home gateway, for example, by associating the two with the same user, or by establishing a mapping relationship between the two.

[0045] In some embodiments, during registration, the home gateway can report basic information about all smart home devices associated with it to the cloud. This basic information includes, for example, the brand, functions, control parameters (such as speed settings and size), on / off status, and control constraints of the smart home devices. When changes occur to the smart home devices associated with the home gateway, such as adding a new smart home device or deleting an existing one, the home gateway can send an update to the cloud.

[0046] Home gateway 103 is used to receive the second task instruction and distribute the second task instruction to the corresponding smart home devices.

[0047] Smart Home 104 is used to receive the corresponding second task instruction and respond to the execution instruction.

[0048] In the vehicle-cloud integrated system provided in this application embodiment, the cloud can perform scene parsing on fuzzy instructions to obtain atomic operations, and then determine multiple first task instructions based on the atomic operations. In other words, in this solution, users do not need to issue task instructions for each smart home device; instead, the cloud autonomously generates task instructions based on user input information and controls the smart home devices. This enables proactive services that are imperceptible to the user.

[0049] Please refer to Figure 2 , Figure 2 This is a flowchart illustrating a method for remotely controlling a smart home via vehicle-cloud collaboration, as provided in an embodiment of this application. Figure 1 This method is applied to the cloud within a vehicle-cloud integrated system, which also includes the vehicle itself, home gateways, and smart home devices. The cloud is configured with scene planning agents and semantic mapping agents, such as... Figure 2 As shown, the method may include steps 201 to 203.

[0050] Step 201: Receive user input information collected by the vehicle terminal.

[0051] In this context, "vehicle-side 101" typically refers to in-vehicle terminal equipment, such as in-vehicle central control screens, voice assistants, or in-vehicle mobile phone interconnection modules. This equipment is used to collect users' natural language commands or operational behaviors. The user input information collected by the vehicle-side can be in the form of voice (e.g., "I'm off work," "I'm going home"), or it can be structured or unstructured data generated by touch input, gesture recognition, or other interaction methods. The user input information collected by the vehicle-side is uploaded to a cloud server via in-vehicle communication modules (such as 5G, Wi-Fi, Bluetooth, etc.) as the raw input for subsequent processing.

[0052] The user input information collected by the vehicle is highly fuzzy and abstract, often including contextual dependencies, multi-task concurrency requirements, and user personalized preferences. For example, the comfort mode can be automatically turned on after arriving home. The comfort mode is an abstract concept that requires the vehicle's intelligent system to autonomously parse the specific control logic behind the abstract concept.

[0053] In one implementation, the user input information includes not only the content entered by the user (e.g., "turn on the living room lights before arriving home"), but also metadata such as vehicle location, timestamp, and user identity.

[0054] In some embodiments, user input information may be audio data collected through an in-vehicle voice assistant, signals output by the user based on a touchscreen, or gesture signals issued by the user.

[0055] In some embodiments, the cloud can perform preliminary preprocessing on user input information, including noise filtering, language recognition, and sensitive word detection, to provide high-quality input for subsequent intent parsing.

[0056] Step 202: Invoke the scene planning agent to deconstruct the home scene based on the user input information and obtain at least one atomic operation.

[0057] Among them, atomic operations represent the smallest control actions in a home environment.

[0058] In this embodiment, atomic operations are the basic units constituting complex home scene tasks, possessing independence, executableness, and composability. An atomic operation represents an indivisible device control action, such as turning on the living room light, adjusting the air conditioner temperature to 26°C, or closing the bedroom curtains. Each atomic operation corresponds to a specific device and its attribute operations, such as the on / off state of a light fixture, the air conditioner's temperature setting, or the degree of curtain opening / closing. Furthermore, atomic operations not only include the action itself but also implicitly contain control logic such as execution conditions, priorities, and dependencies. For example, adjusting the air conditioner temperature may require first determining whether the current room temperature is higher than a threshold.

[0059] In this embodiment, the Scene Planning Agent (SPA) can utilize its internally deployed lightweight domain-specific model (such as a home IoT-specific model fine-tuned based on Llama-3-8B) to perform semantic parsing of user input information. This model, trained on massive amounts of home scene corpora, possesses the ability to understand natural language intent, remove invalid semantics, and extract core needs.

[0060] For example, if a user inputs information including "going home", the SPA can first identify driving home as the trigger condition and home comfort mode as the abstract goal, and then break it down into multiple atomic operations: disarming security alarms, turning on the entrance lights, adjusting the air conditioner temperature to 26°C, opening the curtains, etc.

[0061] In this embodiment, a scene planning agent precisely deconstructs user input information, outputs a structured task graph containing multiple atomic operations, and clarifies the execution order and constraints between each atomic operation. For example, if a user says they will turn on all the lights in the house after returning home at night, the SPA can decompose this into three atomic operations: turning on the living room light, turning on the dining room light, and turning on the bedroom light, and set them to be executed in parallel, while adding a time constraint that they only execute during nighttime hours.

[0062] By deconstructing user input information into a home scenario, clarifying the task scenario, and identifying multiple atomic operations within the task scenario, an intelligent transformation from vague user intent to a precise and executable task sequence is achieved, preparing for subsequent semantic mapping and device control.

[0063] Step 203: Generate a first task instruction based on at least one atomic operation, and call the semantic mapping agent to perform semantic mapping on the first task instruction to obtain a second task instruction.

[0064] The first task instruction is used to control the smart home system; the second task instruction is a native semantic instruction that the smart home system can recognize.

[0065] In this embodiment, the SPA can generate a structured task graph, where nodes represent atomic operations and edges represent dependencies between tasks, such as sequential dependencies, parallel execution, and mutual exclusion. For example, disarming a security alarm must be performed before turning on the entrance lights to avoid security risks; adjusting the air conditioner temperature and opening the curtains can be performed in parallel.

[0066] In this embodiment, one or more first task instructions can be generated based on each atomic operation, with multiple atomic operations corresponding to multiple first task instructions. The multiple first task instructions need to consider dependencies, parallelism, and mutual exclusion relationships between tasks.

[0067] The first task instruction is a standardized intermediate instruction generated based on atomic operations, used to guide the control of smart home devices. The first task instruction uses an internally unified intermediate semantic layer representation, independent of any specific device or protocol, and typically includes information such as device category, operation type, target attributes, control parameters, execution sequence, and security constraints. For example, for the atomic operation of adjusting the air conditioner temperature to 26°C, the first task instruction might be represented as: {device_type: "air_conditioner", action: "set_temperature", value: 26, timestamp: 1710000000}. This instruction format facilitates transmission and processing between modules within the cloud, but it cannot be directly recognized by smart home devices.

[0068] In this embodiment, for example, different brands of air conditioners may use different field names such as temperature, temp, and heat_level to represent temperature settings; the field brightness represents ambient illuminance in lighting fixtures, but may represent screen backlight brightness in air conditioning equipment. Therefore, there are semantic differences between multi-source heterogeneous devices, and it is necessary to convert the standardized semantic content generated in the cloud into semantics that can be recognized by each smart home device.

[0069] This application proposes to invoke a Semantic Mapping Agent (SMA) to perform semantic mapping on a first task instruction to obtain a second task instruction, wherein the second task instruction is an instruction encapsulated based on a private protocol of smart home.

[0070] The core function of SMA is to solve the semantic confusion problem between heterogeneous devices from multiple sources. By integrating the Sentence-BERT semantic embedding model, the Graph Attention Network (GAT) dynamic reasoning mechanism, and the KL divergence semantic drift detection algorithm, SMA can automatically align the device-specific property model with the general intermediate semantic layer in the cloud.

[0071] Specifically, SMA first encodes multi-dimensional contextual information such as the original attribute name, data type, value range, device category, and application scenario of the device, generating a 128-dimensional high-precision semantic embedding vector to accurately capture the differentiated semantics of similar fields in different smart home devices. Subsequently, SMA constructs a dynamic and iterative device semantic knowledge graph, employing a two-layer stacked GAT architecture and combining the LeakyReLU activation function to dynamically calculate the attention coefficients aij between attributes. It adjusts the semantic association weights according to the scene context, achieving accurate semantic disambiguation in complex scenarios. For example, in the task of adjusting the air conditioner temperature, SMA can distinguish the meaning of "brightness" in the context of backlighting in an air conditioner through semantic embedding, avoiding mismapping it as brightness control for lighting devices. Finally, SMA can map standard task commands to native semantic commands (secondary task commands) that smart home devices can recognize, such as converting standard task commands into the Matter protocol's SetTemperatureCommand or the HomeKit protocol's SetTargetTemperature command.

[0072] In this embodiment of the application, the first task instruction includes multiple first task instructions, each of which may correspond to different types or brands of smart home devices. SMA can obtain the first task instruction and the original device attribute name, data type, value range, device category, application scenario, and other multi-dimensional context information of the smart home device corresponding to the first task instruction, and perform accurate semantic conversion on the first task instruction to generate a second task instruction that conforms to the native protocol specification of the target device.

[0073] This application embodiment uses SMA to perform precise semantic conversion on the first task instruction, generating a second task instruction that conforms to the native protocol specification of the target device. This achieves seamless semantic alignment across protocols and ecosystem devices, ensuring high accuracy and high reliability of the task instructions.

[0074] Step 204: Forward the second task instruction to the smart home via the home gateway so that the smart home can respond.

[0075] As a physical bridge connecting the cloud and smart homes, a home gateway performs functions such as command forwarding, local caching, protocol conversion, and self-regulation during network outages. Home gateways are typically deployed in a home network environment, possess strong local processing capabilities, and support multiple communication protocols (such as Wi-Fi, Zigbee, and Bluetooth Mesh). In this embodiment, the home gateway can receive a second task command from the cloud and, based on the target device address and protocol type specified in the second task command, perform localized forwarding of the command.

[0076] Furthermore, the home gateway also possesses the ability to self-regulate during network outages. When a network interruption or cloud connectivity failure is detected, the home gateway can automatically switch to local autonomous mode. Relying on locally cached high-frequency scene rules, device mapping relationships, and control policies, it can independently complete basic linkage control tasks, ensuring the normal operation of critical services such as home security, lighting, and device management. After the network is restored, the home gateway synchronizes the status of multiple smart home nodes through a vector clock mechanism, automatically repairs data conflicts, and achieves seamless network recovery.

[0077] For example, if a user enters "go home", the cloud will process the information and generate a second task instruction, which will be forwarded to the air conditioning unit via the home gateway. The air conditioning unit will then preheat or cool the air before the user arrives.

[0078] This solution achieves precise conversion from natural language to structured tasks by decomposing fuzzy or abstract instructions into multiple atomic operations of the smallest granularity. Then, based on these atomic operations, a first task instruction is generated and converted into native semantic instructions recognizable by smart homes through a semantic mapping agent. This solves the control failure problem caused by protocol incompatibility and semantic inconsistency between heterogeneous devices. Finally, a second task instruction is sent to the target smart home device through the home gateway, achieving seamless cross-ecosystem and cross-protocol linkage. This process proactively analyzes user information and provides proactive services to users. It eliminates the need for users to send task instructions to each specific smart home device, significantly improving the efficiency and accuracy of interconnection between multi-brand and multi-protocol devices. Furthermore, through cloud-based intelligent agent collaborative processing, it effectively avoids the single point of failure risk inherent in traditional centralized architectures.

[0079] Based on the above embodiments, please refer to Figure 3 , Figure 3 This is a flowchart illustrating a method for remotely controlling a smart home via vehicle-cloud collaboration, as provided in an embodiment of this application. Figure 2 The method includes: Step 301: Receive user input information collected by the vehicle terminal; invoke the scene planning agent to deconstruct the home scene based on the user input information to obtain at least one atomic operation.

[0080] The user inputs the natural language command "Drive home, activate home comfort mode in advance." After receiving the user's input, the SPA uses a lightweight Llama-3-8B LLM model fine-tuned for home scenarios to deconstruct the ambiguous user input, remove invalid semantics, determine the home comfort scenario, and then break it down into atomic operations such as "vehicle entry and location recognition, turning on entrance lights, adjusting air conditioning temperature, opening curtains, and turning off security alarms." Based on these atomic operations, a structured task graph is generated. Nodes in the structured task graph represent atomic operations, and edges represent dependencies, parallelism, or mutual exclusion relationships between tasks. The timing constraint of "disabling security alarms before turning on devices" is clearly defined.

[0081] Step 302: Determine the equipment capability requirements based on at least one atomic operation.

[0082] Among them, equipment capability requirements indicate the equipment capabilities required for atomic operations.

[0083] In this embodiment, when there is only one atomic operation, the device capability requirement is determined based on that atomic operation. The device capability requirement is simply the device capability required for that atomic operation.

[0084] When there are multiple atomic operations, the equipment capability requirements can be determined for each atomic operation separately, and then the equipment capability requirements of multiple atomic operations can be combined to obtain the final equipment capability requirements.

[0085] Equipment capability requirements refer to the functional, performance, or state conditions of the target equipment required to complete a specific atomic operation. Equipment capability requirements typically include multi-dimensional information such as equipment type, supported operation instruction sets, response latency requirements, energy consumption limits, and security access levels. For example, if the atomic operation is to adjust the air conditioner temperature to 26°C, the corresponding equipment capability requirements might include: supporting temperature adjustment functionality, having a remote control interface, being currently powered on, and having authorized user access permissions.

[0086] In this embodiment, the Spatial Utility Model (SPA) can extract the implicit device attributes and behavioral constraints of atomic operations by parsing their specific content, thereby generating a structured description of device capability requirements. For example, the SPA can automatically derive the device capability requirements description for each atomic operation after deconstructing the user intent based on a lightweight LLM model, and output it in a standardized format.

[0087] This application embodiment dynamically generates the device attributes and behavioral constraints corresponding to each atomic operation, as well as the device capability requirements, based on each atomic operation. This allows for precise identification of the device characteristics required to execute at least one atomic operation, thereby avoiding control failures due to device mismatch and improving the accuracy and reliability of the system's instruction generation.

[0088] Step 303: Based on the device capability requirements, obtain the device capability status of the smart home through the home gateway.

[0089] In the embodiments of this application, device capability requirements refer to the description of the capabilities that a smart home needs to possess in order to complete atomic operations.

[0090] In one implementation, when the SPA receives a device capability requirement, the cloud, triggered by this requirement, can generate a device capability request and send it to the home gateway. A device capability request is a standardized request generated by the SPA based on the current scenario plan or user instructions, used to query the functional attributes and operating status of specific devices. It instructs the SPA to retrieve the device capability status of all smart home devices associated with the home gateway.

[0091] After receiving a device capability request from the cloud, the home gateway interacts with each smart home device in the home environment to obtain the current device capability status of each smart home device. Specifically, the home gateway can establish connections with local smart home devices via communication protocols such as Wi-Fi, Zigbee, and Bluetooth Mesh, monitor the operating status of smart home devices in real time, and collect operational capability data from smart home devices periodically or as needed to determine the current device capability status.

[0092] In this embodiment, each smart home device can return its operational capability data based on its firmware and configuration. The home gateway integrates, cleans, and formats this scattered operational capability data to form a unified reporting data packet. Then, the home gateway can report the device capability status of each smart home device to the cloud. In this way, the cloud can obtain the device capability status of all online smart homes associated with the home gateway.

[0093] The device capability status can include whether the smart home device is online, whether it supports remote control, whether it has specific functional modules (such as voice control, timed on / off, scene linkage, etc.), the current working mode (such as energy-saving mode, standard mode), and the value range of various parameters (such as temperature adjustment range, brightness adjustment level, etc.).

[0094] In another implementation, there are multiple smart home devices. After obtaining the device capability requirements, the SPA can determine the first smart home device based on these requirements. The first smart home device is the one associated with the home gateway and related to the atomic operation. Then, the cloud can generate a device capability request, which carries the target device identifier; the target device identifier is the identifier of the first smart home device.

[0095] In this embodiment of the application, the device capability request may also carry key parameters such as the list of attributes that each first smart home needs to query (such as whether it supports remote control, whether it has temperature control function, etc.), query priority, response time requirements, etc.

[0096] After receiving a device capability request from the cloud, the home gateway interacts with the first smart home device corresponding to the target device identifier to obtain the current device capability status of each first smart home device. The home gateway then reports the device capability status of each first smart home device to the cloud. In this way, the cloud can obtain the device capability status of the smart home device corresponding to the target device identifier.

[0097] In this embodiment, the device capability request is generated by the SPA, which can broadcast the request based on a shared blackboard mechanism. The cloud can invoke a semantic mapping agent to convert the device capability request into a data format recognizable by the home gateway based on the semantic mapping relationship of the object model established by the agent, and then send it to the home gateway.

[0098] Step 304: Generate a first task instruction based on at least one atomic operation and the device capability status of the smart home.

[0099] In this embodiment, the generation of the first task instruction depends on two key inputs: the semantic content of the atomic operation and the device capability status of the smart home obtained through the home gateway. The cloud can determine whether the smart home has the capability to execute the operation corresponding to the semantic definition of the atomic operation. If it does, a specific task instruction is generated; if it does not, an exception handling mechanism is triggered, such as selecting an alternative smart home device or adjusting the operation strategy.

[0100] In another implementation, the cloud can determine whether smart home devices need adjustment based on their device capabilities and atomic operations. For example, an atomic operation might be adjusting the air conditioner temperature to 26°C. If the device capability status obtained from the home gateway already shows the air conditioner at 26°C, then the air conditioner has fulfilled the atomic operation requirement and therefore no adjustment is needed. In this case, the atomic operation that doesn't require adjustment and the corresponding smart home device can be removed. The cloud can then generate task instructions based on the removed atomic operations and device capability status.

[0101] For example, for the atomic operation of closing curtains, the cloud can first confirm that the smart home (curtains) supports the closing command and is currently online, and then generate the closing operation command (i.e., the first task command).

[0102] The first task instruction is generated by combining atomic operations with the current capability status of the smart home. This ensures that the generated first task instruction matches the actual functional capabilities of the smart home, thereby avoiding the issuance of invalid or erroneous instructions.

[0103] Step 305: Invoke the semantic mapping agent to perform semantic mapping on the first task instruction to obtain the second task instruction, and forward the second task instruction to the smart home through the home gateway so that the smart home can respond.

[0104] In this embodiment of the application, after receiving the first task instruction, the cloud can call SMA to convert the first task instruction into a second task instruction to facilitate smart home recognition.

[0105] This application embodiment generates a first task instruction by combining atomic operations with the current capability status of the smart home. This ensures that the generated first task instruction matches the actual functional capabilities of the smart home, thereby avoiding the issuance of invalid or erroneous instructions. Based on this, the first task instruction is converted into a second task instruction for easy smart home recognition, achieving seamless semantic alignment across protocols and ecosystem devices, and guaranteeing high accuracy and reliability of the task instructions.

[0106] Based on the above embodiments, please refer to Figure 4 , Figure 4 This is a flowchart illustrating a method for remotely controlling a smart home via vehicle-cloud collaboration, as provided in an embodiment of this application. Figure 3 The method includes: Step 401: Receive user input information collected by the vehicle terminal; invoke the scene planning agent to deconstruct the home scene based on the user input information to obtain at least one atomic operation.

[0107] Step 402: Determine at least one target atomic operation from at least one atomic operation based on the device capability status of the smart home.

[0108] The target atomic operation is the atomic operation that needs to be performed.

[0109] In this embodiment, the atomic operations determined based on the home scene deconstruction are determined by the cloud based on user input information and are not matched with actual smart home devices. Therefore, there may be cases where the smart home corresponding to the atomic operation does not exist. For example, the atomic operation is to close the curtains, but the smart home devices associated with the home gateway do not include smart curtains. Therefore, this atomic operation cannot be executed. Another example is that the atomic operation is to turn on the air conditioner, but the air conditioner in the smart home is already on; in this case, the atomic operation does not need to be executed. For instance, if the air conditioner is offline due to a malfunction, the atomic operation of turning on the air conditioner will be excluded.

[0110] In this embodiment of the application, atomic operations can be filtered according to the device capability status of the smart home. When filtering, it is necessary to consider multiple dimensions of information such as the current status of the smart home (e.g., whether it is online or in standby mode), protocol compatibility, physical limitations (e.g., power limit) and security policy constraints. Atomic operations that do not meet the execution conditions or do not need to be executed are eliminated. The atomic operations retained after filtering are defined as target atomic operations. Target atomic operations are the basic units that constitute the final task instruction.

[0111] In this embodiment, the Smart Home Automation (SMA) can assess the feasibility of each atomic operation based on the device capability status of the smart home and the atomic operation list parsed by the Smart Home Application Spectroscopy (SPA) (the atomic operation list includes at least one of the aforementioned atomic operations). The assessment dimensions include device online status, resource usage, permission authorization status, and historical execution success rate. Only atomic operations that meet all constraints will be selected as target atomic operations. For example, in a vehicle-home interconnection scenario, when a vehicle enters a residential area, if the home gateway is detected to be connected normally, the lighting equipment is online and not occupied by other scenarios, then turning on the entrance lights is determined as the target atomic operation; however, if the air conditioner is offline at this time, the atomic operation of adjusting the air conditioner temperature will not be included in the target atomic operations.

[0112] By filtering atomic operations, only operations that can be actually executed in the current environment can be selected, thereby avoiding equipment misoperation or resource waste caused by invalid command issuance, and thus improving the reliability and stability of the control process.

[0113] Step 403: Generate a structured task graph containing task dependencies based on at least one target atomic operation.

[0114] In the embodiments of this application, the structured task graph is a directed graph model used to represent the logical relationship of multi-device collaborative control tasks, where nodes represent atomic operations and edges represent the dependencies between tasks, such as sequential dependencies, parallel execution, mutual exclusion, etc.

[0115] The structured task graph is automatically generated by the scene planning agent after parsing the user's natural language intent using a lightweight domain-specific large model (a finely tuned version of Llama-3-8B). The structured task graph includes multiple subtasks. Each subtask is the smallest independently executable unit with contextual relationships. Each subtask can contain metadata such as task ID, target device, operation type, parameter values, a list of dependent prerequisite tasks, and conflict checking rules. Subtasks are precisely scheduled through task dependencies.

[0116] For example, if a user inputs that they want to turn on the air conditioner and set it to 26°C before going home, the remote system can break this down into two sub-tasks: starting the air conditioner and setting the temperature to 26°C. A sequential dependency relationship can be established—the power must be started before the temperature adjustment can be performed. Correspondingly, two task instructions can be generated: one is the task instruction to start the air conditioner and the other is the task instruction to adjust the temperature to 26°C.

[0117] For example, turning off the security system must be done before turning on the lights to avoid security vulnerabilities; opening the curtains and turning on the air conditioner can be done in parallel to shorten the overall response time.

[0118] This application embodiment generates a structured task graph, which transforms complex cross-device linkage requirements into a clear and computable task logic structure, thereby achieving efficient and orderly multi-task collaborative execution and reducing task scheduling complexity.

[0119] In this embodiment, before generating a structured task graph based on the target atomic operations, conflict detection is required for multiple target atomic operations. Conflict detection refers to determining whether there are potential or already occurring conflict behaviors by analyzing the relationships between target atomic operations, such as resource consumption, timing logic, and functional mutual exclusion. For example, when home security arming and a high-temperature warning are triggered simultaneously, conflict detection can identify a conflict between these two air conditioning-related task instructions.

[0120] In this embodiment of the application, a conflict arbitration intelligent agent is also deployed in the cloud. The conflict arbitration intelligent agent is a dedicated intelligent agent that is specifically used to solve the problems of device resource competition and state conflict caused by concurrent triggering of rules in multiple scenarios and parallel execution of multiple tasks.

[0121] After obtaining the target atomic operation, the cloud can acquire information about the corresponding task to be executed, including task identifier, target device, operation type, execution time, and required resources. Then, it invokes a conflict arbitration agent to construct a conflict detection graph, where nodes represent task instances and edges represent conflict relationships. A graph traversal algorithm quickly locates sets of conflicting tasks, providing a basis for subsequent adjudication. Conflict relationships include, for example, device resource contention, task sequence reversal, and functional mutual exclusion.

[0122] In this embodiment of the application, when a conflicting set of tasks is detected, conflict arbitration is performed on at least two target atomic operations included in the task set.

[0123] In one implementation, after a task conflict is detected, the task execution sequence is replanned by adjusting the task execution order, modifying task parameters, merging some operations, etc., so that the task execution sequence can be coordinated and executed without violating system constraints.

[0124] For example, when there is a conflict between turning on the air conditioner for cooling and turning it off in energy-saving mode, a task orchestration strategy can be used to retain high-priority tasks (such as turning on the air conditioner in home mode) and postpone or downgrade the execution of low-priority tasks (such as energy-saving mode) (such as raising the air conditioner temperature setting to 26°C instead of 24°C), thereby resolving the conflict.

[0125] In another implementation, upon detecting a task conflict, the execution of one or more tasks is directly canceled or terminated. This is typically for low-priority, non-critical, or deferable tasks. For example, in an emergency, if remote unlocking and whole-house arming are triggered simultaneously, the arming task may be automatically removed to prioritize the user's security needs.

[0126] In some embodiments, task removal requires a secondary review by a security audit agent to ensure that no new security risks are introduced.

[0127] In another implementation, after a task conflict is detected, the tasks are rescheduled. If the conflict cannot be resolved through orchestration, the execution of one or more tasks is canceled or terminated.

[0128] In this embodiment of the application, when orchestrating or removing target atomic operations, a comprehensive evaluation can be performed from multiple dimensions, including priority, device security level, task time urgency, historical execution frequency of tasks, resource consumption, permission authorization status, and historical execution success rate.

[0129] In this embodiment, the cloud can invoke a conflict arbitration agent to identify conflict relationships between target atomic operations and use a multi-dimensional comprehensive priority adjudication model to calculate task priorities in order to achieve accurate adjudication.

[0130] Step 404: Generate the corresponding first task instruction based on each subtask.

[0131] The first task instruction is used to regulate the smart home corresponding to at least one target atomic operation.

[0132] Step 405: Invoke the semantic mapping agent to perform semantic mapping on the first task instruction to obtain the second task instruction, and forward the second task instruction to the smart home through the home gateway so that the smart home can respond.

[0133] This application embodiment filters atomic operations to determine target atomic operations, generates a structured task graph based on the target atomic operations to clarify the interdependencies between the target atomic operations, and generates a standardized first task instruction based on this. Through this method, dynamic perception and precise matching of the device capability status of smart home devices can be achieved, thereby optimizing the dependency and scheduling logic between multiple tasks, and ultimately improving the automation level, response speed, and execution reliability of the smart home system in complex scenarios.

[0134] Based on the above embodiments, please refer to Figure 5 , Figure 5 This is a flowchart illustrating a method for remotely controlling a smart home via vehicle-cloud collaboration, as provided in an embodiment of this application. Figure 4 The method includes: Step 501: Receive user input information collected by the vehicle terminal; invoke the scene planning agent to deconstruct the home scene based on the user input information to obtain at least one atomic operation.

[0135] Step 502: Generate the first task instruction based on at least one atomic operation.

[0136] In this embodiment of the application, at least one atomic operation can be screened to obtain a target atomic operation, and then a first task instruction can be generated based on the target atomic operation.

[0137] Step 503: Obtain the semantic mapping relationship.

[0138] The semantic mapping relationship includes the correspondence between the private protocols and standard protocols of smart homes on different platforms.

[0139] Semantic mapping refers to a standardized set of mapping rules established between heterogeneous devices from multiple sources. It addresses semantic inconsistencies caused by proprietary communication protocols used by devices from different platforms and brands. Semantic mapping encompasses not only protocol-level conversion rules but also semantic alignment at the object model level, including device attribute names, data types, and value ranges. For example, a temperature setting attribute of one brand of air conditioner might be referred to as a target temperature control value in another brand's device. Semantic mapping maps these two attributes to a unified intermediate semantic identifier, ensuring the cloud can correctly understand and execute cross-device operations.

[0140] In this embodiment of the application, the process of obtaining semantic mapping relationships includes: A protocol conversion agent is deployed in the cloud, which receives access requests from the home gateway. The cloud then invokes the protocol conversion agent to construct a semantic mapping relationship based on the access information of the home gateway and the smart home devices associated with it.

[0141] In one implementation, a cross-protocol device access layer is built in the cloud. This layer is natively compatible with mainstream IoT communication protocols such as MQTT, CoAP, HTTP, Zigbee, and Bluetooth Mesh. It also includes lightweight and scalable Matter, OCF, and HomeKit protocol bridging and adaptation plugins, eliminating the need for manually developed custom adaptation tools and automatically recognizing the communication message formats and command rules of various smart home devices. The cloud can receive raw object model data, device status data, and fault alarm data reported by various smart home devices in real time based on the cross-protocol device access layer. After data cleaning, deduplication, and format preprocessing, the data is standardized and broadcast to various smart agents (such as semantic mapping agents or protocol conversion agents).

[0142] The access request carries the access information of the home gateway and the smart home devices associated with the home gateway. The access information includes at least one of the following: device private property model parameters, attribute definitions, data value parameters, and device protocol specifications. The semantic mapping relationship includes the correspondence between the private protocols and standard protocols of smart home devices on different platforms.

[0143] When a user connects a smart air conditioner to the cloud for the first time via a home gateway, the gateway automatically collects information such as the smart air conditioner's device model, supported task command set, and attribute definitions (e.g., cooling mode, fan speed). This information is then packaged into an access request and uploaded to the cloud. The cloud can then invoke a semantic mapping agent to determine the smart air conditioner's category and protocol based on the access information. This process initiates subsequent semantic alignment, generating semantic mapping relationships during the alignment process.

[0144] In this process, the cloud can invoke a semantic mapping agent to map the private attributes and operation commands of smart home devices from different platforms and brands to a one-to-one or multiple mappings with the general intermediate semantic layer. This mapping includes not only the conversion of field names (e.g., power_on → turn on), but also deep alignment at the semantic level (e.g., distinguishing between lighting brightness and screen backlight brightness), ensuring that the semantics corresponding to the general intermediate semantic layer and the smart home devices are accurately aligned.

[0145] For example, for two different brands of lights, one brand uses brightness_level as the brightness parameter and the other brand uses light_intensity as the brightness parameter. Through semantic mapping, they are uniformly mapped to standard semantic brightness and automatically converted to the corresponding private protocol instructions during execution.

[0146] Step 504: Invoke the semantic mapping agent to obtain the target smart home corresponding to the first task instruction.

[0147] In this embodiment, the cloud can invoke a semantic mapping agent to parse the first task instruction, determine the target smart home corresponding to the first task instruction, and obtain the communication protocol type of the target smart home and its corresponding private instruction format specification. Then, based on the communication protocol type of the target smart home and its corresponding private instruction format specification, the cloud can obtain the mapping entry corresponding to the target smart home from the semantic mapping relationship. The mapping entry includes the conversion relationship between general semantics and the device private protocol instructions of the target smart home.

[0148] Based on the first task instruction and the mapping entry corresponding to the target smart home, the first task instruction is converted into the target smart home's private protocol instruction (second task instruction).

[0149] Step 505: Convert the first task instruction into a second task instruction that conforms to the private protocol of the target smart home according to the semantic mapping relationship, and forward the second task instruction to the smart home through the home gateway so that the smart home can respond.

[0150] This application's embodiments achieve precise mapping from general semantic commands to the target smart home's private protocol commands by acquiring semantic mapping relationships step by step, invoking a semantic mapping intelligent agent to identify the target smart home device, and completing protocol conversion based on the semantic mapping relationships. This enables lossless semantic alignment of smart home devices across different platforms and protocols.

[0151] It should be understood that the steps in the aforementioned accompanying drawings are not necessarily performed in the order indicated in the drawings. Unless otherwise expressly stated herein, there is no strict order restriction on the execution of these steps, and they may be performed in other orders. Moreover, at least some of the steps in these drawings may include multiple sub-steps or multiple stages, which are not necessarily completed at the same time, but may be executed at different times, and the execution order of these sub-steps or stages is not necessarily sequential, but may be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0152] In another embodiment of this application, a device for remotely controlling smart homes via vehicle-cloud collaboration is provided; please refer to [reference needed]. Figure 6 , Figure 6 This is a logic block diagram of a device for remotely controlling a smart home via vehicle-to-cloud (V2X) according to an embodiment of this application. The V2X remote control device 600 may include: a receiving module 601 and a processing module 602, wherein: The receiving module 601 is used to receive user input information collected by the vehicle terminal; The processing module 602 is used to call the scene planning agent to deconstruct the home scene based on the user input information and obtain at least one atomic operation, which represents the minimum control action in the home scene; generate a first task instruction based on the at least one atomic operation; call the semantic mapping agent to perform semantic mapping on the first task instruction to obtain a second task instruction, which is used to control the smart home; the second task instruction is a native semantic instruction that the smart home can recognize; and forward the second task instruction to the smart home through the home gateway for the smart home to respond.

[0153] In one embodiment, the processing module 602 is specifically used for: The equipment capability requirement is determined based on at least one atomic operation, and the equipment capability requirement indicates the equipment capability required by the atomic operation. Based on device capability requirements, obtain the device capability status of smart home devices through the home gateway; The first task instruction is generated based on at least one atomic operation and the device capability status of the smart home.

[0154] In one embodiment, there are multiple smart home devices, and the processing module 602 is specifically used for: Based on the device capability requirements, send a device capability request to the home gateway; Receive the device capability status of each smart home device corresponding to the home gateway, as reported by the home gateway.

[0155] In one embodiment, there are multiple smart home devices, and the processing module 602 is specifically used for: Based on the device capability requirements, a device capability request is sent to the home gateway, and the device capability request carries the target device identifier. Receive the device capability status of the smart home corresponding to the target device identifier reported by the home gateway.

[0156] In one embodiment, the processing module 602 is specifically used for: Based on the device capability status of the smart home, determine at least one target atomic operation from at least one atomic operation, where the target atomic operation is the atomic operation to be executed; A structured task graph containing task dependencies is generated based on at least one target atomic operation. The structured task graph includes multiple subtasks. A first task instruction is generated for each subtask, and the first task instruction is used to control the smart home corresponding to at least one target atomic operation.

[0157] In one embodiment, the processing module 602 is specifically used for: Obtain semantic mapping relationships, which include the correspondence between proprietary protocols and standard protocols of smart home systems on different platforms; Invoke the semantic mapping agent to obtain the target smart home corresponding to the first task instruction; Based on semantic mapping, the first task instruction is converted into a second task instruction that conforms to the private protocol of the target smart home.

[0158] In one embodiment, the cloud is also configured with a protocol conversion agent, and the processing module 602 is specifically used for: Receive an access request sent by the home gateway. The access request carries access information of the home gateway and the smart home devices associated with the home gateway. The access information includes at least one of the following: device private property model parameters, attribute definitions, data value parameters, and device protocol specifications. The protocol conversion agent constructs a semantic mapping relationship based on the home gateway and the access information of the smart home devices associated with the home gateway. The semantic mapping relationship includes the correspondence between the private protocols and standard protocols of smart home devices on different platforms.

[0159] In one embodiment, the cloud is also configured with a conflict arbitration intelligent agent, and the processing module 602 is specifically used for: Invoke the conflict arbitration agent to perform conflict detection on at least one target atomic operation; For at least two conflicting target atomic operations, perform task orchestration or task elimination.

[0160] The various modules in the aforementioned device for remotely controlling smart homes via vehicle-cloud collaboration can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of the cloud server in hardware form or independent of it, or stored in the memory of the cloud server in software form, so that the processor can call and execute the corresponding operations of each module.

[0161] In yet another embodiment of this application, please refer to Figure 7 , Figure 7 This is a schematic diagram of the hardware structure of a cloud server provided in an embodiment of this application. The cloud server may include a communication interface 701, a memory 702, and a processor 703; the various components are coupled together through a bus system 704. It is understood that the bus system 704 is used to realize the connection and communication between these components. In addition to a data bus, the bus system 704 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 7 The general designated all buses as Bus System 704.

[0162] In this embodiment, the communication interface 701 is used to send and receive information with other external devices; the memory 702 is used to store a computer program that can run on the processor 703; the processor 703 is used to execute the steps of the method for remotely controlling a smart home via vehicle-cloud connection as described in any of the foregoing embodiments when running the computer program.

[0163] It is understood that the memory 702 in this embodiment can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate Synchronous DRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memory 702 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0164] The processor 703 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 703 or by software instructions. The processor 703 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory 702, and the processor 703 reads the information in memory 702 and, in conjunction with its hardware, completes the steps of the above method.

[0165] It is also understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described herein, or combinations thereof.

[0166] For software implementation, the techniques described herein can be implemented through modules (e.g., procedures, functions, etc.) that perform the functions described herein. Software code can be stored in memory and executed by a processor. The memory can be implemented in the processor or externally. Wherein, if implemented as a software functional module and not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0167] In another embodiment of this application, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the method for remotely controlling a smart home via vehicle-cloud collaboration as described in the foregoing embodiments.

[0168] In another embodiment of this application, a computer program product is also provided, including a computer program or instructions that, when executed by a processor, implement the steps of the method for remotely controlling a smart home via vehicle-cloud collaboration as described in the foregoing embodiments.

[0169] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, devices, or computer program products. Therefore, this application can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage and optical storage) containing computer-usable program code.

[0170] It should be noted that, in this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0171] The sequence numbers of the embodiments in this application are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. The features disclosed in the several product embodiments provided in this application can be arbitrarily combined to obtain new product embodiments without conflict. Similarly, the features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined to obtain new method or device embodiments without conflict. The above descriptions are merely specific implementations of this application, but the protection scope of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the protection scope of this application.

Claims

1. A method for remotely controlling smart homes via vehicle-cloud collaboration, characterized in that, The cloud-based system is applied to a vehicle-cloud integrated system, which also includes vehicle terminals, home gateways, and smart home devices. The cloud is configured with scene planning agents and semantic mapping agents. The method includes: Receive user input information collected by the vehicle terminal; The scene planning agent is invoked to deconstruct the home scene based on the user input information, and at least one atomic operation is obtained. The atomic operation represents the minimum control action in the home scene. A first task instruction is generated based on the at least one atomic operation, and the semantic mapping agent is invoked to perform semantic mapping on the first task instruction to obtain a second task instruction. The first task instruction is used to control the smart home. The second task instruction is a native semantic instruction that the smart home can recognize. The second task instruction is forwarded to the smart home device through the home gateway for the smart home device to respond to.

2. The method according to claim 1, characterized in that, The step of generating the first task instruction based on the at least one atomic operation includes: A device capability requirement is determined based on the at least one atomic operation, the device capability requirement indicating the device capability required by the atomic operation; Based on the device capability requirements, the device capability status of the smart home is obtained through the home gateway; The first task instruction is generated based on the at least one atomic operation and the device capability status of the smart home.

3. The method according to claim 2, characterized in that, The number of smart home devices is multiple, and the step of obtaining the device capability status of the smart home devices through the home gateway according to the device capability requirements includes: Based on the device capability requirements, a device capability request is sent to the home gateway; Receive the device capability status of each smart home device corresponding to the home gateway, as reported by the home gateway.

4. The method according to claim 2, characterized in that, The number of smart home devices is multiple, and the step of obtaining the device capability status of the smart home devices through the home gateway according to the device capability requirements includes: Based on the device capability requirements, a device capability request is sent to the home gateway, and the device capability request carries the target device identifier. Receive the device capability status of the smart home corresponding to the target device identifier reported by the home gateway.

5. The method according to claim 2, characterized in that, The step of generating the first task instruction based on the at least one atomic operation and the device capability status of the smart home includes: Based on the device capability status of the smart home, at least one target atomic operation is determined from the at least one atomic operation, wherein the target atomic operation is the atomic operation to be executed; A structured task graph containing task dependencies is generated based on the at least one target atomic operation, and the structured task graph includes multiple subtasks; The first task instruction is generated according to each of the sub-tasks, and the first task instruction is used to control the smart home corresponding to the at least one target atomic operation.

6. The method according to claim 1, characterized in that, The step of invoking the semantic mapping agent to perform semantic mapping on the first task instruction to obtain the second task instruction includes: Obtain semantic mapping relationships, which include the correspondence between private protocols and standard protocols of smart homes on different platforms; Invoke the semantic mapping agent to obtain the target smart home corresponding to the first task instruction; Based on the semantic mapping relationship, the first task instruction is converted into a second task instruction that conforms to the private protocol of the target smart home.

7. The method according to claim 6, characterized in that, The cloud platform is also configured with a protocol conversion agent, and the acquisition of semantic mapping relationships includes: The system receives an access request sent by the home gateway. The access request carries access information of the home gateway and the smart home devices associated with the home gateway. The access information includes at least one of the following: device private property model parameters, attribute definitions, data value parameters, and device protocol specifications. The protocol conversion agent is invoked to construct a semantic mapping relationship based on the home gateway and the access information of the smart home associated with the home gateway. The semantic mapping relationship includes the correspondence between the private protocols and standard protocols of smart homes on different platforms.

8. The method according to claim 5, characterized in that, The cloud platform is also configured with a conflict arbitration agent. Before generating a structured task graph containing task dependencies based on the at least one target atomic operation, the method further includes: The conflict arbitration agent is invoked to perform conflict detection on the at least one target atomic operation. For at least two conflicting target atomic operations, perform task orchestration or task elimination.

9. A device for remotely controlling smart homes via vehicle-cloud collaboration, characterized in that, The cloud-based system is used in a vehicle-cloud integrated system, which also includes a vehicle, a home gateway, and a smart home device. The cloud is configured with a scene planning agent and a semantic mapping agent. The device includes: The receiving module is used to receive user input information collected by the vehicle terminal; The processing module is used to invoke the scene planning agent to deconstruct the home scene based on the user input information, obtaining at least one atomic operation, wherein the atomic operation represents the minimum control action in the home scene; generate a first task instruction based on the at least one atomic operation; invoke the semantic mapping agent to perform semantic mapping on the first task instruction to obtain a second task instruction, wherein the first task instruction is used to regulate the smart home; the second task instruction is a native semantic instruction that the smart home can recognize; and forward the second task instruction to the smart home through the home gateway for the smart home to respond to.

10. A cloud server comprising a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the method for remotely controlling a smart home via vehicle-cloud collaboration as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program or computer-executable instructions, which, when executed by a processor, implement the steps of the method for remotely controlling a smart home via vehicle-cloud collaboration as described in any one of claims 1 to 8.

Images