Smart home control method, device, equipment, medium and product

Abstract

The invention discloses a smart home control method, device and equipment, a medium and a product. The method comprises the following steps: acquiring multi-modal sensing data, and converting the multi-modal sensing data into a discrete feature descriptor sequence; constructing first prompt information according to the situation state definition information and the discrete feature descriptor sequence, and guiding a micro language model to generate a current situation state based on the first prompt information; and generating a control instruction sequence corresponding to the controlled household equipment according to the current situation state, and controlling the controlled household equipment to execute corresponding operation according to the control instruction sequence. By introducing a hierarchical context awareness mechanism, conversion from an original perception signal to a discrete semantic state is completed by using low-power-consumption signal processing and a miniature semantic model, and accurate recognition of a complex life situation under a low-computing-power condition is realized by using a zero-sample logical reasoning capability of the miniature semantic model. And the energy efficiency ratio and the response speed of edge calculation are improved.

Description

Technical Field

[0001] This invention relates to the field of smart home control technology, and in particular to a smart home control method, device, equipment, medium and product. Background Technology

[0002] With the deep integration of the Internet of Things (IoT) and artificial intelligence (AI) technologies, smart home systems are gradually evolving from passive remote control to proactive intelligent services. Large Language Models (LLMs), due to their superior intent understanding and reasoning capabilities, are being widely applied to the scheduling and control of home devices at the edge.

[0003] Deploying general-purpose large models directly in resource-constrained edge environments and applying them to complex physical control scenarios faces significant technical bottlenecks and challenges: Existing technologies typically convert sensor-collected data streams (such as light intensity streams, continuous temperature and humidity values, and pulse signals from human presence sensors) into text descriptions before inputting them into large models. However, sensor data from home environments is multimodal, high-frequency, unstructured, and accompanied by noise. While general-purpose large models excel at processing high-semantic-density text information, they struggle to efficiently extract high-level contextual semantics, such as "the user is reading" or "preparing to sleep," directly from high-frequency, abruptly changing raw numerical streams. Directly inputting raw data into the model not only rapidly consumes the extremely limited context window resources of edge devices, leading to significant inference delays, but also easily causes data noise to interfere with model judgments, resulting in decision illusions or misjudgments.

[0004] Therefore, existing technologies that directly input raw sensor data into large models lead to a waste of computing resources and high inference latency. Summary of the Invention

[0005] This invention provides a smart home control method, device, equipment, medium, and product to solve the problem of existing technologies that directly input raw sensor data into large models, resulting in wasted computing resources and high inference latency.

[0006] In a first aspect, embodiments of the present invention provide a smart home control method, comprising: Acquire multimodal sensing data and convert the multimodal sensing data into a sequence of discrete feature descriptors; A first prompt message is constructed based on the context state definition information and the discrete feature descriptor sequence, and the micro language model is guided to generate the current context state based on the first prompt message. Based on the current situation state, a control instruction sequence corresponding to the controlled home device is generated, and the controlled home device is controlled to perform the corresponding operation according to the control instruction sequence.

[0007] Secondly, embodiments of the present invention provide a smart home control device, comprising: The perception data acquisition module is used to acquire multimodal perception data and convert the multimodal perception data into a sequence of discrete feature descriptors; The context state determination module is used to construct first prompt information based on the context state definition information and the discrete feature descriptor sequence, and guide the micro language model to generate the current context state based on the first prompt information; The control module is used to generate a sequence of control instructions for the controlled home appliances based on the current situation and context information, and to control the controlled home appliances to perform corresponding operations based on the sequence of control instructions.

[0008] Thirdly, embodiments of the present invention provide an electronic device, the electronic device comprising: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the smart home control method according to any embodiment of the present invention.

[0009] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing computer instructions, which are used to cause a processor to execute and implement the smart home control method described in any embodiment of the present invention.

[0010] Fifthly, embodiments of the present invention provide a computer program product including a computer program, which, when executed by a processor, implements the smart home control method described in any embodiment of the present invention.

[0011] The technical solution of this invention acquires multimodal sensing data and converts it into a sequence of discrete feature descriptors. Based on the contextual state definition information and the discrete feature descriptor sequence, a first prompt is constructed. This first prompt guides a micro-language model to generate the current contextual state. A control command sequence corresponding to the controlled home appliance is generated based on the current contextual state, and the controlled home appliance is controlled to perform the corresponding operation based on the control command sequence. By introducing a hierarchical contextual awareness mechanism, low-power signal processing and a micro-semantic model are used to complete the conversion from raw sensing signals to discrete semantic states. This solves the "semantic gap" between raw multimodal data and high-level semantic decision-making, addressing the problem of wasted computing resources and high inference latency caused by inputting raw sensor data into a large model. Furthermore, by utilizing the zero-shot logical reasoning capability of the micro-language model, accurate identification of complex life situations is achieved under low computing power conditions, improving the energy efficiency and response speed of edge computing.

[0012] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a flowchart of a smart home control method provided in Embodiment 1 of the present invention; Figure 2 This is a flowchart of a smart home control method provided in Embodiment 2 of the present invention; Figure 3 This is a schematic diagram of the structure of a smart home control system provided in this embodiment; Figure 4 This is a schematic diagram of the structure of a smart home control device provided in Embodiment 3 of the present invention; Figure 5 A schematic diagram of the structure of an electronic device for implementing the smart home control method of this invention. Detailed Implementation

[0015] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0016] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0017] It is understood that before using the technical methods disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0018] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose, based on the prompt message, whether to provide personal information to the software or hardware such as the electronic device, application, server, or storage medium performing the operations of this disclosed technology.

[0019] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0020] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0021] Example 1 Figure 1This is a flowchart of a smart home control method provided in Embodiment 1 of the present invention. This embodiment is applicable to the automatic control of smart homes based on multimodal sensing data. The method can be executed by a smart home control device, which can be implemented in hardware and / or software and can be configured in an electronic device.

[0022] like Figure 1 As shown, the method includes: S110. Acquire multimodal sensing data and convert the multimodal sensing data into a sequence of discrete feature descriptors.

[0023] Multimodal sensing data can be considered as a collection of data acquired through different types of sensors, possessing multiple forms of representation or information sources, and capable of describing the same target or environment from complementary perspectives. Types of multimodal sensing data can include images, audio, text, and electrical signal data. For example, multimodal sensing data can include decibel values, temperature, and brightness. A discrete feature descriptor sequence can be considered as a sequence of discrete feature descriptors (token-like descriptors), enabling a structured and compact representation of the original data.

[0024] In this embodiment, raw multimodal sensing data can be collected by multiple different types of sensors. The raw multimodal sensing data is preprocessed using signal processing techniques, such as noise reduction, filtering, and cleaning. Then, the preprocessed multimodal sensing data is converted into a sequence of discrete feature descriptors.

[0025] For example, converting multimodal sensing data into a sequence of discrete feature descriptors can be achieved by mapping the multimodal sensing data into a sequence of discrete feature descriptors based on a threshold mapping table. Specifically, the mapping logic corresponding to each type of sensor can be determined in the threshold mapping table, and this mapping logic maps multimodal sensing data with different numerical ranges into feature descriptors with semantic information.

[0026] Alternatively, a sequence of discrete feature descriptors can be obtained by performing logical reasoning on multimodal sensing data based on a fuzzy logic controller. Specifically, this can involve defining fuzzy variables (containing semantic description labels corresponding to each type of sensing data), designing membership functions, and establishing a fuzzy rule base. For each sensing data, the membership degree of it belonging to all relevant fuzzy sets is calculated, and logical reasoning and aggregation are performed using the fuzzy rule base to convert the comprehensive fuzzy set into explicit discrete feature descriptors.

[0027] Alternatively, the K-Means clustering algorithm can be used to cluster the sensory data to obtain a sequence of discrete feature descriptors. Specifically, the K-Means clustering algorithm can be used to cluster multimodal sensory data of one modality, and a matching discrete feature descriptor can be assigned to each cluster.

[0028] S120. Construct first prompt information based on the context state definition information and the discrete feature descriptor sequence, and guide the micro language model to generate the current context state based on the first prompt information.

[0029] Contextual state definition information can be considered as a set of information that defines a specific environment or situation using a structured description. The current contextual state can be considered as the contextual state reflected by multimodal perception data. A small language model (SLM) can be considered as a lightweight language model with relatively small parameter size, low computational requirements, and specifically optimized for a particular task or domain. The first cue information can be considered as structured input information used to guide the small language model in generating specific outputs.

[0030] In this embodiment, a first prompt information template is predefined, which includes context state definition information. The discrete feature descriptor sequence corresponding to the multimodal perception data is input into the first prompt information template to obtain the first prompt information. The first prompt information is input into the micro language model to guide the micro language model to generate the current context state corresponding to the discrete feature descriptor sequence based on the context state definition information.

[0031] S130. Generate a sequence of control instructions for the controlled home appliances based on the current situation, and control the controlled home appliances to perform the corresponding operations based on the sequence of control instructions.

[0032] In this embodiment, the controlled home appliance can be considered as a home appliance connected to the control system, and the control command sequence can be considered as a set of control commands for controlling the home appliance. In this embodiment, the controlled home appliance can be one home appliance or multiple different home appliances. The control command sequence contains at least one control command, and may also contain multiple control commands controlling one home appliance, or multiple control commands controlling multiple home appliances respectively.

[0033] In this embodiment, after obtaining the current situation state according to the aforementioned steps, a control instruction sequence corresponding to the controlled home device is generated according to the current situation state, and the corresponding controlled home device is controlled to perform the corresponding operation according to each control instruction in the control instruction sequence.

[0034] For example, the method of generating the control command sequence corresponding to the controlled home device based on the current context state can be to determine the control command sequence corresponding to the current context state based on a mapping table between context state and control commands; or it can be to generate the control command sequence corresponding to the controlled home device based on retrieval and generation (RAG) technology and context learning technology. For example, it can be to match similar habitual cases from a habitual case library based on the current context state, use the habitual cases as context information, and construct a second prompt information based on the current context state and context information; and guide the large language model to generate the control command sequence based on the second prompt information.

[0035] Optionally, to further ensure that the generated control command sequence is safe for the controlled home device, the initially generated control command sequence can be intercepted and risk-assessed and corrected. For example, the control command sequence can be predicted to have risks based on the physical performance fingerprint of the controlled home device, or the control command sequence can be judged to have risks based on preset risk constraints. The control commands with risks can be corrected before controlling the controlled home device.

[0036] The technical solution of this invention acquires multimodal sensing data and converts it into a sequence of discrete feature descriptors. Based on the contextual state definition information and the discrete feature descriptor sequence, a first prompt is constructed. This first prompt guides a micro-language model to generate the current contextual state. A control command sequence corresponding to the controlled home appliance is generated based on the current contextual state, and the controlled home appliance is controlled to perform the corresponding operation based on the control command sequence. By introducing a hierarchical contextual awareness mechanism, low-power signal processing and a micro-semantic model are used to complete the conversion from the original sensing signal to the discrete semantic state, solving the "semantic gap" problem between multimodal raw data and high-level semantic decision-making. Furthermore, by utilizing the zero-shot logical reasoning capability of the micro-language model, accurate identification of complex life situations is achieved under low computing power conditions, improving the energy efficiency and response speed of edge computing.

[0037] Example 2 Figure 2This is a flowchart of a smart home control method provided in Embodiment 2 of the present invention. This embodiment further refines the process of generating the control command sequence in the above embodiment. Specifically, the step of generating a control command sequence corresponding to the controlled home device based on the current situation state, and controlling the controlled home device to perform corresponding operations based on the control command sequence, includes: generating an initial control command sequence corresponding to the controlled home device based on the current situation state; performing feedforward prediction on the initial control command sequence based on the physical performance fingerprint of the controlled home device, and determining a target control command sequence based on the feedforward prediction result and the initial control command sequence; and controlling the controlled home device to perform corresponding operations based on the target control command sequence.

[0038] like Figure 2 As shown, the method includes: S210. Acquire multimodal sensing data and convert the multimodal sensing data into a sequence of discrete feature descriptors.

[0039] S220. Construct first prompt information based on the context state definition information and the discrete feature descriptor sequence, and guide the micro language model to generate the current context state based on the first prompt information.

[0040] S230. Generate the initial control command sequence corresponding to the controlled home device based on the current situation.

[0041] In this embodiment, the initial control command sequence can be considered as a sequence of control commands generated based on the current situation state. The initial control command sequence can be used as the target control command sequence after risk assessment, or it can be obtained by modifying the target control command sequence according to the risk assessment results.

[0042] Smart home scenarios demand highly personalized experiences, with vastly different users having varying preferences for device states (such as light color temperature and curtain opening angles). Existing technologies primarily rely on full or fine-tuning techniques (such as SFT or RLHF) to adapt models to specific user habits. However, on single-board computers with limited computing and storage resources, performing parameter updates based on gradient backpropagation is computationally expensive and time-consuming, making it difficult to meet real-time requirements. This results in existing edge agents typically exhibiting "static" characteristics, struggling to evolve in real-time with changing user habits; or they must rely on cloud training, introducing the risk of data privacy leaks and failing to achieve low-cost local adaptation.

[0043] To address the aforementioned issues, this invention proposes a habit evolution mechanism based on non-parametric memory, which addresses the problem of existing technologies relying on expensive model fine-tuning to adapt to user habits. This mechanism stores historically successful triples (contextual state, thought chain, and control sequence) in a habit case library. When the current contextual state is obtained, the most matching habit case is retrieved through vector similarity and used as an in-context learning tool to generate an initial control instruction sequence that aligns with the user's current preferences.

[0044] As an optional embodiment of the present invention, the step of generating an initial control command sequence corresponding to the controlled home device based on the current situational state includes: A1. Based on the current situational state, retrieve the habitual case library to obtain habitual cases that match the current situational state; the habitual case is a triple containing the situational state, thought chain, and control instruction sequence.

[0045] The habit case library contains historical cases of home device control related to the user's situational state and habits. In this embodiment, the habit case library may include positive habit cases and negative habit cases. Positive habit cases serve as role models to be imitated, providing verified and effective behavioral paths for generating initial control command sequences; negative habit cases serve as warning signs, providing avoidable common failure modes for generating initial control command sequences.

[0046] In this embodiment, a habitual case can be defined as a triple containing a situational state, a thought chain, and a sequence of control instructions, such as: ; in, For the i-th habit case, This is a vectorized representation of the contextual state in the i-th habitual case, used for fast retrieval; A thought process chain for reasoning based on the context; It is a sequence of control instructions that were successfully executed in the context state.

[0047] In this embodiment, the current context state is transformed into a query vector through embedding representation; the similarity between the current context state and the context state vectors in each triplet of the habitual case library is calculated, and the TOP-K most similar habitual cases are retrieved according to the similarity score.

[0048] This embodiment utilizes the semantic generalization capability of vector space to identify that they belong to the same situational state even if there are slight differences in sensor values ​​between the current scene (such as "late weekend night") and the historical scene (such as "late weekday night").

[0049] A2. Use the habitual examples as contextual information, and construct a second prompt message based on the current situation and the contextual information.

[0050] In this embodiment, the retrieved TOP-K habitual cases are written together with the current situational state as contextual information into the second prompt information template to construct the second prompt information.

[0051] A3. Based on the second prompt information, guide the large language model to generate an initial control instruction sequence.

[0052] In this embodiment, the second prompt information is input into the preset large language model, which guides the large language model to determine the initial control instruction sequence by combining contextual information (situational state, thought chain and control instruction sequence in habitual cases) and the current situational state. The large language model can also output the thinking process of generating the initial control instruction sequence based on contextual information and the current situational state, i.e., the thought chain.

[0053] This embodiment, based on a habitual case library and Retrieval Enhancement Generation (RAG) technology, replaces "gradient update" with "retrieval and reuse." It can learn users' personalized preferences in real time from historical interactions without performing gradient backpropagation, achieving training-free continuous learning. It can automatically evolve as user data accumulates, becoming more and more in line with users' personalized habits. At the same time, it avoids expensive backpropagation calculations on computing nodes with limited computing power, significantly reducing system deployment costs and energy consumption. It also eliminates the dependence of edge devices on high-performance training hardware (such as GPUs with high video memory), significantly reducing hardware costs, and achieving high productivity of "learning as you use."

[0054] S240. Perform feedforward prediction on the initial control command sequence based on the physical performance fingerprint of the controlled home appliance, and determine the target control command sequence based on the feedforward prediction result and the initial control command sequence.

[0055] Existing large-scale control schemes typically focus only on the logical correctness of instructions (e.g., generating a control instruction to "set the brightness to 10%)", while ignoring the inherent physical response characteristics of heterogeneous home appliances. Real-world home appliances come from a wide variety of brands, with significant differences in their electrical characteristics. For example, some dimming drivers may flicker under certain low loads, or some motors may have start-up dead zones and response delays. Existing systems lack mechanisms for system identification of specific devices, making it impossible to build performance characteristic models for these devices. Control strategies often employ reactive feedback control, meaning corrections are only made after execution failures or anomalies occur. This hysteretic control mechanism, relying on fault feedback, poses a high risk in the physical world, easily leading to abnormal wear and tear on equipment, discontinuous user experience, or even substantial hardware damage.

[0056] To address the aforementioned issues, this embodiment introduces system identification and feedforward control technology. The system constructs a physical performance fingerprint for each heterogeneous home appliance, recording its physical response limits, dead time, and instabilities under specific operating conditions. Before issuing control commands, the output of the planning layer is intercepted, and feedforward prediction is performed based on the performance fingerprint of the target device. If a command is predicted to potentially trigger hardware risks (such as low-brightness flicker or motor stall), the command is reconstructed or smoothed (e.g., by inserting delays or adjusting the slope) based on best practice logic in the fingerprint, generating a final safe command.

[0057] As an optional embodiment of the present invention, the step of performing feedforward prediction on the initial control command sequence based on the physical performance fingerprint of the controlled home appliance, and determining the target control command sequence based on the feedforward prediction result and the initial control command sequence, includes: B1. Based on the physical performance fingerprint of the controlled home device, perform feedforward prediction on the initial control command sequence to obtain the feedforward prediction result.

[0058] Physical performance fingerprints can be considered as a collection of multi-dimensional physical performance parameters, enabling a quantitative feature system for identifying and diagnosing the uniqueness, state, or history of an object. For example, physical performance fingerprints may include physical limit parameters (such as maximum safe input voltage, overheat threshold, and minimum strobe intensity), response dead zone data (such as start-up dead zone, saturation dead zone, and air step dead zone), and historical fault modes (abnormal performance under specific operating conditions, such as current whistling intervals).

[0059] In this embodiment, a file is configured for each controlled home appliance to store its physical performance fingerprint. The generated initial control command sequence is intercepted, and the performance parameters in the physical performance fingerprint of the controlled home appliance are verified one by one to predict whether there is any risk when the controlled home appliance executes the commands in the initial control command sequence.

[0060] B2. If the feedforward prediction result indicates a risk, the initial control command sequence is corrected to obtain the target control command sequence.

[0061] In this embodiment, if the feedforward prediction result of the instructions in the initial control instruction sequence is that there is a risk, then the instructions in the initial control instruction sequence are fedforward processed so that the control instructions can meet the requirements in the physical performance fingerprint, thereby ensuring that the control instructions can be converted into effective physical actions.

[0062] B3. If the feedforward prediction result indicates that there is no risk, then the initial control command sequence is determined as the target control command sequence.

[0063] In this embodiment, if the feedforward prediction result is that there is no risk, it means that the controlled home device can effectively execute the instructions in the initial instruction sequence. Therefore, the initial control instruction sequence can be determined as the target control instruction sequence.

[0064] In this embodiment, by constructing a hardware performance fingerprint and introducing a feedforward protection mechanism, predictive verification and correction are performed based on physical characteristics before the instruction is issued, upgrading the traditional reactive feedback control to feedforward preventive control. This solves the problem of mismatch between general logic instructions and specific hardware physical characteristics, fundamentally eliminating the risk of device damage or experience gaps caused by control instructions exceeding hardware physical limits (such as frequency, voltage, dead zone), effectively reducing the hardware maintenance cost of smart home systems, and improving the stability and reliability of collaborative work of complex heterogeneous devices.

[0065] S250: Control the controlled home appliances to perform corresponding operations based on the target control command sequence.

[0066] The technical solution of this invention involves acquiring multimodal sensing data and converting it into a discrete feature descriptor sequence; constructing first prompt information based on contextual state definition information and the discrete feature descriptor sequence; guiding a micro-language model to generate the current contextual state based on the first prompt information; generating an initial control command sequence corresponding to the controlled home device based on the current contextual state; performing feedforward prediction on the initial control command sequence based on the physical performance fingerprint of the controlled home device; and determining a target control command sequence based on the feedforward prediction result and the initial control command sequence; and controlling the controlled home device to perform corresponding operations based on the target control command sequence. By introducing a hierarchical contextual awareness mechanism, and utilizing low-power signal processing and a micro-semantic model to complete the conversion from raw sensing signals to discrete semantic states, this solves the "semantic gap" problem between multimodal raw data and high-level semantic decision-making. Furthermore, by leveraging the zero-shot logical reasoning capability of the micro-language model, it achieves accurate identification of complex life situations under low computing power conditions, significantly improving the energy efficiency and response speed of edge computing. Meanwhile, by constructing a hardware performance fingerprint and introducing a feedforward protection mechanism, predictive verification and correction are performed based on physical characteristics before the command is issued, which effectively reduces the hardware maintenance cost of smart home systems and improves the stability and reliability of collaborative work of complex heterogeneous devices.

[0067] As an optional embodiment of the present invention, after controlling the controlled home appliance to perform the corresponding operation according to the control command sequence, the method further includes: C1. Detect whether user feedback operations associated with the operations performed by the controlled home appliance have been obtained.

[0068] User feedback operations can be considered as actions performed by a user after controlling a controlled home appliance according to a sequence of target control commands. User feedback operations can be performed on the controlled home appliance responding to the target control command, or on other controlled home appliances connected to the smart home system; they can be operations on one or more controlled home appliances. User feedback operations can be explicit confirmation behaviors such as voice or rating, or they can be similar to or the opposite of performing operations on the home appliance. For example, after controlling a light to turn on, the user can adjust the brightness or turn the light off.

[0069] In this embodiment, after controlling the controlled home appliance to perform the corresponding operation according to the control command sequence, it is detected at a preset time whether user feedback operation associated with the operation performed by the controlled home appliance can be obtained.

[0070] C2. If no user feedback operation is obtained or the obtained user feedback operation is a positive feedback operation, then the target triple corresponding to the current situational state is written into the habit case library as a positive habit case; the target triple includes the target control instruction sequence corresponding to the current situational state and the thought chain that generates the target control instruction sequence.

[0071] Positive feedback operations can be considered as subsequent interactive behaviors that a user actively takes after the controlled home appliance performs a control operation (such as "turn on the living room lights" or "adjust the air conditioner temperature"), in order to express satisfaction or approval of the result of the operation, and which can be recorded and interpreted by the system. Positive feedback operations can be minor adjustments based on the control operation performed by the controlled home appliance (such as the user increasing the playback volume after controlling the TV to turn on), or performing other operations that are logically consistent with the current operation (such as the user playing soothing music after controlling the ambient lights to turn on).

[0072] In this embodiment, if no user feedback is received or the received user feedback is positive after controlling the controlled home appliance to perform an operation for a period of time, it can be considered that the user is relatively satisfied with the control behavior determined based on the current situation. Therefore, the target triple corresponding to the current situation is identified as a positive habit case and written into the habit case library. The target triple may include the target control instruction sequence generated in the current situation, as well as the thought chain of the target control instruction sequence generated by the large model in the current situation.

[0073] C3. If the user feedback operation is obtained and the user feedback operation is a negative feedback operation, then the target triple corresponding to the current situation state is written into the habit case library as a negative habit case.

[0074] Negative user feedback can be considered as a subsequent interactive behavior that can be captured by the system, initiated by the user after an operation is performed on the controlled device to express dissatisfaction, rejection, or a need for correction. Reverse feedback can be an operation that directly restores the device state to its state before the operation, or an operation that is logically mutually exclusive with the current system state, or an operation involving continuous correction and repeated fine-tuning.

[0075] In this embodiment, if a user feedback operation is obtained and the user feedback operation is a negative feedback operation, it can be considered that the user is not satisfied with the control behavior determined based on the current situation. Therefore, the target triple corresponding to the current situation is determined as a negative habitual case and written into the habitual case library to reduce the retrieval weight of similar historical cases in this situation, thereby suppressing the regeneration of such erroneous strategies in future retrievals.

[0076] In this embodiment, as usage time increases, the case library accumulates user preferences for different seasons and time periods, allowing the system to achieve personalization without fine-tuning and possessing a certain degree of self-correction capability.

[0077] Optionally, after controlling the controlled home appliance to perform the corresponding operation according to the control command sequence, the method further includes: C4. Monitor the command response status of the controlled home appliance; update the corresponding performance parameters in the physical performance fingerprint of the controlled home appliance based on the command response status.

[0078] The command response status can be considered as the state of the controlled home appliance after receiving a control command in the corresponding control command sequence. Command response statuses can include, for example, successful operation, timeout, device malfunction, or interruption.

[0079] In this embodiment, after controlling the controlled home device to perform the corresponding operation according to the control command sequence, the command response status of the controlled home device is continuously monitored, and the corresponding performance parameters in the physical performance fingerprint of the controlled home device are updated and maintained according to the command response status, so as to adapt to the aging of the home device or changes in the network environment.

[0080] This embodiment employs a dual closed-loop adaptive update strategy based on user feedback and device response. Based on user feedback, generated control commands are automatically stored as habit cases in a habit case library, enabling the expansion and maintenance of user habits. Based on the command responses of the controlled home appliances, hardware performance fingerprint parameters are updated online to adapt to the aging of home appliances or changes in the network environment. This ensures the system has full lifecycle adaptive capabilities, adapting not only to changes in user behavior but also to the aging or characteristic drift of home appliances over time, guaranteeing the system's long-term robustness.

[0081] In one specific embodiment, the smart home control method provided in the above embodiments is applied to a smart home control system that runs on a general-purpose single-board computer (SBC) or a high-performance edge gateway (e.g., an embedded computing platform based on ARM architecture with 4GB or more of memory). This system does not rely on fine-tuning of large cloud models, but instead achieves adaptive control of home devices (such as smart lights, curtain motors, HVAC equipment, and audio-visual entertainment equipment) through local hierarchical perception, vectorized memory retrieval, and hardware fingerprint protection.

[0082] Figure 3 This is a schematic diagram of the structure of a smart home control system provided in this embodiment, as shown below. Figure 3As shown, the smart home control system 30 running on the edge gateway includes three core layers: the perception layer 310, the planning and memory layer 320, and the protection and execution layer 330.

[0083] (1) The perception layer 310 includes a signal processing module 311 and a micro context reasoning module 312. The module obtains a discrete feature descriptor sequence by performing signal processing on the original sensing data and feature discretization based on threshold matching. The module generates a structured first prompt information based on the discrete feature descriptor sequence and inputs the first prompt information into the micro language model for context reasoning. The module transforms the underlying perception data into the current context state with high-level semantics, thereby reducing the computational load of the planning model.

[0084] Signal processing module 311: Acquires real-time sensing data from environmental sensors via the edge gateway's I / O interface. A lightweight data cleaning program runs internally within the signal processing module, performing the following steps: (a) Multimodal data cleaning: Preprocessing of data from light sensors (Lux), decibel meters (dB), and human presence sensors (mmWave / PIR). For example, using a sliding window (e.g., a 5-second window) weighted average algorithm to filter out burst noise from the sensor data.

[0085] (b) Semantic mapping (Tokenization): Based on the preset mapping rule configuration file, continuous numerical tactile data is mapped into discrete semantic tokens.

[0086] For example, the mapping logic can be: Illumination intensity I: If I < 50 Lux, the mapping is as follows <light:dim>TV power P: If P > 30W, it is mapped as<Device_TV: Active> Current time T: If T falls within the period from 23:00 to 06:00 the next day (i.e., T>23:00 or T<06:00), it is mapped as follows:<Time:Late_Night> Sound decibel S: If S < 40 dB, it is mapped as<Sound: Quiet> Output: A sequence of feature descriptors Vt consisting of a series of discrete semantic tokens corresponding to the current time step.<Time:Late_Night> , <light:dim>,<Device_TV: Active> ,<Sound: Quiet> .

[0087] The Micro-Context Inference module 312 deploys a resident memory micro-language model (SLM) (with 0.5B-1.5B parameters, quantized in 4-bit / 8-bit). This module uses structured prompts for zero-shot classification, transforming a sequence of feature descriptors into a single label: the current context state.

[0088] For example, an example of an SLM Prompt (i.e., the first prompt message) is as follows: "You are a context-aware engine running on an edge gateway. Your task is to select the most matching unique state from a predefined list based on sensor characteristics through logical reasoning."

[0089] [State Definition] - Movie_Time (Viewing Mode): The TV must be on and the lighting must be dim. The sound may be intermittent.

[0090] - Reading_Mode: TV off, sufficient light (or supplemental light required), quiet environment.

[0091] - Sleeping_Mode: All entertainment devices are turned off, the body is still, and it is nighttime.

[0092] - Night_Walk (Night Wake Mode): In the dead of night, when the light is extremely dim, brief human movement was detected.

[0093] <|user|> [Feature descriptor sequence] -time:<Time: Late_Night> -illumination:<Light: Dim> -television:<Device_TV: Active> -sound:<Sound: Quiet> Please analyze the relationships between features and output the results in JSON format, including the reasoning process and the final state. The SLM output is as follows in JSON: {"reasoning":"Detected that it was late at night and the light was dim, but the TV was active, which excludes sleep mode and night wake-up mode. This matches the user's characteristics of watching movies late at night.","state": "Movie_Time"}.

[0094] The system extracts "Movie_Time" as the current context state.

[0095] (2) The planning and memory layer 320 includes: a habit case library construction module 321, a retrieval module 322, and a planning agent 323. The planning and memory layer 320 utilizes retrieval-based generation (RAG) technology and in-context learning to enable the planning agent to adapt to the user's personalized lifestyle habits by referring to historical successful cases without parameter fine-tuning (training-free).

[0096] (a) Habitual Case Library Construction Module: The system maintains a lightweight vector database (e.g., ChromaDB or Faiss-lite) locally on the edge gateway. Storage units in the database are defined as triples: ; in, For the i-th habit case, This is a vectorized representation of the contextual state in the i-th habitual case, used for fast retrieval; A thought process chain for reasoning based on the context; It is a sequence of control instructions that were successfully executed in the context state.

[0097] Vectorization implementation techniques: The system deploys a lightweight text embedding model. When the perception layer outputs a state (e.g., {"state": "Movie_Time", "reasoning": "Detected late at night..."}), the system concatenates the "state label + reasoning" into a text string, inputs it into the embedding model, and transforms it into a high-dimensional (e.g., 384-dimensional) dense vector.

[0098] (b) Retrieval module 322: When the perception layer outputs the current context state At that time, perform the following steps: The planning agent calls the locally embedded model to display the current situation state. Convert to query vector ; Calculate using cosine similarity Vectors of all historical cases in the library The distance is calculated using the following formula: Based on the similarity scores, the system retrieves the Top-K (e.g., K=2) most similar habitual cases. Utilizing the semantic generalization capability of vector space, even if there are slight differences in sensor values ​​between the current scene (e.g., "late weekend night") and historical scenes (e.g., "late weekday night"), the system can still identify that they belong to the same contextual state.

[0099] (c) Planning agent 323: Fill the retrieved Top-K cases into the structured second prompt information template as a reference example.

[0100] "You are a personalized home manager. Based on the retrieved 'historical habit cases,' please generate the control intent for home devices in the current scenario."

[0101] [Important Principle] You only need to decide "which home appliance to operate" and "what state to set (color / brightness / on / off)", without having to consider physical protection issues such as hardware voltage, flicker or delay, which will be handled by the underlying system.

[0102] [Historical Habits and Case Studies] Case 1 (96% similarity): - Context: Movie_Time (Late-night movie viewing) - User preference: "Leave only the ambient lighting on, turn off the main light." -Historical operations: [{"target": "Main_Light", "action": "OFF"}, {"target": "Floor_Lamp", "action": "ON", "params": {"color": "Dark_Blue","brightness": "5%"}}] Case 2 (82% similarity): - Context: Reading Mode (Late Night Reading) -Historical operations: [{"target": "Main_Light", "action": "OFF"}, {"target":"Floor_Lamp", "action": "ON", "params": {"color": "Warm_White","brightness": "50%"}}] [Current Context] Movie_Time (Detected late at night and TV active) Please generate a preliminary control sequence (JSON format):.

[0103] It should be noted that the above-described prompt message is only a preferred implementation of this embodiment. In practical applications, changes in the structure, keyword definitions, language (e.g., English / Chinese), and format (JSON / XML / YAML) of the prompt message, as long as they follow the logical paradigm of 'generating a new strategy based on contextual description and historical reference', should be included within the scope of protection of this invention.

[0104] Output the initial instruction sequence: [{"target": "Main_Light", "action": "OFF"},{"target": "Floor_Lamp", "action": "ON", "params": {"color": "Dark_Blue","brightness": "5%"}}].

[0105] (3) Guardian and Execution Layer 330: Responsible for resolving physical compatibility issues of heterogeneous hardware. The system does not assume that the instructions generated by the planning layer are physically safe, but intercepts and corrects them through a fingerprint database built from "system identification" data. Guardian and Execution Layer 330 includes: physical performance fingerprint construction module 331 and guardian agent 332.

[0106] (a) Physical performance fingerprint building module 331: Stores a description file of physical performance fingerprint in JSON format for each connected home device. This description file can be automatically generated by offline testing or online running data.

[0107] For example, the physical performance fingerprint of a floor lamp may include: physical limit parameters, response dead zone data, and historical failure modes.

[0108] Physical limit parameters are used to define the safe boundaries of device operation, such as: maximum safe input voltage max_safe_voltage: 240; overheat throttling threshold: thermal_throttling_temp: 85; minimum non-flicker brightness, below which the device can still light up but will damage the eyes: min_non_flicker_brightness: 8 / / (8%).

[0109] Response dead-time data is used to define the invalid interval between the input signal (0-100%) and the actual response of the device. Start-up dead-time / low-side cutoff: For example, an LED driver chip has a turn-on voltage; when the input instruction is 1%-7%, the voltage is insufficient to break down the PN junction, and the actual brightness is 0. Saturation dead-time / high-side saturation: For example, when the input reaches 92%, the drive current has reached its maximum value. Continuing to increase the instruction to 100% will not improve the physical brightness. Control step dead-time: For example, due to DAC resolution limitations, the device cannot respond to minute instruction changes of less than 1%.

[0110] Historical fault modes are used to record abnormal behavior under specific operating conditions, such as current howling range, brightness and triggering conditions (e.g., triggering when the color temperature reaches 4000K).

[0111] (b) The guardian agent 332 is used to intercept the initial control command sequence of the planning and memory layer, and load the physical performance fingerprint of the corresponding controlled home device for verification one by one, so as to realize the feedforward prediction and correction of the initial control command sequence. The guardian agent can be a micro model or logic algorithm of 0.5B-1.5B level, with the advantages of low power consumption and low latency.

[0112] Step 1: Feedforward Prediction The system intercepts the generated initial control command sequence and verifies each performance parameter in the physical performance fingerprint of the controlled home device to predict whether there is a risk when the controlled home device executes the commands in the initial control command sequence.

[0113] Step 2: Refining: The guardian agent rewrites the instructions based on the built-in "best practice algorithm".

[0114] For example, regarding the handling of startup dead zones: When the initial control command to activate the low-light mode is issued at the planning and memory layer, the guardian agent reads the input low-level cutoff threshold of 7% from the physical performance fingerprint of the controlled home device. The system determines that 5% is in the cutoff zone, meaning the device cannot be physically lit, but execution would result in the system status displaying an error of "on". The guardian agent automatically clamps the parameters to the physical minimum conduction value of 7% (or increases it to 8% to prevent flicker), ensuring that the light fixture can actually emit light after the command is executed, maintaining state synchronization.

[0115] Handling saturation dead zones: When the current brightness is 92%, and the planning and memory layer issues the initial control command to adjust the brightness, the guardian agent reads the maximum brightness threshold of 92% from the physical performance fingerprint of the controlled home device. The system determines that the current brightness has reached the physical limit. The guardian agent intercepts and discards this invalid command to avoid unnecessary network communication consuming bandwidth, and reports "maximum brightness reached" to the system to prevent the planning and memory layer from continuously attempting invalid adjustments.

[0116] To avoid fault ranges: When the planning and memory layer issues an initial control command to adjust the brightness to 45% and the color temperature to 4500K, the guardian agent reads the howling range in the physical performance fingerprint of the controlled home device as 40%-50% and greater than 4000K. The guardian agent then fine-tunes the parameters, modifying the command to BRIGHTNESS=45% and COLOR=3900K (lowering the color temperature to avoid howling conditions), thus avoiding hardware noise while ensuring lighting effects.

[0117] After executing the target control command sequence, the home control system can also enter a dual closed-loop update phase to achieve full lifecycle self-adaptation.

[0118] (4) Cognitive Loop: If the user does not perform a reverse operation (such as manually turning off the floor lamp) within a certain period of time after the instruction is executed (e.g., 15 minutes), the system determines that the "context-instruction" match is successful. The system vectorizes the new triple (current context state-thought chain-control instruction sequence) and stores it in the habit case library. At the same time, the system has a negative correction mechanism: If the user performs a reverse operation within a short period of time after the instruction is executed (e.g., within 1 minute) (e.g., the system automatically turns on the light, and the user immediately turns it off manually), the system determines that the strategy does not match the user's intention. At this time, the new triple is marked as a "negative habit case" and stored in the habit case library, reducing the retrieval weight of similar habit cases in this context state, thereby suppressing the regeneration of such erroneous strategies in future retrievals. As usage time increases, the case library accumulates user preferences for different seasons and time periods, and the system can achieve personalization without fine-tuning and has self-correction capabilities.

[0119] (5) Physical adaptation closed loop: The system backend continuously monitors the actual response of the controlled home appliances. If the system detects that the ACK latency of the Floor_Lamp increases from 300ms to 600ms when executing commands (possibly due to Wi-Fi channel congestion or device aging), the guardian agent automatically updates the dead_zone_ms parameter in the physical performance fingerprint of the controlled home appliance to 600ms. The next time the guardian agent processes the control command sequence of the controlled home appliance, it will automatically increase the interval between control commands to prevent packet loss caused by command backlog and ensure continuous control stability.

[0120] Example 3 Figure 4 This is a schematic diagram of the structure of a smart home control device provided in Embodiment 3 of the present invention. Figure 4 As shown, the device includes: a sensing data acquisition module 410, a situation state determination module 420, and a control module 430; wherein: The perception data acquisition module 410 is used to acquire multimodal perception data and convert the multimodal perception data into a sequence of discrete feature descriptors; The context state determination module 420 is used to construct first prompt information based on the context state definition information and the discrete feature descriptor sequence, and guide the micro language model to generate the current context state based on the first prompt information; The control module 430 is used to generate a control instruction sequence corresponding to the controlled home device based on the current situation state and context information, and to control the controlled home device to perform the corresponding operation based on the control instruction sequence.

[0121] This invention provides a smart home control device that acquires multimodal sensing data and converts it into a discrete feature descriptor sequence. A first prompt message is constructed based on the context state definition information and the discrete feature descriptor sequence. This prompt message guides a micro-language model to generate the current context state. An initial control command sequence corresponding to the controlled home device is generated based on the current context state. The initial control command sequence is then fed forward based on the physical performance fingerprint of the controlled home device, and a target control command sequence is determined based on the feedforward prediction result and the initial control command sequence. The controlled home device is then controlled to perform corresponding operations based on the target control command sequence. By introducing a hierarchical context awareness mechanism, low-power signal processing and a micro-semantic model are used to complete the conversion from raw sensing signals to discrete semantic states, solving the "semantic gap" problem between multimodal raw data and high-level semantic decision-making. Furthermore, the zero-shot logical reasoning capability of the micro-language model enables accurate identification of complex life situations under low computing power conditions, significantly improving the energy efficiency and response speed of edge computing. Meanwhile, by constructing a hardware performance fingerprint and introducing a feedforward protection mechanism, predictive verification and correction are performed based on physical characteristics before the command is issued, which effectively reduces the hardware maintenance cost of smart home systems and improves the stability and reliability of collaborative work of complex heterogeneous devices.

[0122] Optionally, the control module includes: An initial instruction determination unit is used to generate an initial control instruction sequence corresponding to the controlled home device based on the current situation state; The target instruction determination unit is used to perform feedforward prediction on the initial control instruction sequence based on the physical performance fingerprint of the controlled home device, and determine the target control instruction sequence based on the feedforward prediction result and the initial control instruction sequence. The control unit is used to control the controlled home appliance to perform corresponding operations based on the target control command sequence.

[0123] Optionally, the initial instruction determining unit is specifically used for: Based on the current situational state, a habitual case library is retrieved to obtain a habitual case that matches the current situational state; the habitual case is a triple containing the situational state, the thought chain, and the sequence of control instructions. The habitual examples are used as contextual information, and a second prompt is constructed based on the current situation and the contextual information; Based on the second prompt information, the large language model is guided to generate an initial control instruction sequence.

[0124] Optionally, the target instruction determining unit is specifically used for: Based on the physical performance fingerprint of the controlled home device, the initial control command sequence is fed forward to predict the feedforward prediction result. If the feedforward prediction result indicates a risk, the initial control command sequence is corrected to obtain the target control command sequence. If the feedforward prediction result indicates that there is no risk, then the initial control command sequence is determined as the target control command sequence.

[0125] Optional, also includes: The feedback operation detection module is used to detect whether a user feedback operation associated with the operation performed by the controlled home device is obtained after the controlled home device is controlled to perform a corresponding operation according to the control instruction sequence. The positive habit case recording module is used to write the target triple corresponding to the current situational state as a positive habit case into the habit case library if no user feedback operation is obtained or the obtained user feedback operation is a positive feedback operation; the target triple includes the target control instruction sequence corresponding to the current situational state and the thought chain that generates the target control instruction sequence; The negative habit case recording module is used to record the target triple corresponding to the current situation state as a negative habit case in the habit case library if the user feedback operation is obtained and the user feedback operation is a negative feedback operation.

[0126] Optional, also includes: The response status monitoring module is used to monitor the command response status of the controlled home device after controlling the controlled home device to perform the corresponding operation according to the control command sequence. The performance parameter update module is used to update the corresponding performance parameters in the physical performance fingerprint of the controlled home device according to the instruction response status.

[0127] The smart home control device provided in the embodiments of the present invention can execute the smart home control method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method.

[0128] Example 4 Figure 5 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, edge gateways, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0129] like Figure 5 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0130] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0131] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as smart home control methods.

[0132] In some embodiments, the smart home control method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the smart home control method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the smart home control method by any other suitable means (e.g., by means of firmware).

[0133] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0134] In some embodiments, the smart home control method may be implemented as a computer program, which is implicitly included in a computer program product. When executed by a processor, the computer program implements the smart home control method of the present invention. The computer program product can be understood as a software product that primarily implements its solution through a computer program. The computer program used to implement the method of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer program causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer program may be executed entirely on a machine, partially on a machine, partially on a remote machine as a standalone software package, or entirely on a remote machine or server.

[0135] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0136] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0137] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0138] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0139] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0140] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.< / light:dim> < / light:dim>

Claims

1. A smart home control method, characterized in that, include: Acquire multimodal sensing data and convert the multimodal sensing data into a sequence of discrete feature descriptors; A first prompt message is constructed based on the context state definition information and the discrete feature descriptor sequence, and the micro language model is guided to generate the current context state based on the first prompt message. Based on the current situation state, a control instruction sequence corresponding to the controlled home device is generated, and the controlled home device is controlled to perform the corresponding operation according to the control instruction sequence.

2. The method according to claim 1, characterized in that, The step of generating a control command sequence corresponding to the controlled home device based on the current situation state, and controlling the controlled home device to perform corresponding operations according to the control command sequence, includes: Generate an initial control command sequence corresponding to the controlled home appliances based on the current situation state; The initial control command sequence is fed forward based on the physical performance fingerprint of the controlled home device, and the target control command sequence is determined based on the feedforward prediction result and the initial control command sequence. The controlled home appliances are controlled to perform corresponding operations based on the target control command sequence.

3. The method according to claim 2, characterized in that, The step of generating the initial control command sequence corresponding to the controlled home device based on the current situation state includes: Based on the current situational state, a habitual case library is retrieved to obtain a habitual case that matches the current situational state; the habitual case is a triple containing the situational state, the thought chain, and the sequence of control instructions. The habitual examples are used as contextual information, and a second prompt is constructed based on the current situation and the contextual information; Based on the second prompt information, the large language model is guided to generate an initial control instruction sequence.

4. The method according to claim 2, characterized in that, The step of performing feedforward prediction on the initial control command sequence based on the physical performance fingerprint of the controlled home appliance, and determining the target control command sequence based on the feedforward prediction result and the initial control command sequence, includes: Based on the physical performance fingerprint of the controlled home device, the initial control command sequence is fed forward to predict the feedforward prediction result. If the feedforward prediction result indicates a risk, the initial control command sequence is corrected to obtain the target control command sequence. If the feedforward prediction result indicates that there is no risk, then the initial control command sequence is determined as the target control command sequence.

5. The method according to claim 1, characterized in that, After controlling the controlled home appliance to perform the corresponding operation according to the control command sequence, the method further includes: Detect whether user feedback operations associated with the operations performed by the controlled home appliance have been obtained; If no user feedback operation is obtained or the obtained user feedback operation is a positive feedback operation, the target triple corresponding to the current situational state is written into the habit case library as a positive habit case; the target triple includes the target control instruction sequence corresponding to the current situational state and the thought chain that generates the target control instruction sequence; If the user feedback operation is obtained and the user feedback operation is a negative feedback operation, then the target triple corresponding to the current situation state is written into the habit case library as a negative habit case.

6. The method according to claim 1, characterized in that, After controlling the controlled home appliance to perform the corresponding operation according to the control command sequence, the method further includes: Monitor the command response status of the controlled home appliances; Update the corresponding performance parameters in the physical performance fingerprint of the controlled home device according to the instruction response status.

7. A smart home control device, characterized in that, include: The perception data acquisition module is used to acquire multimodal perception data and convert the multimodal perception data into a sequence of discrete feature descriptors; The context state determination module is used to construct first prompt information based on the context state definition information and the discrete feature descriptor sequence, and guide the micro language model to generate the current context state based on the first prompt information; The control module is used to generate a sequence of control instructions for the controlled home appliances based on the current situation and context information, and to control the controlled home appliances to perform corresponding operations based on the sequence of control instructions.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, which is then executed by the at least one processor to enable the at least one processor to perform the smart home control method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the smart home control method according to any one of claims 1-6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the smart home control method according to any one of claims 1-6.

Images