Device control method, apparatus, device, and medium

Abstract

The application discloses a device control method, device, equipment and medium. The method comprises the following steps: constructing a causal knowledge graph; listening to a user intention signal in a current environment to determine a target intention of the user; determining a target intelligent device according to the target intention and the causal knowledge graph; in the case that the target intelligent device is abnormal, determining a function replacement set corresponding to the target intelligent device according to the causal knowledge graph; each function replacement set comprises one or more function replacement devices; determining a function replacement cost corresponding to each function replacement set; determining a target function replacement device from the function replacement set according to the function replacement cost; and generating a control instruction according to the target intention, so as to control the target function replacement device according to the control instruction. The function replacement device capable of realizing the target intention of the user can be automatically found and controlled in the case that the device is abnormal, and the robustness of the smart home system is improved.

Description

Technical Field

[0001] This application belongs to the field of smart home, specifically relating to a device control method, apparatus, equipment, and medium. Background Technology

[0002] The smart home automation control technology in related technologies is based on trigger conditions and execution actions. The control logic is strongly coupled with the physical hardware. When the corresponding execution device fails, goes offline, or is replaced, the entire automation scenario will fail directly, making the smart home system lack fault tolerance and self-healing capabilities. Summary of the Invention

[0003] The purpose of this application is to provide a device control method, apparatus, device, and medium that can solve or at least partially solve the problem that smart home systems lack fault tolerance and self-healing capabilities, and that the corresponding automated scenarios will directly fail when the executing device malfunctions.

[0004] To solve the above-mentioned technical problems, this application is implemented as follows: In a first aspect, embodiments of this application provide a device control method, the method comprising: Constructing a causal knowledge graph; Listen for user intent signals in the current environment to determine the user's target intent; The target intelligent device is determined based on the target intent and the causal knowledge graph; In the event that the target intelligent device is abnormal, a set of functional alternatives corresponding to the target intelligent device is determined based on the causal knowledge graph; each set of functional alternatives includes one or more functional alternative devices. Determine the functional substitution cost corresponding to each set of said functional substitutions; The target functional replacement device is determined from the set of functional replacements based on the functional replacement cost; Control instructions are generated based on the target intent to control the target function replacement device according to the control instructions.

[0005] Optionally, the causal knowledge graph includes: a device operation node, an intent node, an environmental physical variable node, a first causal edge between the intent node and the environmental physical variable node, and a second causal edge between the environmental physical variable node and the device operation node; the first causal edge is used to indicate the environmental physical variable node that each intent node acts upon; the second causal edge is used to indicate the environmental physical variable node that each device operation node acts upon.

[0006] Optionally, the construction of the causal knowledge graph includes: Scan the current local area network to identify online devices in the current local area network; Obtain the object model file of the online device; Generate the device operation node corresponding to the online device based on the object model file; Obtain the user's historical interaction data corresponding to the online device; Determine the historical interaction intent based on the user's historical interaction data; Generate intent nodes based on the historical interaction intents; The environmental physical variable node corresponding to the intent node is determined based on the preset environmental physical variables. The causal knowledge graph is constructed based on the intent node, the environmental physical variable node, and the device operation node.

[0007] Optionally, constructing the causal knowledge graph based on the intent node, the environmental physical variable node, and the device operation node includes: Establish the first causal edge between the intent node and the environmental physical variable node; Determine the probability distribution of environmental physical variables corresponding to the intent node based on the user's historical interaction data; The probability distribution of the environmental physical variables is used as the edge attribute of the first causal edge; Establish a second causal edge between the environmental physical variable node and the device operation node; Determine the direction of the influence of the device operation node on the environmental physical variable node; The edge attributes of the second causal edge are determined based on the direction of influence; The causal knowledge graph is constructed based on the intent node, the environmental physical variable node, the device operation node, the first causal edge, and the second causal edge.

[0008] Optionally, determining the probability distribution of environmental physical variables corresponding to the intent node based on the user's historical interaction data includes: Obtain the baseline probability distribution of the environmental physical variables corresponding to the intent node; Based on the user's historical interaction data, a Bayesian update is performed on the baseline probability distribution of the environmental physical variables to obtain the probability distribution of the environmental physical variables corresponding to the intent node.

[0009] Optionally, determining the set of functional substitutes corresponding to the target intelligent device based on the causal knowledge graph includes: Based on the target intelligent device and the target intent, the corresponding target device operation node is determined from the causal knowledge graph; Traverse the causal knowledge graph to determine the environmental physical variable node corresponding to the target device operation node as the target environmental physical variable node; Based on the target environment physical variable nodes, search for available device operation nodes in the causal knowledge graph; The set of functional alternatives is generated based on the available device operation nodes.

[0010] Optionally, determining the functional substitution cost corresponding to each set of functional substitutions includes: Obtain the estimated operating power of the functional alternative devices in each of the functional alternative sets; Determine the probability distribution of the predicted environmental physical variables for the functional replacement device; The first causal edge corresponding to the target intent is determined based on the causal knowledge graph. The probability distribution of the physical variables of the target environment is obtained based on the first causal edge corresponding to the target intent; The environmental distribution similarity corresponding to the functional replacement device is determined based on the probability distribution of the predicted environmental physical variables and the probability distribution of the target environmental physical variables. Determine the second causal edge corresponding to the functional replacement device based on the causal knowledge graph; The device penalty value of the functional replacement device is determined based on the second causal edge corresponding to the functional replacement device. The functional substitution cost corresponding to each functional substitution set is determined based on the estimated operating power, the environmental distribution similarity, and the equipment penalty value.

[0011] Optionally, determining the probability distribution of predicted environmental physical variables for the functional replacement device includes: Obtain the current environment state vector and the physical model of the functional replacement device; Determine the state transition matrix corresponding to the functional replacement device based on the object model; The probability distribution of predicted environmental physical variables for the functional replacement device is determined based on the environmental state vector and the state transition matrix.

[0012] Optionally, determining the device penalty value of the functional replacement device based on the second causal edge corresponding to the functional replacement device includes: Obtain the edge attributes of the second causal edge corresponding to the functional replacement device; From the edge attributes, determine the environmental physical variable nodes whose influence direction is negative corresponding to the functional replacement device as target penalty nodes; Determine the preset penalty parameter benchmark based on the target intent and the target penalty node; Obtain the environmental physical variable parameters corresponding to the target penalty node when the functional replacement device is running; The device penalty value of the functional replacement device is determined based on the difference between the penalty parameter benchmark and the environmental physical variable parameter, as well as the preset unit penalty value.

[0013] Secondly, embodiments of this application provide a device control apparatus, the apparatus comprising: Modules for building causal knowledge graphs; The listening module is used to listen for user intent signals in the current environment to determine the user's target intent; A target intelligent device determination module is used to determine the target intelligent device based on the target intent and the causal knowledge graph. A function substitution set determination module is used to determine the function substitution set corresponding to the target intelligent device based on the causal knowledge graph when the target intelligent device is abnormal; each function substitution set includes one or more function substitution devices; A function substitution cost determination module is used to determine the function substitution cost corresponding to each set of function substitutions; A target function replacement device determination module is used to determine a target function replacement device from the function replacement set based on the function replacement cost; A control module is used to generate control commands based on the target intent, so as to control the target function replacement device according to the control commands.

[0014] Thirdly, embodiments of this application provide an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0015] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0016] In this embodiment, a causal knowledge graph can be constructed; user intent signals in the current environment can be monitored to determine the user's target intent; then, based on the target intent and the causal knowledge graph, a target smart device for realizing the user's target intent can be determined; if the target smart device malfunctions, a set of functional alternatives corresponding to the target smart device can be automatically determined based on the causal knowledge graph, each set of functional alternatives may include one or more functional alternative devices, thereby obtaining one or more alternative solutions to replace the faulty device in realizing the user's target intent; then, the functional alternative cost corresponding to each set of functional alternatives can be determined; then, based on the functional alternative cost, a target functional alternative device is determined from the set of functional alternatives, resulting in the final device object used to realize the user's target intent; finally, a control command is generated based on the target intent to control the target functional alternative device. This solves the problems of fragile and low fault tolerance of smart home automation rules in related technologies. In the event of malfunction of the execution device in the smart home system, a target functional alternative device for replacing the malfunctioning device's function can be dynamically determined based on the functional alternative cost, without requiring manual configuration or adjustment by the user, thus improving the robustness of the smart home system. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the steps of an embodiment of a device control method according to this application; Figure 2 This is a schematic diagram of the causal knowledge graph hierarchy of an embodiment of the device control method of this application; Figure 3 This is a flowchart illustrating the steps of another embodiment of the device control method of this application; Figure 4 This is a flowchart illustrating the seamless self-healing process under home appliance malfunctions, according to an embodiment of the device control method of this application. Figure 5 This is a structural block diagram of an embodiment of a device control apparatus according to this application; Figure 6 This is a structural block diagram of an embodiment of an electronic device according to this application. Detailed Implementation

[0019] To make the above objectives, features and advantages of the present application more clearly understood, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0020] The terms "first", "second" and the like in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific sequence or order. It should be understood that the data used in this way may be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / " generally means that the associated objects before and after are in an "or" relationship.

[0021] The device control method provided by the embodiments of the present application will be described in detail below through specific embodiments and application scenarios with reference to the accompanying drawings.

[0022] With reference to Figure 1 , which is a step flowchart of an embodiment of the device control method of the present application, the method can be applied to a gateway or a user terminal.

[0023] The gateway may be a home smart gateway (Edge Gateway), which is a smart central device deployed at the edge of the home network, integrating the functions of multi-protocol conversion, local data processing, edge computing and security management and control, and can connect local IoT smart devices with the cloud / internet to realize off-grid availability, low-latency linkage and privacy protection. The user terminal may include, but is not limited to, a mobile terminal such as a user's smart phone, tablet computer, laptop portable computer, desktop computer, etc., and a related APP (Application) may be installed on the terminal for executing the device control method of the present application. The gateway or the user terminal may establish a connection with the smart devices in the home through a local area network wireless communication protocol such as Wi-Fi, Bluetooth, Zigbee, etc., and issue control instructions silently.

[0024] The embodiment of the device control method provided by the present application may include the following steps: Step 101: Construct a causal knowledge graph; The causal knowledge graph may be a knowledge representation form that explicitly stores the directional logical relationship of "cause-effect" in a graph structure, which can efficiently store, manage and infer the causal relationship between graph nodes.

[0025] Step 102: Listen for user intent signals in the current environment to determine the user's target intent; The current environment can refer to the current home environment corresponding to the smart home system. The current home environment can be a digital living space composed of a perception layer, a network layer, a central hub, and an execution layer. The perception layer can include various sensors, the network layer can include a network composed of one or more of Wi-Fi, Bluetooth, and Zigbee, the central hub can refer to a local gateway, and the execution layer can include one or more smart devices in the smart home system.

[0026] User intent signals can be control signals or sensory signals that reflect user intent. For example, user intent signals can include explicit signals and implicit signals. Explicit signals can be control signals sent by the user through a terminal click, such as a user selecting "sleep mode" through a mobile app, or voice commands sent by the user, such as a user issuing the voice command "I want to sleep" to a central controller or mobile app. Implicit signals can be sensory data fed back from sensors or actuators in the current environment, such as time information sent by a clock sensor (current time is 23:00), brightness perception information sent by a bedroom illuminance sensor (bedroom brightness dims), and heart rate perception information reported by a smart wearable device (user's heart rate decreases to resting state).

[0027] A target intent can reflect the abstract needs that a user wants to achieve. For example: getting up, leaving home for work, deep sleep, working from home, eating, etc.

[0028] For example, for explicit signals, after the gateway or terminal receives the display signal, it can directly perform text matching on the causal knowledge graph, filter out the intent node with the highest matching degree and the matching degree exceeding the preset matching degree threshold, take the user intent corresponding to the intent node as the target intent, and activate the corresponding intent node in the causal knowledge graph.

[0029] For example, for implicit signals, after receiving an implicit signal, the gateway or terminal can package one or more received implicit signals into a multi-dimensional feature vector and input it into a pre-trained intent classification engine or LLM (Large Language Model). The intent classification engine or LLM can calculate the semantic distance between the feature vector and each intent node in the causal knowledge graph, and take the user intent corresponding to the intent node with the smallest semantic distance to the feature vector and a matching degree exceeding a preset matching degree threshold as the target intent. The intent classification engine or LLM can be an existing model or a combination of models, and this application does not impose specific limitations on it. The calculation of semantic distance and matching degree can refer to the methods in related technologies, and this application will not elaborate on them further.

[0030] Step 103: Determine the target intelligent device based on the target intent and the causal knowledge graph; The target smart device can be a smart device used to achieve the target intent.

[0031] Step 104: In the event that the target intelligent device is abnormal, determine the set of functional alternatives corresponding to the target intelligent device based on the causal knowledge graph; Each set of functional alternatives may include one or more functional alternative devices. A target smart device may have one or more sets of functional alternatives, depending on the type, function and number of smart devices in the current environment. This application does not impose any specific restrictions on this.

[0032] Step 105: Determine the functional substitution cost corresponding to each set of said functional substitutions; Among them, the cost of functional substitution can represent the compensatory cost required to achieve the function of the target intelligent device by using a set of functional substitutions.

[0033] Step 106: Determine the target functional replacement device from the functional replacement set based on the functional replacement cost; Among them, the target function replacement device can be a function replacement device in the set of function replacements with the lowest function replacement cost.

[0034] Step 107: Generate control instructions according to the target intent, so as to control the target function replacement device according to the control instructions.

[0035] In this embodiment, a causal knowledge graph can be constructed; user intent signals in the current environment can be monitored to determine the user's target intent; then, based on the target intent and the causal knowledge graph, a target smart device for realizing the user's target intent can be determined; if the target smart device malfunctions, a set of functional alternatives corresponding to the target smart device can be automatically determined based on the causal knowledge graph, each set of functional alternatives may include one or more functional alternative devices, thereby obtaining one or more alternative solutions to replace the faulty device in realizing the user's target intent; then, the functional alternative cost corresponding to each set of functional alternatives can be determined; then, based on the functional alternative cost, a target functional alternative device is determined from the set of functional alternatives, resulting in the final device object used to realize the user's target intent; finally, a control command is generated based on the target intent to control the target functional alternative device. This solves the problems of fragile and low fault tolerance of smart home automation rules in related technologies. In the event of malfunction of the execution device in the smart home system, a target functional alternative device for replacing the malfunctioning device's function can be dynamically determined based on the functional alternative cost, without requiring manual configuration or adjustment by the user, thus improving the robustness of the smart home system.

[0036] In some embodiments of this application, the causal knowledge graph includes: a device operation node, an intent node, an environmental physical variable node, a first causal edge between the intent node and the environmental physical variable node, and a second causal edge between the environmental physical variable node and the device operation node; the first causal edge is used to indicate the environmental physical variable node that each intent node acts upon; the second causal edge is used to indicate the environmental physical variable node that each device operation node acts upon.

[0037] In this embodiment of the application, the causal knowledge graph may include device operation nodes, intent nodes, environmental physical variable nodes, a first causal edge, and a second causal edge.

[0038] Among them, device operation nodes can include a set of device operations for all networked devices in the current network environment. For example, the device operation nodes corresponding to an air conditioner can include air conditioner dehumidification nodes, air conditioner heating nodes, and air conditioner cooling nodes. Intent nodes can represent the user's abstract control requirements for the smart device. Environmental physical variable nodes can represent preset environmental physical variables. For example, environmental physical variables can include, but are not limited to, temperature, humidity, carbon dioxide concentration, and noise decibels.

[0039] The first causal edge is a causal edge between the intent node and the environmental physical variable node. It can represent the dependency relationship between the intent node and the environmental physical variable node. The direction of the first causal edge is from the intent node to the environmental physical variable node. The first causal edge is used to indicate the environmental physical variable node that each intent node affects, and can represent which environmental physical variables in the current environment will be affected by realizing a certain user intent.

[0040] The second causal edge is a causal edge between environmental physical variable nodes and device operation nodes. It can represent the dependency relationship between environmental physical variable nodes and device operation nodes. The direction of the second causal edge is from the environmental physical variable node to the device operation node. The second causal edge is used to indicate the environmental physical variable node that each device operation node acts upon, and can represent the device operation that can be used to change a certain environmental physical variable.

[0041] Through the causal knowledge graph of this application, the capabilities of devices can be semantically mapped, and specific device operations can be abstracted into causal chains of physical environment adjustment intentions. This breaks the one-way binding between instructions and devices in related technologies, endows smart home systems with the ability to understand the physical environment through common sense, and realizes the improvement from human device control to smart home systems automatically meeting user intentions.

[0042] In some embodiments of this application, the construction of a causal knowledge graph includes: Scan the current local area network to identify online devices in the current local area network; Obtain the object model file of the online device; Generate the device operation node corresponding to the online device based on the object model file; Obtain the user's historical interaction data corresponding to the online device; Determine the historical interaction intent based on the user's historical interaction data; Generate intent nodes based on the historical interaction intents; The environmental physical variable node corresponding to the intent node is determined based on the preset environmental physical variables. The causal knowledge graph is constructed based on the intent node, the environmental physical variable node, and the device operation node.

[0043] In this embodiment of the application, the current local area network (LAN) can be automatically scanned using LAN device discovery protocols to identify online devices within the LAN. LAN device discovery protocols may include SSDP (Simple Service Discovery Protocol), mDNS (Multicast Domain Name System), BLE (Bluetooth Low Energy) broadcast, etc.

[0044] Standard IoT (Internet of Things) protocols, such as Matter (a smart home connectivity standard) or MQTT (Message Queuing Telemetry Transport), can be used to send a request to the cloud-based IoT platform to obtain the object model files of online devices in the current local area network. The cloud-based IoT platform can pre-store a large number of smart device object model files. Object model files can be standardized description files used to describe the device's functions, states, and interactions, and can be stored in JSON format on the cloud-based IoT platform. For example, if a smart device has its own object model file stored internally, a gateway or terminal can directly request its object model file from the corresponding smart device. The object model can describe the device from three dimensions: attributes, services, and events. Attributes describe the device's runtime state, services refer to instructions that the device can invoke externally, and events are information actively reported by the device during runtime.

[0045] Device operation nodes corresponding to online devices can be generated based on the object model file. For example, the object model file can be parsed to obtain the functional API (Application Programming Interface) dictionary and control logic for each online device. The functional API dictionary describes which functions or services the corresponding online device can provide that can be invoked, used to determine the corresponding device operation node. For example, the functional API dictionary for an air conditioner may include `AirConditioner.SetMode(Dehumidify)`, indicating that the air conditioner is switched to dehumidification mode, which corresponds to the device operation of air conditioner dehumidification; the functional API dictionary for a humidifier may include `TurnOn,SetMistLevel(1-3)`, indicating that the humidifier can be turned on at levels 1-3, which corresponds to the device operation of humidifier level adjustment.

[0046] Through the above implementation process, compared to treating the device as an independent node in the causal knowledge graph, this embodiment breaks down and refines the device into multiple API operations that the device possesses, obtaining multiple device operation nodes corresponding to each device. This helps to establish more accurate causal edges in the future, improves the fine granularity of reasoning in the causal knowledge graph, and thus ensures the accuracy of subsequent target function replacement device selection.

[0047] In this embodiment, historical user interaction data corresponding to online devices can be obtained. This historical user interaction data can be a log of multimodal interactions between the user and the smart home system over a historical period. For example, historical user interaction data may include the frequency of scene mode clicks in a smart home app on the user's mobile phone over a past period, records of manual adjustments made by the user to smart devices in the environment over a past period, and records of voice control commands given by the user to a smart speaker over a past period. The time range corresponding to the historical user interaction data can be set according to actual needs; for example, it can be set to obtain historical user interaction data from the past 30 days. This application does not impose specific limitations on this.

[0048] After acquiring historical user interaction data of online devices, the historical interaction intent can be determined based on this data. For example, historical interaction text from a past period can be extracted from the historical interaction data. This text can include voice interaction text and device status control command text. Voice interaction text could be a user complaining about dryness during interaction with a smart speaker, while device status control commands could be a user manually issuing a command to "adjust the humidifier's operating mode from level one to level two," or other interaction text; this application does not impose specific limitations on these. Then, the extracted historical interaction text can be converted into high-dimensional semantic vector embeddings to obtain historical interaction semantic vectors. These vectors are then subjected to unsupervised clustering using a clustering algorithm to obtain one or more clusters. The clustering algorithm can be K-Means or DBSCAN (Density-Based Spatial Clustering of Applications with Noise), and this application does not impose specific limitations on these. Subsequently, based on the clustering results, the historical interaction semantic vectors in the same cluster are input into a pre-trained LLM for semantic extraction and analysis, so as to obtain the historical interaction intent represented by the cluster.

[0049] Historical interaction intents can be used as node names in a causal knowledge graph to generate corresponding intent nodes. For example, after clustering, a certain cluster may contain multiple historical interaction semantic vectors such as "turn off the main light," "draw the curtains," "turn down the air conditioner," and "turn on Do Not Disturb for your phone." Inputting these vectors into a pre-trained LLM for semantic extraction and analysis yields the historical interaction intent "deep sleep," which can then be used to generate an intent node named "deep sleep" in the causal knowledge graph. Besides intent nodes obtained from user historical interaction data, the causal knowledge graph can also include pre-defined intent nodes; this application does not impose specific limitations on this.

[0050] Through the above implementation process, the intent nodes in the causal knowledge graph can be dynamically expanded based on the user's historical interaction data over a period of time, making the constructed causal knowledge graph more closely match the user's personalized intent.

[0051] The gateway or terminal can store multiple preset environmental physical variables, such as temperature, humidity, PM2.5, CO2 concentration, noise level, illuminance, TVOC (Total Volatile Organic Compounds), air pressure, etc. The environmental physical variable node corresponding to the intent node can be filtered from the preset environmental physical variables. For example, the environmental physical variable node corresponding to the intent node can be obtained through preset rule matching or through LLM inference; this application does not impose specific limitations on this.

[0052] Then, a causal knowledge graph can be constructed based on intent nodes, environmental physical variable nodes, and device operation nodes.

[0053] Through the above implementation process, intent nodes, environmental physical variable nodes, and device operation nodes can be identified to construct a causal knowledge graph, thereby associating user intent, environmental physical variables, and device operation. This breaks the one-way binding between instructions and devices in related technologies and endows the smart home system with the ability to understand the physical environment through common sense.

[0054] In some embodiments of this application, constructing the causal knowledge graph based on the intent node, the environmental physical variable node, and the device operation node includes: Establish the first causal edge between the intent node and the environmental physical variable node; Determine the probability distribution of environmental physical variables corresponding to the intent node based on the user's historical interaction data; The probability distribution of the environmental physical variables is used as the edge attribute of the first causal edge; Establish a second causal edge between the environmental physical variable node and the device operation node; Determine the direction of the influence of the device operation node on the environmental physical variable node; The edge attributes of the second causal edge are determined based on the direction of influence; The causal knowledge graph is constructed based on the intent node, the environmental physical variable node, the device operation node, the first causal edge, and the second causal edge.

[0055] In this embodiment, the environmental physical variable nodes that each intent node affects can be connected to establish a first causal edge between the intent node and the environmental physical variable nodes. The environmental physical variable nodes that the intent node affects can be determined by preset influence rules. For example, the preset influence rules can define that the intent node "deep sleep" needs to affect environmental physical variable nodes such as "humidity" and "temperature". Alternatively, a pre-trained LLM can be used to infer from common sense that the intent node "deep sleep" will affect environmental physical variable nodes such as "humidity" and "temperature". This application does not impose specific limitations on this.

[0056] For example, the current intent node and all environmental physical variable nodes can be input into a pre-trained LLM, so that the LLM outputs the correlation score between the intent node and each environmental physical variable node. For example, for the intent node of deep sleep, a correlation score result of "humidity 0.92, temperature 0.88, noise level 0.85, light intensity 0.80, CO2 concentration 0.65, PM2.5 0.12, TVOC 0.08" can be output. Then, environmental physical variable nodes whose correlation scores exceed a preset correlation score threshold can be marked as core relevant environmental nodes of the intent node. For example, the preset correlation score threshold can be set to 0.5, or it can be set separately according to actual needs; this application does not impose specific restrictions on this. A first causal edge can be established between the core relevant environmental nodes and the intent node. Through the above implementation process, the number of first causal edges can be effectively compressed, significantly reducing subsequent computational overhead while preserving the semantic information of the intent.

[0057] The probability distribution of environmental physical variables corresponding to an intent node can be determined based on historical user interaction data, and this distribution can be used as an edge attribute of the first causal edge. The probability distribution of environmental physical variables reflects the distribution probability of the corresponding environmental physical variable within multiple vector parameter intervals, and can be used to define the target state requirement of a specific intent for that environmental physical variable. For example, the environmental physical variable of humidity has multiple distribution intervals. Based on the edge attribute of the first causal edge between the intent node "deep sleep" and the environmental physical variable node "humidity," the distribution probability of humidity is highest in the 50%-55% range. This indicates that the optimal environmental physical state corresponding to the user intent "deep sleep" is humidity maintained at 50%-55%.

[0058] The device operation nodes that can be used to adjust each environmental physical variable node can be connected to establish a second causal edge between the environmental physical variable node and the device operation node. The device operation nodes that can be used by each environmental physical variable node can be determined by preset influence rules. For example, the environmental physical variable node "humidity" can be defined in the preset influence rules as being able to be adjusted by "air conditioning dehumidification" or "humidifier humidification", or it can be obtained by common sense reasoning using a pre-trained LLM. This application does not impose specific restrictions on this.

[0059] The direction of influence of device operation nodes on environmental physical variable nodes can be determined. The direction of influence of device operation nodes on environmental physical variable nodes can be positive or negative. A positive influence indicates that the device operation can purposefully adjust the parameter values ​​of the environmental physical variable, while a negative influence indicates that the device operation has a non-purposeful negative impact on the parameter values ​​of the environmental physical variable. For example, turning on a robot vacuum cleaner can purposefully increase the cleanliness of the floor; the direction of influence corresponding to the second causal edge between the robot vacuum cleaner turning on node and the floor cleanliness node is positive. However, turning on a robot vacuum cleaner can also non-purposefully increase the noise level in the environment; the direction of influence corresponding to the second causal edge between the robot vacuum cleaner turning on node and the noise level node is negative. Subsequently, the direction of influence of the device operation node on the environmental physical variable node can be used as an edge attribute of the second causal edge. Thus, a causal knowledge graph is constructed based on the intent node, the environmental physical variable node, the device operation node, the first causal edge, and the second causal edge. For example, the edge attribute of the second causal edge can also include the target of the device operation node's influence on the environmental physical variable node, which can include increasing or decreasing. For instance, the device operation of air conditioning dehumidification will decrease the value of the environmental physical variable parameter humidity, and its corresponding target is decrease. The device operation of air conditioning heating will increase the value of the environmental physical variable parameter temperature, and its corresponding target is increase.

[0060] Through the above implementation process, causal edges between nodes can be further constructed based on the nodes. The edge attributes of the causal edges can be used to represent the user's intentions on environmental physical variables and the impact of various device operations on environmental physical variables, thereby obtaining a causal knowledge graph. This allows for semantic mapping of device capabilities and abstracts specific device operations into causal chains of physical environment adjustment intentions. This breaks the one-way binding between instructions and devices in related technologies, endows smart home systems with common sense understanding of the physical environment, and realizes the improvement from human device control to smart home systems automatically meeting user intentions.

[0061] In some embodiments of this application, determining the probability distribution of environmental physical variables corresponding to the intent node based on the user's historical interaction data includes: Obtain the baseline probability distribution of the environmental physical variables corresponding to the intent node; Based on the user's historical interaction data, a Bayesian update is performed on the baseline probability distribution of the environmental physical variables to obtain the probability distribution of the environmental physical variables corresponding to the intent node.

[0062] In this embodiment, the baseline probability distribution of environmental physical variables corresponding to an intent node can be obtained. For example, the baseline probability distribution of environmental physical variables corresponding to each intent node can be obtained using an LLM based on a preset common sense knowledge base. If the information in the common sense knowledge base indicates that the optimal humidity for the intent node "deep sleep" is between 50% and 60%, then the humidity distribution within this range can be assigned the highest probability. The baseline distribution range of environmental physical variables corresponding to each intent node can also be determined based on the common sense knowledge base; for example, the baseline distribution range for humidity can be [<40%], [40%-50%], [50%-60%], or [>60%].

[0063] Then, the user's interaction behavior in the intent scenario corresponding to the intent node can be extracted from the user's historical interaction data. The extracted interaction behavior in the intent scenario is used as feedback information. The baseline probability distribution of the environmental physical variables is updated and corrected by the Bayesian update algorithm to obtain the probability distribution of the environmental physical variables corresponding to the intent node.

[0064] The process of updating the baseline probability distribution of environmental physical variables using a Bayesian update algorithm may include the following steps: The baseline probability distribution of environmental physical variables is used as a prior parameter of the Dirichlet distribution. For example, the baseline probability distribution of environmental physical variables can be expressed as "the probability distribution of humidity intervals x1 [<40%], x2 [40%-50%], x3 [50%-60%], x4 [>60%] [0.05, 0.15, 0.70, 0.10]".

[0065] This system can identify the target preference represented by each manual adjustment action in a user's interaction within a given intent scenario. For example, if a user adjusts the humidifier from level 1 to level 3, it can be inferred that the user desires higher humidity. The user's target preference is then defined as the humidity range corresponding to x3 and x4, and this observation is recorded as a positive vote for the x3 and x4 range, incrementing the count of that range by 1. If the user turns on dehumidification mode, it can be inferred that the user desires lower humidity, defining the user's target preference as the humidity range corresponding to x1 and x2, and recording this observation as a positive vote for the x1 and x2 range. Finally, a voting statistics result can be obtained. Based on this voting statistics, a likelihood function can be constructed. This likelihood function describes the probability of a user's actions when faced with a baseline probability distribution of environmental physical variables.

[0066] After accumulating interaction data over a certain period, the positive vote counts corresponding to each interval can be used as observation data to update the posterior parameters of the Dirichlet distribution, recalculate the expected probability of each interval, and achieve a smooth migration of the target distribution to the user's true preference. The accumulation period can be 7 days or 30 days, which can be set according to actual needs, and this application does not impose specific restrictions on it. For example, after accumulating feedback from 20 "adjust humidifier" interactions over 30 days, the updated posterior parameter distribution may become [0.02, 0.08, 0.55, 0.35], with a significant increase in the distribution probability of the x4 interval, reflecting the user's personalized preference for higher humidity.

[0067] Through the above implementation process, the probability distribution of environmental physical variables corresponding to each intent node can be dynamically determined based on common sense and user historical interaction data. This makes the obtained probability distribution of environmental physical variables more aligned with the user's personalized preferences, helping to ensure that device control based on causal knowledge graphs can meet user intents and guarantee user comfort.

[0068] For example, a local area network (LAN) device discovery protocol can periodically monitor the network status of the current LAN to identify newly added or replaced devices in the current environment in real time. Upon detecting an unregistered MAC (Media Access Control) address and device type identifier, it can be determined that a device has been added / replaced. Subsequently, the device model of the added / replaced device can be obtained and the corresponding device operation node can be identified. An LLM (Local Level Management) can be invoked to extract semantic features from the newly added / replaced device operation node, identifying what environmental physical variables the operation node can change, thereby determining the environmental physical variable nodes connected to the newly added device operation node and generating corresponding second causal edges. For example, an LLM can be invoked to extract the semantic features of "SetMistLevel," identifying that this action has the ability to change environmental humidity. Subsequently, a positive causal edge from the "humidifier adjustment node" to the "humidity node" can be automatically generated in the causal knowledge graph. When the old humidifier breaks down and goes offline, the system can use a causal knowledge graph to find nodes that affect "humidity," automatically tracing the causal edges to locate the new humidifier and calling its API. The entire process requires no manual configuration of scene rules by the user.

[0069] Through the above implementation process, when adding or replacing smart devices in a home, the system can automatically scan for available new devices in the home's local area network, enabling adaptive matching of control rules and dynamic updates of the causal knowledge graph, thereby improving the robustness of the smart home system.

[0070] Reference Figure 2 This is a schematic diagram of the causal knowledge graph hierarchy of an embodiment of a device control method of this application.

[0071] In some embodiments of this application, the causal knowledge graph can be divided into a three-layer structure according to the node type: the intent layer, the physical variable layer, and the executor layer.

[0072] The intent layer can be associated with intent nodes in a causal knowledge graph. Intent nodes can be obtained by acquiring user behavior data and parsing it using LLM (Limited Language Management).

[0073] The physical variable layer can correspond to environmental physical variable nodes in a causal knowledge graph. Environmental physical variable nodes can include various types such as humidity, temperature, and PM2.5.

[0074] The actuator layer can correspond to device operation nodes in a causal knowledge graph. A device operation node can contain a collection of device operations from multiple actuators. Actuators can include humidifiers, robot vacuums, and fresh air systems in a smart home system. For example, if a humidifier malfunction is detected, the causal knowledge graph can be used to find relevant alternative devices, such as robot vacuums with wet mopping capabilities and fresh air systems with water-locking functions.

[0075] For example, it can be determined whether the target smart device has malfunctioned based on its feedback signals. These feedback signals may include heartbeat timeout signals, offline signals, or specific error codes. When a gateway or terminal sends a heartbeat signal to the target smart device, but the device fails to respond within the timeout period, a heartbeat timeout signal indicating the target smart device's status can be obtained. When the gateway or terminal detects that the target smart device has lost connection, an offline signal indicating the target smart device's status can be obtained. When the target smart device successfully receives control commands from the gateway or terminal but malfunctions, it can send a specific error code to the gateway or terminal to indicate which function of the target smart device has malfunctioned. For example, if the gateway or terminal sends a humidification control command to the humidifier, and the humidifier returns an error code indicating that the water tank is low or the filter is clogged, it can report an malfunction to the gateway or terminal in the humidifier's humidification operation.

[0076] In some embodiments of this application, determining the set of functional substitutes corresponding to the target intelligent device based on the causal knowledge graph includes: Based on the target intelligent device and the target intent, the corresponding target device operation node is determined from the causal knowledge graph; Traverse the causal knowledge graph to determine the environmental physical variable node corresponding to the target device operation node as the target environmental physical variable node; Based on the target environment physical variable nodes, search for available device operation nodes in the causal knowledge graph; The set of functional alternatives is generated based on the available device operation nodes.

[0077] In this embodiment, the corresponding target device operation node can be determined from the causal knowledge graph based on the target intelligent device and the target intent. For example, during the causal knowledge graph construction phase, the causal connections between the device operation node, intent node, and environmental physical variable node—namely, the first causal edge and the second causal edge—are already stored in the graph database. The corresponding target operation node can be quickly located by querying the graph database based on the target intelligent device and the target intent.

[0078] Subsequently, a graph traversal algorithm can be used to traverse the causal knowledge graph to determine the environmental physical variable nodes corresponding to the target device operation node as the target environmental physical variable nodes. For example, after determining the target device operation node, a breadth-first search can be performed to find all second causal edges of the target device operation node in the causal knowledge graph, thereby determining the target environmental physical variable nodes affected by the target device operation node. Correspondingly, the corresponding first causal edges can be determined based on the target intent and the target environmental physical variable nodes. The probability distribution of the target environmental physical variables can be obtained from the edge attributes of the first causal edges, thus clarifying the specific target that needs to be compensated—that is, finding the corresponding functional alternative device to maintain the desired environmental physical variable distribution for the intended intent.

[0079] After identifying the target environment physical variable node, the device operation nodes in the actuator layer can be traversed in the causal knowledge graph centered on the target environment physical variable node to find available device operation nodes. For example, a breadth-first search can be used to find all available device operation nodes in the causal knowledge graph, excluding the target device operation node, that can have a matching influence on the target environment physical variable node. Whether the influence of the available device operation node and the target device operation node on the target environment physical variable node matches can be determined based on the edge attributes of the corresponding second causal edge. If the edge attributes are the same, then the influence of the available device operation node and the target device operation node on the target environment physical variable node can be considered to match.

[0080] Then, the searched available device operation nodes can be idempotently generated to produce one or more sets of function substitutions. Idempotent generation refers to traversing all possible subset combinations of available device operation nodes. For example, if the searched available device operation nodes include operation node A of device 1 and operation node B of device 2, then the combinations of available device operation nodes obtained after idempotent generation are S1={A}, S2={B}, and S3={A,B}. Correspondingly, the function substitution sets are device 1, device 2, device 1, and device 2, respectively.

[0081] Through the above implementation process, a set of functional alternatives that can replace the original target smart device to achieve the user's intended purpose can be searched from the devices in the smart home system based on causal knowledge graphs. Providing as many alternatives as possible, including combined alternatives, can avoid situations where a single device is insufficient to achieve the intended purpose.

[0082] Reference Figure 3 This is a flowchart illustrating the steps of another embodiment of the device control method of this application. In some embodiments of this application, determining the functional substitution cost corresponding to each set of functional substitutions includes: Step 301: Obtain the estimated operating power of the functional replacement device in each functional replacement set; The estimated operating power can be the power consumption value of the functional replacement equipment per unit time. The unit time can be 1 minute, 1 hour, or can be set according to actual needs; this application does not impose specific restrictions on this.

[0083] Step 302: Determine the probability distribution of predicted environmental physical variables for the functional replacement device; Among them, the predicted probability distribution of environmental physical variables can represent the expected probability distribution of environmental physical variables that can be achieved by using functional replacement devices to replace the abnormal target intelligent devices.

[0084] Step 303: Determine the first causal edge corresponding to the target intent based on the causal knowledge graph; Step 304: Obtain the probability distribution of the physical variables of the target environment based on the first causal edge corresponding to the target intent; Step 305: Determine the environmental distribution similarity corresponding to the functional replacement device based on the predicted environmental physical variable probability distribution and the target environmental physical variable probability distribution; Among them, the environmental distribution similarity can reflect the difference between the probability distribution of environmental physical variables that can be achieved by the operation of the control function substitute equipment and the probability distribution of the ideal environmental physical variables corresponding to the target intention.

[0085] Step 306: Determine the second causal edge corresponding to the functional replacement device based on the causal knowledge graph; Step 307: Determine the device penalty value of the functional replacement device based on the second causal edge corresponding to the functional replacement device; The equipment penalty value reflects the negative effects of replacing equipment with a function. These negative effects can include increased noise levels and enhanced vibration.

[0086] Step 308: Determine the function substitution cost corresponding to each function substitution set based on the estimated operating power, the environmental distribution similarity, and the equipment penalty value.

[0087] In this embodiment, the physical model corresponding to the functional replacement device in each functional replacement set can be parsed, and the rated power of each functional replacement device obtained from the parsing can be used as the estimated operating power of each functional replacement device. Alternatively, the historical average operating power of the functional replacement devices in each functional replacement set over a past period can be read from a smart socket in the current environment and used as the estimated operating power of each functional replacement device. For example, the time range read is typically in days and can be set according to actual needs; this application does not impose specific limitations on this.

[0088] It is possible to determine the probability distribution of predicted environmental physical variables for functional replacement devices, and also to determine the first causal edge corresponding to the target intent based on the causal knowledge graph, and obtain the probability distribution of target environmental physical variables based on the first causal edge corresponding to the target intent. For example, the corresponding first causal edge can be determined based on the target intent and the target environmental physical variable nodes, and the probability distribution of target environmental physical variables can be obtained from the edge attributes of the first causal edge.

[0089] Subsequently, the environmental distribution similarity corresponding to the functional replacement device can be determined based on the probability distributions of the predicted and target environmental physical variables. For example, the environmental distribution similarity corresponding to the functional replacement device can be determined by calculating the distribution divergence divergence (KL divergence) between the probability distributions of the predicted and target environmental physical variables. The environmental distribution similarity can be expressed by the following formula:

[0090] in, It can represent the divergence between the probability distributions of predicted environmental physical variables and the probability distributions of target environmental physical variables, that is, the similarity of the environmental distributions corresponding to the functional replacement equipment; It can represent the probability distribution of physical variables in the target environment; This can represent the probability distribution of predicted environmental physical variables; x can represent the discretized state interval of that environmental physical variable. For example, for the environmental physical variable of humidity, its discretized state interval can be divided into x1 (40%-50%) representing the current humidity is relatively dry, x2 (50%-60%) representing the current humidity is comfortable, and x3 (60%-70%) representing the current humidity is relatively humid. The smaller the value of this distributional divergence, the closer the environmental physical variable adjustment effect achieved by the corresponding functional alternative device or combination of functional alternative devices is to the effect intended to be achieved.

[0091] For example, suppose a perfect humidifier can stabilize a bedroom in a comfortable zone, with the corresponding probability distribution of the target environmental physical variables being x1=0.1 (slightly dry), x2=0.8 (comfortable), and x3=0.1 (slightly damp). Meanwhile, a robot vacuum cleaner's mopping effect is weaker, with the corresponding probability distribution of the predicted environmental physical variables being x1=0.5 (slightly dry), x2=0.4 (comfortable), and x3=0.1 (slightly damp). Then, the environmental distribution similarity of the corresponding functional substitute devices is: .

[0092] The calculated value of 0.568 represents the environmental distribution similarity corresponding to using a robotic vacuum cleaner as a single-function replacement device. If the environmental distribution similarity obtained from other functional replacement sets is lower than 0.568, it is considered that the environmental physical variable adjustment effect achieved by other functional replacement sets is closer to the ideal state.

[0093] The second causal edge corresponding to the functional replacement device can be determined based on the causal knowledge graph, and then the device penalty value of the functional replacement device can be determined based on the second causal edge corresponding to the functional replacement device.

[0094] The function substitution cost for each set of function substitutions can be determined based on estimated operating power, environmental distribution similarity, and equipment penalty values. For example, the function substitution cost can be represented by the following parameters:

[0095] in, It can represent the cost of functional substitution corresponding to the set of functional substitutions S. It can represent the estimated operating power of each functional replacement device in the functional replacement set S; It can represent the similarity of environmental distribution corresponding to functional replacement devices; It can represent the device penalty value of each functional substitute device in the functional substitute set S; It can represent the energy consumption weighting coefficient, reflecting the system's sensitivity to the electrical energy consumption of equipment; It can represent the environmental distribution similarity weight coefficient, reflecting the system's sensitivity to the environmental distribution that the equipment can achieve; It can represent the negative penalty weight coefficient, which can reflect the system's tolerance limit for negative environmental physical variables; i can represent the order of functional substitution devices in the functional substitution set S.

[0096] For example, in In this system, α, β, and γ are multiplied by sub-items of different dimensions. The energy consumption item corresponding to the estimated operating power is in watts, while the environmental distribution similarity item and the equipment penalty value item are dimensionless. The weighting coefficients independently adjust the contribution intensity of each sub-item to the total cost. Therefore, the weighting parameters α, β, and γ do not need to satisfy normalization constraints and can be pre-set with reasonable weighting ranges to ensure that the contribution proportion of the three sub-items in the total cost is within a reasonable range. For example, the weighting parameters can be set as integers from 0 to 100. The weighting parameters α, β, and γ can be dynamically determined based on the real-time state of the current environment. Multidimensional state parameters of the current environment can be obtained in real time. These parameters include, but are not limited to, the current timestamp, the current total household power load, and resting heart rate data from user status sensors such as smartwatches. The multidimensional state parameters of the current environment can be input into the system's built-in fuzzy logic controller, allowing the controller to determine the current weighting parameters based on a preset fuzzy control rule base. The preset fuzzy control rule base can be obtained by combining preset expert rules with LLM based on long-term user habits determined by user historical interaction data.

[0097] For example, when the current time is determined to be 23:00 and the user's heart rate is at rest based on the multidimensional state parameters of the current environment, the "Do Not Disturb" sleep rule is triggered. The fuzzy logic controller can output an instruction to directly increase γ to 80 and decrease α to 10, thereby guiding the function substitution cost calculation to favor the strategy of "preferring higher power consumption for absolute silence". Several examples of weight parameter settings in different scenarios are given below, but they are not intended to limit this application.

[0098] Table 1, Example Table of Scene-Weight Parameters.

[0099]

[0100] In some embodiments of this application, determining the probability distribution of predicted environmental physical variables for the functional replacement device includes: Obtain the current environment state vector and the physical model of the functional replacement device; Determine the state transition matrix corresponding to the functional replacement device based on the object model; The probability distribution of predicted environmental physical variables for the functional replacement device is determined based on the environmental state vector and the state transition matrix.

[0101] In this embodiment, an environmental state vector of the current environment and a physical model of a functional replacement device can be obtained. The environmental state vector of the current environment can be an environmental state corresponding to the probability distribution of physical variables of the target environment, collected by sensors or smart devices. For example, the humidity of the current environment can be collected by a humidity sensor, obtaining a humidity level of 42% in the bedroom, which falls within the x1 relatively dry range, thus forming the corresponding environmental state vector S of the current environment. current =[1,0,0] represents a probability of 1 for being in x1. The object model of the functional replacement device can be obtained by sending an object model file retrieval request to the cloud IoT platform.

[0102] The operating state of the functional replacement device within a preset operating time can be determined based on the physical model. The impact of the functional replacement device on the environmental physical variables of the current environment within the preset operating time can then be abstracted into a state transition matrix based on the operating state. The preset operating time can be set to half an hour, one hour, or otherwise, depending on actual needs; this application does not impose specific limitations on this. For example, the operating state can be related to the probability distribution of the target environmental physical variables. For instance, for the humidity probability distribution, the corresponding operating state could be humidification, dehumidification, or ventilation. The impact of the functional replacement device on the environmental physical variables of the current environment within the preset operating time can be determined in advance based on an equipment capability-environmental impact mapping table obtained from experimental data or expert experience. The equipment capability-environmental impact mapping table can be obtained by comprehensively fitting standard test data under laboratory conditions, equipment physical parameters (nominal values ​​such as power, air volume, and humidification capacity), and historical operating data of similar equipment to obtain a standard transition matrix template for each equipment category, and then adjusting it linearly based on the physical model parameters of the specific equipment.

[0103] For example, if the ambient humidity in the bedroom is determined to be in the slightly dry range (x1) by sensors, by obtaining the object model of the robot vacuum cleaner, it can be determined that the robot vacuum cleaner has a small water tank capacity and no spray function. Combined with a preset device capability-environmental impact mapping table, it can be determined that the probability of the robot vacuum cleaner maintaining the environment at x1 after working for half an hour is 60%, the probability of improving to x2 (comfortable range) is 40%, and the probability of reaching x3 (slightly humid range) is 0%. The state transition matrix can then be obtained. The state transition matrix can be represented as:

[0104] in This can represent the state transition matrix for a robotic vacuum cleaner. The i-th row and j-th column of the matrix represents the probability of transitioning from state i to state j after the device's action. The specific values ​​of each probability in the transition matrix are determined based on a preset device capability-environmental influence mapping table.

[0105] The predicted environmental physical variable probability distribution of the functional replacement device can then be obtained by multiplying the environmental state vector and the state transition matrix. For example, the predicted environmental physical variable probability distribution can be calculated using the following formula:

[0106] in, It can represent the probability distribution of predicted environmental physical variables. It can represent an environment state vector; It can represent the state transition matrix corresponding to the robot vacuum cleaner.

[0107] For example, when the set of functional substitutions includes multiple functional substitution devices, the comprehensive state transition matrix corresponding to the set of functional substitutions can be determined by multiplying the state transition matrices of the multiple functional substitution devices. Suppose a set of functional substitutions includes a robot vacuum cleaner and a fresh air system, corresponding to the robot vacuum cleaner's wet mopping operation and the fresh air system's reduced air exchange rate operation, respectively. Then, the comprehensive state transition matrix corresponding to this set of functional substitutions can be expressed as:

[0108] in, It can represent the comprehensive state transition matrix corresponding to the functional substitution set; This can represent the state transition matrix corresponding to the fresh air system; This can represent the state transition matrix corresponding to the robot vacuum cleaner. The order of multiplication of functional replacement devices is usually arranged according to the nature of the influence of the functional replacement devices on environmental physical variables. First, the direct effects are executed (such as the robot vacuum cleaner increasing the ambient humidity by wet mopping), and then the maintenance effects are superimposed (such as the fresh air system reducing the air exchange rate to lock in the humidity).

[0109] Through the above implementation process, the probability distribution of predicted environmental physical variables of functional replacement equipment can be dynamically determined in combination with the current environmental state, thereby determining the possible impact of adopting different sets of functional replacements on the environmental physical variables of the current environment, which helps to improve the selection accuracy of target functional replacement equipment.

[0110] In some embodiments of this application, determining the device penalty value of the functional replacement device based on the second causal edge corresponding to the functional replacement device includes: Obtain the edge attributes of the second causal edge corresponding to the functional replacement device; From the edge attributes, determine the environmental physical variable nodes whose influence direction is negative corresponding to the functional replacement device as target penalty nodes; Determine the preset penalty parameter benchmark based on the target intent and the target penalty node; Obtain the environmental physical variable parameters corresponding to the target penalty node when the functional replacement device is running; The device penalty value of the functional replacement device is determined based on the difference between the penalty parameter benchmark and the environmental physical variable parameter, as well as the preset unit penalty value.

[0111] In this embodiment, the edge attributes of the second causal edge corresponding to the functional replacement device can be obtained from the causal knowledge graph. Then, the environmental physical variable node with a negative influence direction corresponding to the functional replacement device can be determined as the target penalty node from the edge attributes. The target penalty node can typically be a node with a negative effect, such as CO2 concentration, noise decibels, PM2.5, or TVOC. The influence target in the edge attributes of the second causal edge corresponding to the functional replacement device is usually to increase.

[0112] After determining the target penalty point, a preset penalty parameter benchmark can be determined based on the target intent and the target penalty point. The penalty parameter benchmark can be predetermined based on expert experience or common sense. For example, when the target intent is related to sleep, since 40dB is considered a relatively quiet and normal environment, in acoustic and health standards, 40dB is often regarded as the threshold between quiet and disturbance, and is the recommended upper limit volume for ensuring quality sleep. Therefore, the corresponding noise decibel penalty parameter benchmark can be set to 40dB.

[0113] Obtain the environmental physical variable parameters corresponding to the target penalty node when the alternative device is running. Assuming the alternative device is a robotic vacuum cleaner and the target penalty node is a noise level in decibels, the environmental physical variable parameters corresponding to the target penalty node when the alternative device is running are the operating decibels of the robotic vacuum cleaner. For example, the environmental physical variable parameters corresponding to the target penalty node can be obtained by parsing the object model of the alternative device. For instance, the operating decibel parameters of the robotic vacuum cleaner after startup (e.g., noiseLevel: 45dB) can be determined by parsing the object model of the robotic vacuum cleaner. For example, if the object model of the alternative device does not contain relevant parameter fields, an impact assessment value can be generated based on a common-sense knowledge base according to the device type of the alternative device.

[0114] The device penalty value for the functional replacement device is determined based on the difference between the penalty parameter benchmark and the environmental physical variable parameter, as well as a preset unit penalty value. The preset unit penalty value can be set to 1, or it can be set according to the system's sensitivity to negative environmental impacts; this application does not impose specific limitations on this. For example, the environmental physical variable parameter corresponding to the robotic vacuum cleaner could be 45dB, the corresponding penalty parameter benchmark could be 40dB, and the preset unit penalty value could be set to 1, resulting in a device penalty value of 5.

[0115] Through the above implementation process, the negative environmental impact of using various functional replacement devices can be determined based on the causal knowledge graph, and the corresponding device penalty value can be calculated as a component of the compensation cost of using different replacement schemes. This can effectively avoid schemes with excessive negative environmental impact when selecting target functional replacement devices, and improve the accuracy of target functional replacement device selection.

[0116] Reference Figure 4 This is a flowchart illustrating the seamless self-healing process for home appliances malfunctions, according to an embodiment of the device control method of this application.

[0117] Anomaly Detection: Once the system determines the user's target intent, it can identify the master device that can satisfy the user's intent from the causal knowledge graph as the corresponding target smart device, and attempt to wake up the corresponding target smart device as originally planned. If the system receives anomaly information from the sensor after attempting to wake up the target smart device, it can determine that the corresponding master device is offline or malfunctioning. For example, if the user's target intent is to increase the humidity in the bedroom, the system can search the causal knowledge graph and attempt to wake up the bedroom humidifier as originally planned, but the sensor reports a "water tank low" alarm. At this point, the system can determine that the original main execution path has been interrupted.

[0118] Path Search: The system can use a graph traversal algorithm to perform path search in the causal knowledge graph, obtaining candidate solutions that can functionally replace the abnormal target intelligent device system, resulting in a set of functional replacements. For example, the system can immediately search the causal knowledge graph for nodes that positively influence "humidity," and based on other available devices within the home network, generate the following candidate replacement paths (i.e., the set of functional replacements): Path A (Air Conditioner Dehumidification to Air Supply Mode): Utilizing residual condensate from the air conditioner evaporator. Path B (Robot Mopping): Controlling the robot to perform a "high-volume" silent mopping along the edge of the bedroom. Path C (Robot Mopping + Lowering the Fresh Air System's Air Exchange Rate): The robot mops while reducing the fresh air volume to lock in humidity.

[0119] Cost calculation: Once the set of functional alternatives is determined, the cost of each functional alternative can be calculated for each set of functional alternatives. For example, path A, estimated. Path B, estimated Path C, combined action prediction Combining the estimated operating power of the air conditioner, robot vacuum cleaner, and fresh air system along the three paths determined from the object model, and incorporating the high noise penalty weight of the current sleep scenario, a global [process] is performed. Calculation. Because the humidification effect of path A is too weak, it cannot maintain humidity ( With a divergence as high as 0.6, the cost of haptic distortion becomes too high. For path B, relying solely on a robotic vacuum cleaner to achieve the required humidity, frequent mopping is necessary, generating mechanical noise. )exist High pressure can lead to excessively high overall costs. Path C involves the coordinated adjustments of two devices: fresh air helps retain moisture, and the robot vacuum only needs to replenish water at low frequencies, thus ensuring proper humidity distribution. Extremely low, and the adjustment of the fresh air system is almost imperceptible. The overall noise level will not be too high, making it the lowest value for a general agent.

[0120] Dynamic arbitration: The system ultimately selects path C, which has the lowest value among the general agents, as the optimal compensation path.

[0121] Execution and Feedback: The home gateway silently sends commands to the corresponding devices in path C, simultaneously initiating wet mopping and modifying the fresh air configuration. Users require absolutely no intervention while asleep, maintaining a consistently comfortable humidity level, achieving seamless self-healing in the event of device malfunction.

[0122] Through the above implementation process, when the main control device that meets the user's intentions in the home environment malfunctions, the system can automatically search for and combine other available cross-category devices through a dynamically constructed causal knowledge graph to replace the malfunctioning device, ensuring that the user's core needs are not interrupted.

[0123] It should be noted that the device control method provided in this application embodiment can be executed by a device control device, or a control module within the device control device for executing the loading device control method. This application embodiment uses the execution of the loading device control method by a device control device as an example to illustrate the device control method provided in this application embodiment.

[0124] Reference Figure 5 This is a structural block diagram of an embodiment of a device control apparatus according to this application. The apparatus includes: Module 501 is used to construct causal knowledge graphs; The listening module 502 is used to listen to user intent signals in the current environment to determine the user's target intent; The target intelligent device determination module 503 is used to determine the target intelligent device based on the target intent and the causal knowledge graph. Functional substitution set determination module 504 is used to determine the functional substitution set corresponding to the target intelligent device based on the causal knowledge graph when the target intelligent device is abnormal; each functional substitution set includes one or more functional substitution devices. Functional substitution cost determination module 505 is used to determine the functional substitution cost corresponding to each set of said functional substitutions; The target function replacement device determination module 506 is used to determine a target function replacement device from the function replacement set based on the function replacement cost; The control module 507 is used to generate control instructions according to the target intent, so as to control the target function replacement device according to the control instructions.

[0125] The causal knowledge graph includes: a device operation node, an intent node, an environmental physical variable node, a first causal edge between the intent node and the environmental physical variable node, and a second causal edge between the environmental physical variable node and the device operation node; the first causal edge is used to indicate the environmental physical variable node that each intent node acts upon; the second causal edge is used to indicate the environmental physical variable node that each device operation node acts upon.

[0126] The building module includes: The scanning submodule is used to scan the current local area network (LAN) to identify online devices in the LAN. The object model file acquisition submodule is used to acquire the object model file of the online device; The device operation node generation submodule is used to generate the device operation node corresponding to the online device based on the object model file. The user history interaction data acquisition submodule is used to acquire the user history interaction data corresponding to the online device; The historical interaction intent determination submodule is used to determine the historical interaction intent based on the user's historical interaction data. The intent node generation submodule is used to generate intent nodes based on the historical interaction intents; The environmental physical variable node determination submodule is used to determine the environmental physical variable node corresponding to the intent node based on preset environmental physical variables. A submodule is constructed to build the causal knowledge graph based on the intent node, the environmental physical variable node, and the device operation node.

[0127] The construction submodule includes: The first causal edge establishment unit is used to establish a first causal edge between the intention node and the environmental physical variable node; An environmental physical variable probability distribution determination unit is used to determine the environmental physical variable probability distribution corresponding to the intent node based on the user's historical interaction data. The first edge attribute determination unit is used to use the probability distribution of the environmental physical variables as the edge attribute of the first causal edge. The second causal edge establishment unit is used to establish a second causal edge between the environmental physical variable node and the device operation node; An influence direction determination unit is used to determine the influence direction of the device operation node on the environmental physical variable node; The second edge attribute determination unit is used to determine the edge attribute of the second causal edge according to the influence direction; The construction unit is used to construct the causal knowledge graph based on the intent node, the environmental physical variable node, the device operation node, the first causal edge, and the second causal edge.

[0128] The environmental physical variable probability distribution determination unit is further used for: Obtain the baseline probability distribution of the environmental physical variables corresponding to the intent node; Based on the user's historical interaction data, a Bayesian update is performed on the baseline probability distribution of the environmental physical variables to obtain the probability distribution of the environmental physical variables corresponding to the intent node.

[0129] The functional substitution set determination module includes: The target device operation node determination submodule is used to determine the corresponding target device operation node from the causal knowledge graph based on the target intelligent device and the target intent; The target environment physical variable node determination submodule is used to traverse the causal knowledge graph to determine the environmental physical variable node corresponding to the target device operation node as the target environment physical variable node; The search submodule is used to search for available device operation nodes in the causal knowledge graph based on the target environment physical variable nodes. The function alternative set generation submodule is used to generate the function alternative set based on the available device operation nodes.

[0130] The function substitution cost determination module includes: The estimated operating power acquisition submodule is used to acquire the estimated operating power of the functional alternative device in each of the functional alternative sets; The submodule for determining the probability distribution of predicted environmental physical variables is used to determine the probability distribution of predicted environmental physical variables of the functional replacement device; The first causal edge determination submodule is used to determine the first causal edge corresponding to the target intent based on the causal knowledge graph. The target environment physical variable probability distribution determination submodule is used to obtain the target environment physical variable probability distribution based on the first causal edge corresponding to the target intent; The environmental distribution similarity determination submodule is used to determine the environmental distribution similarity corresponding to the functional replacement device based on the probability distribution of the predicted environmental physical variables and the probability distribution of the target environmental physical variables; The second causal edge determination submodule is used to determine the second causal edge corresponding to the functional replacement device based on the causal knowledge graph. The device penalty value determination submodule is used to determine the device penalty value of the functional replacement device based on the second causal edge corresponding to the functional replacement device. The function substitution cost determination submodule is used to determine the function substitution cost corresponding to each function substitution set based on the estimated operating power, the environmental distribution similarity, and the equipment penalty value.

[0131] The submodule for determining the probability distribution of predicted environmental physical variables includes: The object model acquisition unit is used to acquire the environmental state vector of the current environment and the object model of the functional replacement device; A state transition matrix determination unit is used to determine the state transition matrix corresponding to the functional replacement device based on the object model. The predictive environmental physical variable probability distribution determination unit is used to determine the predictive environmental physical variable probability distribution of the functional replacement device based on the environmental state vector and the state transition matrix.

[0132] The device penalty value determination submodule includes: An edge attribute acquisition unit is used to acquire the edge attributes of the second causal edge corresponding to the functional replacement device; The target penalty node determination unit is used to determine, from the edge attributes, the environmental physical variable node with a negative influence direction corresponding to the functional replacement device as the target penalty node; The penalty parameter benchmark determination unit is used to determine a preset penalty parameter benchmark based on the target intent and the target penalty node; An environmental physical variable parameter acquisition and determination unit is used to acquire environmental physical variable parameters corresponding to the target penalty node when the functional replacement device is running; The equipment penalty value determination unit is used to determine the equipment penalty value of the functional replacement equipment based on the difference between the penalty parameter benchmark and the environmental physical variable parameter and a preset unit penalty value.

[0133] The device control device in this application embodiment can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not impose specific limitations.

[0134] The device control device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system used.

[0135] The device control apparatus provided in this application embodiment can achieve... Figures 1 to 4 The various processes implemented by the device control device in the method embodiment will not be described again here to avoid repetition.

[0136] The device control apparatus provided in this application can construct a causal knowledge graph and monitor user intent signals in the current environment to determine the user's target intent. Then, based on the target intent and the causal knowledge graph, a target intelligent device for realizing the user's target intent is determined. In the event of an anomaly in the target intelligent device, a set of functional alternatives corresponding to the target intelligent device can be automatically determined based on the causal knowledge graph. Each set of functional alternatives may include one or more functional alternative devices, thereby obtaining one or more alternative solutions to replace the faulty device in realizing the user's target intent. Then, the functional alternative cost corresponding to each functional alternative set can be determined. Based on the functional alternative cost, a target functional alternative device is determined from the functional alternative set, resulting in the final device object used to realize the user's target intent. Finally, a control command is generated based on the target intent to control the target functional alternative device. This solves the problems of fragile and low-tolerance rules in smart home automation systems in related technologies. In the event of an anomaly in the execution device of a smart home system, a target functional alternative device for replacing the abnormal device's function can be dynamically determined based on the functional alternative cost, without requiring manual configuration or adjustment by the user, thus improving the robustness of the smart home system.

[0137] Reference Figure 6 This is a structural block diagram of an embodiment of an electronic device according to this application. This application also provides an electronic device including a processor 602, a memory 601, and a program or instruction 6011 stored in the memory 601 and executable on the processor 602. When executed by the processor 602, this program or instruction implements the various processes of the above-described device control method embodiments and achieves the same technical effects. To avoid repetition, further details are omitted here.

[0138] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0139] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described device control method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0140] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0141] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0142] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0143] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A device control method, characterized in that, The method includes: Constructing a causal knowledge graph; Listen for user intent signals in the current environment to determine the user's target intent; The target intelligent device is determined based on the target intent and the causal knowledge graph; In the event that the target intelligent device is abnormal, a set of functional alternatives corresponding to the target intelligent device is determined based on the causal knowledge graph; each set of functional alternatives includes one or more functional alternative devices. Determine the functional substitution cost corresponding to each set of functional substitutions; the functional substitution cost is a weighted sum of the estimated operating power of the functional substitution equipment, the environmental distribution similarity, and the equipment penalty value. The target functional replacement device is determined from the set of functional replacements based on the functional replacement cost; Control instructions are generated based on the target intent to control the target function replacement device according to the control instructions.

2. The method according to claim 1, characterized in that, The causal knowledge graph includes: a device operation node, an intent node, an environmental physical variable node, a first causal edge between the intent node and the environmental physical variable node, and a second causal edge between the environmental physical variable node and the device operation node; the first causal edge is used to indicate the environmental physical variable node that each intent node acts upon; the second causal edge is used to indicate the environmental physical variable node that each device operation node acts upon.

3. The method according to claim 1 or 2, characterized in that, The construction of the causal knowledge graph includes: Scan the current local area network to identify online devices in the current local area network; Obtain the object model file of the online device; Generate the device operation node corresponding to the online device based on the object model file; Obtain the user's historical interaction data corresponding to the online device; Determine the historical interaction intent based on the user's historical interaction data; Generate intent nodes based on the historical interaction intents; The environmental physical variable node corresponding to the intent node is determined based on the preset environmental physical variables. The causal knowledge graph is constructed based on the intent node, the environmental physical variable node, and the device operation node.

4. The method according to claim 3, characterized in that, The step of constructing the causal knowledge graph based on the intent node, the environmental physical variable node, and the device operation node includes: Establish the first causal edge between the intent node and the environmental physical variable node; Determine the probability distribution of environmental physical variables corresponding to the intent node based on the user's historical interaction data; The probability distribution of the environmental physical variables is used as the edge attribute of the first causal edge; Establish a second causal edge between the environmental physical variable node and the device operation node; Determine the direction of the influence of the device operation node on the environmental physical variable node; The edge attributes of the second causal edge are determined based on the direction of influence; The causal knowledge graph is constructed based on the intent node, the environmental physical variable node, the device operation node, the first causal edge, and the second causal edge.

5. The method according to claim 4, characterized in that, Determining the probability distribution of environmental physical variables corresponding to the intent node based on the user's historical interaction data includes: Obtain the baseline probability distribution of the environmental physical variables corresponding to the intent node; Based on the user's historical interaction data, a Bayesian update is performed on the baseline probability distribution of the environmental physical variables to obtain the probability distribution of the environmental physical variables corresponding to the intent node.

6. The method according to claim 2, characterized in that, The step of determining the set of functional substitutes corresponding to the target intelligent device based on the causal knowledge graph includes: Based on the target intelligent device and the target intent, the corresponding target device operation node is determined from the causal knowledge graph; Traverse the causal knowledge graph to determine the environmental physical variable node corresponding to the target device operation node as the target environmental physical variable node; Based on the target environment physical variable nodes, search for available device operation nodes in the causal knowledge graph; The set of functional alternatives is generated based on the available device operation nodes.

7. The method according to claim 2, characterized in that, Determining the functional substitution cost corresponding to each set of functional substitutions includes: Obtain the estimated operating power of the functional alternative devices in each of the functional alternative sets; Determine the probability distribution of the predicted environmental physical variables for the functional replacement device; The first causal edge corresponding to the target intent is determined based on the causal knowledge graph. The probability distribution of the physical variables of the target environment is obtained based on the first causal edge corresponding to the target intent; The environmental distribution similarity corresponding to the functional replacement device is determined based on the probability distribution of the predicted environmental physical variables and the probability distribution of the target environmental physical variables. Determine the second causal edge corresponding to the functional replacement device based on the causal knowledge graph; The device penalty value of the functional replacement device is determined based on the second causal edge corresponding to the functional replacement device. The functional substitution cost corresponding to each functional substitution set is determined based on the estimated operating power, the environmental distribution similarity, and the equipment penalty value.

8. The method according to claim 7, characterized in that, Determining the probability distribution of predicted environmental physical variables for the functional replacement device includes: Obtain the current environment state vector and the physical model of the functional replacement device; Determine the state transition matrix corresponding to the functional replacement device based on the object model; The probability distribution of predicted environmental physical variables for the functional replacement device is determined based on the environmental state vector and the state transition matrix.

9. The method according to claim 7, characterized in that, Determining the device penalty value of the functional replacement device based on the second causal edge corresponding to the functional replacement device includes: Obtain the edge attributes of the second causal edge corresponding to the functional replacement device; From the edge attributes, determine the environmental physical variable nodes whose influence direction is negative corresponding to the functional replacement device as target penalty nodes; Determine the preset penalty parameter benchmark based on the target intent and the target penalty node; Obtain the environmental physical variable parameters corresponding to the target penalty node when the functional replacement device is running; The device penalty value of the functional replacement device is determined based on the difference between the penalty parameter benchmark and the environmental physical variable parameter, as well as the preset unit penalty value.

10. A device control apparatus, characterized in that, The device includes: Modules for building causal knowledge graphs; The listening module is used to listen for user intent signals in the current environment to determine the user's target intent; A target intelligent device determination module is used to determine the target intelligent device based on the target intent and the causal knowledge graph. A function substitution set determination module is used to determine the function substitution set corresponding to the target intelligent device based on the causal knowledge graph when the target intelligent device is abnormal; each function substitution set includes one or more function substitution devices; A function substitution cost determination module is used to determine the function substitution cost corresponding to each set of function substitutions; the function substitution cost is a weighted sum of the estimated operating power of the function substitution device, the environmental distribution similarity, and the device penalty value. A target function replacement device determination module is used to determine a target function replacement device from the function replacement set based on the function replacement cost; A control module is used to generate control commands based on the target intent, so as to control the target function replacement device according to the control commands.

11. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the device control method as described in claims 1-9.

12. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the device control method as described in claims 1-9.

Images