A method, apparatus, device, medium and product for intelligent home device control

Abstract

This invention provides a method, apparatus, device, medium, and product for controlling smart home devices, applied in the field of artificial intelligence technology. The method includes: acquiring user voice and determining target entities and target intentions based on the user voice; determining target nodes mapping the target entities in the knowledge graph of the current smart home environment; when the target intention has missing slot information or ambiguous parameter values, reasoning based on the relational edges of the target nodes to complete the missing slot information or map the ambiguous parameter values ​​to specific values; when the target intention contains indicated or omitted expressions, completing the slot information or parameter values ​​corresponding to the indicated or omitted expressions based on preset historical context; and generating device control commands based on the completed target intentions to control the smart home devices, thereby achieving automatic reasoning and supplementation of user intentions to improve the user experience.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device, medium, and product for controlling smart home devices. Background Technology

[0002] With the development of artificial intelligence technology in smart home scenarios, users can control smart home devices via voice commands. However, when controlling smart home devices via voice commands, users need to describe the specific smart home device to be controlled and the specific control parameters. If the description is unclear, it will lead to control errors and affect the user experience. Summary of the Invention

[0003] In view of the above problems, a method, apparatus, device, medium, and product for controlling smart home devices are proposed to overcome or at least partially solve the above problems, including: In a first aspect, the present invention provides a method for controlling smart home devices, the method comprising: Acquire user voice and determine target entity and target intent based on the user voice; wherein the target intent includes at least one slot information and each slot information has a corresponding parameter value; In the knowledge graph of the current smart home environment, the target node mapped to the target entity is determined; wherein, the knowledge graph includes multiple nodes and the relationship edges between the nodes; When there is missing slot information or ambiguous parameter values ​​in the target intent, reasoning is performed based on the relational edges of the target node to fill in the missing slot information or map the ambiguous parameter values ​​to specific values. When the target intent contains a specified or omitted expression, the slot information or parameter value corresponding to the specified or omitted expression is completed according to the preset historical context. Based on the completed target intent, device control commands are generated to control smart home devices.

[0004] Optionally, before acquiring the user's voice and determining the target entity and target intent based on the user's voice, the method further includes: In the knowledge graph, device nodes for smart home devices in the current smart home environment and room nodes for rooms in the current smart home environment are established, and relationship edges between the device nodes and the room nodes are established. In the knowledge graph, relationship edges between the device nodes are established based on the device relationships between the smart home devices; In the knowledge graph, scene nodes are established for scene templates, and relationship edges between scene nodes and device nodes corresponding to device state combinations are established based on the device state combinations corresponding to the scene templates. In the knowledge graph, user nodes are established for users in the current smart home environment, and relationship edges between user nodes and device nodes or scene nodes are established based on the user's preference attributes. In the knowledge graph, rule nodes are established for dialogue rules, and relationship edges between the rule nodes and device nodes or scene nodes are established based on the content of the dialogue rules.

[0005] Optionally, when the target intent contains missing slot information or ambiguous parameter values, reasoning is performed based on the relational edges of the target node to complete the missing slot information or map the ambiguous parameter values ​​to specific numerical values, including: When there is missing slot information in the target intent, if the target node includes a target room node, the list of device nodes under the target room node is determined according to the relationship edge between the target room node and the device node, and the target device node is filtered from the list of device nodes to complete the missing target device information in the slot information; When there are ambiguous parameter values ​​in the target intent, if the target node contains a target rule node, the content of the dialogue rule is determined according to the relationship between the target rule node and the device node or scene node, and the ambiguous parameter values ​​are mapped to specific values ​​according to the content of the dialogue rule. When slot information is missing in the target intent, if the target node includes a target user node, the preference attribute of the target user node for the device node or scene node is determined based on the relationship between the target user node and the device node or scene node, so as to complete the default value of the corresponding parameter value of the slot information.

[0006] Optionally, when the target intent contains indicated or omitted expressions, the slot information or parameter values ​​corresponding to the indicated or omitted expressions are completed according to a preset historical context, including: When the target intent contains an expression that is either explicitly stated or omitted, the time decay weights of multiple preset historical contexts are determined, and a candidate historical context is determined from the multiple historical contexts based on the time decay weights. Based on the candidate historical context, the slot information or parameter values ​​corresponding to the indicated or omitted expressions are completed.

[0007] Optionally, based on the candidate historical context, the slot information or parameter values ​​corresponding to the indicated or omitted expression are completed, including: When there are multiple candidate historical contexts and the multiple candidate historical contexts are contradictory, the priority of the multiple candidate historical contexts is determined; Based on the priority, the target historical context is determined from the plurality of candidate historical contexts; Based on the target historical context, the slot information or parameter values ​​corresponding to the indicated or omitted expressions are completed.

[0008] Optionally, determine the time decay weights of multiple preset historical contexts, including: The time difference is determined based on the current time and the time of the historical context. The time decay weight is determined based on the time difference.

[0009] Optionally, the historical context includes any one or more of the following: session-level context, entry-level context, and user-level context; The session-level context includes the intent recognition result, slot information, and dialogue text of the current session; The entry-level context includes the high-frequency intent patterns and device preference information of the current entry point; The user-level context contains global preference statistics for the current user.

[0010] Optionally, the first time decay weight of the session-level context decreases as the time difference increases, and the first time decay weight is zero when the time difference exceeds a time threshold; The second time decay weight of the entry-level context decays exponentially as the time difference increases, and the decay rate is adjusted according to user activity. The decay rate of the third time decay weight of the user-level context is slower than that of the first time decay weight and the second time decay weight.

[0011] Optionally, if the user's voice was collected through a first entry point, the plurality of historical contexts include: The current user's entry-level context and user-level context are corresponding to the first entry point; The current user has an entry-level context and a user-level context corresponding to other entry points; where other entry points are entry points other than the first entry point.

[0012] Optionally, based on the completed target intent, device control commands are generated to control smart home devices, including: Based on the completed target intent, generate equipment control commands; When the device control command does not contain a timed execution condition or a periodic execution condition, the device control command will be directly sent to the smart home device for control. When the device control command includes a timed execution condition or a periodic execution condition, a task instance is created according to the device control command, and when the timed execution condition or periodic execution condition of the task instance is met, the device control command is sent to the smart home device for control.

[0013] Secondly, the present invention provides a device for controlling smart home devices, the device comprising: An entity and intent determination module is used to acquire user voice and determine target entities and target intents based on the user voice; wherein the target intent includes at least one slot information, and each slot information has a corresponding parameter value; The target node determination module is used to determine the target node mapped by the target entity in the knowledge graph of the current smart home environment; wherein, the knowledge graph includes multiple nodes and the relationship edges between the nodes; The first completion module is used to perform reasoning based on the relational edges of the target node when there is missing slot information or ambiguous parameter values ​​in the target intent, so as to complete the missing slot information or map the ambiguous parameter values ​​to specific values. The second completion module is used to complete the slot information or parameter value corresponding to the indicated or omitted expression according to the preset historical context when there is an indicated or omitted expression in the target intent. The device control module is used to generate device control commands based on the completed target intent in order to control smart home devices.

[0014] Thirdly, the present invention provides an electronic device including a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the method described above.

[0015] Fourthly, the present invention provides a computer-readable storage medium on which a computer program is stored, wherein the computer program, when executed by a processor, implements the method described above.

[0016] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the method described above.

[0017] The embodiments of the present invention have the following advantages: In this embodiment of the invention, user voice is acquired, and target entities and target intentions are determined based on the user voice. The target intention includes at least one slot information, and each slot information has a corresponding parameter value. In the knowledge graph of the current smart home environment, target nodes mapping the target entity are determined. The knowledge graph includes multiple nodes and the edges between nodes. When slot information is missing or parameter values ​​are ambiguous in the target intention, reasoning is performed based on the edges between the target nodes to complete the missing slot information or map the ambiguous parameter values ​​to specific values. When the target intention contains indicated or omitted expressions, the slot information or parameter values ​​corresponding to the indicated or omitted expressions are completed based on a preset historical context. Based on the completed target intention, device control commands are generated to control the smart home devices. This achieves the combined use of the knowledge graph and historical context to reason and supplement the user's intention, generating device control commands that meet the user's expectations, thus improving the response accuracy of smart home device control and the user experience. Attached Figure Description

[0018] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description of the present invention will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the architecture of a smart home device control system provided in some embodiments of the present invention; Figure 2 This is a flowchart of the steps of a method for controlling smart home devices provided in some embodiments of the present invention; Figure 3 This is a flowchart of the steps of a second method for controlling smart home devices provided in some embodiments of the present invention; Figure 4 This is a structural block diagram of a smart home device control device provided in some embodiments of the present invention. Detailed Implementation

[0020] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0021] With the development of artificial intelligence technology in smart home scenarios, users can control smart home devices via voice commands. However, when controlling smart home devices via voice commands, users need to describe the specific smart home device to be controlled and the specific control parameters. If the description is unclear, it will lead to control errors and affect the user experience.

[0022] Secondly, although some smart home devices can create scheduled tasks, periodic tasks, and delayed tasks, these tasks are easily lost due to system restarts, network fluctuations, device disconnections, etc., since they are only stored in memory.

[0023] Based on this, this invention proposes to construct a five-layer structured, dedicated knowledge graph for smart home scenarios, unifying device knowledge, scenario knowledge, user preferences, and interaction rules into a graph structure. This upgrades intent understanding from simple text semantic matching to a dual understanding mechanism of text semantics and knowledge reasoning. Combined with a three-layer hierarchical context management mechanism and full lifecycle task persistence capabilities, it enables cross-session and cross-entry context association and reliable task execution, improves the accuracy of understanding fuzzy expressions, omitted expressions, and referential expressions, and enhances the execution reliability of scheduled and periodic tasks.

[0024] like Figure 1 A three-tiered cloud-edge-device deployment architecture can be adopted. On the device side, devices such as smart speakers, mobile apps, and large-screen terminals are responsible for voice acquisition, playback, and local recognition of simple commands. On the edge side (home gateways or regional edge nodes), home-level context storage, local task scheduling, and caching of commonly used knowledge graph fragments are handled to achieve millisecond-level response. On the cloud side (all business modules), complex intent understanding, knowledge graph reasoning, long-term context storage, and cross-regional task scheduling are centrally processed. The cloud-side business process is completed collaboratively by six core processing modules: speech recognition, intent understanding, knowledge graph, context management, task management, and command execution.

[0025] The knowledge graph can be a five-layer structured dialogue knowledge graph, and context management can include three layers of context management. The five layers of the knowledge graph, from bottom to top, are: device entity layer (with device model as the node, storage type, function list, parameter value range), device relationship layer (linkage relationship, mutual exclusion relationship, hierarchical relationship), scene semantic layer (mapping from sleep / home / cinema, etc. modes to device state combinations), user preference layer (preferred temperature / brightness / fan speed, historical operation behavior statistics), and interaction rule layer (fuzzy / omission / reference resolution rules, such as "too hot" + room temperature > 28°C mapping to lowering the air conditioner temperature).

[0026] The three-layer context management includes: session-level context (stores the intent, slots, and dialogue text of each round of the current session, which can be linearly decayed by 1-Δt / 30min), entry-level context (such as aggregating recent high-frequency intent patterns and common device and parameter preferences by entry point, which can be exponentially decayed by e^(-λ·days)), and user-level context (stores global user preference statistics and long-term usage habits across entry points, which can be extremely slow decayed with a half-life of 90 days).

[0027] Based on the above architecture, through knowledge graph reasoning and three-layer contextual query, it is possible to accurately understand and complete ambiguous expressions, omitted expressions and symbolic expressions in user speech, and to achieve persistent storage and reliable execution of timed / periodic tasks through the task management module.

[0028] Reference Figure 2 The diagram illustrates a flowchart of a method for controlling a smart home device according to some embodiments of the present invention, which may specifically include the following steps: Step 201: Obtain user voice and determine target entity and target intent based on user voice; wherein the target intent includes at least one slot information, and each slot information has a corresponding parameter value.

[0029] In some examples, user voice can be natural language statements spoken by the user, captured by voice capture devices in a smart home system (such as smart speakers, mobile phone microphones, smart panels, etc.). The target entity can be an object related to the home environment identified from the user's voice, which can include one or more of the following: device names (such as air conditioners), room names (such as living rooms), and parameter numerical entities (such as 26 degrees).

[0030] For example, when a user says "turn on the living room air conditioner", the target entities could be the living room and the air conditioner; when a user says "set the temperature to 26 degrees", the target entities could be the temperature and 26 degrees.

[0031] In some examples, the target intent can be a category of action that the user wishes to perform, identified from the user's voice; where the action category can include one or more of the following: turning on the device, turning off the device, adjusting the temperature, adjusting the brightness, and setting a scene mode.

[0032] In some examples, slot information can be parameter fields that need to be obtained when executing a target intent. These parameter fields can vary depending on the intent type. For example, for an intent to adjust temperature, slot information might include three fields: target device, operation type, and target temperature value; for an intent to turn on a device, slot information might include only one field: target device.

[0033] The parameter values ​​are the specific information corresponding to each slot. For example, when a user says "set the living room air conditioner to 26 degrees," the parameter value corresponding to the target device slot is "living room air conditioner," the parameter value corresponding to the operation type is "temperature adjustment," and the parameter value corresponding to the target temperature is 26 degrees.

[0034] In practical applications, after a user speaks, the smart home system can acquire that voice through a voice acquisition device. By inputting the user's voice into an Automatic Speech Recognition (ASR) module, the ASR module converts the speech signal into text. Natural language processing is then performed on the converted text; this natural language processing can include intent recognition and entity recognition.

[0035] During intent recognition, text can be input into a pre-trained intent classification model. This model can be based on the BERT (Bidirectional Encoder Representations from Transformers) architecture and fine-tuned on a home-related corpus. The intent classification model can output candidate intents and confidence scores corresponding to the text, and the candidate intent with the highest confidence score can be identified as the target intent.

[0036] In entity recognition, text can be processed by Named Entity Recognition (NER) to identify entity words in the home furnishing domain from the text. These entity words can include one or more of the following: device entities (such as air conditioner), room entities (such as living room), parameter value entities (such as 26 degrees), and time entities (such as 7 a.m. every day). The identified entities can be determined as target entities.

[0037] Step 202: In the knowledge graph of the current smart home environment, determine the target node mapped by the target entity; wherein, the knowledge graph includes multiple nodes and the relationship edges between the nodes.

[0038] In some examples, a knowledge graph can be a knowledge base organized in a graph structure, containing multiple nodes and edges connecting those nodes. Nodes can represent entity objects, such as devices, rooms, scenes, users, rules, etc.; edges can represent the connections between entities, such as a device belonging to a room, devices interacting with each other, or a user having a preference for a device.

[0039] In some examples, the target node can be the graph node corresponding to the target entity in the knowledge graph. The graph node can have node type (such as device node, room node, scene node, etc.) and node attributes (such as device model, device type, list of supported functions, etc.).

[0040] In some examples, mapping can refer to matching the entity name of the identified target entity with the node name in the knowledge graph. This matching can include one or more matching methods, such as exact matching and fuzzy matching.

[0041] Fuzzy matching can refer to finding the closest node when the entity name in the user's speech does not completely match the node name in the knowledge graph (e.g., the user says "cabinet air conditioner", but the graph stores "standing air conditioner"). This can be achieved by calculating string similarity, such as edit distance or Jaccard similarity coefficient.

[0042] After identifying the target entity and target intent, the target entity can be used for querying within the knowledge graph. Queries use the target entity's name as keywords and match them against nodes in the knowledge graph. When a unique node is matched with the target entity in the knowledge graph, that node can be used as the target node.

[0043] In practical applications, when a target entity matches multiple candidate nodes, disambiguation can be performed by combining context information. For example, the context information of the current session can be obtained, and one of the multiple candidate nodes can be selected as the target node based on the context information.

[0044] For example, when a user says "turn that off", the target entity "that" cannot be directly matched to a specific node, but the target node can be determined from nodes such as the living room air conditioner in the historical context; when a user says "turn on the light" and the most recent mention of the living room in the current context, the living room light node can be determined as the target node.

[0045] In some examples, nodes in the knowledge graph have node types. Depending on the target entity, a target node can belong to different node types. When the target entity in the user's speech is a device name, the target node is a device node; when the target entity is a room name, the target node is a room node; when the target entity is a scene name, the target node is a scene node; and when the target entity is a user identifier, the target node is a user node.

[0046] In some examples, relationship edges in a knowledge graph can connect different nodes. Relationship edges can include one or more of the following: attribution relationship between device nodes and room nodes, linkage relationship and mutual exclusion relationship between device nodes, hierarchical relationship between room nodes and device nodes, mapping relationship between scene nodes and device nodes, preference relationship between user nodes and device nodes, and association relationship between rule nodes and device nodes or scene nodes.

[0047] Step 203: When there is missing slot information or ambiguous parameter values ​​in the target intent, reasoning is performed based on the relational edges of the target node to complete the missing slot information or map the ambiguous parameter values ​​to specific values.

[0048] In some examples, missing slot information can refer to the absence of corresponding parameter values ​​for one or more slots within the target intent. For instance, a user says "also open," and the target intent is to open the device. This intent includes the slot information for the target device, but "also open" does not specify the target device, resulting in missing parameter values ​​for the target device; that is, there is missing slot information.

[0049] In some examples, parameter ambiguity can refer to the use of imprecise expressions for parameter values ​​in one or more slots within the target intent's slot information. For instance, a user might say "raise it a little," specifying the operation type as "raise," but "a little" is an imprecise value and cannot be directly used to generate device control commands; thus, parameter ambiguity exists.

[0050] In some examples, the relationship edges of a target node can be connections between the target node and other nodes in a knowledge graph. Depending on the type of the target node, the relationship edges can have different types: when the target node is a room node, the room node can be connected to the various device nodes under that room through hierarchical relationship edges; when the target node is a device node, the device node can be connected to the corresponding rule node through association relationship edges; when the target node is a user node, the user node can be connected to the corresponding device node or scene node through preference relationship edges.

[0051] In some examples, inference can refer to the process of starting from a target node, obtaining information about other nodes associated with the target node along relational edges, and using the obtained information to fill in missing slot information or map fuzzy parameter values. Inference can include one-hop inference (directly obtaining adjacent nodes along the relational edges of the target node) and multi-hop inference (obtaining more distant nodes by following their relational edges from adjacent nodes).

[0052] In practical applications, when the slot information in the target intent is found to be missing or the parameter value is found to be ambiguous, the target node can be used as the starting point for reasoning. The knowledge graph can be traversed along the relation edges of the target node to obtain information about other nodes associated with the target node. The obtained information can be used to fill in the missing slot information or map the ambiguous parameter value.

[0053] Step 204: When the target intent contains a specified or omitted expression, the slot information or parameter value corresponding to the specified or omitted expression is completed according to the preset historical context.

[0054] In some examples, a referential term can be a word or phrase in a user's speech that refers to an entity or action, and the referential term does not contain the specific name of the entity being referred to. For example, when a user says "turn that off," "that" is a referential term used to refer to a device; when a user says "turn it up a bit," "it" is also a referential term used to refer to a device or parameter.

[0055] In some examples, ellipsis can be a way of expressing information in the user's speech that omits one or more of the following: the target device, the type of operation, or the parameter value. Ellipsis can include omitting one or more of the following: device name (e.g., when the user says "turn on the living room one too," omitting the name of the device such as the air conditioner or lights), operation parameters (e.g., when the user says "adjust it too," omitting the specific adjustment value), and operation object (e.g., when the user says "turn it on too," omitting the target device).

[0056] In some examples, the historical context can be a collection of user interaction information recorded before the current round. The historical context can include one or more of the following: records of each round of dialogue that have occurred in the current session, operation preferences and intent patterns recorded by the current entry point (such as a smart speaker or mobile app) in the historical session, and long-term preference statistics of users across entry points.

[0057] In practical applications, when a specified expression (such as "that", "it", "this") or an omitted expression (such as "open also", "living room", "adjust") is detected in the target intent, the historical context can be obtained as a data source for query matching to complete the slot information or parameter value corresponding to the specified expression or the omitted expression.

[0058] Step 205: Based on the completed target intent, generate device control commands to control smart home devices.

[0059] In some examples, smart home devices may include air conditioners, lights, curtains, air purifiers, etc.

[0060] In practical applications, the completed target intent can contain complete slot information, including fields such as target device, operation type, and parameter values. By taking the target intent as input, device control commands are generated according to the protocol format of the device control interface.

[0061] For example, the target device field can be extracted from the completed target intent and converted into a device identification field in the device control command; the device identification field may include one or more of the following: smart home device ID, device address, or device name.

[0062] For example, an operation type field can be extracted from the completed target intent and converted into a command field in the device control instruction; the command field may include one or more of the following: turn on, turn off, set temperature, set brightness.

[0063] For example, parameter value fields can be extracted from the completed target intent and converted into parameter fields in the device control command; the parameter fields may include one or more of temperature value, brightness value, and fan speed level.

[0064] After generating device control commands, they can be sent to smart home devices through the device control interface. The device control interface can be implemented based on one or more of the following protocols: Wi-Fi (Wireless Fidelity), Bluetooth, ZigBee, or infrared.

[0065] After receiving a device control command, a smart home device can parse the command and parameter fields in the command and execute the corresponding operation. Upon completion, it can return an execution result, which may include one or more of the following: execution success status, execution failure status, or the device status value after execution.

[0066] In some specific implementations, user voice can be acquired through voice acquisition devices. After voice recognition, entity recognition and intent classification, the target entity (such as living room air conditioner) and target intent (such as setting temperature, including slot {target device = living room air conditioner, operation = cooling, temperature = 26°C}) can be determined.

[0067] After identifying the target entity and target intent, the corresponding node (such as the living room air conditioner node) can be found in a pre-built knowledge graph. If there are missing slots in the target intent (such as only saying "turn that on" without specifying the device) or vague parameters (such as "turn it up a bit"), reasoning can be performed from the located target node along the relationship edges in the knowledge graph (such as the linkage relationship in the device relationship layer, the rules in the interaction rule layer, and the preferences in the user preference layer) to complete the missing device information or map "a bit higher" to a specific numerical value (such as 2°C).

[0068] If the target intent contains a referential expression (such as "that") or an abbreviated expression (such as "open also"), the preset historical context (such as devices operated by the user in previous sessions) can be queried to obtain the referent or the device name that needs to be supplemented to complete the slot information.

[0069] After completing the slot information, standardized device control commands, such as JSON (JavaScript Object Notation) format, can be generated based on the completed target intent and sent to the corresponding smart home devices for execution.

[0070] In this embodiment of the invention, user voice is acquired, and target entities and target intentions are determined based on the user voice. The target intention includes at least one slot information, and each slot information has a corresponding parameter value. In the knowledge graph of the current smart home environment, target nodes mapping the target entity are determined. The knowledge graph includes multiple nodes and the edges between nodes. When slot information is missing or parameter values ​​are ambiguous in the target intention, reasoning is performed based on the edges between the target nodes to complete the missing slot information or map the ambiguous parameter values ​​to specific values. When the target intention contains indicated or omitted expressions, the slot information or parameter values ​​corresponding to the indicated or omitted expressions are completed based on a preset historical context. Based on the completed target intention, device control commands are generated to control the smart home devices. This achieves the combined use of the knowledge graph and historical context to reason and supplement the user's intention, generating device control commands that meet the user's expectations, thus improving the response accuracy of smart home device control and the user experience.

[0071] In the above embodiments, by combining knowledge graph reasoning, natural language understanding, and contextual association, the smart home system can generate complete device control commands based on the vague expressions, omitted expressions, and directives in the user's voice. This eliminates the need for the user to fully describe the target device and specific parameter values ​​each time, reducing the user's expressive burden and repetitive input costs when using the smart home system.

[0072] In some embodiments of the present invention, before acquiring user voice and determining the target entity and target intent based on the user voice, steps S1-S5 are further included: Step S1: In the knowledge graph, establish device nodes for smart home devices in the current smart home environment and room nodes for rooms in the current smart home environment, and establish relationship edges between the device nodes and the room nodes.

[0073] In some examples, a device node can be a graph node representing an instance of a smart home device in the current smart home environment within the knowledge graph. A device node can contain a device node identifier and device node attributes; the device node identifier can be a device model ID (Identifier) ​​or a device instance ID. Device node attributes can include one or more of the following: device type (e.g., air conditioner, light, curtains, air purifier), device brand, a list of supported functions (e.g., on / off, temperature control, speed control, dimming), and parameter value ranges (e.g., temperature 16-30°C, brightness 0-100%).

[0074] In some examples, a room node can be a graph node representing a room space in the current smart home environment within the knowledge graph. A room node can contain a room node identifier (such as a room name ID) and node attributes. Room node attributes can include one or more of the following: room type (such as living room, bedroom, kitchen, study), room area, and room location information.

[0075] In practical applications, the device management platform can store device information for each smart home device in the current smart home environment. The device information may include one or more of the following: device model, device type, device installation location (room to which it belongs), device function list, and parameter value range.

[0076] Secondly, the device management platform can also store room information in the current smart home environment. Room information can include one or more of the following: room name (such as living room, bedroom), room type, and room location.

[0077] During the smart home environment initialization phase, room names can be used as node primary keys to create corresponding room nodes in the knowledge graph. Each room node stores attribute information such as room type and room location. During the new device addition phase, device model IDs or device instance IDs can be used as node primary keys to create corresponding device nodes in the knowledge graph. Each device node stores attribute information such as device type, brand, supported function list, and parameter value range.

[0078] For each device node, an affiliation edge can be established between the device node and the room node based on the device's installation location (i.e., which room the device is installed in). The affiliation edge points from the device node to the room node, indicating that the device belongs to that room.

[0079] After the attribution relationship is established, during a query, you can reach the room node to which the device belongs by following the attribution relationship edge from the device node, or you can query all device nodes under the room by following the attribution relationship edge from the room node.

[0080] Step S2: In the knowledge graph, establish relationship edges between the device nodes based on the device relationships between the smart home devices.

[0081] In some examples, device relationships can refer to the logical or physical connections between different smart home devices in the current smart home environment. Device relationships can include one or more of the following: linkage, mutual exclusion, and hierarchical relationships.

[0082] Interlocking relationships can refer to the relationship where the operation of one device triggers a corresponding operation in another device; for example, curtains automatically close when the air conditioner is turned on. Mutual exclusion relationships can refer to conflicts between different functions of the same device or between different devices; for example, an air conditioner cannot simultaneously execute cooling and heating commands, or there is a functional conflict when the cooling mode of an air conditioner and the humidifying mode of a humidifier are turned on simultaneously. Hierarchical relationships can refer to the inclusion of sub-devices within a room; for example, a living room contains sub-devices such as air conditioners, lights, and curtains.

[0083] In practical applications, the device management platform stores the relationships between various devices in the current smart home environment. These device relationships can be pre-configured in the device management platform or automatically extracted through device co-installation rules.

[0084] When there is a linkage between two devices (such as between an air conditioner and curtains, or between an air conditioner and a humidifier), a linkage edge is established between these two device nodes. This linkage edge connects the two device nodes, indicating that there is a linkage triggering relationship between the operation of one device and the operation of the other device.

[0085] When there is a functional conflict between two different devices, a mutual exclusion edge can be established between the two device nodes to indicate that the two devices cannot be in a certain operating state at the same time.

[0086] When a hierarchical relationship exists from room nodes to device nodes, the hierarchical relationship between devices can also be represented by edges connecting the device nodes. For example, all device nodes under the living room node (living room air conditioner, living room lights, living room curtains) can be connected by hierarchical relationship edges, indicating that they belong to the same level. Hierarchical relationship edges differ from belonging relationship edges; belonging relationship edges connect device nodes to room nodes, while hierarchical relationship edges connect device nodes to each other.

[0087] Step S3: In the knowledge graph, establish scene nodes for scene templates, and establish relationship edges between scene nodes and device nodes corresponding to device state combinations based on the device state combinations corresponding to the scene templates.

[0088] In some examples, a scene template can refer to a predefined template that contains a set of device state combinations. For example, a scene template may include one or more of the following: Sleep mode, Home mode, Away mode, and Cinema mode. Each scene template corresponds to a set of device state combinations.

[0089] A scene node can be a graph node representing a scene template in a knowledge graph. A scene node can contain a scene node identifier (such as scene name ID) and scene node attributes. Scene node attributes can include one or more of the following: scene type (such as sleep scene, home scene), scene description information, and scene triggering conditions.

[0090] In this context, a device state combination can refer to a set of target states or target parameter values ​​for each device defined in a scene template. For example, the device state combination corresponding to the sleep mode can include the correspondence between three sets of devices and states: lights - off, air conditioner - 26 degrees sleep mode, and curtains - closed.

[0091] In some examples, the device node corresponding to a device state combination can refer to the device node in the knowledge graph corresponding to each device involved in the device state combination. For example, the device state combination for sleep mode includes two devices: light and air conditioner. These two devices correspond to the light device node and the air conditioner device node, respectively, in the knowledge graph.

[0092] In practical applications, multiple scene templates can be predefined (such as sleep mode, home mode, away mode, and cinema mode). Each scene template includes a scene name and a combination of device states corresponding to that scene. For each device in the device state combination corresponding to the scene template, a mapping edge can be established between the scene node and the corresponding device node. The mapping edge points from the scene node to the device node, indicating that the device participates in the scene, and the mapping edge can store the target state or target parameter value of the device in that scene as an attribute of the mapping edge.

[0093] Step S4: In the knowledge graph, establish user nodes for users in the current smart home environment, and establish relationship edges between user nodes and device nodes or scene nodes based on the user's preference attributes.

[0094] In some examples, a user node can be a graph node representing a user in a knowledge graph. A user node can contain a user node identifier (such as a user ID) and user node attributes. User node attributes can include one or more of the following: user identity information (such as user name) and user preference attributes (such as preferred temperature, preferred brightness, and preferred fan speed).

[0095] User preference attributes can refer to a user's personalized preference settings for a particular device or scenario. These attributes may include one or more of the following: preferred temperature value (e.g., 26°C), preferred brightness value (e.g., 60%), commonly used fan speed setting (e.g., stroke mode), and commonly used scenario mode (e.g., sleep mode). User preference attributes may also include statistics on the user's historical device operation behavior, which may include one or more of the following: the time distribution of device operations, and the frequency statistics of device operations.

[0096] In practical applications, when a user registers or uses a smart home environment for the first time, after the user's identity information (such as user ID) is entered, the user ID can be used as a node identifier to create a corresponding user node in the knowledge graph.

[0097] During operation, user actions and preferences on the device can be continuously recorded. Records can include one or more of the following: user-defined preferences (e.g., setting the preferred temperature to 26°C via an app), statistical results of user history (e.g., the user most frequently set the air conditioner to 26°C in the past 7 days), and frequency statistics of user history (e.g., the number of times the user uses the air conditioner each day and the distribution of time periods).

[0098] When a user has a preference attribute for a device (such as a preferred temperature value of 26°C), a preference relationship edge can be established between the user node and the device node. The preference attribute value (such as preferred temperature: 26°C) can be stored on the preference relationship edge.

[0099] When a user has a preference for a certain scenario (such as a user frequently using sleep mode), a preference relationship edge can be established between the user node and the scenario node. The preference relationship edge can store preference attributes (such as preferred scenario: sleep mode).

[0100] In some implementations, during the cold start phase (i.e., when a new user has no historical operation records), the group preference statistics of similar users can be used as the default preference attribute value for the new user. The determination of similar users can be based on one or more of the following dimensions: apartment type (e.g., three-bedroom apartment), family composition (e.g., a family of three), climate zone (e.g., subtropical climate), and similarity of device category combinations (e.g., Jaccard similarity of device type sets). After being determined to be similar users, the group preference statistics of similar users can be recorded in the preference attributes of the new user node, or connected to the corresponding device node or scene node through preference relationship edges.

[0101] Step S5: In the knowledge graph, establish rule nodes for dialogue rules, and establish relationship edges between the rule nodes and device nodes or scene nodes according to the content of the dialogue rules.

[0102] In some examples, dialogue rules can refer to predefined judgment logic that defines the correspondence between user voice expressions and device operations in a smart home scenario. These dialogue rules are stored in the form of "trigger condition → reasoning conclusion".

[0103] For example, a dialogue rule could include: if a user says "It's too hot" and the current ambient temperature is above 28°C, the inference conclusion is to lower the air conditioning temperature; or it could include: if a user says "It's too hot" and the current ambient temperature is below or equal to 28°C, the inference conclusion is to turn on the fan. A dialogue rule can contain a set of trigger conditions, and a set of trigger conditions can include one or more trigger conditions.

[0104] In some examples, a rule node can be a graph node representing a dialogue rule in a knowledge graph. A rule node can contain a rule node identifier (such as a rule ID) and rule node attributes. Rule node attributes can include a trigger condition field and an inference conclusion field. The trigger condition field can store the triggering conditions of the dialogue rule (e.g., the user's statement contains the keyword "too hot" and the ambient temperature is above 28°C), and the inference conclusion field stores the inference conclusion after the dialogue rule is triggered (e.g., target device = air conditioner, operation = cooling, parameter value = temperature decrease of 2°C).

[0105] The content of the dialogue rule can refer to the specific values ​​of the trigger condition field and the inference conclusion field stored in the rule node. The content may include one or more of the following: keywords in the trigger condition, environmental parameter thresholds in the trigger condition, operation type in the inference conclusion, target device in the inference conclusion, and parameter values ​​in the inference conclusion.

[0106] In practical applications, business experts can define multiple dialogue rules based on users' actual communication habits and device functional boundaries in home scenarios. These business experts can be individuals familiar with the smart home business field and possessing knowledge of product functions and user usage scenarios, such as product managers, device function engineers, user experience researchers, or beta tester representatives—one or more of these roles.

[0107] The dialogue rules can be defined in a structured form of "trigger condition → inference conclusion". The trigger condition includes one or more of the keywords or phrases that may be contained in the user's expression (such as "too hot", "turn it up", "too bright") and environmental conditions (such as the current room temperature value, the current brightness value). The inference conclusion may include one or more of the following: the type of operation to be performed when the trigger condition is met, the target device, and the parameter value.

[0108] In practical applications, rule IDs can be used as node identifiers to create corresponding rule nodes in the knowledge graph, and trigger condition fields and inference conclusion fields can be stored in the rule node attributes. The trigger condition field can store the original text or structured expression of the trigger condition of the dialogue rule, and the inference conclusion field can store the original text or structured expression of the inference conclusion of the dialogue rule.

[0109] For devices or scenarios involved in a dialogue rule, an association edge can be established between the rule node and the device node or scenario node. This association edge can point from the rule node to the device node or scenario node, indicating that "this dialogue rule applies to this device or scenario." Alternatively, an association edge can point from the device node or scenario node to the rule node, indicating that "the dialogue rules applicable to this device or scenario include this rule."

[0110] When multiple dialogue rules involve the same device or the same scenario, the device node or scenario node can connect to multiple rule nodes, and each rule node is connected to the device node or scenario node through an association edge.

[0111] In some specific implementations, a knowledge graph can be a five-layer structured dialogue knowledge graph, and the construction process of a knowledge graph can include: 1. Device Entity Data Collection and Input. Import structured information such as model, type, brand, supported function list, and parameter value range of all categories of smart devices from the device management platform in batches. Create device entity nodes in the knowledge graph using the device model ID as the node primary key. Each device entity node includes the following attributes: device type (air conditioner, lighting, curtains, air purifier, etc.), supported function list (switch, temperature control, speed control, dimming, etc.), and parameter range (temperature 16-30°C, brightness 0-100%, etc.).

[0112] For each room in the current smart home environment (living room, bedroom, kitchen, etc.), a corresponding room node can be created in the knowledge graph using the room name as the primary key. Based on the actual installation location of the device, an "attribution" edge is established between the device entity node and the corresponding room node, indicating that the device belongs to that room. The device entity node and its attributes, the room node and its attributes, and the attribution edge between them together construct the device entity layer.

[0113] 2. Equipment Relationship Extraction and Establishment. Based on equipment co-installation rules (such as air conditioners and curtains in the same room usually having a linkage relationship) and equipment function mutual exclusion rules (such as an air conditioner cannot execute cooling and heating commands at the same time), the relationship edges between equipment are automatically extracted.

[0114] The extracted relationship edges include: linkage edges, indicating that an operation of one device triggers a corresponding operation of another device; mutual exclusion edges, indicating that there is a conflict between different functions of the same device or between different devices; and hierarchical edges, indicating that a room contains sub-devices. These relationship edges connect the corresponding device nodes in the knowledge graph, constructing a device relationship layer.

[0115] 3. Scene Semantic Definition and Mapping. Predefine scene templates based on business needs. Scene templates can include one or more of the following: Sleep Mode, Home Mode, Away Mode, and Cinema Mode. Each scene template defines a corresponding combination of device states. For example, the device state combination corresponding to Sleep Mode is: all lights off, curtains closed, and air conditioning set to 26 degrees Celsius for sleep mode.

[0116] For example, scene nodes can be created in the knowledge graph using the scene name as the node primary key. Based on the device state combination corresponding to the scene template, mapping edges are established between the scene node and the device nodes contained in the device state combination. The mapping edges store the target state or target parameter value of the device in that scene. The scene nodes and the mapping edges between them and the device nodes together construct the scene semantic layer.

[0117] 4. User Preference Data Accumulation. During operation, continuously record users' personalized preference settings for each device (such as preferred temperature, preferred brightness, and frequently used fan speed), as well as the time distribution and frequency statistics of device operations. Create user nodes in the knowledge graph using the user ID as the node primary key.

[0118] After creating a user node, preference relationship edges can be established between the user node and the corresponding device node or scene node based on the user's preference attributes. Preference attribute values ​​(such as preferred temperature value, preferred brightness value, and commonly used fan speed) are stored on the preference relationship edges. The user node and the preference relationship edges between it and the device node or scene node together construct the user preference layer.

[0119] 5. Interaction Rule Definition and Import. Business experts define dialogue rules specific to the home scenario. Dialogue rules are stored in the form of "trigger condition → inference conclusion". For example, rule "User says 'It's too hot' + current temperature > 28°C → Intent = lower air conditioner temperature, slot {target device = air conditioner, operation = cool down}"; rule "User says 'Also' + X + context includes operation Y → Intent = perform operation Y on X".

[0120] In practical applications, business experts can use a visual rule editor to enter dialogue rules using a structured template of "trigger condition → inference conclusion". The editor can convert natural language rules into the form of "field + operator + threshold / value" and automatically perform conflict detection and priority sorting.

[0121] Taking the rule "Too hot + room temperature > 28°C → lower air conditioner temperature" as an example: In the trigger condition area, check "User statement contains keywords = too hot" and set "Environmental condition = current room temperature > 28°C", with the logical relationship between the two being AND; In the reasoning conclusion area, select "Intent type = adjust device", "Target device = air conditioner", "Operation = cool down", and "Parameter = temperature decrease by 2°C"; When saving, the editor automatically verifies and confirms that it does not conflict with the existing rule "Too hot + room temperature ≤ 28°C → turn on fan" (the two conditions are mutually exclusive), assigns priority, and writes it into the interaction rule layer for subsequent multi-hop reasoning calls.

[0122] After defining the dialogue rules, rule nodes can be created in the knowledge graph using the rule ID as the node primary key. The rule node's attributes store trigger condition fields and inference conclusion fields. For the devices or scenarios involved in the dialogue rules, association edges are established between the rule nodes and the corresponding device or scenario nodes. The rule nodes and their attributes, along with the association edges between the rule nodes and device or scenario nodes, together construct the interaction rule layer.

[0123] 6. Graph Storage and Index Construction. Nodes and relationship edges across five layers—device entity layer, device relationship layer, scene semantic layer, user preference layer, and interaction rule layer—are uniformly stored in a graph database, such as Neo4j (a graph database management system). Full-text search indexes are built in the graph database, corresponding to device names to device entity nodes, room names to room nodes, and scene names to scene nodes, to support efficient queries during online inference.

[0124] In some embodiments of the present invention, in step 203 above, when there is missing slot information or ambiguous parameter values ​​in the target intent, reasoning is performed based on the relational edges of the target node to complete the missing slot information or map the ambiguous parameter values ​​to specific numerical values, including sub-steps 2031-2033: Sub-step 2031: When there is missing slot information in the target intent, if the target node includes a target room node, determine the list of device nodes under the target room node according to the relationship edge between the target room node and the device node, and filter the target device nodes from the list of device nodes to complete the missing target device information in the slot information.

[0125] In practical applications, when the target intent is checked and there is a missing slot information for the "target device", and the target node is the target room node, the target room node can be used as the starting point to query along the relationship edge between the target room node and the device node in the knowledge graph, obtain all the device nodes connected by the relationship edge, and build a device node list from all the obtained device nodes.

[0126] Each device node in the device node list has device node attributes, which may include one or more of the following: device type (such as air conditioner, light, curtain), supported function list (such as supporting one or more of the following: turn on, turn off, temperature adjustment, speed adjustment, dimming), and current operation status (such as currently in the on or off state).

[0127] After determining the list of device nodes, you can filter out the device nodes that meet the filtering criteria as target device nodes based on the operation type in the current round's target intent.

[0128] The filtering criteria may include: the device node’s supported function list contains the current operation type (i.e., the device supports the operation type), and the device node’s current operation status does not conflict with the current operation type (e.g., when the current operation type is “on”, the device is currently in the off state; when the current operation type is “off”, the device is currently in the on state).

[0129] After the target device node is determined, the node name or node identifier of the selected target device node can be extracted and used as the parameter value for the missing slot information of the "target device" slot, which is then filled into the slot information, so that the slot information changes from a missing state to a complete state.

[0130] When multiple device nodes in the device node list meet the filtering criteria, further selection can be made from multiple device nodes that meet the criteria by combining the context information of the current session (such as the type of device mentioned by the user in the previous round of conversation), or by combining user preferences (such as the user's historical frequency of operation of each device in the room). The selected device node is used as the target device node.

[0131] Sub-step 2032: When there are ambiguous parameter values ​​in the target intent, if the target node contains a target rule node, determine the content of the dialogue rule based on the relationship between the target rule node and the device node or scene node, and map the ambiguous parameter values ​​to specific numerical values ​​based on the content of the dialogue rule.

[0132] In practical applications, when the target intent is examined and the parameter values ​​are ambiguous, and the target node is a target rule node, the target rule node can be used as a starting point to obtain the rule node attributes of the target rule node in the knowledge graph. By obtaining the current environmental information (such as the current ambient temperature value and the current ambient brightness value), the current environmental information is matched with the trigger condition fields in the rule node attributes.

[0133] Specifically, when the current environmental information meets the conditions in the trigger condition field (such as "the current ambient temperature is higher than 28°C"), the rule node is matched. After a successful match, the inference conclusion can be obtained from the inference conclusion field of the target rule node, and the parameter value in the inference conclusion can be extracted as the specific numerical value after the fuzzy representation mapping. This value is then filled into the slot information corresponding to the fuzzy parameter value, so that the slot information changes from a fuzzy state to a complete state with a specific numerical value.

[0134] When the trigger condition field of a target rule node contains multiple trigger conditions, all trigger conditions must be met for a match to be considered successful. If any trigger condition in the trigger condition field is not met, it means that the rule node is not matched, and other rule nodes can be retrieved for matching.

[0135] In practical applications, the relationship edges between the target rule node and the device node or scene node can be used to determine the device range or scene range to which the rule applies. Once a rule node is matched, the device node or scene node to which the rule applies can be obtained by following the relationship edges between the rule node and the device node or scene node, and the obtained device node or scene node can be used as a candidate for the target device or target scene.

[0136] Sub-step 2033: When there is missing slot information in the target intent, if the target node includes a target user node, determine the target user node's preference attribute for the device node or scene node based on the relationship between the target user node and the device node or scene node, so as to complete the default value of the corresponding parameter value of the slot information.

[0137] In practical applications, when the target intent is checked and there is a missing parameter value corresponding to a certain slot information (such as missing target device, missing target temperature value, missing target brightness value, or one or more of these), and the target node is a target user node, the target user node can be used as the starting point to determine the preference relationship edge between the target user node and the device node or scene node in the knowledge graph.

[0138] After determining the preference relationship edges, the stored preference attribute values ​​can be obtained from these edges. These preference attribute values ​​can include one or more of the following: preferred temperature value, preferred brightness value, commonly used fan speed setting, and commonly used scene mode for a particular device. By determining the preference type corresponding to the missing slot information in the target intent (e.g., if the missing value is the target temperature value, the preference type is preferred temperature value; if the missing value is the target brightness value, the preference type is preferred brightness value), preference values ​​matching the obtained preference attribute values ​​are extracted.

[0139] After obtaining the preference value, it can be used as the default value for the missing parameter value and filled into the corresponding slot information, changing the slot information from a missing state to a complete state with default values. The parameter value in the filled slot information can be marked as "default value" to distinguish it from the explicit parameter value directly given in the user's voice.

[0140] In practical applications, when there are preference relationship edges between a user node and multiple device nodes, the corresponding preference relationship edge can be selected to obtain the preference attribute value based on other slot information already determined in the target intent (such as target device type, current room).

[0141] For example, when a user says "turn it up a little", and the target temperature value is missing in the target intent and the target device type is not determined, the preferred temperature value can be obtained from the preference relationship between the user node and the device node in the current room; when the target device has been determined to be the living room air conditioner, the preferred temperature value can be obtained from the preference relationship between the user node and the living room air conditioner device node.

[0142] In some implementations, during the cold start phase, when user nodes have not yet recorded actual preference attribute values, the user nodes can pre-store the group preference statistics of similar users as default preference attribute values. The determination of similar users can be based on one or more of the following dimensions: apartment type (e.g., three-bedroom apartment), family composition (e.g., a family of three), climate zone (e.g., subtropical climate), and similarity of equipment category combinations (e.g., Jaccard similarity of equipment type sets). The default preference attribute values ​​can be stored between user nodes and corresponding device nodes or scene nodes via preference relationship edges.

[0143] In some embodiments of the present invention, in step 204 above, when the target intent contains a specified or omitted expression, the slot information or parameter value corresponding to the specified or omitted expression is completed according to a preset historical context, including: When the target intent contains a specified or omitted expression, the time decay weights of multiple preset historical contexts are determined, and a candidate historical context is determined from the multiple historical contexts according to the time decay weights; and the slot information or parameter value corresponding to the specified or omitted expression is completed according to the candidate historical context.

[0144] In some examples, the time decay weight can be a numerical value. The time decay weight represents the priority of a historical context record being referenced at the current moment, and it decreases as the difference between the record's timestamp and the current time increases. For example, older records have lower decay weights, while more recent records have higher decay weights.

[0145] Here, candidate historical context refers to candidate records selected from all historical context records based on time decay weights to complete missing information in the current round. Each candidate record can be one or more, and each has a relatively high time decay weight.

[0146] In practical applications, when a specified or omitted expression is detected in the target intent, the current timestamp can be obtained, and the timestamps of each record in the preset historical context can be retrieved. For each record in the historical context, the time difference between the current timestamp and the timestamp of that record can be calculated, and the time decay weight of that record can be determined based on the time difference.

[0147] The method for determining the time decay weight varies depending on the context level to which the record belongs. For records in a session-level context, the time decay weight decreases linearly with the time difference, and the decay weight returns to zero when the time difference exceeds a preset timeout. For records in an entry-level context, the time decay weight decays exponentially with the time difference. For records in a user-level context, the decay rate of the time decay weight is slower than that of records in session-level and entry-level contexts.

[0148] After determining the time decay weights of all records, the records can be sorted in descending order of their time decay weights, and one or more records at the top of the sort can be selected as candidate historical contexts.

[0149] After obtaining the candidate historical context, the missing slot information or ambiguous parameter values ​​in the current round can be filled in based on the information in the candidate historical context (such as target device, operation type, parameter value).

[0150] For example, extract the parameter value corresponding to the currently missing slot from the candidate historical context and fill it into the slot; or extract the operation type from the candidate historical context as the operation type of the current round; or extract the device name from the candidate historical context as the target device of the current round.

[0151] In some examples, when there are multiple candidate historical contexts, the records can be used sequentially in sorted order to complete the context, or the subsequent matching can be stopped when the first record that can be completed is encountered.

[0152] In some embodiments of the present invention, based on the candidate historical context, the slot information or parameter values ​​corresponding to the indicated or omitted expression are completed, including: When multiple candidate historical contexts exist and the multiple candidate historical contexts are contradictory, the priority of the multiple candidate historical contexts is determined; based on the priority, a target historical context is determined from the multiple candidate historical contexts; based on the target historical context, the slot information or parameter value corresponding to the indicated expression or omitted expression is completed.

[0153] In some examples, conflicting candidate historical contexts can occur when multiple candidate historical context records have different values ​​on the same information dimension, making it impossible to use them simultaneously.

[0154] For example, one candidate record targets the living room air conditioner, while another targets the bedroom air conditioner; both point to different devices under the same operation type. Similarly, one candidate record indicates the operation type is "on," while another indicates it's "off," both pointing to different operations under the same target device. When these situations occur, one candidate record can be selected as the target record from multiple candidate records to resolve the information conflict.

[0155] In some examples, priority can be determined based on one or more of the time dimension, confidence dimension, and source dimension. In the time dimension, the most recent candidate record is given priority; in the confidence dimension, candidate records with higher source confidence are given priority (e.g., confirmed operations are given priority over inferred operations); in the source dimension, candidate records from the same entry point are given priority over candidate records from different entry points.

[0156] In practical applications, when there are multiple candidate historical contexts and there are contradictions among them, the attribute information corresponding to each candidate record can be obtained. The attribute information may include one or more of the following: record timestamp, record source (such as session level, entry level, user level), record entry (such as smart speaker entry, mobile APP entry), and record confirmation status (such as confirmed operation or inferred operation).

[0157] After obtaining the attribute information, the priority of multiple candidate records can be determined based on the attribute information. The priority comparison can be performed according to one or more of the following orders: First, compare the timestamps of the records. Records with later timestamps have higher priority than records with earlier timestamps (i.e., the most recent timestamps take precedence); when the timestamps are the same, compare the confirmation status of the records. Records with confirmed operations have higher priority than records with inferred operations (i.e., records with higher source confidence take precedence); when the above rules cannot distinguish between them, the source entry of the records can be compared. Records with the same entry as the current round have higher priority than records with different entry points (i.e., records with the same entry point take precedence).

[0158] After determining the priority, the highest priority candidate record can be selected from multiple candidate records as the target historical context. Information corresponding to the current round's indicated or omitted expression (such as target device, operation type, parameter value) can be extracted from the target historical context. The extracted information is then filled into the missing slot information or ambiguous parameter values ​​of the current round.

[0159] In some embodiments of the present invention, determining the time decay weights of a plurality of preset historical contexts includes: determining a time difference based on the current time and the time of the historical context; and determining the time decay weights based on the time difference.

[0160] In some examples, the current time can refer to the timestamp when the current round of dialogue was received. This timestamp can be based on the cloud system clock to ensure time consistency across entry points (such as smart speakers and mobile apps).

[0161] In some examples, the time of historical context can refer to the timestamp when the historical context record was written. Specifically, when each historical context record is written, the timestamp of that round can be recorded together as the time identifier of that record.

[0162] In some examples, the time difference can refer to the difference between the current time and the time of a historical context record. For example, if the timestamp of a historical context record is 19:30:00 and the current time is 19:45:00, then the time difference is 15 minutes.

[0163] In some examples, the determination of the time decay weight may include one or more of the following: a decay strategy based on a linear function (the time decay weight decreases linearly with the time difference), a decay strategy based on an exponential function (the time decay weight decays exponentially with the time difference), or a decay strategy based on a step function (remains unchanged within a preset threshold and resets to zero after exceeding the threshold).

[0164] In practical applications, different attenuation strategies can be used for different context levels (session level, entry level, user level), or the same attenuation strategy can be used but different attenuation parameters can be set.

[0165] In some embodiments of the present invention, the historical context includes any one or more of the following: session-level context, entry-level context, and user-level context; the session-level context includes the intent recognition result, slot information, and dialogue text of the current session; the entry-level context includes the high-frequency intent pattern and device preference information of the current entry; and the user-level context includes the global preference statistics of the current user.

[0166] In practical applications, historical context can be pre-defined and stored in a graph database or an in-memory database. The storage structure of historical context includes three independent storage areas: session-level context, entry-level context, and user-level context.

[0167] Within the session-level context, dialogue records for each round within the session can be stored using the session ID as the key. Each dialogue record includes the intent recognition result, slot information, dialogue text, and a timestamp for that round. When a new dialogue round arrives, the dialogue record for that round can be appended to the current session's dialogue record list. The session-level context can be retained for a period after a session ends for reference resolution and omission completion within that session. The dialogue text stored in the session-level context can be the raw text after ASR recognition or preprocessed text.

[0168] Within the entry-level context, entry identifiers (e.g., `smart_speaker` for smart speakers, `mobile_app` for mobile apps) can be used as keys to store statistical information for a given entry point over a preset time period. This statistical information can include high-frequency intent patterns (the execution frequency ranking of intents under this entry point) and device preference information (the operation frequency ranking of each device under this entry point and the commonly used parameter values ​​for each device). When a new conversation round occurs through an entry point, the intent and device information for that round are used to update the entry-level context statistics for that entry point. Entry-level contexts are used for scenarios involving cross-session but the same entry point (e.g., a user's actions on the speaker today can be referenced from yesterday's operation history).

[0169] Within the user-level context, global preference statistics can be stored using the user ID as the key. These statistics can include user preferences for various devices across different parameters (e.g., preferred temperature 26°C, preferred brightness 60%), frequency of user actions on each device (e.g., 100 actions on the living room air conditioner), and frequency of user calls to various scenarios (e.g., 20 calls to sleep mode). The user-level context can be used across entry points and sessions (e.g., what a user says on the speaker today can be referenced from their actions on the app yesterday).

[0170] In some embodiments of the present invention, the first time decay weight of the session-level context decreases as the time difference increases, and the first time decay weight is zero when the time difference exceeds a time threshold; the second time decay weight of the entry-level context decays exponentially as the time difference increases, and the decay rate is adjusted according to user activity; the decay rate of the third time decay weight of the user-level context is slower than the decay rates of the first time decay weight and the second time decay weight.

[0171] In some examples, the first time decay weight can refer to the time decay weight value corresponding to each record in the session-level context. The second time decay weight can refer to the time decay weight value corresponding to each record in the entry-level context. The third time decay weight can refer to the time decay weight value corresponding to each record in the user-level context.

[0172] Records in the session-level context can employ a linear decay strategy. The first-time decay weight decreases linearly as the time difference increases. When the time difference does not exceed the time threshold, the first-time decay weight remains at a non-zero value; when the time difference is equal to or exceeds the time threshold, the first-time decay weight returns to zero.

[0173] The time difference can refer to the difference between the current time and the historical context record time; the time threshold can be a preset time length value. When the time difference reaches the time threshold, the decay weight of the session-level context record is reduced to zero.

[0174] The records in the entry-level context can employ an exponential decay strategy. The second time decay weight can decrease exponentially as the time difference increases, with the base of the exponential function being the natural constant (approximately 2.718). The exponential decay rate can be adjusted based on user activity. When user activity is high, the decay rate is slower, allowing active users' entry-level historical records to be retained for a longer period; when user activity is low, the decay rate is faster, causing inactive users' entry-level historical records to be eliminated more quickly.

[0175] Among them, records in the user-level context can adopt a slow decay strategy. The decay rate of the third time decay weight is slower than that of the first and second time decay weights. That is, the weight of the user-level context record decreases slowly as the time difference increases, and the decay rate is lower than that of the session-level context and entry-level context records.

[0176] In some implementations, the session-level context employs a linear decay strategy. For each record in the session-level context, its time decay weight is calculated as follows: Weight = 1 - (Current Time - Round Time of the Record) / Time Threshold. The default value for the time threshold is 30 minutes. When (Current Time - Round Time) exceeds the session timeout, the decay weight of the record is reset to zero. Records exceeding the session timeout are no longer considered as candidate records.

[0177] The entry-level context employs an exponential decay strategy. For each record in the entry-level context, its time decay weight is calculated as follows: Weight = e^(-λ × number of days). Here, the number of days is the number of days elapsed between the current time and the timestamp of the record; λ is the decay coefficient, which is dynamically adjusted based on user activity. When user activity is high, λ takes a smaller value, resulting in a slower decay rate and allowing active users' entry-level historical records to be retained for a longer period. When user activity is low, λ takes a larger value, resulting in a faster decay rate and allowing inactive users' entry-level historical records to be eliminated more quickly.

[0178] The user-level context employs an extremely slow decay strategy, which decays more slowly than the session-level context and the entry-level context. For example, the user-level context uses a 90-day half-life strategy, meaning that every 90 days, the weight recorded in the user-level context decays to half of its current value.

[0179] In some embodiments of the present invention, when the user's voice is collected through a first entry point, the plurality of historical contexts include: the entry-level context and user-level context corresponding to the current user at the first entry point; the entry-level context and user-level context corresponding to the current user at other entry points; wherein, other entry points are entry points other than the first entry point.

[0180] In some examples, the primary entry point can refer to the user's currently used interactive device or interface. The primary entry point can include any one of a smart speaker, mobile app, or large-screen terminal. When the user issues a voice command through this entry point, it can serve as the primary entry point. Other entry points can refer to interactive devices or interfaces other than the primary entry point. Other entry points can include one or more of a smart speaker, mobile app, or large-screen terminal.

[0181] In practical applications, when the primary entry point is a smart speaker, other entry points can be a mobile app and a large-screen terminal; when the primary entry point is a mobile app, other entry points can be a smart speaker.

[0182] In some specific implementations, when a user initiates the current round of dialogue through the first entry point (such as a smart speaker), and the target intent of the current round involves explicit or omitting expressions that require completion information from the historical context, the user's ID can be used as the association key to query within a pre-defined historical context. The query scope includes the user's entry-level context and user-level context across all entry points (including the first entry point and other entry points).

[0183] For each context record retrieved, a time decay weight can be determined for each record. The records are then sorted in descending order of their time decay weights, and the top K records are selected as candidate historical context records. The value of K can be a preset positive integer, such as 5.

[0184] After identifying candidate historical context records, the relevance between each candidate historical context record and the current round of dialogue can be calculated. The relevance can be determined by one or more of intent similarity and entity overlap.

[0185] Intent similarity refers to the semantic similarity between the intent category in a candidate record and the target intent of the current round; entity overlap refers to the degree of overlap between the device entity, room entity involved in the candidate record and the target entity already identified in the current round. When the intent similarity or entity overlap of a candidate record exceeds a preset threshold, the record can be determined to be relevant to the current dialogue, and thus the record is included in the understanding context window of the current round to complete the slot information or parameter values ​​corresponding to the indicated or omitted expressions.

[0186] In some embodiments of the present invention, step 205 above, generating device control instructions based on the completed target intent to control smart home devices, includes: Based on the completed target intent, a device control command is generated. When the device control command does not contain a timed execution condition or a periodic execution condition, the device control command is directly sent to the smart home device for control. When the device control command contains a timed execution condition or a periodic execution condition, a task instance is created based on the device control command, and when the timed execution condition or periodic execution condition of the task instance is met, the device control command is sent to the smart home device for control.

[0187] In some examples, the timed execution condition can be a condition that specifies the execution of device control commands at a specific point in the future. This timed execution condition can include a specific time point (e.g., 19:00) or a relative time offset (e.g., 10 minutes later).

[0188] In some examples, periodic execution conditions can be conditions that specify the repeated execution of device control instructions at fixed time intervals. These periodic execution conditions can include one or more of the following: period type (e.g., daily, weekly), time point within the period (e.g., 7:00 AM daily), and number of repetitions (e.g., repeating 5 times).

[0189] In some examples, a task instance can refer to an execution unit created according to device control instructions. A task instance can contain the entire content of the device control instructions (target device identifier, operation commands, operation parameters) and triggering conditions (time points or periodic rules).

[0190] In practical applications, when a user's voice request includes timed execution conditions, periodic execution conditions, or delayed execution conditions, the task parameters can be parsed by the intent understanding module. Task parameters may include one or more of the following: trigger time (e.g., 19:00 or 10 minutes later), execution frequency (e.g., once a day or only once), target device (e.g., bedroom curtains), and operation content (e.g., opening).

[0191] After determining the task parameters, a task instance can be generated based on the task parameters, and the status of the task instance can be set to pending confirmation. The task confirmation information can be read to the user via voice (such as "A task to open the bedroom curtains at 7 am every day has been created for you. Please confirm"). After the user confirms, the status of the task instance changes from pending confirmation to confirmed.

[0192] After a user confirms a task instance, the confirmed task instance can be simultaneously written to both the hot data area in memory and the cold data area of ​​the distributed database to ensure data consistency during dual writes. The hot data area in memory can use a time-wheel data structure for storage, supporting O(1) time complexity (i.e., constant time complexity) triggered queries for fast scheduling when tasks are triggered. The cold data area of ​​the distributed database can store all fields and status history of the task instance for data recovery and historical queries after a system restart.

[0193] After storing the task instance, when the time wheel reaches the trigger time of the task instance, the task instance is triggered, and its status changes from confirmed to executing. The target smart home device identifier and operation instructions are extracted from the task instance's data, and the control instructions are sent to the target smart home device for execution via the device control interface. After execution, the task instance's status can be changed to completed (if execution was successful) or failed (if execution failed) based on the execution result returned by the target device.

[0194] For task instances that are marked as failed, an exponential backoff retry strategy can be initiated. This means the waiting interval for the first retry is 30 seconds, the second is 60 seconds, and the third is 120 seconds, with a maximum of three retries. After exceeding the maximum number of retries, the task instance can be marked as failed, and the user will be notified via a messaging channel (such as app push notifications or voice announcements).

[0195] In some implementations, the task table in the distributed database can be scanned to filter out task instances with statuses of "confirmed" and "in execution," and these task instances can be loaded into the memory time wheel to resume execution. For trigger times missed due to system crashes, a handling strategy can be determined based on the task type: one-time tasks are executed immediately and a notification is sent to the user; periodic tasks skip the missed period and wait for the next trigger time to execute normally.

[0196] Each state transition of a task instance can be appended to the execution log, which can record one or more of the following: task ID, transition time, source state, target state, execution result, and error message. The written execution log supports historical queries and tracing based on one or more conditions, including task ID, time range, and task type. Completed task instances can be archived from memory and the main database to a historical data table after 30 days of retention.

[0197] Through the above task management methods, scheduled tasks, periodic tasks, and delayed tasks can be persistently stored in memory and database after user confirmation. Task instances can be loaded from the database and resumed after system restart. Failed tasks are automatically retried through an exponential backoff retry strategy, and state transitions are recorded through execution logs, thus achieving reliable execution of scheduled, periodic, and delayed tasks.

[0198] Reference Figure 3 The diagram illustrates a flowchart of another method for controlling smart home devices according to some embodiments of the present invention, which may specifically include the following steps: Step 301: In the knowledge graph, establish device nodes for smart home devices in the current smart home environment and room nodes for rooms in the current smart home environment, and establish relationship edges between the device nodes and the room nodes.

[0199] Step 302: In the knowledge graph, establish relationship edges between the device nodes based on the device relationships between the smart home devices.

[0200] Step 303: In the knowledge graph, establish scene nodes for scene templates, and establish relationship edges between scene nodes and device nodes corresponding to device state combinations based on the device state combinations corresponding to the scene templates.

[0201] Step 304: In the knowledge graph, establish user nodes for users in the current smart home environment, and establish relationship edges between user nodes and device nodes or scene nodes based on the user's preference attributes.

[0202] Step 305: In the knowledge graph, establish rule nodes for dialogue rules, and establish relationship edges between the rule nodes and device nodes or scene nodes according to the content of the dialogue rules.

[0203] Step 306: Acquire user voice and determine target entity and target intent based on user voice; wherein the target intent includes at least one slot information and each slot information has a corresponding parameter value.

[0204] Step 307: In the knowledge graph of the current smart home environment, determine the target node mapped by the target entity; wherein the knowledge graph includes multiple nodes and the relationship edges between the nodes.

[0205] Step 308: When there is missing slot information or ambiguous parameter values ​​in the target intent, reasoning is performed based on the relational edges of the target node to complete the missing slot information or map the ambiguous parameter values ​​to specific values.

[0206] Step 309: When the target intent contains a specified or omitted expression, the slot information or parameter value corresponding to the specified or omitted expression is completed according to the preset historical context.

[0207] Step 310: Generate device control commands based on the completed target intent to control smart home devices.

[0208] In this embodiment of the invention, by establishing relationship edges between device nodes, room nodes, rule nodes, and scene nodes in the knowledge graph, when the user's target intent has missing slots or ambiguous parameters, reasoning can be performed based on the relationship edges of the knowledge graph to complete the task, which can effectively improve the accuracy of understanding, reduce the number of follow-up questions to the user, and optimize the interactive experience.

[0209] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0210] Reference Figure 4 The diagram illustrates a structural schematic of a smart home device control apparatus according to some embodiments of the present invention, which may specifically include the following modules: The entity and intent determination module 401 is used to acquire user voice and determine target entity and target intent based on the user voice; wherein the target intent includes at least one slot information, and each slot information has a corresponding parameter value; The target node determination module 402 is used to determine the target node mapped by the target entity in the knowledge graph of the current smart home environment; wherein, the knowledge graph includes multiple nodes and the relationship edges between the nodes; The first completion module 403 is used to perform reasoning based on the relational edges of the target node when there is missing slot information or ambiguous parameter values ​​in the target intent, so as to complete the missing slot information or map the ambiguous parameter values ​​to specific values. The second completion module 404 is used to complete the slot information or parameter value corresponding to the indicated or omitted expression according to the preset historical context when there is an indicated or omitted expression in the target intent. The device control module 405 is used to generate device control commands based on the completed target intent in order to control smart home devices.

[0211] In some embodiments of the present invention, the apparatus further includes: The first relation edge establishment module is used to establish device nodes for smart home devices in the current smart home environment and room nodes for rooms in the current smart home environment in the knowledge graph, and to establish relation edges between the device nodes and the room nodes. The second relationship edge establishment module is used to establish relationship edges between device nodes in the knowledge graph based on the device relationships between the smart home devices. The third relation edge establishment module is used to establish scene nodes for scene templates in the knowledge graph, and establish relation edges between scene nodes and device nodes corresponding to device state combinations based on the device state combinations corresponding to the scene templates. The fourth relationship edge establishment module is used to establish user nodes for users in the current smart home environment in the knowledge graph, and establish relationship edges between user nodes and device nodes or scene nodes according to the user's preference attributes. The fifth relationship edge establishment module is used to establish rule nodes for dialogue rules in the knowledge graph, and establish relationship edges between the rule nodes and device nodes or scene nodes according to the content of the dialogue rules.

[0212] In some embodiments of the present invention, the first completion module 403 is used for: When there is missing slot information in the target intent, if the target node includes a target room node, the list of device nodes under the target room node is determined according to the relationship edge between the target room node and the device node, and the target device node is filtered from the list of device nodes to complete the missing target device information in the slot information; When there are ambiguous parameter values ​​in the target intent, if the target node contains a target rule node, the content of the dialogue rule is determined according to the relationship between the target rule node and the device node or scene node, and the ambiguous parameter values ​​are mapped to specific values ​​according to the content of the dialogue rule. When slot information is missing in the target intent, if the target node includes a target user node, the preference attribute of the target user node for the device node or scene node is determined based on the relationship between the target user node and the device node or scene node, so as to complete the default value of the corresponding parameter value of the slot information.

[0213] In some embodiments of the present invention, the second completion module 404 is used for: When the target intent contains an expression that is either explicitly stated or omitted, the time decay weights of multiple preset historical contexts are determined, and a candidate historical context is determined from the multiple historical contexts based on the time decay weights. Based on the candidate historical context, the slot information or parameter values ​​corresponding to the indicated or omitted expressions are completed.

[0214] In some embodiments of the present invention, based on the candidate historical context, the slot information or parameter values ​​corresponding to the indicated or omitted expression are completed, including: When there are multiple candidate historical contexts and the multiple candidate historical contexts are contradictory, the priority of the multiple candidate historical contexts is determined; Based on the priority, the target historical context is determined from the plurality of candidate historical contexts; Based on the target historical context, the slot information or parameter values ​​corresponding to the indicated or omitted expressions are completed.

[0215] In some embodiments of the present invention, determining the time decay weights of a plurality of preset historical contexts includes: The time difference is determined based on the current time and the time of the historical context. The time decay weight is determined based on the time difference.

[0216] In some embodiments of the present invention, the historical context includes any one or more of the following: session-level context, entry-level context, and user-level context; The session-level context includes the intent recognition result, slot information, and dialogue text of the current session; The entry-level context includes the high-frequency intent patterns and device preference information of the current entry point; The user-level context contains global preference statistics for the current user.

[0217] In some embodiments of the present invention, the first time decay weight of the session-level context decreases as the time difference increases, and the first time decay weight is zero when the time difference exceeds a time threshold. The second time decay weight of the entry-level context decays exponentially as the time difference increases, and the decay rate is adjusted according to user activity. The decay rate of the third time decay weight of the user-level context is slower than that of the first time decay weight and the second time decay weight.

[0218] In some embodiments of the present invention, when the user's voice is collected through a first entry point, the plurality of historical contexts include: The current user's entry-level context and user-level context are corresponding to the first entry point; The current user has an entry-level context and a user-level context corresponding to other entry points; where other entry points are entry points other than the first entry point.

[0219] In some embodiments of the present invention, the device control module 405 is used for: Based on the completed target intent, generate equipment control commands; When the device control command does not contain a timed execution condition or a periodic execution condition, the device control command will be directly sent to the smart home device for control. When the device control command includes a timed execution condition or a periodic execution condition, a task instance is created according to the device control command, and when the timed execution condition or periodic execution condition of the task instance is met, the device control command is sent to the smart home device for control.

[0220] Some embodiments of the present invention also provide an electronic device, including a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the method described above.

[0221] Some embodiments of the present invention also provide a computer-readable storage medium on which a computer program is stored, and which, when executed by a processor, implements the method described above.

[0222] Some embodiments of the present invention also provide a computer program product, including a computer program that, when executed by a processor, implements the method described above.

[0223] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0224] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this invention are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0225] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0226] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0227] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0228] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0229] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0230] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0231] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes the aforementioned element.

[0232] The above provides a detailed description of the method, apparatus, device, medium, and product for controlling smart home devices. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for controlling smart home devices, characterized in that, The method includes: Acquire user voice and determine target entity and target intent based on the user voice; wherein the target intent includes at least one slot information and each slot information has a corresponding parameter value; In the knowledge graph of the current smart home environment, the target node mapped to the target entity is determined; wherein, the knowledge graph includes multiple nodes and the relationship edges between the nodes; When there is missing slot information or ambiguous parameter values ​​in the target intent, reasoning is performed based on the relational edges of the target node to fill in the missing slot information or map the ambiguous parameter values ​​to specific values. When the target intent contains a specified or omitted expression, the slot information or parameter value corresponding to the specified or omitted expression is completed according to the preset historical context. Based on the completed target intent, generate device control commands to control smart home devices; Before acquiring user voice and determining the target entity and target intent based on the user voice, the process also includes: In the knowledge graph, device nodes for smart home devices in the current smart home environment and room nodes for rooms in the current smart home environment are established, and relationship edges between the device nodes and the room nodes are established. In the knowledge graph, relationship edges between the device nodes are established based on the device relationships between the smart home devices; In the knowledge graph, scene nodes are established for scene templates, and relationship edges between scene nodes and device nodes corresponding to device state combinations are established based on the device state combinations corresponding to the scene templates. In the knowledge graph, user nodes are established for users in the current smart home environment, and relationship edges between user nodes and device nodes or scene nodes are established based on the user's preference attributes. In the knowledge graph, rule nodes are established for dialogue rules, and relationship edges between the rule nodes and device nodes or scene nodes are established according to the content of the dialogue rules; wherein, the dialogue rule is a predefined judgment logic, the judgment logic is the correspondence between user voice expression and device operation in a smart home scene, the rule node includes a rule node identifier and rule node attributes, and the rule node attributes include a trigger condition field and a reasoning conclusion field; When the target intent contains missing slot information or ambiguous parameter values, reasoning is performed based on the relational edges of the target node to complete the missing slot information or map the ambiguous parameter values ​​to specific numerical values, including: When there is missing slot information in the target intent, if the target node includes a target room node, the list of device nodes under the target room node is determined according to the relationship edge between the target room node and the device node, and the target device node is filtered from the list of device nodes to complete the missing target device information in the slot information; When there are ambiguous parameter values ​​in the target intent, if the target node contains a target rule node, the content of the dialogue rule is determined according to the relationship between the target rule node and the device node or scene node, and the ambiguous parameter values ​​are mapped to specific values ​​according to the content of the dialogue rule. When slot information is missing in the target intent, if the target node includes a target user node, the preference attribute of the target user node for the device node or scene node is determined based on the relationship between the target user node and the device node or scene node, so as to complete the default value of the corresponding parameter value of the slot information.

2. The method according to claim 1, characterized in that, When the target intent contains indicated or omitted expressions, the slot information or parameter values ​​corresponding to the indicated or omitted expressions are completed according to the preset historical context, including: When the target intent contains an expression that is either explicitly stated or omitted, the time decay weights of multiple preset historical contexts are determined, and a candidate historical context is determined from the multiple historical contexts based on the time decay weights. Based on the candidate historical context, the slot information or parameter values ​​corresponding to the indicated or omitted expressions are completed.

3. The method according to claim 2, characterized in that, Based on the candidate historical context, the slot information or parameter values ​​corresponding to the indicated or omitted expressions are completed, including: When there are multiple candidate historical contexts and the multiple candidate historical contexts are contradictory, the priority of the multiple candidate historical contexts is determined; Based on the priority, the target historical context is determined from the plurality of candidate historical contexts; Based on the target historical context, the slot information or parameter values ​​corresponding to the indicated or omitted expressions are completed.

4. The method according to claim 2, characterized in that, Determine the time decay weights of multiple pre-defined historical contexts, including: The time difference is determined based on the current time and the time of the historical context. The time decay weight is determined based on the time difference.

5. The method according to claim 4, characterized in that, The historical context includes one or more of the following: session-level context, entry-level context, and user-level context; The session-level context includes the intent recognition result, slot information, and dialogue text of the current session; The entry-level context includes the high-frequency intent patterns and device preference information of the current entry point; The user-level context contains global preference statistics for the current user.

6. The method according to claim 5, characterized in that, The first time decay weight of the session-level context decreases as the time difference increases, and the first time decay weight returns to zero when the time difference exceeds a time threshold. The second time decay weight of the entry-level context decays exponentially as the time difference increases, and the decay rate is adjusted according to user activity. The decay rate of the third time decay weight of the user-level context is slower than that of the first time decay weight and the second time decay weight.

7. The method according to claim 5, characterized in that, When the user's voice is collected through the first entry point, the multiple historical contexts include: The current user's entry-level context and user-level context are corresponding to the first entry point; The current user has an entry-level context and a user-level context corresponding to other entry points; where other entry points are entry points other than the first entry point.

8. The method according to any one of claims 1, characterized in that, Based on the completed target intent, generate device control commands to control smart home devices, including: Based on the completed target intent, generate equipment control commands; When the device control command does not contain a timed execution condition or a periodic execution condition, the device control command will be directly sent to the smart home device for control. When the device control command includes a timed execution condition or a periodic execution condition, a task instance is created according to the device control command, and when the timed execution condition or periodic execution condition of the task instance is met, the device control command is sent to the smart home device for control.

9. A device for controlling smart home devices, characterized in that, The device includes: An entity and intent determination module is used to acquire user voice and determine target entities and target intents based on the user voice; wherein the target intent includes at least one slot information, and each slot information has a corresponding parameter value; The target node determination module is used to determine the target node mapped by the target entity in the knowledge graph of the current smart home environment; wherein, the knowledge graph includes multiple nodes and the relationship edges between the nodes; The first completion module is used to perform reasoning based on the relational edges of the target node when there is missing slot information or ambiguous parameter values ​​in the target intent, so as to complete the missing slot information or map the ambiguous parameter values ​​to specific values. The second completion module is used to complete the slot information or parameter value corresponding to the indicated or omitted expression according to the preset historical context when there is an indicated or omitted expression in the target intent. The device control module is used to generate device control commands based on the completed target intent in order to control smart home devices; The device further includes: The first relation edge establishment module is used to establish device nodes for smart home devices in the current smart home environment and room nodes for rooms in the current smart home environment in the knowledge graph, and to establish relation edges between the device nodes and the room nodes. The second relationship edge establishment module is used to establish relationship edges between device nodes in the knowledge graph based on the device relationships between the smart home devices. The third relation edge establishment module is used to establish scene nodes for scene templates in the knowledge graph, and establish relation edges between scene nodes and device nodes corresponding to device state combinations based on the device state combinations corresponding to the scene templates. The fourth relationship edge establishment module is used to establish user nodes for users in the current smart home environment in the knowledge graph, and establish relationship edges between user nodes and device nodes or scene nodes according to the user's preference attributes. The fifth relation edge establishment module is used to establish rule nodes for dialogue rules in the knowledge graph, and to establish relation edges between the rule nodes and device nodes or scene nodes according to the content of the dialogue rules; wherein, the dialogue rules are predefined judgment logic, the judgment logic is the correspondence between user voice expression and device operation in a smart home scene, the rule nodes include rule node identifiers and rule node attributes, and the rule node attributes include trigger condition fields and reasoning conclusion fields; The first completion module is used for: When there is missing slot information in the target intent, if the target node includes a target room node, the list of device nodes under the target room node is determined according to the relationship edge between the target room node and the device node, and the target device node is filtered from the list of device nodes to complete the missing target device information in the slot information; When there are ambiguous parameter values ​​in the target intent, if the target node contains a target rule node, the content of the dialogue rule is determined according to the relationship between the target rule node and the device node or scene node, and the ambiguous parameter values ​​are mapped to specific values ​​according to the content of the dialogue rule. When slot information is missing in the target intent, if the target node includes a target user node, the preference attribute of the target user node for the device node or scene node is determined based on the relationship between the target user node and the device node or scene node, so as to complete the default value of the corresponding parameter value of the slot information.

10. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the method as described in any one of claims 1 to 8.

12. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 8.

Images