A Smart Home Control Method and System Based on Graph Retrieval Enhanced Generation and Multi-Agent Game Theory

Abstract

This invention discloses a smart home control method and system based on graph retrieval-enhanced generation and multi-agent game theory. The method includes: constructing a basic knowledge graph based on the current smart home environment, semantically enriching the service entities within it to generate an enhanced knowledge graph, and then storing it in a vector database; constructing an example library containing several formal observation-logic hypothesis pairs; obtaining a snapshot of the user's context, and outputting a structured scenario analysis report based on the context snapshot and the vector database through a front-end LLM; collaboratively deliberating through abductive reasoning agent, a user preference agent, and a conflict detection agent in conjunction with the scenario analysis report and the example library, with the core decision-making LLM outputting the final decision and mapping it into device control commands to invoke the corresponding devices for execution. This invention combines the user's fuzzy commands with the knowledge graph to improve the accuracy of intent recognition; the multi-agent parallel deliberation mechanism enhances the robustness and security of the decision.

Description

Technical Field

[0001] This invention relates to the field of smart home control technology, and in particular to a smart home control method and system based on graph retrieval-enhanced generation and multi-agent game theory. Background Technology

[0002] In recent years, with the popularization of smart home devices, users have placed higher demands on the automation and intelligent control of their home environments. Currently, there are two main types of smart home control methods:

[0003] 1. Methods based on fixed rules or simple intent recognition: These systems typically pre-set "if...then..." rules or identify user intent through keyword matching. For example, when a user says "turn on the living room light," the system can execute the command accurately. However, this method has significant limitations; it cannot understand vague, colloquial, or context-dependent instructions. For instance, if a user says "I feel a little cold" late at night, a simple system might not understand that this implies "close the windows and turn on the air conditioner's heating mode," nor can it make personalized judgments based on current room temperature, the user's past habits, and other information.

[0004] 2. End-to-end control methods based on a single large language model (LLM): To overcome the rigidity of rule-based systems, some solutions attempt to directly leverage the powerful natural language understanding capabilities of large language models for end-to-end decision-making. LLMs are better able to understand fuzzy instructions and context. However, this approach also has serious drawbacks:

[0005] (1) The problem of “illusion”: LLM may have “illusion” that is inconsistent with the facts during the reasoning process, and make operations that are not in line with physical logic or are unsafe, such as turning on the heating equipment when no one is around;

[0006] (2) Logical inconsistency: When facing complex and multi-constrained scenarios, a single LLM lacks structured reasoning and verification in its decision-making process, which is prone to logical contradictions;

[0007] (3) Lack of interpretability: The decision-making process of LLM is like a "black box". When it makes an unexpected decision, it is difficult to trace the cause and to make targeted optimizations.

[0008] (4) Limited personalization capabilities: Although LLM possesses a vast amount of knowledge, its understanding of the preferences and dynamic changes of specific users and specific family environments remains insufficient.

[0009] Therefore, existing technologies suffer from key problems such as inaccurate understanding, insecure decision-making, lack of personalization, and poor interpretability when dealing with complex instructions that are often ambiguous, personalized, and context-dependent in smart home scenarios. Summary of the Invention

[0010] This invention provides a smart home control method and system based on graph retrieval-enhanced generation and multi-agent game theory, to solve the problems of inaccurate understanding, insecure decision-making, lack of personalization and poor interpretability in the existing technology when dealing with complex instructions that are often fuzzy, personalized and context-dependent in smart home scenarios.

[0011] Firstly, a smart home control method based on graph retrieval-enhanced generation and multi-agent game theory is provided, including the following steps:

[0012] S1: Instantiate all entities and their relationships in the current smart home environment to build a basic knowledge graph;

[0013] S2: Enrich the semantics of the service entities in the basic knowledge graph, generate an enhanced knowledge graph, vectorize it, and store it in a vector database;

[0014] S3: Build an example library containing several formal observation-logic hypothesis pairs, where a formal observation is a description of the state of the environment and a logical hypothesis is a logical expression that can explain the cause of the formal observation or the user's intention.

[0015] S4: Obtain a snapshot of the user's context, refine user needs based on the context snapshot and vectorized database through the front-end LLM, and output a structured scenario analysis report, which includes the current formal observations;

[0016] S5: The abductive reasoning agent generates multiple candidate logical hypotheses based on the scenario analysis report and example library, and gives a logical rationality score for each candidate logical hypothesis; the user preference agent gives a personalized compliance score for the service sequence corresponding to each candidate logical hypothesis; the conflict detection agent gives a conflict risk score for the service sequence corresponding to each candidate logical hypothesis; the scenario analysis report and the review report including the logical rationality score, personalized compliance score and conflict risk score are input into the core decision LLM, output the final decision and generate an explanation of the decision;

[0017] S6: Map the final decision to device control commands and call the corresponding device to execute them.

[0018] Furthermore, step S2 specifically includes:

[0019] For each service entity in the basic knowledge graph, an LLM is used to generate relevant semantic tags for it, and a confidence score is provided for each semantic tag. These semantic tags and their relationships with the corresponding services are added back to the basic knowledge graph to form an enhanced knowledge graph.

[0020] All entities and relationships in the enhanced knowledge graph are given corresponding high-quality text descriptions, which are then converted into vectors using a text embedding model and stored in a vector database.

[0021] Furthermore, step S4 specifically includes:

[0022] Acquire a contextual snapshot of the user, which includes commands, sensor data, and user context information;

[0023] Using instruction keywords and sensor data, the most relevant knowledge graph entities are retrieved from a vector database;

[0024] The front-end LLM integrates the most relevant knowledge graph entities from the context snapshot and retrieval, refines user needs, and outputs a structured scenario analysis report, which includes current formal observations.

[0025] Furthermore, the scenario analysis report also includes a judgment on the complexity of the current instruction. If the instruction complexity is simple, the instruction is directly used as the final decision and the process jumps to step S6; if the instruction complexity is complex, the process proceeds to step S5.

[0026] Further, in step S5, the abductive reasoning agent receives formal observations from the scenario analysis report, searches for similar examples from the example library, constructs a hint based on the current formal observations, and then calls an inference LLM to generate multiple candidate logical hypotheses and gives a logical reasonableness score for each candidate logical hypothesis.

[0027] The user preference agent constructs a long-term memory model based on the Mem0 concept to analyze users' historical behavior and preferences, and provides a personalized compliance score for the service sequence corresponding to each candidate logical hypothesis.

[0028] The conflict detection agent combines the service sequence corresponding to each candidate logical hypothesis with the overall state of the current home to construct a temporary automated graph, detects whether there are potential conflicts, and then gives a conflict risk score to the service sequence corresponding to each candidate logical hypothesis.

[0029] All candidate logical assumptions and their corresponding logical rationality scores, personalized compliance scores, and conflict risk scores are compiled into a structured review report;

[0030] Input the scenario analysis report and deliberation report into the core decision-making LLM, output the final decision, and generate an explanation of the decision.

[0031] Furthermore, it also includes:

[0032] Online learning and evolution: Obtain subsequent user feedback and fine-tune and optimize the decision-making strategy of core decision-making LLM based on the feedback.

[0033] Furthermore, the process of online learning and evolution specifically includes:

[0034] Obtain user's subsequent behavior as implicit feedback or receive explicit feedback from the user;

[0035] When the received implicit or explicit feedback is negative, the entire decision chain leading to failure is triggered by the meta-LLM analysis, and the decision strategy (system prompt words) of the core decision LLM is fine-tuned and optimized.

[0036] Secondly, a smart home control system based on graph retrieval-enhanced generation and multi-agent game theory is provided, including:

[0037] The knowledge graph construction module is used to instantiate all entities and their relationships in the current smart home environment and build a basic knowledge graph; it also semantically enriches the service entities in the basic knowledge graph, generates an enhanced knowledge graph, vectorizes it, and stores it in a vector database.

[0038] The example library building module is used to build example libraries containing several formal observation-logic hypothesis pairs, where the formal observation is a description of the environment state and the logical hypothesis is a logical expression that can explain the cause of the formal observation or the user's intention.

[0039] The scenario analysis module is used to obtain a snapshot of the user's context. Based on the context snapshot and vectorized database, the front-end LLM refines the user's needs and outputs a structured scenario analysis report, which includes the current formal observations.

[0040] The multi-agent collaborative decision-making module is used to generate multiple candidate logical hypotheses based on the scenario analysis report and example library through the abductive reasoning agent, and to give a logical rationality score for each candidate logical hypothesis; to give a personalized compliance score for the service sequence corresponding to each candidate logical hypothesis through the user preference agent; to give a conflict risk score for the service sequence corresponding to each candidate logical hypothesis through the conflict detection agent; to input the scenario analysis report and the review report including the logical rationality score, personalized compliance score and conflict risk score into the core decision LLM, output the final decision, and generate an explanation of the decision;

[0041] The instruction issuance module is used to map the final decision into device control instructions and call the corresponding device to execute them.

[0042] Furthermore, it also includes an online learning module, used to obtain subsequent user feedback and to fine-tune and optimize the decision-making strategy of the core decision-making LLM based on the feedback.

[0043] Thirdly, a smart home control system based on graph retrieval-enhanced generation and multi-agent game theory is provided, including a control module and interactive devices, data acquisition devices, and execution devices that are connected to it in communication.

[0044] The interactive device is used to collect user input commands and transmit them to the control module;

[0045] The data acquisition device is used to collect current smart home environment status data and current status data of the execution device, and transmit them to the control module;

[0046] The control module is configured to execute the smart home control method based on graph retrieval-enhanced generation and multi-agent game theory as described above;

[0047] The execution device is used to receive device control commands issued by the control module and execute corresponding operations.

[0048] This invention proposes a smart home control method, system, and medium based on graph retrieval-enhanced generation and multi-agent game theory, which has the following advantages compared with existing technologies:

[0049] 1. Higher intent recognition accuracy: By combining the user's fuzzy instructions with a structured knowledge graph, the system can penetrate the linguistic appearance and accurately capture the user's real needs in a specific context, which is far superior to simple keyword matching or guessing by a single LLM.

[0050] 2. Significantly improved robustness and safety of decision-making: The innovative multi-agent parallel deliberation mechanism cross-validates candidate solutions from three key dimensions: logic, personalization, and safety. This effectively avoids the "illusion" and logical fallacies that may occur with a single model, and greatly reduces the risk of executing errors or dangerous operations.

[0051] 3. Achieved deep personalized services: The user preference agent constructs dynamic user profiles through long-term memory and learning, making each decision more in line with the habits and preferences of specific users, thus improving the user experience.

[0052] 4. Enhanced system interpretability: The structured "deliberation report" and the decision explanation generated by the core decision LLM make the entire reasoning process transparent, allowing users to understand why the system makes a certain decision and enhancing the trust between humans and machines.

[0053] 5. Balancing efficiency and depth: The dynamic path selection mechanism can intelligently distinguish between simple and complex tasks, respond quickly to simple instructions, and conduct in-depth deliberation on complex instructions, thus optimizing the system's response efficiency while ensuring decision-making quality.

[0054] 6. Possesses continuous evolution capabilities: The introduced online learning mechanism can automatically iterate its core decision-making logic based on user feedback. This "meta-learning" capability can adapt to changes in the environment and user habits, achieving true intelligent growth, and the optimization cost is far lower than retraining the entire model. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 This is a flowchart of a smart home control method based on graph retrieval enhancement generation and multi-agent game theory provided by an embodiment of the present invention;

[0057] Figure 2 This is a schematic diagram of a smart home control system based on graph retrieval-enhanced generation and multi-agent game theory, provided by an embodiment of the present invention. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be described in detail below. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other implementation methods obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0059] Existing technologies suffer from key problems such as inaccurate understanding, insecure decision-making, lack of personalization, and poor interpretability when dealing with complex commands that are often ambiguous, personalized, and context-dependent in smart home scenarios. Therefore, this invention provides a smart home control method, system, and medium based on graph retrieval-enhanced generation and multi-agent game theory, aiming to achieve the following objectives:

[0060] 1. Accurately understand ambiguous intentions: By combining knowledge graphs and large language models, deeply analyze users' real and potential needs in specific contexts.

[0061] 2. Enhance the logic and security of decision-making: By using multiple agents to conduct parallel and multi-dimensional deliberation and evaluation of candidate solutions, the final decision is ensured to be logical and risks are avoided.

[0062] 3. Achieve highly personalized services: By building a long-term memory model for users, the system's decisions can be aligned with users' personal preferences and habits.

[0063] 4. Enhance the system's explainability and continuous optimization capabilities: Through structured decision-making processes and online learning mechanisms based on user feedback, the system can evolve itself and continuously improve service quality.

[0064] The technical solution of the present invention will be described in detail below with reference to specific embodiments.

[0065] Example 1

[0066] like Figure 1 As shown, this embodiment discloses a smart home control method based on graph retrieval-enhanced generation and multi-agent game theory, including the following steps:

[0067] S1: Instantiate all entities and their relationships in the current smart home environment to build a basic knowledge graph.

[0068] Specifically, based on predefined patterns, all entities (such as devices, services, locations, users, sensors, etc.) and their relationships in the smart home environment are instantiated to construct a basic knowledge graph.

[0069] In this embodiment, the following scenario will be used as an example: On a winter night, user Xiaoming has just returned to his bedroom from outside and feels a little cold; the current temperature in the bedroom is 18°C ​​and the window is open; Xiaoming says to the smart speaker: "I'm so cold."

[0070] In this scenario, it is necessary to construct a basic knowledge graph that includes the bedroom environment, where entities include "air conditioner" (services: heating, cooling), "window" (status: open / closed), "temperature sensor", "user Xiaoming", etc.

[0071] S2: Enrich the semantics of the service entities in the basic knowledge graph, generate an enhanced knowledge graph, vectorize it, and store it in a vector database.

[0072] The specific process includes:

[0073] For each service entity (e.g., "air conditioner.heating") in the basic knowledge graph, a Large Language Model (LLM) is used to generate relevant semantic tags, and a confidence score is provided for each semantic tag. For example, for "desk lamp.night light mode", LLM may generate semantic tags such as "sleep-aiding atmosphere" and "night safety"; for the "air conditioner.heating" service, LLM may generate semantic tags such as "warm feeling" and "cold-dispelling". A confidence score is provided for each semantic tag. These semantic tags and their relationship with the corresponding service (with confidence weights) are added back to the basic knowledge graph to form an enhanced knowledge graph.

[0074] RAG (Retrieval-Augmented Generation) vectorization: Generates high-quality text descriptions of all entities and relations in the enhanced knowledge graph, and uses a text embedding model to convert them into vectors, which are then stored in a vector database to support fast semantic retrieval.

[0075] S3: Construct an example library containing several formal observation-logic hypothesis pairs, where the formal observation is a description of the environmental state, and the logical hypothesis is a logical expression that can explain the possible cause or user intent of the formal observation; this example library is used for subsequent in-context learning by the agent. For example: Formal observation: {"Need type": "I'm so cold", "Entity": "Warm feeling", "Location": "Bedroom"}, with corresponding logical hypotheses: {The user wants to increase the bedroom temperature} (corresponding to the user intent), or {The bedroom window is open} (corresponding to a possible cause).

[0076] S4: Obtain a snapshot of the user's context, refine user needs based on the context snapshot and vectorized database through the front-end LLM, and output a structured scenario analysis report, which includes the current formal observations.

[0077] This step is the GraphRAG-driven scene parsing process, which specifically includes:

[0078] Acquire a context snapshot of the user, which includes instructions (input by the user, such as "I'm so cold"), sensor data (such as temperature 18°C), and user context information (such as location: bedroom); therefore, a context snapshot can be represented as: {Instruction: "I'm so cold", Temperature: "18°C", Location: "Bedroom", User: "Xiaoming"}.

[0079] Using instruction keywords (such as "cold") and sensor data (such as temperature 18°C), the most relevant knowledge graph entities are retrieved from the vector database, such as the semantic entity "warm feeling" and the physical entities "temperature sensor" and "air conditioner".

[0080] The front-end LLM integrates the context snapshot and the most relevant knowledge graph entity information retrieved to refine user needs and output a structured scenario analysis report, which includes the current formal observations. In some embodiments, the output scenario analysis report can be directly input into step S5 for in-depth review. However, user input instructions are not all vague (e.g., "I'm so cold"); many instructions are simple and explicit (e.g., "Close the window," "Turn up the lights," etc.). If multi-agent collaborative decision-making is used for all of them, the response efficiency will be reduced. Therefore, in some preferred embodiments, in order to balance efficiency and depth, a dynamic path selection mechanism is used to intelligently distinguish between simple and complex tasks, responding quickly to simple instructions and conducting in-depth review of complex instructions. This ensures decision quality while optimizing response efficiency. Specifically, in addition to formal observations, the scenario analysis report also includes a judgment on the complexity of the current instruction (simple / complex). If the instruction complexity is simple, the instruction is directly used as the final decision and the process jumps to step S6 to execute an explicit service sequence. If the instruction complexity is complex, the process proceeds to step S5 for multi-agent review.

[0081] In this embodiment, by refining user needs, the core content of the output scenario analysis report includes: Formal observation: {"Need type": "I'm so cold", "Entity": "Warm feeling", "Location": "Bedroom"}; Instruction complexity: Complex (because "cold" is a vague concept and requires complex judgment based on the environmental state). Due to the complex complexity, proceed to step S5 to initiate multi-agent deliberation.

[0082] S5: The abductive reasoning agent generates multiple candidate logical hypotheses based on the scenario analysis report and example library, and gives a logical rationality score for each candidate logical hypothesis; the user preference agent gives a personalized compliance score for the service sequence corresponding to each candidate logical hypothesis; the conflict detection agent gives a conflict risk score for the service sequence corresponding to each candidate logical hypothesis; the scenario analysis report and the review report including the logical rationality score, personalized compliance score and conflict risk score are input into the core decision LLM, output the final decision, and generate a natural language explanation of the decision.

[0083] The abductive reasoning agent receives formal observations from the scene analysis report, searches for similar examples in the example library, constructs a prompt based on the current formal observations, and then calls an inference LLM (as the inference engine) to generate multiple candidate logical hypotheses (i.e., possible user intentions, such as "the user wants to increase the bedroom temperature"), and gives a logical plausibility score for each candidate logical hypothesis. For example, it generates the following two candidate logical hypotheses H1 and H2:

[0084] H1: The user wants to increase the bedroom temperature. (Logical rationality: 0.95)

[0085] H2: The user wants to wear thicker clothes. (Logical validity: 0.6, because the system cannot control the clothing).

[0086] Now focus on H1 and deduce the specific service sequence C1 that satisfies H1: Turn on the bedroom air conditioner and set it to heating mode.

[0087] The user preference agent is built based on the Mem0 concept to construct a long-term memory model to analyze users' historical behavior and preferences. The user preference agent receives candidate logical hypotheses generated by the abductive reasoning agent and assigns a personalized compliance score to the service sequence corresponding to each candidate logical hypothesis (e.g., "turn on the bedroom air conditioner to heating mode at 25 degrees"). Specifically, the user preference agent queries user Xiaoming's historical records and finds that Xiaoming tends to set the air conditioner to 25°C when he feels cold in winter. Therefore, it assigns a high personalized compliance score (0.9) to "turn on the bedroom air conditioner to heating mode at 25 degrees".

[0088] The conflict detection agent combines the service sequence corresponding to each candidate logical hypothesis with the overall state of the current home to construct a temporary automated graph. It then detects potential conflicts and assigns a conflict risk score to the service sequence corresponding to each candidate logical hypothesis. For example, upon receiving service sequence C1, the automated graph detects that the current "window" state is "open," which conflicts with "turning on the bedroom air conditioner," resulting in energy waste. Therefore, it assigns a high conflict risk score (0.85) to C1 and reports the conflict source: {Conflict type: "energy waste", involved entities: ["air conditioner", "window"]}.

[0089] Summary and Decision-Making: All candidate logical assumptions and their corresponding logical rationality scores, personalization compliance scores, and conflict risk scores are summarized into a structured review report. Then, the scenario analysis report and review report are input into the core decision-making LLM. The core decision-making LLM weighs the following: the user's core need is heating (high logical rationality, high personalization compliance), but a conflict exists. The optimal solution is to resolve the conflict first, then satisfy the need. Finally, the core decision-making LLM outputs the final, most reasonable decision and generates a natural language explanation of the decision: "Detecting that the window is still open, I have closed it for you and turned on the air conditioner to your preferred temperature of 25 degrees Celsius."

[0090] S6: Map the final decision to specific device control commands (such as device ID, gateway ID, command parameters), and then execute them through API calls. In this embodiment, the command to close the window is executed first, and then the command to turn on the air conditioner for heating is executed.

[0091] To enable continuous evolution, some preferred embodiments further include:

[0092] S7: Online Learning and Evolution: Obtain subsequent user feedback and fine-tune and optimize the decision-making strategy of the core decision-making LLM based on the feedback.

[0093] The specific process of online learning and evolution includes:

[0094] Acquire subsequent user behavior as implicit feedback (e.g., the user manually turns the air conditioner up to 27°C a few minutes later) or receive explicit user feedback;

[0095] When the received implicit or explicit feedback is negative, the negative feedback signal (negative reward value) is captured; this triggers a meta-LLM analysis of the entire decision chain leading to failure, and fine-tunes and optimizes the decision strategy (system prompts) of the core decision LLM. For example, in this embodiment, it is found that the strategy of "the user's preferred 25 degrees" may not be perfect. It will fine-tune the system prompts of the core decision LLM, possibly adding a rule: "...when the outside temperature is below 5°C, the user's preferred temperature can be appropriately increased by 2°C for recommendation..." In this way, more accurate decisions will be made in similar scenarios in the future.

[0096] The smart home control method based on graph retrieval-enhanced generation and multi-agent game theory disclosed in the above embodiments has the following beneficial effects: Higher intent recognition accuracy: By combining fuzzy user commands with structured knowledge graphs through GraphRAG technology, the system can penetrate linguistic representations and accurately capture the user's true needs in specific situations, far superior to simple keyword matching or guessing by a single LLM; Significantly improved robustness and security of decision-making: The innovative multi-agent parallel deliberation mechanism cross-validates candidate solutions from three key dimensions: logic, personalization, and security, effectively avoiding the "illusion" and logical fallacies that may occur with a single model, greatly reducing the risk of executing errors or dangerous operations; Achieved deep personalized services: The user preference agent constructs a dynamic user profile through long-term memory and learning. This approach makes each decision more aligned with the habits and preferences of specific users, enhancing the user experience; it also improves the system's interpretability: structured "deliberation reports" and decision explanations generated by the core decision-making LLM make the entire reasoning process transparent, allowing users to understand why the system makes a certain decision and enhancing trust between humans and machines; it balances efficiency and depth: the dynamic path selection mechanism intelligently distinguishes between simple and complex tasks, responding quickly to simple instructions and conducting in-depth deliberation on complex instructions, ensuring decision quality while optimizing system response efficiency; and it possesses continuous evolution capabilities: the introduced online learning mechanism can automatically iterate its core decision logic based on user feedback. This "meta-learning" capability adapts to changes in the environment and user habits, achieving true intelligent growth, and the optimization cost is far lower than retraining the entire model.

[0097] Example 2

[0098] This embodiment discloses a smart home control system based on graph retrieval-enhanced generation and multi-agent game theory, including:

[0099] The knowledge graph construction module is used to instantiate all entities and their relationships in the current smart home environment and build a basic knowledge graph; it also semantically enriches the service entities in the basic knowledge graph, generates an enhanced knowledge graph, vectorizes it, and stores it in a vector database.

[0100] The example library building module is used to build example libraries containing several formal observation-logic hypothesis pairs, where the formal observation is a description of the environment state and the logical hypothesis is a logical expression that can explain the possible causes or user intentions of the formal observation.

[0101] The scenario analysis module is used to obtain a snapshot of the user's context. Based on the context snapshot and vectorized database, the front-end LLM refines the user's needs and outputs a structured scenario analysis report, which includes the current formal observations.

[0102] The multi-agent collaborative decision-making module is used to generate multiple candidate logical hypotheses based on the scenario analysis report and example library through the abductive reasoning agent, and to give a logical rationality score for each candidate logical hypothesis; to give a personalized compliance score for the service sequence corresponding to each candidate logical hypothesis through the user preference agent; to give a conflict risk score for the service sequence corresponding to each candidate logical hypothesis through the conflict detection agent; to input the scenario analysis report and the review report including the logical rationality score, personalized compliance score and conflict risk score into the core decision LLM, output the final decision, and generate a natural language explanation of the decision;

[0103] The instruction issuance module is used to map the final decision into device control instructions and call the corresponding device to execute them.

[0104] In some preferred embodiments, the system further includes an online learning module for obtaining subsequent user feedback and fine-tuning and optimizing the decision-making strategy of the core decision-making LLM based on the feedback.

[0105] It should be understood that the functional unit modules in the various embodiments of the present invention can be concentrated in one processing unit, or each unit module can exist physically separately, or two or more unit modules can be integrated into one unit module, and can be implemented in hardware or software.

[0106] Example 3

[0107] This embodiment discloses a smart home control system based on graph retrieval-enhanced generation and multi-agent game theory, such as... Figure 2 As shown, it includes a control module 1 and an interactive device 2, a data acquisition device 3, and an execution device 4 that are connected to it in communication.

[0108] The interactive device 2 is used to collect user input commands and transmit them to the control module 1;

[0109] The data acquisition device 3 is used to collect current smart home environment status data and current status data of the execution device, and transmit them to the control module 1;

[0110] The control module 1 is configured to execute the smart home control method based on graph retrieval enhancement generation and multi-agent game as described above;

[0111] The execution device 4 is used to receive the device control commands issued by the control module 1 and execute the corresponding operations.

[0112] Among them, the interactive device 2 can be a smart speaker or an application for a terminal device; the data acquisition device 3 includes a home gateway and various sensors (temperature sensor, humidity sensor, light sensor, air quality sensor, human body detection sensor, etc.); the execution device 4 includes lights, air conditioners, fans, electric curtains, televisions, electric doors and windows, etc.

[0113] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0114] In the technical solution disclosed herein, the collection, storage, use, processing, transmission, provision, disclosure, and application of user personal information all comply with relevant laws and regulations, necessary confidentiality measures have been taken, and they do not violate public order and good morals. In the technical solution disclosed herein, user authorization or consent has been obtained before acquiring or collecting user personal information.

[0115] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A smart home control method based on graph retrieval-enhanced generation and multi-agent game theory, characterized in that, Includes the following steps: S1: Instantiate all entities and their relationships in the current smart home environment to build a basic knowledge graph; S2: Enrich the semantics of the service entities in the basic knowledge graph, generate an enhanced knowledge graph, vectorize it, and store it in a vector database; S3: Build an example library containing several formal observation-logic hypothesis pairs, where a formal observation is a description of the state of the environment and a logical hypothesis is a logical expression that can explain the cause of the formal observation or the user's intention. S4: Obtain a snapshot of the user's context, refine user needs based on the context snapshot and vectorized database through the front-end LLM, and output a structured scenario analysis report, which includes the current formal observations; S5: The abductive reasoning agent generates multiple candidate logical hypotheses based on the scenario analysis report and example library, and gives a logical rationality score for each candidate logical hypothesis; The user-preference-based intelligent agent provides a personalized compliance score for the service sequence corresponding to each candidate logical hypothesis. The conflict detection agent assigns a conflict risk score to the service sequence corresponding to each candidate logical hypothesis. Input the scenario analysis report and the review report, which includes logical rationality score, personalized compliance score and conflict risk score, into the core decision-making LLM, output the final decision, and generate an explanation of the decision; S6: Map the final decision to device control commands and call the corresponding device to execute them.

2. The smart home control method based on graph retrieval-enhanced generation and multi-agent game theory according to claim 1, characterized in that, Step S2 specifically includes: For each service entity in the basic knowledge graph, an LLM is used to generate relevant semantic tags for it, and a confidence score is provided for each semantic tag. These semantic tags and their relationships with the corresponding services are added back to the basic knowledge graph to form an enhanced knowledge graph. All entities and relationships in the enhanced knowledge graph are given corresponding high-quality text descriptions, which are then converted into vectors using a text embedding model and stored in a vector database.

3. The smart home control method based on graph retrieval-enhanced generation and multi-agent game theory according to claim 1, characterized in that, Step S4 specifically includes: Acquire a contextual snapshot of the user, which includes commands, sensor data, and user context information; Using instruction keywords and sensor data, the most relevant knowledge graph entities are retrieved from a vector database; The front-end LLM integrates the most relevant knowledge graph entities from the context snapshot and retrieval, refines user needs, and outputs a structured scenario analysis report, which includes current formal observations.

4. The smart home control method based on graph retrieval-enhanced generation and multi-agent game theory according to claim 1, characterized in that, The scenario analysis report also includes a judgment on the complexity of the current instruction. If the instruction complexity is simple, the instruction is directly used as the final decision and the process jumps to step S6; if the instruction complexity is complex, the process proceeds to step S5.

5. The smart home control method based on graph retrieval-enhanced generation and multi-agent game theory according to claim 1, characterized in that, In step S5, the abductive reasoning agent receives formal observations from the scenario analysis report, searches for similar examples from the example library, constructs a hint based on the current formal observations, and then calls an inference LLM to generate multiple candidate logical hypotheses and gives a logical reasonableness score for each candidate logical hypothesis. The user preference agent constructs a long-term memory model based on the Mem0 concept to analyze users' historical behavior and preferences, and provides a personalized compliance score for the service sequence corresponding to each candidate logical hypothesis. The conflict detection agent combines the service sequence corresponding to each candidate logical hypothesis with the overall state of the current home to construct a temporary automated graph, detects whether there are potential conflicts, and then gives a conflict risk score to the service sequence corresponding to each candidate logical hypothesis. All candidate logical assumptions and their corresponding logical rationality scores, personalized compliance scores, and conflict risk scores are compiled into a structured review report; Input the scenario analysis report and deliberation report into the core decision-making LLM, output the final decision, and generate an explanation of the decision.

6. The smart home control method based on graph retrieval-enhanced generation and multi-agent game theory according to any one of claims 1 to 5, characterized in that, Also includes: Online learning and evolution: Obtain subsequent user feedback and fine-tune and optimize the decision-making strategy of core decision-making LLM based on the feedback.

7. The smart home control method based on graph retrieval-enhanced generation and multi-agent game theory according to claim 6, characterized in that, The process of online learning and evolution specifically includes: Obtain user's subsequent behavior as implicit feedback or receive explicit feedback from the user; When the received implicit or explicit feedback is negative, the meta-LLM analysis is triggered, leading to the failure of the entire decision chain, and the decision-making strategy of the core decision LLM is fine-tuned and optimized.

8. A smart home control system based on graph retrieval-enhanced generation and multi-agent game theory, characterized in that, include: The knowledge graph construction module is used to instantiate all entities and their relationships in the current smart home environment and build a basic knowledge graph. The service entities in the basic knowledge graph are semantically enriched to generate an enhanced knowledge graph, which is then vectorized and stored in a vector database. The example library building module is used to build example libraries containing several formal observation-logic hypothesis pairs, where the formal observation is a description of the environment state and the logical hypothesis is a logical expression that can explain the cause of the formal observation or the user's intention. The scenario analysis module is used to obtain a snapshot of the user's context. Based on the context snapshot and vectorized database, the front-end LLM refines the user's needs and outputs a structured scenario analysis report, which includes the current formal observations. Multi-agent collaborative decision-making module: used to generate multiple candidate logical hypotheses based on scenario analysis reports and example libraries through abductive reasoning agents, and to give a logical rationality score for each candidate logical hypothesis; The user-preference-based intelligent agent provides a personalized compliance score for the service sequence corresponding to each candidate logical hypothesis. The conflict detection agent assigns a conflict risk score to the service sequence corresponding to each candidate logical hypothesis. Input the scenario analysis report and the review report, which includes logical rationality score, personalized compliance score and conflict risk score, into the core decision-making LLM, output the final decision, and generate an explanation of the decision; The instruction issuance module is used to map the final decision into device control instructions and call the corresponding device to execute them.

9. The smart home control system based on graph retrieval-enhanced generation and multi-agent game theory according to claim 8, characterized in that, Also includes: The online learning module is used to obtain subsequent user feedback and to fine-tune and optimize the decision-making strategy of the core decision-making LLM based on the feedback.

10. A smart home control system based on graph retrieval-enhanced generation and multi-agent game theory, characterized in that, This includes the control module and the interactive devices, data acquisition devices, and execution devices that are connected to it in communication. The interactive device is used to collect user input commands and transmit them to the control module; The data acquisition device is used to collect current smart home environment status data and current status data of the execution device, and transmit them to the control module; The control module is configured to execute the smart home control method based on graph retrieval-enhanced generation and multi-agent game theory as described in any one of claims 1 to 7; The execution device is used to receive device control commands issued by the control module and execute corresponding operations.

Images