Smart home control method and system based on graph retrieval enhanced generation and multi-agent game

Abstract

The invention discloses a smart home control method and system based on graph retrieval enhanced generation and multi-agent gaming, and the method comprises the steps: constructing a basic knowledge graph based on a current smart home environment, carrying out the semantic enrichment of service entities in the basic knowledge graph, generating an enhanced knowledge graph, and storing the enhanced knowledge graph in a vector database; constructing an example library containing a plurality of formalized observation-logic hypothesis pairs; acquiring a situation snapshot of the user, and outputting a structured scene analysis report based on the situation snapshot and the vectorization database through a front-end LLM; collaborative review is carried out by combining a factor-tracing reasoning agent, a user preference agent and a conflict detection agent with a scene analysis report and an example library, and a final decision is output by a core decision LLM and mapped into an equipment control instruction to call corresponding equipment for execution. According to the method, the fuzzy instruction of the user is combined with the knowledge graph, so that the intention recognition accuracy is improved; the multi-agent parallel auditing mechanism improves the robustness and safety of decision making.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of smart home control, and in particular to a smart home control method and system based on graph retrieval enhanced generation and multi-agent game. BACKGROUND

[0002] In recent years, with the popularization of smart home devices, users have higher requirements for the automation and intelligent control of home environments. There are mainly two kinds of current mainstream smart home control methods:

[0003] 1. Method based on fixed rules or simple intent recognition: This kind of system usually presets "if... then..." rules or recognizes user intent through keyword matching. For example, when the user says "turn on the living room light", the system can accurately execute. However, the limitations of this method are very obvious, it cannot understand ambiguous, colloquial or context-dependent instructions. For example, when the user says "I feel a little cold" at night, a simple system may not understand that it is a suggestion to "close the window and start the air conditioning heating mode", and it cannot make personalized judgments combined with current room temperature, user past habits, etc.

[0004] 2. End-to-end control method based on single large language model (LLM): In order to overcome the rigidity of rule systems, some solutions try to directly use the powerful natural language understanding ability of large language models for end-to-end decision making. LLM can better understand ambiguous instructions and context. However, this method also has serious defects: (1) "hallucination" problem: LLM may produce "hallucinations" that are inconsistent with reality during reasoning, making unsafe operations such as turning on heating devices when no one is present; (2) Logical inconsistency: When facing complex, multi-constrained scenarios, the decision-making process of a single LLM lacks structured reasoning and verification, and is prone to logical contradictions; (3) Lack of explainability: The decision-making process of LLM is like a "black box", when it makes an unexpected decision, it is difficult to trace the reason and difficult to optimize; (4) Limited personalization: Although LLM has a vast amount of knowledge, it still lacks understanding of the preferences and dynamic changes of specific users and specific home environments.

[0005] Therefore, the existing technology has the key problems of inaccurate understanding, unsafe decision making, lack of personalization, and poor explainability when dealing with complex instructions that are often ambiguous, personalized, and context-dependent in smart home scenarios. SUMMARY

[0006] The application provides a smart home control method and system based on graph search enhanced generation and multi-agent game, to solve the problems of inaccurate understanding, unsafe decision-making, lack of personalization and poor interpretability in the prior art when dealing with complex instructions that are often fuzzy, personalized and context-dependent in smart home scenarios.

[0007] In a first aspect, a smart home control method based on graph search enhanced generation and multi-agent game is provided, comprising the following steps: S1: instantiate all entities and their relationships in the current smart home environment, and construct a basic knowledge graph; S2: semantically enrich the service entities in the basic knowledge graph, generate an enhanced knowledge graph, and vectorize and store it in a vector database; S3: construct an example library containing a plurality of formalized observation-logical hypothesis pairs, wherein the formalized observation is a description of the environment state, and the logical hypothesis is a logical expression that can explain the reason or user intent of the formalized observation; S4: obtain a context snapshot of the user, purify the user's demand based on the context snapshot and the vectorized database through the front-end LLM, and output a structured scene analysis report, which includes the current formalized observation; S5: generate a plurality of candidate logical hypotheses based on the scene analysis report and the example library through a causal reasoning agent, and give a logical rationality score for each candidate logical hypothesis; through a user preference agent, give an individualized compliance score for the service sequence corresponding to each candidate logical hypothesis; through a conflict detection agent, give a conflict risk score for the service sequence corresponding to each candidate logical hypothesis; input the scene analysis report and the review report including the logical rationality score, the individualized compliance score and the conflict risk score into the core decision LLM, output the final decision, and generate an explanation for the decision; S6: map the final decision to a device control instruction and call the corresponding device to execute.

[0008] Further, step S2 specifically comprises: traverse each service entity in the basic knowledge graph, use an LLM to generate relevant semantic tags for it, and provide a confidence score for each semantic tag; these semantic tags and their relationship with the corresponding service are added back to the basic knowledge graph to form an enhanced knowledge graph; generate high-quality text descriptions of all entities and relationships in the enhanced knowledge graph, and convert them into vectors using a text embedding model and store them in a vector database.

[0009] Further, step S4 specifically comprises: obtain a context snapshot of the user, which includes instructions, sensor data and user context information; Retrieving the most relevant knowledge graph entities from the vector database using instruction keywords and sensor data; The front-end LLM synthesizes the context snapshot and the most relevant knowledge graph entities retrieved, purifies the user demand, and outputs a structured scenario analysis report, which includes the current formalized observations.

[0010] Further, the scenario analysis report also includes a judgment of the current instruction complexity. If the instruction complexity is simple, the instruction is directly taken as the final decision and jumps to step S6. If the instruction complexity is complex, it enters step S5.

[0011] Further, in step S5, the abductive reasoning agent receives the formalized observations in the scenario analysis report, searches for similar examples from the example library, and constructs a hint combining the current formalized observations, and then calls a reasoning LLM to generate multiple candidate logical hypotheses and gives a logical rationality score for each candidate logical hypothesis. The user preference agent constructs a long-term memory model based on the Mem0 idea to analyze the user's historical behavior and preferences, and gives a personalized compliance score for each candidate logical hypothesis corresponding service sequence; The conflict detection agent combines the service sequence corresponding to each candidate logical hypothesis with the overall state of the current home to construct a temporary automation graph, detects whether there is a potential conflict, and then gives a conflict risk score for each candidate logical hypothesis corresponding service sequence; All candidate logical hypotheses and corresponding logical rationality scores, personalized compliance scores, and conflict risk scores are summarized into a structured review report; The scenario analysis report and the review report are input into the core decision LLM, and the final decision is output, and an explanation of the decision is generated.

[0012] Further, it also includes: Online learning and evolution: obtaining the user's subsequent feedback, and fine-tuning and optimizing the decision strategy of the core decision LLM based on the feedback.

[0013] Further, the process of online learning and evolution specifically includes: Obtaining the user's subsequent behavior as implicit feedback or receiving the user's explicit feedback; When the received implicit feedback or explicit feedback is negative feedback, trigger the meta-LLM to analyze the entire decision chain that leads to failure, and fine-tune and optimize the decision strategy (system prompt words) of the core decision LLM.

[0014] In the second aspect, an intelligent home control system based on graph retrieval enhanced generation and multi-agent game is provided, which includes: A knowledge graph construction module is configured to instantiate all entities and their relationships in the current smart home environment, construct a basic knowledge graph, enrich the service entities in the basic knowledge graph in semantics, generate an enhanced version of the knowledge graph, vectorize the enhanced version of the knowledge graph, and store the vectorized enhanced version of the knowledge graph in a vector database; An example library construction module is configured to construct an example library containing a plurality of pairs of formalized observations and logical hypotheses, wherein the formalized observation is a description of an environment state, and the logical hypothesis is a logical expression that can explain the reason or user intention of the formalized observation; A scene analysis module is configured to obtain a context snapshot of a user, purify the user demand based on the context snapshot and the vectorized database by using a front-end LLM, and output a structured scene analysis report, which includes a current formalized observation. A multi-agent collaborative decision-making module is configured to generate a plurality of candidate logical hypotheses based on the scene analysis report and the example library by using a backward reasoning agent, give a logical rationality score of each candidate logical hypothesis, give a personalized compliance score of a service sequence corresponding to each candidate logical hypothesis by using a user preference agent, give a conflict risk score of the service sequence corresponding to each candidate logical hypothesis by using a conflict detection agent, input the scene analysis report and a deliberation report including the logical rationality score, the personalized compliance score, and the conflict risk score into a core decision-making LLM, output a final decision, and generate an explanation of the decision. An instruction issuing module is configured to map the final decision into a device control instruction and call a corresponding device to execute the device control instruction.

[0015] Further, the system further comprises an online learning module configured to obtain subsequent feedback of the user and fine-tune and optimize the decision-making strategy of the core decision-making LLM based on the feedback.

[0016] In a third aspect, an intelligent home control system based on graph retrieval enhanced generation and multi-agent game is provided, which comprises a control module and an interactive device, a data acquisition device, and an execution device in communication with the control module. The interactive device is configured to acquire a user input instruction and transmit the instruction to the control module. The data acquisition device is configured to acquire current smart home environment state data and execution device current state data and transmit the data to the control module. The control module is configured to execute the intelligent home control method based on graph retrieval enhanced generation and multi-agent game as described above. The execution device is configured to receive a device control instruction issued by the control module and execute a corresponding operation.

[0017] The present application provides an intelligent home control method and system based on graph retrieval enhanced generation and multi-agent game, which has the following advantages compared with the prior art: 1. Higher intent recognition accuracy: Combining users' ambiguous instructions with structured knowledge graphs enables the system to penetrate language representations and accurately capture users' real needs in specific situations, far surpassing simple keyword matching or single LLM guessing.

[0018] 2. Robustness and safety of decision-making significantly improved: The innovative multi-agent parallel deliberation mechanism cross- validates candidate solutions from three key dimensions: logic, individualization, and safety, effectively avoiding the "hallucinations" and logical fallacies that single models may have, greatly reducing the risk of execution errors or dangerous operations.

[0019] 3. Achieving deep individualized service: User preference agents build dynamic user portraits through long-term memory and learning, making each decision more in line with the habits and preferences of specific users, improving user experience.

[0020] 4. Enhanced system explainability: Structured "deliberation reports" and decision explanations generated by core decision LLMs make the entire reasoning process transparent, allowing users to understand why the system made a certain decision, enhancing trust between humans and machines.

[0021] 5. Balancing efficiency and depth: Dynamic path selection mechanism intelligently distinguishes between simple and complex tasks, responding quickly to simple instructions and thoroughly deliberating on complex instructions, ensuring decision quality while optimizing system response efficiency.

[0022] 6. Ability to continuously evolve: Online learning mechanism introduced can automatically iterate its core decision logic based on user feedback. This "meta-learning" capability can adapt to changes in the environment and user habits, enabling true intelligent growth with significantly lower optimization costs than retraining the entire model. BRIEF DESCRIPTION OF DRAWINGS

[0023] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor based on these drawings.

[0024] Figure 1 is a flow chart of an intelligent home control method based on graph search enhanced generation and multi-agent game provided by an embodiment of the present application; Figure 2 is a structural schematic diagram of an intelligent home control system based on graph search enhanced generation and multi-agent game provided by an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described in detail below. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0026] The prior art has the key problems of inaccurate understanding, unsafe decision-making, lack of personalization and poor explainability when dealing with complex instructions that are common in smart home scenarios, which are fuzzy, personalized and context-dependent. Based on this, the present application provides a smart home control method and system based on graph retrieval enhanced generation and multi-agent game, and a medium, aiming to achieve the following purposes: 1. Accurate understanding of fuzzy intent: by combining knowledge graph and large language model, the real and potential needs of users in a specific context are deeply analyzed.

[0027] 2. Improve the logicality and safety of decision-making: through multi-agent parallel, multi-dimensional review and evaluation of candidate solutions, ensure that the final decision is logical and risk-averse.

[0028] 3. Achieve highly personalized service: by building a user long-term memory model, the system decision can meet the personal preferences and habits of users.

[0029] 4. Enhance the explainability and continuous optimization ability of the system: through structured decision-making process and online learning mechanism based on user feedback, the system can evolve and continuously improve service quality.

[0030] The technical solutions of the present application will be described in detail below in conjunction with specific embodiments.

[0031] Embodiment 1

[0032] As shown in Figure 1 The present embodiment discloses a smart home control method based on graph retrieval enhanced generation and multi-agent game, comprising the following steps: S1: instantiate all entities and their relationships in the current smart home environment, and construct a basic knowledge graph.

[0033] Specifically, according to the predefined mode, all entities (such as devices, services, locations, users, sensors, etc.) and their relationships in the smart home environment are instantiated, and a basic knowledge graph is constructed.

[0034] In this embodiment, the following scenario will be used as an example: on a winter night, user Xiaoming has just returned to his bedroom from outside and feels a bit cold; the current temperature in the bedroom is 18°C, and the window is open; Xiaoming says to the smart speaker: "I'm so cold."

[0035] In this scenario, a basic knowledge graph containing the bedroom environment needs to be constructed, which includes entities such as "air conditioner" (service: heating, cooling), "window" (status: open / close), "temperature sensor", "user Xiaoming", etc.

[0036] S2: Semantic enrichment of service entities in the basic knowledge graph, generating an enhanced knowledge graph, vectorizing and storing it in the vector database.

[0037] The specific process includes: Traverse each service entity in the basic knowledge graph (such as "air conditioner. heating"), use a large language model (LLM) to generate relevant semantic tags for it, and provide a confidence score for each semantic tag; for example, for "table lamp. night light mode", LLM may generate semantic tags such as "sleep aid atmosphere" and "night safety"; for "air conditioner. heating" service, LLM may generate semantic tags such as "warm feeling" and "cold prevention"; and provide a confidence score for each semantic tag. These semantic tags and their relationship with the corresponding service (with confidence weight) are added back to the basic knowledge graph to form an enhanced knowledge graph; RAG (Retrieval-Augmented Generation) vectorization: generate high-quality text descriptions for all entities and relationships in the enhanced knowledge graph, and convert them into vectors using a text embedding model, and store them in the vector database to support fast semantic retrieval.

[0038] S3: Construct an example library containing several formalized observation-logical hypothesis pairs, where the formalized observation is a description of the environment state, and the logical hypothesis is a logical expression that can explain the possible reason or user intent of the formalized observation; this example library is used for subsequent context learning of the agent. For example: formalized observation: {"demand type": "I'm so cold", "entity": "warm feeling", "location": "bedroom"}, its corresponding logical hypothesis: {user wants to raise the temperature in the bedroom} (corresponding to user intent), or {the bedroom window is open} (corresponding to possible reason).

[0039] S4: Obtain the user's context snapshot, purify the user's demand based on the context snapshot and the vectorized database through the front-end LLM, and output a structured scene analysis report, which includes the current formalized observation.

[0040] This step is a GraphRAG-driven scene analysis process, which specifically includes: Obtain the context snapshot of the user, which includes the instruction (input by the user, such as "I am so cold"), sensor data (such as temperature of 18°C), and user context information (such as location: bedroom); therefore, the context snapshot can be represented as: {instruction: "I am so cold", temperature: "18°C", location: "bedroom", user: "Xiaoming"}.

[0041] Using the instruction keyword (such as "cold") and sensor data (such as temperature of 18°C), retrieve the most relevant knowledge graph entities from the vector database, such as retrieving the semantic entity "warm feeling" and the physical entities "temperature sensor" and "air conditioner".

[0042] The front-end LLM synthesizes the context snapshot and the retrieved most relevant knowledge graph entity information, purifies the user demand, and outputs a structured scene analysis report, which includes the current formalized observation. In some embodiments, the output scene analysis report can be directly input to step S5 for in-depth deliberation. However, not all user input instructions are ambiguous instructions (such as "I am so cold"), many instructions are simple and clear instructions (such as "close the window", "turn on the light", etc.), and if all of them are subject to multi-agent collaborative decision-making, the response efficiency will be reduced. Therefore, in some preferred embodiments, in order to balance efficiency and depth, a dynamic path selection mechanism is used to intelligently distinguish between simple tasks and complex tasks, simple instructions are responded quickly, and complex instructions are deliberated in-depth, thereby ensuring decision-making quality while optimizing response efficiency; specifically, in addition to the formalized observation, the scene analysis report also includes the judgment of the complexity of the current instruction (simple / complex), if the instruction complexity is simple, the instruction is directly taken as the final decision and jumps to step S6 to execute the explicit service sequence; if the instruction complexity is complex, it enters step S5 for multi-agent deliberation.

[0043] In this embodiment, by purifying the user demand, the core content of the output scene analysis report includes: formalized observation: {"demand type": "I am so cold", "entity": "warm feeling", "location": "bedroom"}; instruction complexity: complex (because "cold" is a vague concept and needs to be combined with the environmental state for complex judgment). Since the complexity is complex, it enters step S5 to start multi-agent deliberation.

[0044] S5: The agent of abductive reasoning generates multiple candidate logical hypotheses based on the scene analysis report and the example library, and gives a logical rationality score for each candidate logical hypothesis; the user preference agent gives a personalized compliance score for the service sequence corresponding to each candidate logical hypothesis; the conflict detection agent gives a conflict risk score for the service sequence corresponding to each candidate logical hypothesis; the scene analysis report and the deliberation report including the logical rationality score, the personalized compliance score and the conflict risk score are input into the core decision LLM, and the final decision is output, and a natural language explanation of the decision is generated.

[0045] Agent of abductive reasoning: receives formalized observations in the scene analysis report, searches for similar examples from the example library, and constructs a prompt combining the current formalized observations, then calls a reasoning LLM (as a reasoning engine) to generate multiple candidate logical hypotheses (i.e. possible user intentions, such as "user wants to raise the temperature in the bedroom") and give a logical rationality score for each candidate logical hypothesis. For example, the following two candidate logical hypotheses H1 and H2 are generated: H1: User wants to raise the temperature in the bedroom. (Logical rationality: 0.95); H2: User wants to wear thicker clothes. (Logical rationality: 0.6, because the system cannot control clothing).

[0046] At this point, the focus is on H1, and the specific service sequence C1 that satisfies H1 is derived: turn on the bedroom air conditioner and set it to heating mode.

[0047] User preference agent: based on the Mem0 idea to build a long-term memory model to analyze the user's historical behavior and preferences, the user preference agent receives the candidate logical hypotheses generated by the agent of abductive reasoning, and gives a personalized compliance score for the service sequence corresponding to each candidate logical hypothesis (such as "turn on the bedroom air conditioner heating mode to 25 degrees"). Specifically, the user preference agent queries the historical records of user Xiaoming and finds that Xiaoming tends to set the air conditioner to 25°C when he feels cold in winter. Therefore, it gives a high personalized compliance score (0.9) to "turn on the bedroom air conditioner heating mode to 25 degrees".

[0048] The conflict detection agent combines the service sequence corresponding to each candidate logical hypothesis with the overall state of the current home to construct a temporary automated graph. It then detects potential conflicts and assigns a conflict risk score to the service sequence corresponding to each candidate logical hypothesis. For example, upon receiving service sequence C1, the automated graph detects that the current "window" state is "open," which conflicts with "turning on the bedroom air conditioner," resulting in energy waste. Therefore, it assigns a high conflict risk score (0.85) to C1 and reports the conflict source: {Conflict type: "energy waste", involved entities: ["air conditioner", "window"]}.

[0049] Summary and Decision-Making: All candidate logical assumptions and their corresponding logical rationality scores, personalization compliance scores, and conflict risk scores are summarized into a structured review report. Then, the scenario analysis report and review report are input into the core decision-making LLM. The core decision-making LLM weighs the following: the user's core need is heating (high logical rationality, high personalization compliance), but a conflict exists. The optimal solution is to resolve the conflict first, then satisfy the need. Finally, the core decision-making LLM outputs the final, most reasonable decision and generates a natural language explanation of the decision: "Detecting that the window is still open, I have closed it for you and turned on the air conditioner to your preferred temperature of 25 degrees Celsius."

[0050] S6: Map the final decision to specific device control commands (such as device ID, gateway ID, command parameters), and then execute them through API calls. In this embodiment, the command to close the window is executed first, and then the command to turn on the air conditioner for heating is executed.

[0051] To enable continuous evolution, some preferred embodiments further include: S7: Online Learning and Evolution: Obtain subsequent user feedback and fine-tune and optimize the decision-making strategy of the core decision-making LLM based on the feedback.

[0052] The specific process of online learning and evolution includes: Acquire subsequent user behavior as implicit feedback (e.g., the user manually turns the air conditioner up to 27°C a few minutes later) or receive explicit user feedback; When the received implicit or explicit feedback is negative, the negative feedback signal (negative reward value) is captured; this triggers a meta-LLM analysis of the entire decision chain leading to failure, and fine-tunes and optimizes the decision strategy (system prompts) of the core decision LLM. For example, in this embodiment, it is found that the strategy of "the user's preferred 25 degrees" may not be perfect. It will fine-tune the system prompts of the core decision LLM, possibly adding a rule: "...when the outside temperature is below 5°C, the user's preferred temperature can be appropriately increased by 2°C for recommendation..." In this way, more accurate decisions will be made in similar scenarios in the future.

[0053] The smart home control method based on graph retrieval-enhanced generation and multi-agent game theory disclosed in the above embodiments has the following beneficial effects: Higher intent recognition accuracy: By combining fuzzy user commands with structured knowledge graphs through GraphRAG technology, the system can penetrate linguistic representations and accurately capture the user's true needs in specific situations, far superior to simple keyword matching or guessing by a single LLM; Significantly improved robustness and security of decision-making: The innovative multi-agent parallel deliberation mechanism cross-validates candidate solutions from three key dimensions: logic, personalization, and security, effectively avoiding the "illusion" and logical fallacies that may occur with a single model, greatly reducing the risk of executing errors or dangerous operations; Achieved deep personalized services: The user preference agent constructs a dynamic user profile through long-term memory and learning. This approach makes each decision more aligned with the habits and preferences of specific users, enhancing the user experience; it also improves the system's interpretability: structured "deliberation reports" and decision explanations generated by the core decision-making LLM make the entire reasoning process transparent, allowing users to understand why the system makes a certain decision and enhancing trust between humans and machines; it balances efficiency and depth: the dynamic path selection mechanism intelligently distinguishes between simple and complex tasks, responding quickly to simple instructions and conducting in-depth deliberation on complex instructions, ensuring decision quality while optimizing system response efficiency; and it possesses continuous evolution capabilities: the introduced online learning mechanism can automatically iterate its core decision logic based on user feedback. This "meta-learning" capability adapts to changes in the environment and user habits, achieving true intelligent growth, and the optimization cost is far lower than retraining the entire model.

[0054] Example 2

[0055] This embodiment discloses a smart home control system based on graph retrieval-enhanced generation and multi-agent game theory, including: The knowledge graph construction module is used to instantiate all entities and their relationships in the current smart home environment and build a basic knowledge graph; it also semantically enriches the service entities in the basic knowledge graph, generates an enhanced knowledge graph, vectorizes it, and stores it in a vector database. The example library building module is used to build example libraries containing several formal observation-logic hypothesis pairs, where the formal observation is a description of the environment state and the logical hypothesis is a logical expression that can explain the possible causes or user intentions of the formal observation. The scenario analysis module is used to obtain a snapshot of the user's context. Based on the context snapshot and vectorized database, the front-end LLM refines the user's needs and outputs a structured scenario analysis report, which includes the current formal observations. The multi-agent collaborative decision-making module is used to generate multiple candidate logical hypotheses based on the scenario analysis report and example library through the abductive reasoning agent, and to give a logical rationality score for each candidate logical hypothesis; to give a personalized compliance score for the service sequence corresponding to each candidate logical hypothesis through the user preference agent; to give a conflict risk score for the service sequence corresponding to each candidate logical hypothesis through the conflict detection agent; to input the scenario analysis report and the review report including the logical rationality score, personalized compliance score and conflict risk score into the core decision LLM, output the final decision, and generate a natural language explanation of the decision; The instruction issuance module is used to map the final decision into device control instructions and call the corresponding device to execute them.

[0056] In some preferred embodiments, the system further includes an online learning module for obtaining subsequent user feedback and fine-tuning and optimizing the decision-making strategy of the core decision-making LLM based on the feedback.

[0057] It should be understood that the functional unit modules in the various embodiments of the present invention can be concentrated in one processing unit, or each unit module can exist physically separately, or two or more unit modules can be integrated into one unit module, and can be implemented in hardware or software.

[0058] Example 3

[0059] This embodiment discloses a smart home control system based on graph retrieval-enhanced generation and multi-agent game theory, such as... Figure 2 As shown, it includes a control module 1 and an interactive device 2, a data acquisition device 3, and an execution device 4 that are connected to it in communication. The interactive device 2 is used to collect user input commands and transmit them to the control module 1; The data acquisition device 3 is used to collect current smart home environment status data and current status data of the execution device, and transmit them to the control module 1; The control module 1 is configured to execute the smart home control method based on graph retrieval enhancement generation and multi-agent game as described above; The execution device 4 is used to receive the device control commands issued by the control module 1 and execute the corresponding operations.

[0060] Among them, the interactive device 2 can be a smart speaker or an application for a terminal device; the data acquisition device 3 includes a home gateway and various sensors (temperature sensor, humidity sensor, light sensor, air quality sensor, human body detection sensor, etc.); the execution device 4 includes lights, air conditioners, fans, electric curtains, televisions, electric doors and windows, etc.

[0061] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0062] In the technical solution disclosed herein, the collection, storage, use, processing, transmission, provision, disclosure, and application of user personal information all comply with relevant laws and regulations, necessary confidentiality measures have been taken, and they do not violate public order and good morals. In the technical solution disclosed herein, user authorization or consent has been obtained before acquiring or collecting user personal information.

[0063] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A smart home control method based on graph search enhanced generation and multi-agent game, characterized in that, Comprise the following steps: S1: instantiate all entities in the current smart home environment and their relationships, build a basic knowledge graph; S2: semantic enrichment of service entities in the basic knowledge graph, generate an enhanced version of the knowledge graph, vectorize and store it in the vector database; S3: build a library of several formal observation-logical hypothesis pairs, where the formal observation is a description of the environment state, and the logical hypothesis is a logical expression that can explain the reason or user intent of the formal observation; S4: obtain the user's context snapshot, purify the user's demand based on the context snapshot and the vector database through the front-end LLM, and output a structured scene analysis report, which includes the current formal observation; S5: through the abductive reasoning agent, generate multiple candidate logical hypotheses based on the scene analysis report and the example library, and give a logical rationality score for each candidate logical hypothesis; Through the user preference agent, give a personalized compliance score for the service sequence corresponding to each candidate logical hypothesis; Through the conflict detection agent, give a conflict risk score for the service sequence corresponding to each candidate logical hypothesis; Input the scene analysis report and the review report including the logical rationality score, the personalized compliance score and the conflict risk score into the core decision LLM, output the final decision, and generate the explanation of the decision; S6: map the final decision to device control instructions and call the corresponding device to execute. 2.The smart home control method based on graph search enhancement generation and multi-agent game according to claim 1, wherein, Step S2 specifically includes: Traverse each service entity in the basic knowledge graph, use an LLM to generate related semantic tags for it, and provide a confidence score for each semantic tag; these semantic tags and their relationships with the corresponding services are added back to the basic knowledge graph to form an enhanced knowledge graph; Generate high-quality text descriptions of all entities and relationships in the enhanced knowledge graph, and convert them into vectors using a text embedding model and store them in a vector database. 3.The smart home control method based on graph search enhancement generation and multi-agent game according to claim 1, characterized in that, Step S4 specifically includes: Obtain the user's context snapshot, which includes instructions, sensor data and user context information; Use instruction keywords and sensor data to retrieve the most relevant knowledge graph entities from the vector database; The front-end LLM integrates the context snapshot and the most relevant knowledge graph entities retrieved to purify the user's demand, and outputs a structured scene analysis report, which includes the current formal observation. 4.The smart home control method based on graph search enhancement generation and multi-agent game according to claim 1, wherein, The scene analysis report also includes a judgment of the complexity of the current instruction. If the instruction complexity is simple, the instruction is directly taken as the final decision and jumps to step S6; if the instruction complexity is complex, it enters step S5. 5.The smart home control method based on graph search enhancement generation and multi-agent game according to claim 1, wherein, In step S5, the abductive reasoning agent receives the formal observation in the scene analysis report, searches for similar examples from the example library, and constructs a prompt based on the current formal observation, then calls a reasoning LLM to generate multiple candidate logical hypotheses and gives a logical rationality score for each candidate logical hypothesis; The user preference agent builds a long-term memory model based on the Mem0 idea to analyze the user's historical behavior and preferences, and gives a personalized compliance score for the service sequence corresponding to each candidate logical hypothesis; The conflict detection agent combines the service sequence corresponding to each candidate logical hypothesis with the overall state of the current home to construct a temporary automation graph, detects whether there is a potential conflict, and then gives a conflict risk score for the service sequence corresponding to each candidate logical hypothesis; All candidate logical hypotheses and corresponding logical rationality scores, individualized fitness scores, and conflict risk scores are summarized into a structured review report; The scene analysis report and the review report are input into the core decision LLM, and the final decision is output, and an explanation of the decision is generated. 6.The smart home control method based on graph search enhancement generation and multi-agent game according to any one of claims 1 to 5, characterized in that, Also includes: Online learning and evolution: obtaining subsequent feedback of the user, and fine-tuning and optimizing the decision strategy of the core decision LLM based on the feedback. 7.The smart home control method based on graph search enhancement generation and multi-agent game according to claim 6, characterized in that, The process of online learning and evolution specifically includes: Obtain the subsequent behavior of the user as implicit feedback or receive explicit feedback from the user; When the received implicit feedback or explicit feedback is negative feedback, trigger the meta-LLM to analyze the entire decision chain that leads to failure, and fine-tune and optimize the decision strategy of the core decision LLM.

8. An intelligent home control system based on graph search reinforcement generation and multi-agent game, characterized in that, It includes: A knowledge graph construction module for instantiating all entities and their relationships in the current smart home environment and constructing a basic knowledge graph; Semantically enriching service entities in the basic knowledge graph to generate an enhanced knowledge graph, which is vectorized and stored in a vector database; An example library construction module for constructing an example library containing a number of formalized observation-logical hypothesis pairs, where the formalized observation is a description of the environment state, and the logical hypothesis is a logical expression that can explain the reason or user intent of the formalized observation; A scene analysis module for obtaining a context snapshot of the user, purifying user needs based on the context snapshot and the vectorized database through the front-end LLM, and outputting a structured scene analysis report including the current formalized observation; A multi-agent collaborative decision-making module: for generating multiple candidate logical hypotheses based on the scene analysis report and the example library through the abductive reasoning agent, and giving a logical rationality score for each candidate logical hypothesis; Through the user preference agent, give an individualized fitness score for the service sequence corresponding to each candidate logical hypothesis; Through the conflict detection agent, give a conflict risk score for the service sequence corresponding to each candidate logical hypothesis; The scene analysis report and the review report including the logical rationality score, the individualized fitness score, and the conflict risk score are input into the core decision LLM, and the final decision is output, and an explanation of the decision is generated. An instruction issuing module for mapping the final decision into device control instructions and calling corresponding devices for execution. 9.The smart home control system based on graph search enhancement generation and multi-agent game according to claim 8, wherein, Also includes: An online learning module for obtaining subsequent feedback of the user, and fine-tuning and optimizing the decision strategy of the core decision LLM based on the feedback.

10. An intelligent home control system based on graph search enhanced generation and multi-agent game, characterized in that, It includes a control module and an interactive device, a data acquisition device, and an execution device in communication with the control module; The interactive device is used to collect user input instructions and transmit them to the control module; The data acquisition device is used to collect current smart home environment state data and execution device current state data and transmit them to the control module; The control module is configured to perform the smart home control method based on graph search enhancement generation and multi-agent game according to any one of claims 1 to 7. The execution device is configured to receive the device control instruction issued by the control module and perform corresponding operation.

Images