Mobile phone terminal-oriented multi-agent illusion detection and reasoning optimization system and method

Abstract

This application provides a multi-agent hallucination detection and reasoning optimization system and method for mobile devices, comprising: a planning agent that generates a solution and detects it via a reasoning hallucination detection submodule; when there is no hallucination, the subtask is transmitted to the executing agent; when there is a hallucination, the solution is regenerated as instructed by the reasoning optimization module; the executing agent generates and executes action instructions; the reflecting agent compares the execution status of the subtasks and makes a preliminary judgment, inputs the reasoning process into the executing hallucination detection submodule for detection, provides feedback on the completion status when there is no hallucination, and makes corrections when there is a hallucination; a hallucination detection agent that includes the above two detection submodules; and a reasoning optimization module that uses an iterative optimization mechanism without carrying historical reasoning processes to optimize for reasoning hallucinations; the system improves the accuracy of hallucination detection and the reliability of task execution by having each module collaboratively and cyclically execute subtasks until completion.

Description

Technical Field

[0001] This application belongs to the field of intelligent data processing technology, and more specifically, relates to a multi-agent illusion detection and reasoning optimization system and method for mobile devices. Background Technology

[0002] With the rapid development of artificial intelligence technology, Large Language Models (LLMs), with their powerful natural language understanding and generation capabilities, are driving the large-scale application of agent technology in the field of terminal devices. Mobile intelligent agents based on LLMs have been widely used in complex scenarios such as voice assistants, intelligent customer service, information retrieval, task planning, and life service scheduling, becoming the core carrier connecting users and digital services.

[0003] Mobile intelligent agents typically employ a multi-agent collaborative architecture. However, existing technologies suffer from the following technical problems: When handling complex reasoning tasks, current intelligent agent systems generally rely on the "Chain of Thought (CoT)" prompting mechanism to guide the model's gradual thinking. However, limited by the core characteristics of LLMs' autoregressive generation and maximum likelihood estimation, the model prioritizes the fluency of language expression, lacking effective endogenous constraints and verification mechanisms for the factual accuracy and logical rigor of the reasoning content. This leads to the system being prone to "illusion" phenomena: Reasoning illusion: Occurs during the planning agent's task planning phase, manifested as a seemingly reasonable plan containing logical errors or lacking supporting evidence, encompassing biases in goal comprehension, errors in intent decomposition, and plan generation fallacies; Execution illusion: Occurs during the reflective agent's judgment phase, manifested as an incorrect judgment that the executing agent has completed a sub-task when it has not actually done so.

[0004] The aforementioned hallucination phenomenon causes the reasoning results of intelligent agents to lose reliability, which may lead to serious consequences in key scenarios such as navigation planning, medical consultation, and financial information inquiry, severely restricting the application of mobile intelligent agents in scenarios with high reliability requirements. In summary, existing methods rely on general, large-scale models for reflection, lacking specialized detection capabilities for specific mobile task scenarios, making it difficult to accurately identify reasoning illusions and execution illusions. Existing correction mechanisms only continuously refine the initial solution, leading to iterative corrections based on flawed logic, causing the correction direction to deviate from the correct path and creating a vicious cycle of "the more you correct, the worse it gets." Summary of the Invention

[0005] The purpose of this application is to provide a multi-agent illusion detection and reasoning optimization system and method for mobile devices, so as to solve at least one of the aforementioned technical problems.

[0006] A first aspect of this application provides a multi-agent hallucination detection and reasoning optimization system for mobile devices, comprising: A planning agent is used to collect user commands and mobile device environment data, generate a solution corresponding to the user commands based on a thought chain mechanism, and input the solution into the inference hallucination detection submodule of the hallucination detection agent, so that the inference hallucination detection submodule performs hallucination detection on the solution and outputs the detection result; the planning agent is also used to: when the detection result indicates that the solution is not a hallucination, transfer the currently to-be-executed subtask in the solution to the execution agent; when the detection result indicates that the solution is a hallucination, receive the instruction from the inference optimization module to regenerate the solution; the solution includes multiple subtasks; The execution agent is connected to the planning agent and is used to generate and execute corresponding mobile terminal action instructions based on the sub-task. The reflecting agent, connected to the executing agent, compares the mobile device status before and after the subtask execution. Based on the comparison result, it preliminarily judges the completion status of the subtask and inputs its reasoning process into the execution hallucination detection submodule of the hallucination detection agent for detection. If no hallucination is detected, the completion status is sent to the planning agent, which then allocates a new subtask based on the completion status. If a hallucination is detected, the reasoning process is corrected and the judgment is re-evaluated. The final completion status of the subtask is fed back to the planning agent, which then allocates a new subtask based on the final completion status. The hallucination detection agent is connected to the planning agent and the reflection agent respectively, and is equipped with the reasoning hallucination detection submodule and the execution hallucination detection submodule. The inference optimization module is connected to the planning agent and the illusion detection agent. It adopts a preset iterative optimization mechanism and only optimizes the illusions detected by the inference illusion detection submodule. It instructs the planning agent to regenerate the solution until the inference illusion detection submodule detects that there are no illusions or reaches a preset number of iterations. The optimization process does not carry the historical inference process. The system, through the collaboration of the planning agent, the execution agent, the reflection agent, the illusion detection agent, and the reasoning optimization module, cyclically executes sub-tasks until all sub-tasks corresponding to the user instruction are completed.

[0007] A second aspect of this application provides a method for optimizing multi-agent hallucination detection and reasoning on mobile devices, including: The planning agent collects user commands and mobile device environment data, generates a solution corresponding to the user commands based on a thought chain mechanism, and inputs the solution into the inference hallucination detection submodule of the hallucination detection agent. The inference hallucination detection submodule performs hallucination detection on the solution and outputs the detection result. The planning agent is also used to: when the detection result indicates that the solution is not a hallucination, transfer the currently to-be-executed subtask in the solution to the execution agent; when the detection result indicates that the solution is a hallucination, receive the instruction from the inference optimization module to regenerate the solution; the solution includes multiple subtasks. The executing agent generates and executes corresponding mobile action commands based on the sub-tasks. By comparing the mobile device status before and after the execution of the subtask, the reflective agent makes a preliminary judgment on the completion status of the subtask based on the comparison results, and inputs its own reasoning process into the hallucination detection submodule of the hallucination detection agent for detection; if no hallucination is detected, the completion status is sent to the planning agent, so that the planning agent can allocate new subtasks based on the completion status; if a hallucination is detected, the reasoning process is corrected and re-judged, and the final completion status of the subtask is fed back to the planning agent, so that the planning agent can allocate new subtasks based on the final completion status; Through the reasoning optimization module, a preset iterative optimization mechanism is adopted to optimize only the illusions detected by the reasoning illusion detection submodule, instructing the planning agent to regenerate the solution until the reasoning illusion detection submodule detects that there are no illusions or the preset number of iterations is reached. The optimization process does not carry the historical reasoning process. The system, through the collaboration of the planning agent, the execution agent, the reflection agent, the illusion detection agent, and the reasoning optimization module, cyclically executes sub-tasks until all sub-tasks corresponding to the user instruction are completed.

[0008] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the above-described method.

[0009] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.

[0010] The beneficial effects of the proposed solution are as follows: by deploying a hallucination detection agent that includes a specially trained mobile hallucination detection visual language model, the reasoning process of the planning agent and the execution result of the reflection agent are accurately detected. Furthermore, a pre-defined iterative optimization mechanism that only transmits the final solution of the previous round and does not carry the historical reasoning process is used to optimize the detected reasoning hallucinations. This not only prevents the accumulation of erroneous logic, but also achieves closed-loop control and reliable execution of the entire task process, significantly improving the hallucination detection accuracy and task execution reliability of the mobile multi-agent system. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A schematic diagram of the structure of a multi-agent hallucination detection and reasoning optimization system for mobile devices provided in an embodiment of this application; Figure 2 A flowchart of an iterative verification strategy algorithm provided in an embodiment of this application; Figure 3 A schematic diagram of an iterative verification strategy that relies on the previous round's final solution, provided as an embodiment of this application; Figure 4 This is an example diagram of an iterative verification process provided in an embodiment of this application; Figure 5 This is a schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0013] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0014] Mobile intelligent agents based on large language models have been widely used in scenarios such as voice assistants, intelligent customer service, information retrieval, task planning, and life service scheduling, becoming the core carrier connecting users and digital services, and significantly improving user interaction efficiency and service access convenience.

[0015] In existing technologies, mobile intelligent agents face severe challenges in terms of reasoning reliability and security. Large Language Models (LLMs) generally rely on "Chain of Thought (CoT)" prompting mechanisms to guide the model's step-by-step thinking when handling complex reasoning tasks. However, due to the core characteristics of LLMs—autoregressive generation and maximum likelihood estimation—the models often prioritize the fluency and coherence of language expression during the generation process, while lacking effective endogenous constraints and verification mechanisms for the factual accuracy and logical rigor of the reasoning content.

[0016] This characteristic directly leads to mobile AI agents being prone to "hallucinations," that is, outputting answers that "seem reasonable but are logically flawed." The hallucinations of mobile AI agents mainly fall into two categories: Reasoning illusion occurs during the planning task phase of a planning agent. It manifests as a plan that appears reasonable on the surface but contains logical errors or lacks supporting evidence. Specifically, it covers three stages: goal understanding (caused by fuzzy goal information or insufficient subjective understanding), intent decomposition (caused by insufficient dependency modeling leading to the generation of irrelevant or infeasible sub-intents), and plan generation (caused by misunderstanding or misuse of planning information). Execution illusion: occurs during the judgment phase of a reflective agent, manifested as an incorrect judgment that the executing agent has completed a certain sub-stage task, when in fact it has not.

[0017] The existence of illusions renders the reasoning results of mobile AI agents unreliable, leading not only to users receiving incorrect information and task failures, but also potentially causing serious consequences in critical scenarios. For example, providing incorrect routes in navigation planning could endanger users' travel, offering incorrect health advice in medical consultations could harm users' health, and providing false data in financial information queries could mislead users' decisions. These erroneous behaviors could even lead to security incidents, severely restricting the application expansion of mobile AI agents in scenarios with high reliability requirements and reducing users' trust in AI agent technology.

[0018] In summary, the existing technology has two major flaws: Poor detection accuracy: The intelligent agent uses a general large model rather than a special hallucination detection model for reflection, and lacks professional detection capabilities for specific task scenarios on mobile devices, making it difficult to accurately identify reasoning hallucinations and execution hallucinations; Errors can accumulate: Illusion correction is based on the initial solution and is continuously modified without considering whether the initial solution has fundamental errors or whether it can complete the user's instructions. This leads to iterative correction based on erroneous logic, causing the correction direction to deviate from the correct path.

[0019] The technical problem this application aims to solve is to overcome the shortcomings of existing technologies, such as low accuracy in illusion detection on mobile devices and the accumulation of erroneous logic during inference correction. This application provides a multi-agent illusion detection and inference optimization technology for mobile devices, improving the accuracy of illusion detection and the reliability of the inference process, ensuring that the agent can accurately respond to and complete user commands. Specifically, this application relates to the fields of artificial intelligence and mobile terminal interaction technology, and is applicable to scenarios that improve the reliability of task execution in multi-agent systems on mobile devices. It aims to achieve accurate illusion detection and inference logic optimization for the agent's inference process and execution results.

[0020] To achieve the above objectives, this application constructs a dedicated illusion detection model and detection agent: a Visual Language Model (VLM) specifically designed for mobile task scenarios is trained, and a reasoning illusion detection submodule and an execution illusion detection submodule are constructed based on the VLM model to accurately detect the reasoning process of the planning agent and the execution result of the reflecting agent, respectively; furthermore, a "Markov-style" iterative optimization mechanism is proposed: an iterative optimization strategy is designed that only transmits the final answer of the previous round and does not carry the historical reasoning process, so as to prevent the accumulation of erroneous logic during the correction process from the root; among which, this mechanism is mainly used to optimize the reasoning illusion generated by the planning agent.

[0021] To make the objectives, technical solutions, and advantages of this application clearer, the following description will be provided in conjunction with the accompanying drawings and specific embodiments.

[0022] Please refer to Figure 1 , Figure 1 This is a schematic diagram of the structure of a multi-agent hallucination detection and reasoning optimization system for mobile devices provided in an embodiment of this application. It can be executed by any electronic device, such as a mobile phone. The system may include: a planning agent 11, an execution agent 12, a reflection agent 13, a hallucination detection agent 14, and a reasoning optimization module 15, wherein: The planning agent 11 is used to collect user commands and mobile terminal environment data, generate a solution corresponding to the user commands based on the thought chain mechanism, and input the solution into the inference hallucination detection submodule in the hallucination detection agent, so that the inference hallucination detection submodule performs hallucination detection on the solution and outputs the detection result; the planning agent is also used to: when the detection result indicates that the solution is not a hallucination, transfer the currently to-be-executed subtask in the solution to the execution agent; when the detection result indicates that the solution is a hallucination, receive the instruction from the inference optimization module to regenerate the solution; the solution includes multiple subtasks; specifically, the solution may include a subtask list of multiple subtasks.

[0023] The execution agent 12 is connected to the planning agent 11 and is used to generate and execute corresponding mobile terminal action instructions based on the sub-task. The reflecting agent 13, connected to the executing agent 12, compares the mobile device status before and after the execution of the subtask, makes a preliminary judgment on the completion status of the subtask based on the comparison result, and inputs its own reasoning process to the execution hallucination detection submodule in the hallucination detection agent for detection; if no hallucination is detected, the completion status is sent to the planning agent, so that the planning agent allocates a new subtask based on the completion status; if a hallucination is detected, the reasoning process is corrected and re-judged, and the final completion status of the subtask is fed back to the planning agent, so that the planning agent allocates a new subtask based on the final completion status; The hallucination detection agent 14 is connected to the planning agent 12 and the reflecting agent 13 respectively, and is equipped with the reasoning hallucination detection submodule and the execution hallucination detection submodule. The reasoning optimization module 15 is connected to the planning agent 11 and the hallucination detection agent 14. It adopts a preset iterative optimization mechanism and only optimizes the hallucinations detected by the reasoning hallucination detection submodule. It instructs the planning agent to regenerate the solution until the reasoning hallucination detection submodule detects that there are no hallucinations or reaches the preset number of iterations. The optimization process does not carry the historical reasoning process. The system, through the collaboration of the planning agent 11, the execution agent 12, the reflection agent 13, the hallucination detection agent 14, and the reasoning optimization module 15, cyclically executes sub-tasks until all sub-tasks corresponding to the user instruction are completed.

[0024] In some optional embodiments of this application, the planning agent includes: The user instruction parsing unit is used to collect and parse natural language instructions input by the user. An environmental data acquisition unit interfaces with the mobile phone system API to acquire environmental data from the mobile phone, including current mobile phone status, screenshots, hardware status, and running process information. Subtask generation unit, used to generate the solution based on the thought chain mechanism; The task progress recording unit is used to record the execution status of each subtask and the overall task progress.

[0025] In some optional embodiments of this application, the executing agent includes: An action instruction generation unit generates corresponding mobile phone operation instructions based on the sub-tasks. The mobile phone operation instructions include click, input, swipe, application switching, and system control instructions. The instruction execution unit interfaces with the mobile phone control system and is used to convert the mobile phone operation instructions into system-level calls and execute them.

[0026] In some optional embodiments of this application, the reflective agent includes: A state comparison unit is used to compare the mobile terminal state before and after the execution of the subtask, and the mobile terminal state includes differences in screen UI and system state. The preliminary judgment unit is used to make a preliminary judgment on the completion status of the sub-task based on the comparison results; The reasoning process organizing unit is used to organize its own reasoning process, which includes reasoning logic and judgment basis, and inputs its own reasoning process into the hallucination detection submodule in the hallucination detection agent for detection. A correction unit is used to correct the reasoning process when the hallucination detection submodule detects a hallucination.

[0027] In some optional embodiments of this application, the reasoning illusion detection submodule is trained through the following steps S101-S104: S101. Collect the original interaction data of the mobile terminal multi-agent system. The original interaction data is a triple containing the original state screenshot, action and future state screenshot, covering multiple task types in multiple application scenarios. Optionally, the aforementioned raw interaction data is the real interaction data of the mobile phone multi-agent system. The aforementioned multiple application scenarios may include four major categories of task scenarios: system operation, online shopping, search applications and other third-party applications. The real interaction data also includes the interaction data of multiple different applications. The actions include common mobile interaction actions such as scrolling, clicking, waiting and returning.

[0028] Optionally, the training dataset involved in this application includes correct / incorrect inference samples and execution result samples for various task scenarios on mobile devices.

[0029] S102. Convert the original interaction data into semantic annotations, the semantic annotations including: converting the action into an action description, generating a state change text description based on the difference between the original state screenshot and the future state screenshot, and generating a true / false question about the future state and its corresponding answer; Specifically, the raw low-level data is converted into semantic annotations, which includes the following three types of annotations: Converting the action into an action description means converting low-level action instructions (such as coordinates "Click (250,120)") into high-level action descriptions. This is done by visually marking actions on a screenshot (clicks are marked with a crosshair, and swipes with an arrow to indicate direction), and then inputting the marked screenshot into a Visual Language Model (VLM) to generate human-understandable action descriptions (such as "Click the search button"). Generating a text description of the state change based on the difference between the original state screenshot and the future state screenshot means: generating a text description of the state change using VLM based on the difference between the original state screenshot and the future state screenshot (such as "The page jumps from the main settings menu to the network settings page, displaying options such as Wi-Fi and airplane mode"). Specifically, the triples can be input into a large language model (such as GPT-4o) to generate multiple true / false questions and corresponding answers based on state differences (such as "Should the shopping cart button be displayed? Answer: Yes").

[0030] S103. Perform manual verification and filtering on the semantic annotations, and retain training samples that meet the quality requirements; The semantically annotated data is manually verified and filtered to retain training samples that meet the quality requirements. The verification criteria for quality requirements include: (1) consistency between the answer to a true / false question about the future state and the actual state; (2) relevance between the question and the action; and (3) predictability of the question (excluding questions that require real-time information or are ambiguous, such as "What is the headline in the sports section?").

[0031] S104. Input the original state screenshot, the future state screenshot, the action description, the state change text description, and the future state true / false question and corresponding answer into the visual language model simultaneously, and train them together to obtain the target visual language model. The reasoning illusion detection submodule is constructed based on the trained target visual language model.

[0032] The VLM model is simultaneously input into the original state screenshot, future state screenshot, action description, state change text description, and a true / false question about the future state and its corresponding answer. The model's semantic prediction ability is jointly optimized to learn the mapping relationship of "current state + action → future state semantic description".

[0033] The trained VLM model (i.e., the target visual language model) is integrated into a multi-agent system as a semantic world model. Inputting hallucination operation instructions can verify the model's ability to recognize hallucinations.

[0034] Optionally, the aforementioned reasoning illusion detection submodule includes a target visual language model.

[0035] In some optional embodiments of this application, the actions include scrolling, clicking, waiting, and returning, and the triples cover interactive scenarios of multiple different applications.

[0036] In some optional embodiments of this application, the inference optimization module includes: A Markov input control unit is used to filter information from the historical reasoning process and send only the final solution from the previous round as input to the planning agent as the basis for regenerating the solution. The budget control unit is used to record the current iteration number and forcibly terminate the iteration process when the preset iteration number is reached, instructing the planning agent to output the solution for the current round.

[0037] Optionally, the aforementioned preset iterative mechanism is a Markov-style iterative mechanism. The execution steps of the Markov-style iterative optimization inference may include the following steps S21-S23: S21, Initial Reasoning.

[0038] The planning agent generates the first round of reasoning results (initial solution) based on user instructions and mobile device environment data, and inputs the initial solution into the reasoning hallucination detection submodule for hallucination detection.

[0039] S22, Illusion Judgment and Iterative Triggering.

[0040] If the reasoning illusion detection submodule detects no illusion, it transmits the initial solution as the final solution to the executing agent; if an illusion is detected, it triggers the reasoning optimization module, which adopts a Markov iterative optimization mechanism and sends only the previous round's final solution (rather than the historical reasoning process) as input to the planning agent, instructing the planning agent to regenerate the solution. S23, Iterative Reasoning and Termination.

[0041] The planning agent regenerates a new solution based on the previous final solution, and then tests it again through the reasoning illusion detection submodule; the above iterative process is repeated until no illusion is detected or the preset number of iterations is reached, and then the final solution is output.

[0042] Before introducing the iterative verification strategy of this application, it is necessary to clarify three core input elements to ensure the stable implementation of the process. Specific descriptions and examples of each input element are shown in Table 1 below: Table 1. Detailed explanations and examples of each input element.

[0043] The iterative verification in this application strictly follows a closed-loop process of "initial solution generation → iterative verification - generation → termination output" (e.g., Figure 2 As shown), it can be broken down into the following steps S31-S35 to achieve end-to-end optimization from seed solution generation to final reliable solution output: S31. Generate the initial solution ( — The “seed” of the iterative process.

[0044] This step aims to generate an initial solution that requires no pre-validation using the standard Chain of Thought (CoT) mechanism, serving as a foundational anchor for subsequent iterative validation. The specific operational process is as follows: (1) Constructing standard CoT prompts: Prompts are constructed using a fixed template. The template content is: "Question Q: [User instruction to be implemented]. Please think step by step and output a standard solution in JSON format containing the four elements of 'observation description + context + reasoning + action'." (2) Model reasoning and solution extraction: Input the above prompt words into the target large language model (M), obtain the model output results, extract the "final solution" (remove redundant content such as intermediate thinking steps and annotations, such as extracting "click the 11th icon"), and record it as the initial solution.

[0045] In this step, solution extraction should focus on the "executable / verifiable end result" (e.g., the specific action instructions to complete the task of "opening the Gmail application"), avoiding the inclusion of non-core content such as verification process descriptions and thought processes. This is to ensure the simplicity and relevance of subsequent verification objects.

[0046] S32, Iterative verification - generation (core closed loop).

[0047] This step is the core of the iterative verification strategy. Each iteration follows a closed-loop logic of "previous solution → verification → new solution" and strictly adheres to the Markov property—that is, it only depends on the final solution of the previous round. It does not carry any historical reasoning process, thus avoiding context overflow and the accumulation of erroneous logic at the source. Optionally, for specific operational procedures, please refer to... Figure 3 As shown.

[0048] Furthermore, the iterative verification strategy may specifically include the following steps S321-S323: S321. Constructing this round of verification - Generating prompts: Prompts are generated using a fixed template. The template content is: "Question Q: [User instruction to be implemented]. Candidate solutions..." [The final solution extracted in the previous round]. Please verify first. To determine if the user's instructions can be met, the correct solution will be derived step by step and output. S322, Model Inference and New Solution Extraction: Input the above prompts into the target large language model (M), obtain the model output, and extract the "final solution" again (removing redundant content such as verification process descriptions and intermediate thinking steps), and record it as the solution for the current round. .

[0049] S323. Iterative Loop: Starting from the first round, repeat the above process of "constructing prompt words → model reasoning → extraction" until the Bth round of iteration is completed.

[0050] The core feature of this step is that the iterative process is a standard Markov process: each round of reasoning "starts from zero", using only the final solution of the previous round as the verification anchor point, without referring to the "error reasons" or "thinking logic" of previous rounds, thus completely solving the problem of context overflow and error accumulation caused by carrying historical reasoning processes in traditional self-correction methods (such as Self-Correction).

[0051] S33. Determine whether the number of iterations has reached the preset budget B. If yes, proceed to S34 to determine the output result. If no, return to the iteration verification-generation step.

[0052] S34. Determine the output result.

[0053] When the number of iterations reaches the preset computational budget (B), the iteration process is immediately terminated, and the solution generated in round B is directly output. As the final result.

[0054] Alternatively, see also details Figure 4 As shown.

[0055] Process Example: If the calculated budget B=3, the complete iterative process is as follows: (Initial Solution) → (verify (Generate) → (verify (Generate), final output Finally, the solution generated in the third round is output.

[0056] The scheme presented in this application also has cost control advantages, with the total computational cost being the product of the number of iterations (B) and the number of tokens per round of inference. Experimental results show that the number of tokens per round of inference in this scheme is only 20% to 50% more than that of the standard CoT (e.g., the standard CoT has 365 tokens per round, while this scheme has approximately 533 tokens per round); when B=3, the total number of tokens is approximately 1599, which is far lower than that of traditional high-cost schemes such as parallel sampling (the total number of tokens exceeds 3650 when sampling 10 candidates in parallel).

[0057] S35. Convert the solution into an execution instruction.

[0058] The final solution output from step 33 is input into a preset instruction conversion function. This function maps the solution into system instructions that can be directly executed by the mobile device, driving the agent to complete the transition from the current state ( ) to the next state ( The transition is to enable task execution.

[0059] The following example, "The mobile smart agent completes the task of sending a text message containing the current location to contact A," illustrates the implementation process in detail: (1) Task planning and reasoning hallucination detection The planning agent collects the user command "send a text message containing the current location to contact A," and obtains the current phone status (such as whether location services are enabled and whether the SMS application is running) and historical screen images (recent contacts interface and SMS interface) through the environmental data acquisition unit. Using a thought chain mechanism, the task is broken down into a list of subtasks: ① obtain the current location; ② find contact A's phone number; ③ edit a text message containing the location; ④ send the text message. This list of subtasks is input into the reasoning illusion detection submodule for detection. If a reasoning illusion is detected (such as mistakenly breaking down "contact A" into "contact B"), the reasoning optimization module uses a Markov-style iterative optimization mechanism (only passing the final subtask list from the previous round, without carrying the historical reasoning process) to instruct the planning agent to regenerate until no illusion is detected.

[0060] (2) Subtask execution and hallucination detection The executing agent receives the first subtask, "Get Current Location," generates and executes the location activation command and location acquisition command. The reflecting agent collects the phone's state before and after execution (location changes from off to on, latitude and longitude information is obtained), initially determining that the subtask is complete, and inputs this reasoning process into the execution illusion detection submodule for detection. If no execution illusion is detected, the completion result is fed back to the planning agent. If an execution illusion is detected (e.g., contact A's number is not actually obtained, only the search interface is displayed), the reflecting agent is triggered to correct its reasoning process and re-evaluate, feeding back the corrected completion result to the planning agent, which then reassigns the subtask.

[0061] (3) Task progress update and cyclic execution The planning agent updates the task progress based on the feedback, marks completed subtasks as completed, and assigns the next subtask (such as "find contact A's phone number") to the executing agent. This execution-reflection-checking process is repeated to complete each subtask sequentially.

[0062] (4) Task completed Once all subtasks have been successfully detected and confirmed through hallucination testing, the SMS message is sent successfully, and the task ends.

[0063] This application improves the accuracy of hallucination detection by employing a specially trained visual language model and a corresponding hallucination detection submodule. Compared to general large models, it can specifically identify reasoning hallucinations and execution hallucinations of multi-agent systems on mobile devices, significantly improving detection accuracy. Secondly, this application's solution avoids the accumulation of erroneous logic. Through a "Markov-style" iterative optimization mechanism, it only transmits the final solution from the previous round, eliminating the carrying over of historical reasoning processes. This mechanism fundamentally prevents the accumulation of initial or intermediate errors, ensuring the correctness of the reasoning optimization direction. Furthermore, this application's solution also guarantees task execution reliability: through a closed-loop process of "detection-optimization-execution-re-detection," it achieves hallucination control and reasoning correction throughout the entire task process, significantly improving the success rate of mobile agents completing user commands and optimizing the user interaction experience.

[0064] This application also provides a method for optimizing multi-agent hallucination detection and reasoning on mobile devices, including: The planning agent collects user commands and mobile device environment data, generates a solution corresponding to the user commands based on a thought chain mechanism, and inputs the solution into the inference hallucination detection submodule of the hallucination detection agent. The inference hallucination detection submodule performs hallucination detection on the solution and outputs the detection result. The planning agent is also used to: when the detection result indicates that the solution is not a hallucination, transfer the currently to-be-executed subtask in the solution to the execution agent; when the detection result indicates that the solution is a hallucination, receive the instruction from the inference optimization module to regenerate the solution; the solution includes multiple subtasks. The executing agent generates and executes corresponding mobile action commands based on the sub-tasks. By comparing the mobile device status before and after the execution of the subtask, the reflective agent makes a preliminary judgment on the completion status of the subtask based on the comparison results, and inputs its own reasoning process into the hallucination detection submodule of the hallucination detection agent for detection; if no hallucination is detected, the completion status is sent to the planning agent, so that the planning agent can allocate new subtasks based on the completion status; if a hallucination is detected, the reasoning process is corrected and re-judged, and the final completion status of the subtask is fed back to the planning agent, so that the planning agent can allocate new subtasks based on the final completion status; Through the reasoning optimization module, a preset iterative optimization mechanism is adopted to optimize only the illusions detected by the reasoning illusion detection submodule, instructing the planning agent to regenerate the solution until the reasoning illusion detection submodule detects that there are no illusions or the preset number of iterations is reached. The optimization process does not carry the historical reasoning process. The system, through the collaboration of the planning agent, the execution agent, the reflection agent, the illusion detection agent, and the reasoning optimization module, cyclically executes sub-tasks until all sub-tasks corresponding to the user instruction are completed.

[0065] For details on how this embodiment is implemented, please refer to the foregoing content, which will not be repeated here.

[0066] See Figure 5 , Figure 5 This is a schematic block diagram of an electronic device provided according to an embodiment of this application. Figure 5 The electronic device 300 in this embodiment may include one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The processors 301, input devices 302, output devices 303, and memories 304 communicate with each other via a communication bus 305. The memories 304 store computer programs, including program instructions. The processors 301 execute the program instructions stored in the memories 304. The processors 301 are configured to invoke the program instructions to perform the functions of the modules or intelligent agents in the above-described device embodiments.

[0067] It should be understood that, in the embodiments of this application, the processor 301 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0068] Input device 302 may include a touchpad, a fingerprint sensor (for collecting the user's fingerprint information and fingerprint orientation information), a microphone, etc., and output device 303 may include a display (LCD, etc.), a speaker, etc.

[0069] The memory 304 may include read-only memory and random access memory, and provides instructions and data to the processor 301. A portion of the memory 304 may also include non-volatile random access memory.

[0070] In specific implementations, the processor 301, input device 302, and output device 303 described in the embodiments of this application can execute the implementation methods described in the embodiments of this application, or they can execute the implementation methods of the electronic devices described in the embodiments of this application, which will not be repeated here.

[0071] In another embodiment of this application, a computer-readable storage medium is provided. This computer-readable storage medium stores a computer program, which includes program instructions. When executed by a processor, the program instructions implement all or part of the processes in the methods described above. Alternatively, the computer program can instruct related hardware to complete the process. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include any entity or device capable of carrying computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0072] The computer-readable storage medium can be an internal storage unit of the electronic device in any of the foregoing embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device. Furthermore, the computer-readable storage medium can include both internal and external storage units of the electronic device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0073] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0074] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the electronic devices and units described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0075] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces or modules, or it may be an electrical, mechanical, or other form of connection.

[0076] The modules described as separate components may or may not be physically separate. Similarly, the components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0077] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0078] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A multi-agent hallucination detection and reasoning optimization system for mobile devices, characterized in that, include: A planning agent is used to collect user commands and mobile device environment data, generate a solution corresponding to the user commands based on a thought chain mechanism, and input the solution into the inference hallucination detection submodule of the hallucination detection agent, so that the inference hallucination detection submodule performs hallucination detection on the solution and outputs the detection result; the planning agent is also used to: when the detection result indicates that the solution is not a hallucination, transfer the currently to-be-executed subtask in the solution to the execution agent; when the detection result indicates that the solution is a hallucination, receive the instruction from the inference optimization module to regenerate the solution; the solution includes multiple subtasks; The execution agent is connected to the planning agent and is used to generate and execute corresponding mobile terminal action instructions based on the sub-task. The reflecting agent is connected to the executing agent and is used to compare the mobile phone status before and after the execution of the subtask. Based on the comparison results, it makes a preliminary judgment on the completion status of the subtask and inputs its own reasoning process into the execution hallucination detection submodule in the hallucination detection agent for detection. If no hallucination is detected, the completion status is sent to the planning agent, which then assigns a new subtask based on the completion status. If a hallucination is detected, the reasoning process is corrected and reassessed, and the final completion status of the subtask is fed back to the planning agent, so that the planning agent can allocate new subtasks based on the final completion status. The hallucination detection agent is connected to the planning agent and the reflection agent respectively, and is equipped with the reasoning hallucination detection submodule and the execution hallucination detection submodule. The inference optimization module is connected to the planning agent and the illusion detection agent. It adopts a preset iterative optimization mechanism and only optimizes the illusions detected by the inference illusion detection submodule. It instructs the planning agent to regenerate the solution until the inference illusion detection submodule detects that there are no illusions or reaches a preset number of iterations. The optimization process does not carry the historical inference process. The system, through the collaboration of the planning agent, the execution agent, the reflection agent, the illusion detection agent, and the reasoning optimization module, cyclically executes sub-tasks until all sub-tasks corresponding to the user instruction are completed.

2. The system according to claim 1, characterized in that, The planning agent includes: The user instruction parsing unit is used to collect and parse natural language instructions input by the user. An environmental data acquisition unit interfaces with the mobile phone system API to acquire environmental data from the mobile phone, including current mobile phone status, screenshots, hardware status, and running process information. Subtask generation unit, used to generate the solution based on the thought chain mechanism; The task progress recording unit is used to record the execution status of each subtask and the overall task progress.

3. The system according to claim 1, characterized in that, The executing intelligent agent includes: An action instruction generation unit generates corresponding mobile phone operation instructions based on the sub-tasks. The mobile phone operation instructions include click, input, swipe, application switching, and system control instructions. The instruction execution unit interfaces with the mobile phone control system and is used to convert the mobile phone operation instructions into system-level calls and execute them.

4. The system according to claim 1, characterized in that, The reflective agent includes: A state comparison unit is used to compare the mobile terminal state before and after the execution of the subtask, and the mobile terminal state includes differences in screen UI and system state. The preliminary judgment unit is used to make a preliminary judgment on the completion status of the sub-task based on the comparison results; The reasoning process organizing unit is used to organize its own reasoning process, which includes reasoning logic and judgment basis, and inputs its own reasoning process into the hallucination detection submodule in the hallucination detection agent for detection. A correction unit is used to correct the reasoning process when the hallucination detection submodule detects a hallucination.

5. The system according to claim 1, characterized in that, The reasoning illusion detection submodule is trained in the following way: Collect raw interaction data of a multi-agent system on a mobile device. The raw interaction data is a triple containing a screenshot of the original state, an action, and a screenshot of the future state, covering multiple task types in various application scenarios. The original interaction data is converted into semantic annotation, which includes: converting the action into an action description, generating a text description of the state change based on the difference between the original state screenshot and the future state screenshot, and generating a true / false question about the future state and its corresponding answer. The semantic annotations are manually verified and filtered to retain training samples that meet the quality requirements; The original state screenshot, the future state screenshot, the action description, the state change text description, and the future state true / false question and corresponding answer are simultaneously input into the visual language model and jointly trained to obtain the target visual language model. The reasoning illusion detection submodule is constructed based on the trained target visual language model.

6. The system according to claim 5, characterized in that, The actions include scrolling, clicking, waiting, and returning, and the triplets cover interactive scenarios from multiple different applications.

7. The system according to claim 1, characterized in that, The inference optimization module includes: A Markov input control unit is used to filter information from the historical reasoning process and send only the final solution from the previous round as input to the planning agent as the basis for regenerating the solution. The budget control unit is used to record the current iteration number and forcibly terminate the iteration process when the preset iteration number is reached, instructing the planning agent to output the solution for the current round.

8. A multi-agent hallucination detection and reasoning optimization method for mobile devices, characterized in that, include: The planning agent collects user commands and mobile device environment data, generates a solution corresponding to the user commands based on a thought chain mechanism, and inputs the solution into the inference hallucination detection submodule of the hallucination detection agent. The inference hallucination detection submodule performs hallucination detection on the solution and outputs the detection result. The planning agent is also used to: when the detection result indicates that the solution is not a hallucination, transfer the currently to-be-executed subtask in the solution to the execution agent; when the detection result indicates that the solution is a hallucination, receive the instruction from the inference optimization module to regenerate the solution; the solution includes multiple subtasks. The executing agent generates and executes corresponding mobile action commands based on the sub-tasks. By reflecting on the intelligent agent and comparing the mobile phone status before and after the execution of the sub-task, the completion status of the sub-task is initially judged based on the comparison results, and the agent inputs its own reasoning process into the hallucination detection sub-module in the hallucination detection intelligent agent for detection. If no hallucination is detected, the completion status is sent to the planning agent, which then assigns a new subtask based on the completion status. If a hallucination is detected, the reasoning process is corrected and reassessed, and the final completion status of the subtask is fed back to the planning agent, so that the planning agent can allocate new subtasks based on the final completion status. Through the reasoning optimization module, a preset iterative optimization mechanism is adopted to optimize only the illusions detected by the reasoning illusion detection submodule, instructing the planning agent to regenerate the solution until the reasoning illusion detection submodule detects that there are no illusions or the preset number of iterations is reached. The optimization process does not carry the historical reasoning process. The system according to any one of claims 1-7, through the collaboration of the planning agent, the execution agent, the reflection agent, the illusion detection agent and the inference optimization module, cyclically executes sub-tasks until all sub-tasks corresponding to the user instruction are completed.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in claim 8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in claim 8.

Images