Multi-round dialogue system, method, device, medium and program product of body-equipped agent

Abstract

The invention provides a multi-round dialogue system, method, equipment, medium and program product for an agent with a body. The multi-round dialogue system for the agent with the body comprises a tool pool, a strategy identification module, an execution tool module and a strategy dialogue module, the tool pool is used for storing tools; the strategy identification module is used for identifying a current task and a current calling tool and storing an execution result into a data structure body; the execution tool module is used for executing circulation of sensing, planning, tool selection and tool execution based on the current task identified by the strategy identification module and the current calling tool agent so as to obtain an output result and a dialogue strategy; and the strategy dialogue module is used for storing a pre-constructed strategy library, and the pre-constructed strategy library comprises a dialogue strategy. And the purposes of dialogue effectiveness and diversity are achieved.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and in particular, relates to an embodied intelligent agent multi-round dialogue system, method, device, medium and program product. Background Art

[0002] Currently, the fine-tuned language model can interact with users in the form of dialogue. Feature engineering technology plays a vital role in extending the capabilities of language models. With fixed language model parameters, this technology gives clear instructions for specific tasks, improves the performance of the language model, and transfers the capabilities of the language model to clear downstream tasks.

[0003] Existing data-driven embodied agents make decisions based on the correlations within the data. This modeling approach cannot enable the model to truly understand the causal relationship between knowledge, behavior, and environment, which makes the existing embodied agent dialogues monotonous and has the problem of invalid dialogues. Summary of the invention

[0004] In response to the problems existing in the prior art, the present invention provides an embodied intelligent multi-round dialogue system, method, device, medium and program product, which at least partially solve the problems of single dialogue and invalid dialogue existing in the prior art.

[0005] In a first aspect, an embodiment of the present disclosure provides an embodied agent multi-round dialogue system, including: a tool pool, a strategy identification module, an execution tool module, and a strategy dialogue module; The tool pool is used to store tools; The strategy identification module is used to identify the current task and the current calling tool, and store the execution result in a data structure; The execution tool module executes a cycle of perception, planning, tool selection and tool execution based on the current task identified by the strategy identification module and the current calling tool agent, thereby obtaining an output result and a dialogue strategy; The strategy dialogue module is used to store a pre-built strategy library, which includes dialogue strategies.

[0006] Optionally, the tool pool is to inject external knowledge of prompt words and recalled information, and then send the information into the speech model for reasoning.

[0007] Optionally, the tool pool includes historical conversation information query, exception handling, value strategy guided conversation and knowledge retrieval.

[0008] Optionally, the description file of each tool in the tool pool includes: name, description, function, optional parameters and required parameters, and the description file is used to help the intelligent agent to make reasonable tool selection, parameter transfer and tool execution.

[0009] Optionally, the pre-built strategy library is constructed based on a multi-round dialogue dataset fine-tuning model, and the dataset of the multi-round dialogue dataset fine-tuning model meets set conditions, and the set conditions include authenticity, richness and accuracy; The authenticity: the value dimension comes from the definition of the embodied agent, the scene information comes from the information of the environment in which the embodied agent is located, and the action sequence comes from the action sequence that the embodied agent can complete; Richness: The data includes the execution of actions driven by different value dimensions, different dialogue strategies, and rich topics; Accuracy: Build a data pipeline. The data pipeline consists of three parts: scenario->strategy->dialogue. Design different system prompts for each part, and make full use of thinking chains and knowledge expansion to supplement and enhance information.

[0010] Optionally, the multi-round dialogue dataset fine-tuning model uses a low-rank adaptive approach to perform parameter fine-tuning.

[0011] Optionally, the strategy identification module combines the methods of cue word engineering and model fine-tuning.

[0012] In a second aspect, the embodiments of the present disclosure further provide an embodied agent multi-round dialogue method, which is applied to any system described in the first method, including: In response to an input instruction, decomposing the input instruction to obtain an intention of the input instruction; Obtain the agent dialogue strategy based on input instructions and pre-built strategy library; Retrieve corresponding data from the pre-built tool pool based on the intent and dialogue strategy of the input command; The output result is obtained based on the intention of the input instruction, the dialogue strategy and the retrieved data.

[0013] Optionally, obtaining an output result based on the obtained intention of the input instruction, the dialogue strategy and the retrieved data further includes: Obtaining a dialogue strategy based on the obtained intention of the input instruction, the dialogue strategy, and the retrieved data; The dialogue strategy is input into a strategy library to dynamically adjust the strategy library.

[0014] In a third aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device comprising: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the embodied intelligent agent multi-round dialogue method described in any one of the second aspects.

[0015] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute any of the embodied intelligent agent multi-round dialogue methods described in the second aspect.

[0016] In a fifth aspect, an embodiment of the present disclosure further provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the embodied intelligent agent multi-round dialogue method described in any one of the second aspects.

[0017] The present invention provides an embodied intelligent multi-round dialogue system, method, device, medium and program product, wherein the embodied intelligent multi-round dialogue system, through a pre-built strategy library, sets and generates a multi-round dialogue data set with strategies with a knowledge base to ensure the effectiveness and diversity of the dialogue, identifies the dialogue strategy through a strategy recognition module, and flexibly selects different strategy models to complete multi-round dialogues, thereby achieving the purpose of dialogue effectiveness and diversity. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, wherein like reference numerals generally represent like components throughout the exemplary embodiments of the present disclosure.

[0019] Figure 1 A principle block diagram of an embodied intelligent multi-round dialogue system provided in an embodiment of the present disclosure; Figure 2 A flowchart of a multi-round dialogue method of an embodied intelligent agent provided in an embodiment of the present disclosure; Figure 3 A functional block diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0020] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0021] It should be clear that the following embodiments of the present disclosure are described by specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other in the absence of conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0022] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0023] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The illustrations only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0024] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0025] In the AI ​​model, the role of prompt is mainly to prompt the AI ​​model with the context of input information and parameter information of the input model. When training a supervised learning or unsupervised learning model, prompt can help the model better understand the intention of the input and respond accordingly. In addition, prompt can also improve the interpretability and accessibility of the model. In layman's terms, it is to provide an AI model with a "hint" or "guidance" to help it better understand and complete tasks. The existing prompts are summarized as follows: 1. Design simple templates to achieve migration to new tasks. The zero-shot prompt defines the description of the task, but lacks labeled data, so that the model can migrate to new tasks based on the defined prompt using existing data; the few-shot prompt consists of high-quality labeled examples to improve the model's ability to solve complex problems. Few-shot prompt is a technical method that guides the model to generate answers by providing a small number of examples. In natural language processing, the model is given a few examples to help it understand the task and generate the correct response. This technique is particularly suitable for situations where training data is limited and can significantly improve the performance and adaptability of the model.

[0026] 2. Design templates with logical reasoning to guide the language model to reason and get answers. Chain-of-thought (cot thinking chain) manually designs templates with reasoning chains, so that the language model generates structured and thoughtful replies; Automatic Chain-of-Thought The manually designed templates are complex and locally optimal, and "let's step by step" is added to guide the model to automatically realize reasoning; Logical chain-of-thought provides a series of logical steps to guide the language model to generate more accurate results; Graph-of-Thoughts organizes templates in the form of building mind maps, improves the model's structured thinking ability, and can realize the process of comparison, contrast and interpretation. ‌Graph of Thoughts (GoT)‌ is an open source framework that aims to solve complex problems by modeling them as operation graphs. The core idea of ​​GoT is to decompose the problem into a series of operation steps and use the language model as an engine to automatically execute these steps.

[0027] 3. Design a retrieval-augmented language model generation that reduces hallucinations. Retrieval Augmented Generation integrates the retrieved information into the original prompt as a knowledge supplement, enriches the contextual knowledge, achieves knowledge update, and overcomes the limitations of static prompts; ReAct prompting integrates the reasoning process with the execution process for the first time, interleaving reasoning, retrieval, and update, thereby reducing the hallucination phenomenon and error accumulation transmission of the language model. ‌Retrieval Augmented Generation (RAG)‌ is a model that combines retrieval and generation techniques, designed to generate answers or content by referencing information from an external knowledge base, thereby improving the quality and accuracy of predictions. The RAG model performs well in handling complex query and generation tasks, and is particularly suitable for multiple natural language processing tasks such as question-answering systems, document generation, and intelligent assistants.

[0028] 4. Dynamically fine-tune and optimize prompts. Unlike existing chained thinking (cot) methods that rely on a fixed set of human-annotated examples, Active Prompting introduces a mechanism to determine the most influential questions for annotation; Automatic Promot Engineer dynamic feature engineering realizes automated prompt construction for specific tasks and improves the reasoning ability of language models. ‌Active Prompting‌ is a prompt engineering method for language models that aims to improve the reasoning ability of models by selecting the most valuable examples through an active learning paradigm. This method draws on uncertainty methods in active learning to mine demonstration examples with high information content, thereby reducing the cost of manually selecting examples and improving the reasoning effect of the model. ‌Automatic Prompt Engineer (APE)‌ is an innovative open source project that aims to improve the smart prompts and completion functions in code editors through machine learning.

[0029] 5. Generative paradigms that rely on knowledge-based reasoning. Automatic Reasoning and Tool-use (ART) implements multi-hop reasoning under external knowledge fusion by dynamically injecting knowledge into tool calls. Automatic Reasoning and Tool-use (ART) is a framework for language models that aims to solve multi-step reasoning and tool use problems in complex task processing. The ART framework helps language models perform complex reasoning tasks with few or no samples by automatically generating intermediate reasoning steps and calling external tools.

[0030] 6. Improve the consistency and orderliness of language model generation. Contrastive CoT prompting introduces contrasting positive and negative arguments in the prompt to reduce the occurrence of discontinuities and irrelevant problems in model generation. ‌Contrastive CoT Prompting‌ is a prompting method that aims to enhance the reasoning ability of generative AI models by comparing different options. This method helps the model better understand and generate accurate answers by providing contrasting examples.

[0031] This implementation is a value- and strategy-based multi-turn dialogue system for embodied agents.

[0032] like Figure 1 As shown, this embodiment discloses an embodied agent multi-round dialogue system, including: a tool pool, a strategy identification module, an execution tool module and a strategy dialogue module; The tool pool is used to store tools; the tool pool is to inject external knowledge of prompt words and recalled information, and then send the information to the speech model for reasoning. The tool pool includes historical dialogue information query, exception handling, value strategy guided dialogue and knowledge retrieval. The description file of each tool in the tool pool includes: name, description, function, optional parameters and required parameters. The description file is used to help the intelligent agent to make reasonable tool selection, parameter transfer and tool execution.

[0033] The role of the tool is to inject external knowledge and recalled information into the prompt, and then feed it into the language model for reasoning. This module is a collection of tools, including historical dialogue information query, exception handling, value strategy guided dialogue, and knowledge retrieval. The description file of each tool includes: name, description, function, optional parameters and required parameters. This description is used to help the agent make reasonable tool selection, parameter transfer and tool execution.

[0034] When building an embodied robot with emotions, the role of tools is crucial. They help the agent inject external knowledge and recall information, and then feed it into the language model for reasoning. These tool sets are auxiliary systems for the agent, and they include functions such as historical dialogue information query, exception handling, value strategy guided dialogue, and knowledge retrieval. The following is a detailed description of these tools, which will help the agent make reasonable tool selection, parameter transmission, and tool execution.

[0035] Historical conversation information query tool: used to retrieve and analyze historical conversation information between the agent and the user, so as to provide context-related answers in the current conversation. It can be used to retrieve historical conversation records, analyze conversation trends and user preferences, and provide information related to the conversation context. Optional parameters: specify the time range of the conversation to be retrieved, and filter the conversation records according to a specific topic. Required parameters: the user's unique identifier, used to retrieve the corresponding historical conversation.

[0036] Exception handling tool: When abnormal or incomprehensible input occurs in the conversation, this tool intervenes to handle these situations to ensure the fluency of the conversation and the responsiveness of the agent. Used to identify anomalies or errors in the conversation, provide error recovery strategies, provide feedback to users, and guide the conversation back on track. Optional parameters: Specify the type of exception, such as misunderstanding, technical problems, etc. Define the response strategy when an error occurs. Required parameters: Input data of the current conversation, used for anomaly detection.

[0037] Value strategy guided conversation tool: Ensures that the conversation follows the preset value strategies, such as positivity, respect, and user privacy protection, while guiding the conversation in a constructive and positive direction. Used to evaluate the conversation content according to the value strategy, provide conversation guidance that complies with the value strategy, and adjust the conversation direction when necessary. Optional parameters: define the set of values ​​that should be followed in the conversation. And set the severity level when the value strategy is violated. Required parameters: the content of the current conversation, used for value evaluation.

[0038] Knowledge retrieval tool: used to retrieve information from external knowledge sources to support the agent in providing accurate and detailed answers. Used to retrieve information from databases and the Internet, provide relevant knowledge based on the conversation content, and ensure the accuracy and timeliness of information. Optional parameters: define the source priority of information retrieval. Specify the type of information to be retrieved, such as facts, data, opinions, etc. Required parameters: retrieval query generated based on the current conversation content.

[0039] The agent will select the appropriate tool and pass the corresponding parameters based on the current conversation context and user needs. For example, if the user asks a question about Qin Shi Huang, the agent may first use the knowledge retrieval tool to retrieve relevant information, then use the value strategy to guide the conversation tool based on the retrieved content to ensure that the answer meets the preset value strategy, and finally use the historical conversation information query tool to check whether there is any relevant historical conversation that can provide additional contextual information.

[0040] During the execution of the tool, if any exception occurs, the exception handling tool will be activated to ensure that the conversation can continue smoothly. The agent will dynamically adjust the dialogue strategy based on the output and feedback of the tool to provide the best user experience.

[0041] Through the integration and collaboration of these tools, the intelligent agent can handle complex conversation scenarios and provide a conversation experience with emotions, knowledge, and values. This modular tool set design not only improves the conversation ability of the intelligent agent, but also provides it with flexibility and scalability, enabling it to adapt to changing conversation needs and environments.

[0042] The strategy identification module is used to identify the current task and the current calling tool, and store the execution result in a data structure; As the brain of the entire Agent, it identifies the current task and the current calling tool in the single-step execution, and stores the execution result in the data structure as the input for the next execution of the tool. The implementation of the agent combines the prompt-engineering and fine-tuning methods.

[0043] The execution tool module executes a cycle of perception, planning, tool selection and tool execution based on the current task identified by the strategy identification module and the current calling tool agent, thereby obtaining an output result and a dialogue strategy; The execution tool module, as the controller of the entire agent, connects various modules through preset trigger conditions. When receiving a new query, the agent executes a cycle of perception, planning, tool selection, and tool execution. The specific process includes: memory update, task planning, tool execution, summary and polishing.

[0044] As the controller of the entire agent, the core task is to ensure that the agent can efficiently and accurately execute the cycle of perception, planning, tool selection and tool execution when receiving new queries. This process involves multiple steps, including memory updating, task planning, tool execution and summary polishing.

[0045] Memory update, triggering conditions: After each interaction with the user, the agent needs to update its conversation memory. Process: Historical conversation record: Store the latest conversation content in the historical conversation database, including the user's questions, the agent's answers, and the timestamp of the interaction. User profile update: Update the user profile based on the user's new interactions, which may include the user's interests, preferences, and historical behavior patterns. Context maintenance: Maintain the conversation context to maintain coherence in multiple rounds of conversation.

[0046] Task planning, trigger conditions: When the agent receives a new user query, it needs to plan how to respond. Process: Intent recognition: Use natural language processing technology to identify the intent of the user query. Goal setting: Based on the identified intent, set the goals and expected results of the conversation. Strategy selection: Select the most appropriate dialogue strategy from the strategy library to achieve these goals.

[0047] Tool selection, triggering conditions: Based on the results of task planning, the agent needs to select the appropriate tool to perform the task. Process: Tool evaluation: Evaluate the available tools and select the most appropriate tool based on the requirements and context of the current task. Parameter configuration: Configure the necessary parameters for the selected tool, such as user ID, query content, etc. Exception handling preparation: Determine how to handle exceptions if the tool fails to execute.

[0048] Tool execution, trigger conditions: Once the tools are selected and configured, the agent will execute these tools. Process: Information retrieval: Retrieve relevant information from the knowledge base. Dialogue guidance: Ensure that the dialogue conforms to the preset value strategy. Historical information query: Retrieve relevant historical dialogue information. Anomaly detection: Monitor possible anomalies during the dialogue process.

[0049] After the tool is executed, the agent needs to summarize and polish the results to provide them to the user. Process: Information integration: Integrate the results of the tool execution into coherent information. Language polishing: Use natural language generation technology to polish the information to make it more natural and easy to understand. Emotion injection: Inject corresponding emotional elements according to the context of the conversation and the emotional state of the user.

[0050] Suppose the user asks a question about Qin Shi Huang: "Please tell me about Qin Shi Huang's achievements." The agent's processing flow is as follows: Memory update: Record the user's questions in the historical conversation database. Update the user profile and mark the user's interest in Chinese history. Maintain the conversation context and prepare to quote it in subsequent conversations.

[0051] Task planning: Identify that the user's query intent is to obtain historical information. Set the goal to provide the main achievements of Qin Shi Huang. Choose an information provision strategy that aims to educate and entertain.

[0052] Tool selection: Evaluate and select a knowledge retrieval tool to retrieve information about Qin Shi Huang. Configure parameters and enter the query "Qin Shi Huang's achievements".

[0053] Tool execution: The knowledge retrieval tool retrieves the achievements of Qin Shi Huang from the knowledge base. The value strategy guides the conversation tool to ensure that the answer respects historical facts and avoids misleading. The historical conversation information query tool checks whether there are related historical conversations that can provide additional information. The exception handling tool monitors the conversation and is ready to handle any exceptions.

[0054] Summarize and polish: Integrate the retrieved information into a coherent answer. Polish the answer to make it more fluent and easy to understand. Inject emotional elements to make the answer more vivid and attractive. Or choose whether to answer based on your own perception. If you are in an entertainment environment, you can choose to answer in an interesting way. If you are in a teaching environment, you can answer in a teaching way. You can also debate with the user or refuse to answer the question based on the identified user.

[0055] The strategy dialogue module is used to store a pre-built strategy library, which includes dialogue strategies.

[0056] Optionally, the pre-built strategy library is constructed based on a multi-round dialogue dataset fine-tuning model, and the dataset of the multi-round dialogue dataset fine-tuning model meets set conditions, and the set conditions include authenticity, richness and accuracy; High-quality datasets bring significant benefits. Fine-tune the model with a high-quality multi-turn dialogue dataset to align the model to the set roles and their possible behavior logic. The dataset meets the following criteria: authenticity, richness, and accuracy.

[0057] The authenticity described: The value dimension comes from the embodied agent definition, and the value dimension in the dataset should come from the embodied agent definition, which means that the dataset should reflect the ethical and value decisions that the agent may face in actual applications. For example, if the embodied agent is designed as a medical assistant, the dataset should contain conversations related to medical ethics and patient care. The scene information comes from the information of the environment in which the embodied agent is located, and the scene information should come from the environment in which the embodied agent is located, ensuring that the content of the conversation matches the actual usage environment. This can be achieved through simulation or actual scene data collection to ensure that the agent can understand and adapt to its operating environment. The action sequence comes from the action sequence that the embodied agent can complete; the action sequence should come from the action sequence that the embodied agent can complete, ensuring that the actions mentioned in the conversation are within the capabilities of the agent. This helps the training model understand executable actions and their significance in the conversation.

[0058] Richness: The data includes the execution of actions driven by different value dimensions, different dialogue strategies, and rich topics; the dataset should include the execution of actions driven by different value dimensions, which helps the model learn how to respond appropriately in different situations. The dataset should include different dialogue strategies so that the model can learn to adopt different communication methods in different situations. The dataset should cover a rich range of topics to improve the generalization ability of the model in diverse scenarios.

[0059] Accuracy: Data generation depends on speech models, and data pipelines are built. The data pipeline contains three parts: scene->strategy->dialogue. Different system prompts are designed for each part, and different system prompts are designed for each part by making full use of thought chains and knowledge expansion. Data generation should rely on advanced models to ensure the natural fluency and semantic accuracy of dialogue. Data pipeline: The built data pipeline should contain three parts: scene, strategy, and dialogue, and different system prompts are designed for each part. This helps to ensure the coherence and contextual relevance of the data. Thought chain and knowledge expansion: Make full use of thought chains and knowledge expansion to supplement and enhance information to improve the information quality and depth of the data set. High-quality multimodal data drives the rapid improvement of embodied intelligence perception, decision-making, and action control capabilities. Data is the key to the rapid adaptation of embodied intelligence to new environments and tasks. High-quality multi-round dialogue datasets that meet the standards of authenticity, richness, and accuracy can significantly improve the performance and adaptability of embodied agents, and provide a solid foundation for more intelligent and natural interactions.

[0060] Optionally, the multi-round dialogue dataset fine-tuning model uses low-rank adaptation to fine-tune parameters. In order to preserve the knowledge of the model itself and meet the rich pre-settings, the parameter fine-tuning scheme selects LoRA (Low-Rank Adaption) to fine-tune parameters. Experiments show that after LoRA fine-tuning Qwen1.5-14b, business needs are met.

[0061] Experimental design: In order to verify the rationality and effectiveness of this embodiment as a prompt solution, the results of CoT-only, few-shots-only, and this embodiment on Qwen1.5-14b were evaluated. 1556 pieces of precise data and 119 pieces of verification data were used. The experimental results show that this embodiment has a high accuracy rate in tool calling, intent understanding, and output structure.

[0062] The ablation experiment aims to evaluate whether the prompting technology is used directly or the voice model needs to be fine-tuned. The results show that fine-tuning the prompt composed of multiple information such as strategy, intent, and tool effectively alleviates the model's hallucinations and achieves high accuracy.

[0063] The decision-making module of the embodied intelligent body is the core of the entire embodied intelligent system. It is responsible for receiving environmental information from the perception module, performing task planning and reasoning analysis, and guiding the action module to generate actions. Specifically, the decision-making module includes the following aspects: Mission planning and reasoning analysis: The decision module needs to complete high-level mission planning and reasoning analysis, and generate step-by-step decision instructions to control actions. This includes understanding the mission objectives, the state of the environment, and path planning for how to reach the target state from the current state.

[0064] Processing of environmental perception information: The decision module receives various information from the perception module, including multimodal data such as vision, hearing, and touch, and further processes it in combination with the task objectives.

[0065] Formulation of action strategies: Based on the processed information, the decision module formulates specific action strategies, which will guide the action module to generate actions to achieve the task objectives.

[0066] Flexibility and adaptability: The decision-making module needs to be highly flexible and adaptable to cope with dynamically changing environments and unknown situations.

[0067] Application of intelligent models: With the development of intelligent model technology, decision-making modules increasingly rely on intelligent models to achieve machine intelligent decision-making. These intelligent models can process and transform multimodal information, perform specific actions, and continuously learn and evolve in interaction with the environment.

[0068] Multimodal information fusion: The decision-making module needs to be able to integrate information from different senses, such as vision, language, touch, hearing, etc., to achieve more accurate decisions and actions.

[0069] Autonomous learning and evolution: The decision-making module also involves the ability to continuously learn and evolve from experience feedback. Through a closed-loop feedback mechanism, the decision-making module can self-optimize and improve its adaptability to the environment and its intelligence level.

[0070] The decision-making module of the embodied intelligent agent handles complex tasks mainly through the following key sub-capability modules: Goal Interpretation: This module is responsible for converting high-level task goals into low-level goals that the agent can understand and execute. It requires the agent to be able to interpret and understand complex task descriptions and convert these descriptions into specific action guidelines.

[0071] Sub-goal decomposition: The task goal is broken down into a series of sub-goals, each of which is also expressed in a linear temporal logic formula. This decomposition helps break down complex tasks into smaller, more manageable parts, so that the final goal can be achieved step by step.

[0072] Action sequence planning: Generate action sequences based on the task objectives and execute them in the environment to achieve the target state. This module requires the agent to be able to plan a series of actions and execute them in sequence to achieve sub-goals and the final goal.

[0073] Transition modeling: Generate preconditions and effects for each action or operator to form a model of the environment transition. This involves understanding how each action changes the state of the environment and how to choose the best action based on the current state and the goal state.

[0074] The goal interpretation module first converts complex task goals into executable sub-goals, then the sub-goal decomposition module further refines these sub-goals, the action sequence planning module plans the specific action sequences to achieve these sub-goals, and finally the transformation modeling module evaluates how these actions affect the environment and guides the agent to adjust its strategy according to environmental changes. Through this modular approach, the embodied agent can gradually solve complex tasks and improve its ability to handle complex environments and tasks.

[0075] The embodied robot disclosed in the embodiment of the present embodiment is an embodied robot with its own value function and personality modeled. When it faces different questions, it can choose to reply or refuse to reply. This depends on its own state at the time and the intention of others. Therefore, when different dialogue strategies are used for the same question, there will be completely different replies. When the intention of others conflicts with one's own intention, by debating with others, if one successfully convinces others, one can continue to complete the task; otherwise, when convinced, one listens to the advice of others to do new tasks.

[0076] like Figure 2 As shown, this embodiment also discloses a multi-round dialogue method of an embodied intelligent agent, including: Step S201: in response to an input instruction, decompose the input instruction to obtain the intention of the input instruction; When receiving input instructions, the input instructions are split. For example, when the input instruction is "I want to know the answer to a certain question", the instruction is decomposed to understand that the input instruction is to answer a question in a certain aspect, thereby understanding the intention of others.

[0077] In response to the input command, the input command is first semantically analyzed and intent recognized to obtain the specific intent of the input command. This step may involve natural language processing techniques such as entity recognition and intent classification to ensure that the agent can accurately understand the user's instructions.

[0078] Step S202: Obtaining the agent dialogue strategy based on the input instruction and the pre-built strategy library; Based on the input instructions and the pre-built policy library, the agent will get a dialogue strategy. This policy library may contain a variety of dialogue scenarios and corresponding response strategies. The agent selects the appropriate dialogue strategy by matching the intention of the input instructions.

[0079] The agent searches for the corresponding dialogue strategy in the pre-built strategy library according to the input instructions. For example, in the pre-built strategy library, the agent perceives the need for a strategy. If the user is a teacher, the dialogue strategy is a direct answer. If the user is a student, a guided approach is used to enable the student to obtain the answer.

[0080] Step S203: Retrieving corresponding data from a pre-built tool pool based on the obtained intention of the input instruction and the dialogue strategy; When the instruction splitting and dialogue strategy are completed, the corresponding knowledge is retrieved in the tool pool. Such as historical knowledge or geographical knowledge related to the problem. According to the intention of the input instruction and the dialogue strategy, the corresponding data is retrieved in the tool pool. This data may include knowledge base information, historical dialogue records, user preferences, etc., to support the intelligent agent to generate appropriate dialogue content.

[0081] Step S204: Obtain an output result based on the obtained intention of the input instruction, the dialogue strategy and the retrieved data.

[0082] For example, the teacher can directly tell the result, while the student can use the question-and-answer method, first give some simple prompts, and when the student answers, guide the student to get the answer. Based on the intention of the input command, the dialogue strategy and the retrieved data, the agent will generate the output result. This output result may be an answer, an action instruction or a series of dialogue turns.

[0083] Optionally, obtaining an output result based on the obtained intention of the input instruction, the dialogue strategy and the retrieved data further includes: Obtaining a dialogue strategy based on the obtained intention of the input instruction, the dialogue strategy, and the retrieved data; The dialogue strategy is input into a strategy library to dynamically adjust the strategy library.

[0084] Based on the intent of the input instructions, the dialogue strategy, and the retrieved data, the agent can further obtain the dialogue strategy and input it into the strategy library to dynamically adjust the strategy library. This means that the agent can continuously optimize and update its dialogue strategy library according to the actual situation of the dialogue and user feedback to improve the quality and adaptability of the dialogue. In some cases, embodied agents may need to improve their dialogue capabilities through cross-modal learning and imitation. For example, agents can learn how to perform tasks in a specific visual environment by observing and imitating the behavior of experts. When conducting multi-round dialogues, embodied agents may need to process inputs from multiple modalities, such as visual, auditory, and textual information. The agent needs to be able to understand and integrate this multimodal information to provide richer and more accurate dialogue responses.

[0085] There is no limitation on the language model used in this embodiment, and it can be a self-developed language model, or a large language model such as GPT (Generative Pre-trained Transformer) can be used for processing.

[0086] The electronic device disclosed in this embodiment includes a memory and a processor. The memory is used to store non-temporary computer-readable instructions. Specifically, the memory may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc.

[0087] The processor may be a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions. In one embodiment of the present disclosure, the processor is used to run the computer-readable instructions stored in the memory, so that the electronic device performs all or part of the steps of the embodied agent multi-round dialogue method of each embodiment of the present disclosure.

[0088] Those skilled in the art should be able to understand that in order to solve the technical problem of how to obtain a good user experience, the present embodiment may also include well-known structures such as a communication bus and an interface, and these well-known structures should also be included in the protection scope of the present disclosure.

[0089] like Figure 3 The present invention provides a schematic diagram of the structure of an electronic device according to an embodiment of the present invention, which is suitable for implementing the electronic device in the embodiment of the present invention. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0090] like Figure 3 As shown, the electronic device may include a processing device (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage device into a random access memory (RAM). In the RAM, various programs and data required for the operation of the electronic device are also stored. The processing device, ROM, and RAM are connected to each other through a bus. The input / output (I / O) interface is also connected to the bus.

[0091] Typically, the following devices can be connected to the I / O interface: input devices such as sensors or visual information acquisition devices; output devices such as display screens; storage devices such as tapes, hard disks, etc.; and communication devices. The communication device allows the electronic device to communicate with other devices (such as edge computing devices) wirelessly or by wire to exchange data. Figure 3 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0092] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processing device, all or part of the steps of the embodied intelligent agent multi-round dialogue method of the embodiment of the present disclosure are executed.

[0093] For detailed description of this embodiment, reference may be made to the corresponding descriptions in the aforementioned embodiments, which will not be repeated here.

[0094] The computer-readable storage medium disclosed in this embodiment stores non-transitory computer-readable instructions. When the non-transitory computer-readable instructions are executed by a processor, all or part of the steps of the embodied agent multi-round dialogue method of each embodiment of the present disclosure are executed.

[0095] The above-mentioned computer-readable storage media include, but are not limited to: optical storage media (e.g., CD-ROM and DVD), magneto-optical storage media (e.g., MO), magnetic storage media (e.g., magnetic tape or mobile hard disk), media with built-in rewritable non-volatile memory (e.g., memory card) and media with built-in ROM (e.g., ROM box).

[0096] For detailed description of this embodiment, reference may be made to the corresponding descriptions in the aforementioned embodiments, which will not be repeated here.

[0097] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0098] In the present disclosure, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. The block diagrams of the devices, devices, equipment, and systems involved in the present disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0099] Additionally, as used herein, "or" used in a list of items beginning with "at least one" indicates a separate list, so that, for example, a list of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not mean that the example described is preferred or better than other examples.

[0100] It should also be noted that in the system and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0101] Various changes, substitutions, and modifications of the techniques described herein may be made without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of the present disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of events, means, methods, and actions described above. Currently existing or later to be developed processes, machines, manufactures, compositions of events, means, methods, or actions that perform substantially the same functions or achieve substantially the same results as the corresponding aspects described herein may be utilized. Thus, the appended claims include such processes, machines, manufactures, compositions of events, means, methods, or actions within their scope.

[0102] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0103] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. An embodied agent multi-turn dialogue system, characterized in that: include: Tool pool, strategy identification module, execution tool module and strategy dialogue module; The tool pool is used to store tools; The strategy identification module is used to identify the current task and the current calling tool, and store the execution result in a data structure; The execution tool module executes a cycle of perception, planning, tool selection and tool execution based on the current task identified by the strategy identification module and the current calling tool agent, thereby obtaining an output result and a dialogue strategy; The strategy dialogue module is used to store a pre-built strategy library, which includes dialogue strategies.

2. The embodied agent multi-turn dialogue system according to claim 1, characterized in that: The tool pool is obtained by injecting external knowledge of prompt words and recalled information, and then sending the information into the speech model for reasoning.

3. The embodied agent multi-round dialogue system according to claim 2, characterized in that: The tool pool includes historical conversation information query, exception handling, value strategy guided conversation and knowledge retrieval.

4. The embodied agent multi-round dialogue system according to claim 2, characterized in that: The description file of each tool in the tool pool includes: name, description, function, optional parameters and required parameters. The description file is used to help the intelligent agent to make reasonable tool selection, parameter transfer and tool execution.

5. The embodied agent multi-turn dialogue system according to claim 1, characterized in that: The pre-built strategy library is constructed based on a multi-round dialogue dataset fine-tuning model, and the dataset of the multi-round dialogue dataset fine-tuning model meets the set conditions, and the set conditions include authenticity, richness and accuracy; The authenticity: the value dimension comes from the definition of the embodied agent, the scene information comes from the information of the environment in which the embodied agent is located, and the action sequence comes from the action sequence that the embodied agent can complete; Richness: The data includes the execution of actions driven by different value dimensions, different dialogue strategies, and rich topics; Accuracy: Build a data pipeline. The data pipeline consists of three parts: scenario->strategy->dialogue. Design different system prompts for each part, and make full use of thinking chains and knowledge expansion to supplement and enhance information.

6. The embodied agent multi-round dialogue system according to claim 5, characterized in that: The multi-round dialogue dataset fine-tuning model uses a low-rank adaptive approach to perform parameter fine-tuning.

7. The embodied agent multi-round dialogue system according to claim 1, characterized in that: The strategy recognition module combines the methods of cue word engineering and model fine-tuning.

8. An embodied agent multi-round dialogue method, applied to the system according to any one of claims 1 to 7, characterized in that: include: In response to an input instruction, decomposing the input instruction to obtain an intention of the input instruction; Obtain the agent dialogue strategy based on input instructions and pre-built strategy library; Retrieve corresponding data from the pre-built tool pool based on the intent and dialogue strategy of the input command; The output result is obtained based on the intention of the input instruction, the dialogue strategy and the retrieved data.

9. The embodied agent multi-round dialogue method according to claim 8, characterized in that: The output result obtained based on the obtained input instruction intention, dialogue strategy and retrieved data also includes: Obtaining a dialogue strategy based on the obtained intention of the input instruction, the dialogue strategy, and the retrieved data; The dialogue strategy is input into a strategy library to dynamically adjust the strategy library.

10. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the embodied intelligent agent multi-round dialogue method described in any one of claims 8-9.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, which are used to enable a computer to execute the embodied intelligent agent multi-round dialogue method described in any one of claims 8-9.

12. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the embodied intelligent agent multi-round dialogue method described in any one of claims 8-9 is implemented.

Images