Multi-agent-based mixed domain dialogue method

Abstract

The invention provides a multi-agent mixed domain dialogue method which comprises the following steps: receiving a document and dialogue content input by a user, judging a dialogue mode to which the dialogue content input by the user belongs, and generating a processing plan to be executed according to the dialogue mode; executing the processing plan so as to call at least one large model agent, and receiving a dialogue feedback result generated by using a generation method corresponding to the dialogue mode when the plurality of large model agents input the optimization result of the dialogue content and the document; and summarizing the dialogue feedback result to generate a final result, and displaying the final result to the user. According to the method, a centralized multi-agent integrated framework can be utilized, a TOD agent and an ODD agent which solve specific problems can be combined, meanwhile, the improved retrieval enhancement generation method based on the knowledge graph is provided for open domain dialogues based on knowledge, and the problem of insufficient knowledge can be effectively solved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of human-computer dialogue, and particularly relates to a hybrid field dialogue method based on multiple agents. BACKGROUND

[0002] An industrial scene dialogue system based on actual needs needs to be like an intelligent assistant such as Alex and Siri in addition to helping users complete a specific task. In actual application scenarios, a dialogue system must have certain open field dialogue capabilities, especially knowledge-grounded open domain dialogue (KODD) capabilities. Directly applying TOD systems such as ZS-TOD and SimpleTOD in an intelligent dialogue scenario has problems such as limited dialogue flexibility and insufficient knowledge coverage. An intelligent dialogue system often needs to simultaneously solve multiple complex dialogue problems, including calling a field service, and in addition, the system needs to be able to use data in a document as external knowledge to perform knowledge-based question answering.

[0003] The prior art document FusedChat: Fusing Task-oriented and Open-domain Dialogues in Conversational Agents [paper]: YOUNG T, XING F, PANDELEA V, et al. Fusing task-oriented and open-domain dialogues in conversational agents [C] / / Proceedings of the AAAI Conference on Artificial Intelligence: vol. 36: 10. 2022: 11622-11629 modifies dialogue data in two ways through a traditional TOD dataset MultiWOZ: one is to modify and add ODD dialogue turns before dialogue, and the other is to add ODD dialogue turns after dialogue to construct a hybrid field dialogue dataset, and two baseline methods are proposed: one is an end-to-end single model, and the other is to first determine whether the dialogue mode is TOD or ODD by additionally introducing a classification model, and then two models respectively complete the corresponding dialogue task.

[0004] The prior art also provides a body agent multi-round dialogue system, method, device, medium and program product. The body agent multi-round dialogue system comprises a tool pool, a strategy identification module, an execution tool module and a strategy dialogue module. The tool pool is used to save tools. The strategy identification module is used to identify a current task and a current calling tool, and store an execution result in a data structure. The execution tool module is used to execute a cycle of perception, planning, tool selection and tool execution based on the current task and the current calling tool identified by the strategy identification module, so as to obtain an output result and a dialogue strategy. The strategy dialogue module is used to save a pre-constructed strategy library, and the pre-constructed strategy library comprises a dialogue strategy. The dialogue effectiveness and diversity are achieved.

[0005] However, the above prior art has the following defects:

[0006] 1. In actual use, a single dialogue round of a user may contain both ODD mode dialogue and TOD mode dialogue, and the above scheme does not consider the mixed field dialogue problem; 2. Researchers tend to solve specific field dialogue problems by constructing special agents, and the existing scheme cannot quickly collect different agent capabilities, and the expansibility of the agent is not considered. SUMMARY

[0007] In order to solve the above problems in the prior art, the application provides a mixed field dialogue method based on multiple agents. The technical problem to be solved by the application is solved by the following technical scheme:

[0008] A mixed field dialogue method based on multiple agents is applied to any intelligent terminal, and the intelligent terminal is integrated with multiple large model agents. The mixed field dialogue method based on multiple agents comprises the following steps:

[0009] S100, receiving a document and dialogue content input by a user, judging a dialogue mode to which the dialogue content input by the user belongs, and generating a processing plan to be executed according to the dialogue mode; the dialogue mode comprises a task-oriented dialogue and an open domain dialogue; wherein the processing plan comprises at least one large model agent to be called and an optimization result of the dialogue content;

[0010] S200, executing the processing plan to call at least one large model agent, and receiving dialogue feedback results generated by multiple large model agents by using a generation method corresponding to the dialogue mode when the multiple large model agents input the optimization result of the dialogue content and the document;

[0011] S300, summarizing the dialogue feedback results to generate a final result, and displaying the final result to the user.

[0012] Advantages:

[0013] The present invention proposes a multi-agent hybrid domain dialogue method to solve the hybrid domain dialogue generation scenario that can mix open domain dialogue and task-oriented dialogue, as well as the problem of insufficient agent scalability of existing agent dialogue methods. The method receives documents and dialogue content input by the user, determines the dialogue mode to which the dialogue content input by the user belongs, and generates a processing plan to be executed based on the dialogue mode; executes the processing plan to call at least one large model agent, and receives dialogue feedback results generated by multiple large model agents using the generation method corresponding to the dialogue mode when inputting the optimization results of the dialogue content and the documents; summarizes the dialogue feedback results to generate a final result and presents it to the user. The present invention can use a centralized multi-agent integrated architecture to solve the TOD agent and ODD agent of specific problems. At the same time, for knowledge-based open domain dialogue, a knowledge graph-based dialogue enhancement generation method is proposed, which can effectively solve the problem of insufficient knowledge.

[0014] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a flowchart of a multi-agent based mixed domain dialogue method provided by the present invention.

[0016] Figure 2 It is a schematic diagram of the dialogue interaction involved in the multi-agent based mixed-domain dialogue method provided by the present invention. DETAILED DESCRIPTION

[0017] The present invention will be further described in detail below with reference to specific examples, but the embodiments of the present invention are not limited thereto.

[0018] The application provides a multi-agent-based mixed-field dialogue method, which is applied to any intelligent terminal integrated with multiple large model agents.

[0019] In combination Figure 1 and Figure 2 , the application provides a multi-agent-based mixed-field dialogue method, which includes:

[0020] S100, receiving a document and dialogue content input by a user, judging a dialogue mode to which dialogue content input by the user belongs, and generating a processing plan to be executed according to the dialogue mode; the dialogue mode includes a task-oriented dialogue and an open domain dialogue; wherein the processing plan includes at least one large model agent to be called and an optimization result of the dialogue content;

[0021] S200, executing the processing plan to call at least one large model agent, and receiving dialogue feedback results generated by multiple large model agents by using a generation method corresponding to a dialogue mode when the dialogue content is input and the document is optimized;

[0022] wherein the generation method of the dialogue mode includes a task-oriented dialogue generation method and an open domain dialogue generation method, the task-oriented dialogue generation method generates dialogue feedback results by using a general instruction following language model, and the open domain dialogue generation method generates dialogue feedback results by using an improved knowledge graph-based retrieval enhancement generation method.

[0023] S300, summarizing the dialogue feedback results to generate a final result and displaying the final result to the user.

[0024] In a specific embodiment of the application, S100 includes:

[0025] S110, receiving dialogue content input by a user;

[0026] S120, using a conversation mode classification model to determine the conversation mode to which the conversation content belongs;

[0027] S130 , calling the dialogue pattern classification model to generate a processing plan to be executed when a dialogue pattern is input.

[0028] The conversation pattern classification model of this invention uses BERT-base-uncased. An important component of the coordinator is the conversation pattern classification model. Here we choose to use BERT-base-uncased to train based on the historical conversations in the scene. Its training parameters are as follows:

[0029] Table 1. Parameter configuration of conversational mode classification model

[0030]

[0031] The coordinator can then determine whether a conversation is task-oriented or open-domain based on the classification judgment given by the conversation mode classification model. For example, M = {M_TOD, M_ODD, M_ODD+M_TOD} autonomously decomposes the conversation tasks and chooses to have any number of conversations with the ODD agent or the TOD agent and give the final conversation response.

[0032] In a specific embodiment of the present invention, the step of generating a dialogue feedback result using the improved knowledge graph-based search enhancement generation method includes:

[0033] In the indexing stage, the document is segmented into text blocks according to predetermined text blocks; a text vector index, a knowledge graph index, and a graph description vector index are generated for all text blocks;

[0034] First, the document needs to be segmented according to the predetermined text block length. Then, for each text block, relationships and entities are extracted through pattern matching. Relatively clear rules can more accurately identify entities, relationships, and attributes in the data, and are less noisy than other methods. At this stage, a clear entity and relationship pattern needs to be given, for example:

[0035] Person = [WORKS_AT, HAS_EVENT]

[0036] Product = [PROVIDES]

[0037] Event=[HAS_EVENT,IN_LOCATION]

[0038] In the above examples, the pattern of each entity type and relationship is given, and the large model will mainly provide these entities and relationships based on the prompt words. In addition, the algorithm extracts entities and relationships in a non-strict matching manner, while allowing the large model to give entities and relationships that partially exceed the pattern rules based on the context. In the experimental section, the present application designs a matching model that fits the actual needs of enterprises.

[0039] Through non-strict pattern matching, the large model is used to extract entities and relationships, supplemented by text block vectorization and text graph description text vectorization, and the entity is de-duplicated to generate text vector index, knowledge graph index and graph description vector index.

[0040] In the search phase, based on the optimization result of the dialogue content, and through three retrievers, the search results are obtained by recalling search in the text vector index, knowledge graph index and graph description vector index, and they are used as the dialogue feedback results.

[0041] Among them, the three retrievers are knowledge graph retriever, text vector retriever and graph description vector retriever. The search phase is relatively simple, and the three retrievers are used for parallel recall, corresponding to the three indexes obtained above, and the three indexes are searched to obtain the dialogue feedback results.

[0042] In a specific embodiment of the present application, generating the text vector index, the knowledge graph index and the graph description vector index for all text blocks includes:

[0043] SA1, generating a text vector for each text block, and combining the text vector and the corresponding text to form a text vector index;

[0044] SA2, extracting the relationships and entities of the text block by pattern matching to obtain an entity set and a relationship set;

[0045] SA3, generating a knowledge graph index and a graph description vector index using the entity set and the relationship set.

[0046] In a specific embodiment of the present application, SS3 includes:

[0047] SA31, combining the entity set and the relationship set to form a knowledge graph, and performing a de-duplication operation on the knowledge graph to obtain a knowledge graph index;

[0048] SA32, for each text block, constructing a triple text containing all relationships and entities of the text block, and generating a graph description vector for each triple text, and combining the graph description vector and the triple text to form a graph description vector index.

[0049] In a specific embodiment of the present invention, performing a deduplication operation on the knowledge graph to obtain a knowledge graph index includes:

[0050] SB1, perform vector similarity search on each entity in the knowledge graph to obtain the similarity between entities;

[0051] SB2, select the top N similar entities and filter them to select entities that meet the set conditions as candidate entity sets;

[0052] Among them, candidate entities with similarity greater than the similarity threshold, edit distance less than the edit distance threshold and exactly the same labels are retained, and all candidate entities are combined into a candidate entity set.

[0053] SB3, merge the entities in the candidate entity set, and perform intersection operations on the relationships corresponding to the entities to obtain a knowledge graph index.

[0054] The knowledge graph index requires an additional deduplication operation: After the first iteration of each document in Algorithm 4.1, a vector similarity search is performed on each entity. After returning the top 10 most similar entities, similar entities are filtered. Entities with a similarity greater than the similarity_threshold (the similarity threshold can be 0.7 to 0.9), an edit distance less than the edit_distance (generally 2-6), and identical labels are retained as candidate entities. Entity fusion is then performed, and the entity correspondences are intersected to obtain the final knowledge graph index. Deduplication operations often lack best practices and rely on repeated experimentation to determine the different steps and variable values.

[0055] In addition to the knowledge graph index, the present invention also constructs two text embedding vector indexes, namely a vector index based on direct text embedding of text blocks, and a vector index based on text embedding of text triples of graph description relationships in text blocks.

[0056] After the indexing process, the algorithm obtains three indexes: knowledge graph index GraphIndex, text vector index VectorIndextext, and graph description vector index V ectorIndexmentions.

[0057] For the specific algorithm, please refer to Algorithm 4.1 shown in Table 2.

[0058]

[0059] In a specific embodiment of the present invention, in the search phase, based on the optimization results of the conversation content, the search results obtained by performing a recall search on the text vector index, the knowledge graph index, and the graph description vector index by three recallers include:

[0060] Vectorizing the optimized result of the conversation content, and performing search and recall on the text vector index to obtain a recall result of the recall node;

[0061] Performing entity recognition on the optimized result of the conversation content to obtain an entity to be queried;

[0062] Using a knowledge graph recaller to search and recall the entity to be queried on the knowledge graph index to obtain a recall result of the recall node;

[0063] Using a graph description vector recaller to search and recall the entity to be queried on the graph description vector index to obtain a recall result of a recall node;

[0064] Score all recall nodes and select the recall results corresponding to the recall nodes with the top-k scores as the dialogue feedback results.

[0065] The following describes the dialog process of the present invention by way of examples.

[0066] For a petrochemical enterprise, a hybrid domain dialogue scenario involves both open domain dialogue based on knowledge and task-oriented dialogue for service calls based on oil and gas field development reports. We need to build two agents to handle ODD and TOD mode dialogues. This paper chooses to use the proposed knowledge graph-based retrieval enhancement generation method to build the ODD agent:

[0067] First, the entity relationship model is given:

[0068] Oil and gas fields = [contains_blocks]

[0069] Block = [belongs to _oil and gas field, belongs to _reservoir type, faces _development issues, carries out _pilot projects]

[0070] Reservoir type = [demand_technical measures]

[0071] Test item = [adaptation_reservoir type]

[0072] Development Issues = []

[0073] Technical measures = [affect_development indicators, solve_development problems]

[0074] Well type = [cause_development problem]

[0075] Monitoring parameters = [Guidance_Technical measures]

[0076] At the same time, the parameters of the retrieval enhancement generation method are set to the text block size chuck_size is 256, the similarity threshold similarity_threshold is 0.8, and the maximum edit distance edit_distance is 6. Thus, we can construct the corresponding RAG agent as the ODD agent.

[0077] Then we also need a TOD agent. In practice, we use the agent built using the task-oriented dialogue method based on the reasoning model mentioned earlier. I will not go into details here and call it ThinkingTOD.

[0078] We then combined the centralized agent structure to obtain a hybrid domain dialogue method. The prompt words of the coordinator agent are as follows:

[0079] Prompt Word Template:MAHD Coordinator

[0080] ###Identity Tips

[0081] As the core coordinator of a multi-agent mixed-domain dialogue system, you need to handle both task-oriented dialogues (such as structured tasks like booking tickets or checking the weather) and open-domain dialogues (such as small talk or knowledge Q&A). Your responsibilities include:

[0082] 1. Analyze the user input type and determine the conversation mode based on the conversation mode classification model's recommendations.

[0083] 2. Generate a processing plan to be executed and generate the current behavior according to the execution plan. Your behavior is the parameter for requesting help from other agents.

[0084] 3. Observe the responses from other agents and update your plan and current behavior based on the responses and update your responses to the user;

[0085] 4. When you have completed your plan, set the current action to empty and set the final response.

[0086] ###Intelligent Agent Function Library

[0087] 1. Task Expert (@task_agent):

[0088] -Maintain global task status

[0089] -Execute preset domain task processes (including hotel reservations / flight inquiries, etc.)

[0090] -Return: Task results | Parameter requirements | Status change notification

[0091] 2. Knowledge Expert (@knowledge_agent):

[0092] -Access to the knowledge base of the knowledge graph, and is a very knowledgeable knowledge expert

[0093] -Return: External knowledge you want to know

[0094] ###Response Protocol

[0095] Output strictly according to this JSON structure:

[0096] "'json{"plan":"Your processing plan","current_action":

[0097] {"agent":"@agent_name","content":"The conversation content you prepared"},"response_cache":"The cached response, you can update it according to the response content of each agent"}"'

[0098] ###Example......

[0099] ###Now please design an appropriate solution and give an appropriate response based on the user's output and context.

[0100] {input}

[0101] The dialogue mode classification model determines the mode of the user's current input as follows: {mode}

[0102] It is worth noting that the terms "first" and "second" in this disclosure are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, features defined as "first" or "second" may explicitly or implicitly include one or more of such features. In the description of this disclosure, "plurality" means two or more, unless otherwise specifically defined.

[0103] Although the present application is described herein with reference to various embodiments, those skilled in the art will be able to understand and implement other variations of the disclosed embodiments in practicing the claimed application by reviewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality.

[0104] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art of the present invention, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should be considered to fall within the scope of protection of the present invention.

Claims

1. A multi-agent based mixed domain dialogue method, characterized in that: Applicable to any intelligent terminal, the intelligent terminal integrates multiple large-model intelligent agents, and the multi-agent-based mixed-domain dialogue method includes: S100, receiving a document and conversation content input by a user, determining the conversation mode to which the conversation content input by the user belongs, and generating a processing plan to be executed based on the conversation mode; the conversation modes include: task-oriented conversation and open-domain conversation; wherein the processing plan includes at least one large model agent to be invoked and the optimization result of the conversation content; S200, executing the processing plan to call at least one large model agent, and receiving dialogue feedback results generated by multiple large model agents using a generation method corresponding to a dialogue mode when inputting the optimized results of the dialogue content and the document; S300: Summarize the dialogue feedback results to generate a final result, and present it to the user.

2. The multi-agent based mixed domain dialogue method according to claim 1, characterized in that: S100 includes: S110, receiving the conversation content input by the user; S120, using a conversation mode classification model to determine the conversation mode to which the conversation content belongs; S130 , calling the dialogue pattern classification model to generate a processing plan to be executed when a dialogue pattern is input.

3. The multi-agent based mixed domain dialogue method according to claim 1, characterized in that: The generation method using the dialogue mode in S200 includes a task-oriented dialogue generation method and an open-domain dialogue generation method. The task-oriented dialogue generation method uses a general instruction-following language model to generate dialogue feedback results; the open-domain dialogue generation method uses an improved knowledge graph-based retrieval enhancement generation method to generate dialogue feedback results.

4. The multi-agent based mixed domain dialogue method according to claim 3, characterized in that: The improved knowledge graph-based retrieval enhancement generation method is used to generate dialogue feedback results, including: In the indexing stage, the document is segmented into text blocks according to predetermined text blocks; a text vector index, a knowledge graph index, and a graph description vector index are generated for all text blocks; In the search phase, based on the optimization results of the conversation content, three recallers are used to perform recall searches on the text vector index, knowledge graph index, and graph description vector index to obtain search results, which are used as conversation feedback results.

5. The multi-agent based mixed domain dialogue method according to claim 4, characterized in that: Generating text vector indexes, knowledge graph indexes, and graph description vector indexes for all text blocks includes: Generate a text vector for each text block, and combine the text vector and the corresponding text into a text vector index; Extracting the relationships and entities of the text block by pattern matching to obtain entity sets and relationship sets; The entity set and relationship set are used to generate a knowledge graph index and a graph description vector index.

6. The multi-agent based mixed domain dialogue method according to claim 5, characterized in that: Generating a knowledge graph index and a graph description vector index using the entity set and relationship set includes: The entity set and the relationship set are combined into a knowledge graph, and a deduplication operation is performed on the knowledge graph to obtain a knowledge graph index; For each text block, all relations and entities contained in the text block are constructed into triple texts, and a graph description vector of each triple text is generated. The graph description vector and the triple text are combined to form a graph description vector index.

7. The multi-agent based mixed domain dialogue method according to claim 6, characterized in that: The performing of a deduplication operation on the knowledge graph to obtain a knowledge graph index includes: Performing a vector similarity search on each entity in the knowledge graph to obtain the similarity between entities; Select the top N similar entities and filter them to select entities that meet the set conditions as candidate entity sets; The entities in the candidate entity set are fused, and the relationships corresponding to the entities are intersected to obtain a knowledge graph index.

8. The multi-agent based mixed domain dialogue method according to claim 7, characterized in that: The filtering of the similar entities to select entities that meet the set conditions as candidate entity sets includes: Candidate entities with similarity greater than the similarity threshold, edit distance less than the edit distance threshold and exactly the same labels are retained, and all candidate entities are grouped into a candidate entity set.

9. The multi-agent based mixed domain dialogue method according to claim 4, characterized in that: The three recallers are respectively a knowledge graph recaller, a text vector recaller and a graph description vector recaller.

10. The multi-agent based mixed domain dialogue method according to claim 9, characterized in that: In the search phase, based on the optimization results of the conversation content, three recallers are used to perform recall searches on the text vector index, knowledge graph index, and graph description vector index to obtain search results including: Vectorizing the optimized result of the conversation content, and performing search and recall on the text vector index to obtain a recall result of the recall node; Performing entity recognition on the optimized result of the conversation content to obtain an entity to be queried; Using a knowledge graph recaller to search and recall the entity to be queried on the knowledge graph index to obtain a recall result of the recall node; Using a graph description vector recaller to search and recall the entity to be queried on the graph description vector index to obtain a recall result of a recall node; Score all recall nodes and select the recall results corresponding to the recall nodes with the top-k scores as the dialogue feedback results.

Images