Intelligent agent monitoring method and multi-intelligent agent cooperation system

Abstract

The application discloses an agent supervision method and a multi-agent cooperation system. The agent supervision method comprises the following steps: obtaining first content generated by a second agent; the first content represents content generated by the second agent when the second agent executes a target task assigned by a first agent; determining a first quality level corresponding to the first content based on the first content and the target task; in response to the first quality level satisfying a specified condition, determining a third agent capable of executing the target task from a plurality of agents; determining real-time output content corresponding to the target task according to second content generated by the third agent; the second content represents content generated based on the first content and the target task; the first agent represents an agent with a supervision function in the plurality of agents; and the second agent and the third agent represent working agents in the plurality of agents.

Description

Technical Field

[0001] This application relates to, but is not limited to, the field of computer technology, and in particular to an agent supervision method and a multi-agent cooperative system. Background Technology

[0002] A multi-agent network (MAN) is a distributed network system consisting of multiple autonomous agents connected through communication. Its core purpose is to enable multiple agents to work together to complete complex tasks that a single agent cannot accomplish.

[0003] Currently, the Supervisor pattern is one of the most widely used centralized collaborative architectures in multi-agent networks. In the Supervisor pattern, the supervisor agent in a multi-agent network undertakes the core responsibility of judging user intent and distributing requests, while multiple worker agents divide the tasks to be distributed by the supervisor agent. Summary of the Invention

[0004] In view of this, this application provides at least one intelligent agent supervision method and a multi-agent cooperative system.

[0005] The technical solution of this application is implemented as follows: On the one hand, this application provides a method for supervising intelligent agents, the method comprising: Obtain the first content generated by the second agent; the first content represents the content generated when the second agent performs the target task assigned by the first agent. Based on the primary content and the target task, determine the primary quality level corresponding to the primary content; In response to the first quality level meeting the specified conditions, a third agent capable of performing the target task is determined from multiple agents; Based on the second content generated by the third intelligent agent, the real-time output content corresponding to the target task is determined; the second content represents the content generated based on the first content and the target task. Among them, the first intelligent agent represents the intelligent agent with the supervisory function among multiple intelligent agents; the second and third intelligent agents represent the working intelligent agents among multiple intelligent agents.

[0006] In some implementations, the first content includes at least one window content; Based on the primary content and the target task, determine the primary quality level corresponding to the primary content, including: For each window's content, determine the corresponding second quality level based on the target task; The first quality level is determined based on at least one second quality level and at least one window content.

[0007] In some implementations, a second quality level is determined based on the target task, including: Determine the relevance level of the window content to the target task; Determine the invalid response level corresponding to the window content; the invalid response level is determined based on the invalid response content contained in the window content; Based on the relevant level and the invalid response level, a second quality level is determined.

[0008] In some implementations, determining a first quality level based on at least one second quality level and at least one window content includes: For each window's content, determine the semantic information corresponding to that content; Clustering is performed on the semantic information and second quality level corresponding to at least one window content to obtain the clustering result; Based on the clustering results, the first quality level is determined.

[0009] In some implementations, the method further includes: For each window's content, identify at least one keyword within the window's content; Clustering is performed on the semantic information and second quality level corresponding to at least one window content to obtain clustering results, including: Clustering is performed on the semantic information, second quality level, and keywords corresponding to at least one window content to obtain the clustering results.

[0010] In some implementations, the clustering results include multiple clusters; Based on the clustering results, the first quality level is determined, including: determining the first quality level based on the distance between different clusters; The method further includes: determining that the first quality level meets specified conditions when the first quality level is less than a level threshold.

[0011] In some implementations, after determining a third agent from a plurality of agents capable of performing the target task, the method further includes: Based on the target task and the third content, generate target prompt information; the third content includes at least the first content, as well as generated content related to the target task that is output before the first content; The target prompt information is sent to a third intelligent agent so that the third intelligent agent can generate the second content.

[0012] In some implementations, the method further includes: If the first quality level meets the specified conditions and there is no working agent among the multiple agents that can perform the target task, determine the real-time output content for the target task based on the fourth content generated by the first agent. The fourth content represents the content generated based on the first content and the target task.

[0013] In some implementations, before outputting the real-time output content corresponding to the target task, the method further includes: generating and outputting fifth content in real time; the fifth content represents the clarification content.

[0014] On the other hand, this application provides a multi-agent cooperative system, including: a first agent, a second agent, and a third agent; the first agent represents an agent with supervisory functions in the multi-agent cooperative system; the second and third agents represent working agents in the multi-agent cooperative system; wherein... The second intelligent agent executes the target task assigned by the first intelligent agent to generate the first content; The first intelligent agent determines a first quality level corresponding to the first content based on the first content and the target task; in response to the first quality level meeting specified conditions, it determines a third intelligent agent from the multi-agent cooperative system that can perform the target task. The third intelligent agent generates the second content based on the first content and the target task; The first intelligent agent also determines the real-time output content corresponding to the target task based on the second content.

[0015] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of this application. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0017] Figure 1 A schematic diagram illustrating the implementation process of an intelligent agent supervision method provided in this application; Figure 2 A schematic diagram illustrating the implementation process of one embodiment provided in this application; Figure 3 A system architecture diagram of a multi-agent cooperative system provided in this application; Figure 4 This is a schematic diagram of the hardware entity of an electronic device provided in this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application are further described in detail below with reference to the accompanying drawings and embodiments. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0020] The terms “first / second / third” are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that “first / second / third” may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. The terminology used herein is for descriptive purposes only and is not intended to limit the scope of this application.

[0022] To better understand the agent supervision method provided in this application, the following explains two typical supervisor models and their existing problems.

[0023] The first mode is the master-round judgment mode, which means that in each round of dialogue with the user, the user input is identified and assigned a domain by the supervisory agent to ensure accurate response.

[0024] However, in the preceding and following rounds of dialogue, the supervisor agent may distribute tasks to different worker agents. As a result, the worker agent that performs the task later will not be able to obtain the complete context (e.g., the user input and response received by the worker agent that performed the task earlier), leading to insufficient dialogue coherence.

[0025] The second mode is the control handover mode. That is, considering that users often have multiple rounds of interaction around a specific topic in real-world dialogues, after the first round is judged by the supervisory agent, the supervisory agent will transfer control to a domain agent. Subsequent interactions will be led by the domain agent until the domain agent can no longer handle the situation or the topic changes, at which point the supervisory agent will be called back.

[0026] Control handover can significantly improve user experience and dialogue efficiency. However, after control is handed over, the supervisory agent has weak global perception of the dialogue and struggles to detect and correct anomalies such as deviations or overstepping of boundaries by the domain agent in a timely manner. This is especially true in streaming output scenarios, where there is a lack of effective real-time detection and dynamic takeover mechanisms.

[0027] Therefore, while maintaining a smooth user experience, how to improve the real-time perception and dynamic control capabilities of supervisory agents regarding the response quality in multi-turn dialogues has become a technical problem that multi-agent networks urgently need to solve.

[0028] This application provides an agent supervision method. First, a first content generated by a second agent is obtained, wherein the first content represents the content generated by the second agent when performing a target task assigned by the first agent. Then, based on the first content and the target task, a first quality level corresponding to the first content is determined. Subsequently, in response to the first quality level meeting specified conditions, a third agent capable of performing the target task is determined from multiple agents. Finally, based on the second content generated by the third agent, the real-time output content corresponding to the target task is determined, wherein the second content represents the content generated based on the first content and the target task, the first agent represents an agent with supervision function among multiple agents, and the second and third agents represent working agents among multiple agents. In this way, on the one hand, since the first intelligent agent is the supervisory intelligent agent responsible for judging user intent and distributing requests, it has more complete and comprehensive contextual information related to the dialogue. Therefore, the first intelligent agent is also used as an intelligent agent for dynamically supervising the quality of generated content, which enables the first intelligent agent to have global perception capabilities. This can improve the accuracy of the first intelligent agent's supervision of the quality of generated content, realize the dynamic capture and task takeover of abnormal generated content, and thus improve the quality and coherence of the dialogue. On the other hand, in the case of streaming the first content, the first intelligent agent is used to perform asynchronous real-time supervision of the first content, realizing the streaming dialogue control strategy.

[0029] The method provided in this application can be executed by an electronic device, which can be a laptop, tablet, desktop computer, set-top box, mobile device (e.g., mobile phone, portable music player, personal digital assistant, dedicated messaging device, portable gaming device), or a server. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0030] The technical solutions in this application will now be clearly and completely described in conjunction with the accompanying drawings.

[0031] Figure 1 A schematic diagram illustrating the implementation process of an intelligent agent supervision method provided in this application is shown below. Figure 1 As shown, the method includes the following steps S11 to S14: Step S11: Obtain the first content generated by the second intelligent agent; the first content represents the content generated when the second intelligent agent executes the target task assigned by the first intelligent agent; wherein, the first intelligent agent represents the intelligent agent with the supervisory function among multiple intelligent agents; the second intelligent agent represents the working intelligent agent among multiple intelligent agents.

[0032] Here, the multiple agents can be any type of multiple agents connected through a communication network. In some implementations, the multiple agents are multiple agents belonging to the same multi-agent network.

[0033] In some implementations, the multiple agents may include any type of agent such as simple reactive agents, model reactive agents, goal-oriented agents, utility-oriented agents, and / or learning agents.

[0034] In some implementations, the multiple agents may include any type of agent such as a tool-invoking agent, a planning agent, an execution agent, a memory agent, and / or a reflection agent.

[0035] In some implementations, when the multiple agents are agents in a multi-agent network, the multiple agents include supervisory agents, working agents, coordinating agents, expert agents, diagnostic agents, etc.

[0036] The first intelligent agent is the one with supervisory functions among multiple intelligent agents; that is, the first intelligent agent is the supervisory intelligent agent. In this way, the first intelligent agent can receive user input, determine user tasks based on user input, and then distribute user tasks to the working intelligent agents among the multiple intelligent agents.

[0037] The second intelligent agent is a working intelligent agent among multiple intelligent agents; that is, the second intelligent agent is responsible for executing user tasks distributed by the first intelligent agent and reporting the execution results to the first intelligent agent. In some embodiments, the second intelligent agent can be a general-purpose working intelligent agent. In some embodiments, the second intelligent agent can be an expert intelligent agent with knowledge of a specific domain, such as an expert intelligent agent with knowledge of the legal domain, medical domain, astronomical domain, or any other domain.

[0038] A target task refers to a subtask that a first intelligent agent determines based on user input, breaks down the user task into at least one subtask, and then distributes to a second intelligent agent. In some implementations, a user task may include one or more subtasks. Therefore, a target task may be the same as the user task, or it may be a part of the user task. For example, when the user task is "summarizing knowledge related to drones," the target task may be "retrieving information related to drones" or "summarizing and generating knowledge from the retrieved drone information."

[0039] Here, a second intelligent agent is used to perform the target task, thereby generating the first content.

[0040] In some implementations, the first content is content generated in real time and streamed during the execution of the target task by the second intelligent agent. For example, when the target task is "to summarize and generate knowledge from the retrieved UAV data," the first content may be a partial knowledge generation result based on the UAV data.

[0041] In some implementations, the first content is the content generated after the second intelligent agent completes the target task. For example, when the target task is "to summarize and generate knowledge from the retrieved UAV data", the first content can be the complete knowledge generation result for the UAV data.

[0042] In some implementations, the first content may be a sequence of tokens generated by the second agent or natural language text.

[0043] After the second agent generates the first content, the first agent retrieves the first content to determine whether it meets specified conditions. In some embodiments, the first content generated by the second agent is cached in real time to a designated cache space, thereby sending the cached content in the cache space to the first agent in real time. In some embodiments, the first content generated by the second agent is cached in real time to a designated cache space, allowing the first agent to read the first content from the cache space in real time.

[0044] Step S12: Based on the first content and the target task, determine the first quality level corresponding to the first content.

[0045] Here, the first intelligent agent evaluates the quality of the first content based on the target task to determine the first quality level corresponding to the first content.

[0046] In some implementations, the first agent can determine a first quality level based on the semantic relevance of the first content to the target task. In some implementations, the semantic relevance of the first content to the target task is used as the first quality level. In some implementations, a mapping relationship between different semantic relevances and different quality levels can be pre-set, and then the first quality level can be determined based on this mapping relationship and the semantic relevance of the first content to the target task.

[0047] In some implementations, the first agent can utilize lightweight vector matching methods to calculate the relevance between the first content and the target task. For example, the relevance between the first content and the target task can be calculated using methods such as Term Frequency – Inverse Document Frequency (TF-IDF), Simple Contrastive Learning of Sentence Embeddings (SimCSE), and a lightweight version of the BERT model (Distilled Bidirectional Encoder Representations from Transformers, DistilBERT).

[0048] In some implementations, a Large Language Model (LLM) can be used to determine the relevance of the first content to the target task.

[0049] In some implementations, the first agent can use a large language model to judge the semantic accuracy of the first content, thereby helping to determine the first quality level corresponding to the first content.

[0050] Step S13: In response to the first quality level meeting the specified conditions, a third intelligent agent capable of performing the target task is determined from multiple intelligent agents; wherein, the third intelligent agent represents the working intelligent agent among the multiple intelligent agents.

[0051] The specified conditions refer to the conditions under which the first intelligent agent takes over the target task. Thus, if the first quality level meets the specified conditions, it means that the first content does not meet the preset quality requirements, and therefore the first intelligent agent needs to take over the target task.

[0052] In some implementations, a first quality level satisfies specified conditions, indicating that the first content has a low semantic relevance to the target task, the first content has low semantic accuracy, or the number or proportion of uncertain content contained in the first content is greater than a specified threshold.

[0053] Therefore, when the first quality level meets the specified conditions, the first agent takes over the target task and determines a third agent from among the multiple agents that can execute the target task, so as to use the third agent to execute the target task again.

[0054] Here, the third intelligent agent is a working intelligent agent among multiple intelligent agents; that is, the third intelligent agent is responsible for executing user tasks distributed by the first intelligent agent and reporting the execution results to the first intelligent agent. In some embodiments, the third intelligent agent can be a general working intelligent agent. In some embodiments, the third intelligent agent can be an expert intelligent agent with knowledge of a specific domain, such as an expert intelligent agent with knowledge of the legal domain, medical domain, astronomical domain, or any other domain.

[0055] In some implementations, after determining the target task, a first agent can identify multiple candidate working agents capable of executing the target task based on their task execution capabilities. These candidate working agents are then ranked according to their matching degree with the target task. Thus, when determining the second agent, the first agent can select the candidate with the highest matching degree; similarly, when determining the third agent, the first agent can select the candidate with the second highest matching degree.

[0056] In some implementations, after determining that the first quality level meets the specified conditions, the first agent determines the working agent with the highest matching degree with the target task based on the task execution capabilities of multiple working agents other than the second agent, and uses the determined working agent as the third agent.

[0057] In some implementations, the task execution capability of a working agent can be determined based on its functionality and idle state. For example, if a working agent has the capability to execute a target task but is currently occupied, its task execution capability is considered low; conversely, if a working agent has the capability to execute a target task and is currently idle, its task execution capability is considered high.

[0058] Step S14: Determine the real-time output content corresponding to the target task based on the second content generated by the third agent; the second content represents the content generated based on the first content and the target task.

[0059] Here, after determining the third intelligent agent, the third intelligent agent generates the second content corresponding to the target task based on the target task and the first content.

[0060] In some implementations, determining the real-time output content corresponding to the target task based on the second content may include using the second content as the real-time output content corresponding to the target task.

[0061] In some implementations, determining the real-time output content corresponding to the target task based on the second content may include generating real-time output content based on the second content and the first content. That is, the first content is proofread or regenerated using the second content to obtain the real-time output content corresponding to the target task. For example, if the first content is generated using an expert AI agent in the legal field, the second content may be generated using an expert AI agent in the patent law field. In this way, proofreading or regenerating the first content using the second content can supplement and improve the patent law-related content in the first content, thereby obtaining a more comprehensive and accurate real-time output content.

[0062] In some implementations, real-time output content can be generated using any method, such as visualization or audio.

[0063] In the intelligent agent supervision method provided in this application, firstly, first content generated by a second intelligent agent is obtained, wherein the first content represents the content generated by the second intelligent agent when executing the target task assigned by the first intelligent agent; then, based on the first content and the target task, a first quality level corresponding to the first content is determined; subsequently, in response to the first quality level meeting specified conditions, a third intelligent agent capable of executing the target task is determined from multiple intelligent agents; finally, based on the second content generated by the third intelligent agent, the real-time output content corresponding to the target task is determined, wherein the second content represents the content generated based on the first content and the target task, the first intelligent agent represents the intelligent agent with supervision function among multiple intelligent agents, and the second and third intelligent agents represent working intelligent agents among multiple intelligent agents. In this way, on the one hand, since the first intelligent agent is the supervisory intelligent agent responsible for judging user intent and distributing requests, it has more complete and comprehensive contextual information related to the dialogue. Therefore, the first intelligent agent is also used as an intelligent agent for dynamically supervising the quality of generated content, which enables the first intelligent agent to have global perception capabilities. This can improve the accuracy of the first intelligent agent's supervision of the quality of generated content, realize the dynamic capture and task takeover of abnormal generated content, and thus improve the quality and coherence of the dialogue. On the other hand, in the case of streaming the first content, the first intelligent agent is used to perform asynchronous real-time supervision of the first content, realizing the streaming dialogue control strategy.

[0064] In some implementations, the first content includes at least one window content.

[0065] Here, window content refers to the content obtained after dividing the first content into segments.

[0066] In some implementations, at least one window content corresponding to the first content can be determined in any way. In some implementations, firstly, semantic analysis can be performed on the first content to determine at least one semantic segment contained within it; then, each semantic segment is treated as a window content, or one or more semantic segments are treated as a window content. In some implementations, a sliding window technique can be used to determine at least one window content corresponding to the first content; wherein, there is semantic overlap or token sequence overlap between adjacent window contents to maintain contextual coherence and prevent information gaps. For example, between two adjacent window contents, the later-generated content in the earlier window content has semantic or token sequence overlap with the earlier-generated content in the later window content.

[0067] In some implementations, when the first content is generated in a streaming manner, a sliding window technique can be used to form a fixed sequence of continuously updated windows based on the first content and tokens, with the content in each window being the corresponding window content.

[0068] Thus, based on the first content and the target task, the first quality level corresponding to the first content is determined, that is, the above step S12 can be implemented as the following steps S121 to S122: Step S121: For each window content, determine the second quality level corresponding to the window content according to the target task.

[0069] Here, for each window's content, a first quality evaluation is performed based on the target task to determine the corresponding second quality level.

[0070] In some implementations, the relevance of the window content to the target task can be determined based on the task objective, context, constraints, user input or output requirements, etc., and then a second quality level can be determined based on the relevance.

[0071] Step S122: Determine the first quality level based on at least one second quality level and at least one window content.

[0072] Here, after determining the second quality level corresponding to each window content, a second quality evaluation is performed on the first content based on at least one second quality level and at least one window content to obtain the first quality level.

[0073] In some implementations, a second quality evaluation can be performed on the first content based on at least one second quality level, at least one window content, and the semantics and / or keywords corresponding to the at least one window content, to obtain a first quality level.

[0074] In some implementations, the weight of each window's content can be determined based on its timestamp, and then the first quality level of the first content can be determined based on the weight and second quality level of each window. In some implementations, at least one window's content can be assigned a weight from high to low according to its timestamp, so that the content generated earlier has a higher weight.

[0075] In some implementations, clustering can be performed on a first quality level and semantic features corresponding to at least one window content to obtain multiple clusters, thereby determining the first quality level based on the multiple clusters.

[0076] In the embodiments provided in this application, on the one hand, by dividing the first content into at least one window content for quality evaluation, the efficiency of quality evaluation of the first content can be improved; on the other hand, by combining at least one second quality level with at least one window content to determine the first quality level, the first content can be evaluated from more dimensions, thereby improving the reliability and accuracy of the first quality level.

[0077] In some implementations, determining the second quality level corresponding to the window content based on the target task in step S121 above can be implemented as the following steps S1211 to S1213: Step S1211: Determine the relevance level of the window content to the target task.

[0078] Here, the relevance level of the window content to the target task can be determined in any way.

[0079] In some implementations, the relevance level between window content and the target task can be determined by assessing the semantic similarity between the window content and the target task. Specifically, a mapping relationship between different semantic similarities and different relevance levels can be pre-established, and the corresponding relevance level can be determined based on this mapping relationship and the semantic similarity between the window content and the target task. In other implementations, the similarity between window content and the target task can be determined by assessing the semantic similarity between the window content and the user input information or task description information corresponding to the target task.

[0080] In some implementations, a large language model can be used to determine the semantic similarity between the window content and the target task.

[0081] In some implementations, lightweight vector matching methods, such as SimCSE or DistilBERT, can be used to determine the semantic similarity between the window content and the target task.

[0082] In some implementations, the cosine similarity between the window content and the target task can be used as the similarity between the window content and the target task; then, the corresponding relevance level is determined based on this similarity, wherein there is a preset mapping relationship between different similarities and different relevance levels. In some implementations, this cosine similarity can be the cosine similarity between the TF-IDF vector of the window content and the TF-IDF vector of the target task.

[0083] Step S1212: Determine the invalid response level corresponding to the window content; the invalid response level is determined based on the invalid response content contained in the window content.

[0084] Here, invalid response content refers to generated content that does not meet task requirements, lacks practical value, or violates basic interaction principles.

[0085] In some implementations, invalid response content includes content in the window that deviates from the intended topic trajectory.

[0086] In some implementations, the predetermined topic trajectory refers to the semantics of the user input corresponding to the target task. Thus, content deviating from the predetermined topic trajectory refers to content that is semantically inconsistent with or has low similarity to the user input.

[0087] In some implementations, a predetermined topic trajectory refers to the logical trajectory used to generate the results corresponding to the target task. For example, in a logical trajectory of "retrieve, summarize retrieval results, determine whether the summarized content meets the user's requirements, and if not, re-execute the retrieval," it is necessary to determine whether to execute the subsequent steps based on the output results of the previous steps. During this process, it is determined whether the generated content continues the predetermined topic trajectory. Thus, content that deviates from the predetermined topic trajectory refers to window content that does not conform to the predetermined logical trajectory. For example, according to the logical trajectory of the example above, if the "retrieval" step is successfully executed and the "summarize" step continues, the generated content corresponding to the "summarize" step is considered to meet the predetermined topic trajectory; if the "retrieval" step fails and the "summarize" step continues, the generated content corresponding to the "summarize" step is considered to deviate from the predetermined topic trajectory.

[0088] In some implementations, a large language model can be used to query the semantic accuracy of the window content in order to determine whether the window content deviates from the predetermined topic trajectory.

[0089] In some implementations, invalid response content includes redundant content. Redundant content refers to text that repeats itself without adding new information or changing the conclusion.

[0090] In some implementations, invalid response content includes vague or evasive content. Vague or evasive content refers to content generated to avoid directly answering sensitive, uncertain, or questions beyond one's capabilities. Examples include phrases like "I'm not sure..." or "I don't know...".

[0091] In some implementations, rule matching can be used to identify invalid response content in the window content. For example, rule matching can be used to identify templated, vague, or low-information content in the window content.

[0092] Therefore, when determining the level of invalid response, firstly, semantic analysis is performed on the window content to identify the invalid response content contained within it; then, the invalid response level is determined based on the amount of data in the invalid response content or the proportion of invalid response content within the window content. It is understandable that the larger the amount of data in the invalid response content, or the higher the proportion of invalid response content within the window content, the higher the level of invalidity.

[0093] Step S1213: Determine the second quality level based on the relevant level and the invalid response level.

[0094] In some implementations, the relevance level and invalid response level corresponding to the window content can be used as the second quality level corresponding to the window. That is, the second quality level includes the relevance level and the invalid response level.

[0095] In some implementations, a second quality level corresponding to the window content can be calculated using a specified calculation method based on the relevance level and invalid response level of the window content. For example, the difference or weighted difference between the relevance level and the invalid response level can be used as the second quality level.

[0096] In the embodiments provided in this application, during the process of the second agent generating the first content, at least one window content sequence is formed according to the content generation time, and each window content is used as a local semantic unit of the generated content to monitor in real time whether the generated content of the second agent has abnormal deviations or low-quality problems (i.e., whether there are abnormal responses). It is evident that, compared to judging the quality of the agent's response (i.e., the quality of the generated content) after each round of dialogue, this application can shorten the response monitoring cycle and achieve real-time intervention in abnormal responses; compared to monitoring the semantics of each token generated by the agent, this application can reduce the huge computational and cost overhead caused by frequent calls to embedding networks, classification models, etc.

[0097] In some implementations, a first quality level is determined based on at least one second quality level and at least one window content; that is, step S122 above can be implemented as steps S1221 to S1223: Step S1221: For each window content, determine the semantic information corresponding to the window content.

[0098] Here, semantic parsing is performed on the content of each window to obtain the semantic information corresponding to each window content. For example, if the window content is a token sequence, the token sequence can be decoded to obtain the corresponding semantic information.

[0099] Step S1222: Perform clustering processing on the semantic information and second quality level corresponding to at least one window content to obtain the clustering result.

[0100] Here, for each window content, the semantic information and the second quality level corresponding to the window content are both used as the content features corresponding to the window content, thereby performing clustering processing on the content features of at least one window content in the first content to obtain clustering results, that is, obtaining one or more clusters.

[0101] In some implementations, clustering can be performed on the content features of at least one window using any method. In some implementations, clustering can be performed using algorithms such as K-means clustering, density-based spatial clustering of applications with noise (DBSCAN), agglomerative clustering, divisive clustering, and Gaussian mixture model-based clustering.

[0102] Step S1223: Determine the first quality level based on the clustering results.

[0103] Here, based on the clustering results after clustering, a first quality level is determined for the first content. For example, when the number of clusters in the clustering result is small, a higher first quality level is assigned to the first content; while when the number of clusters is large, a lower first quality level is assigned to the first content. As another example, when the distance between different clusters in the clustering result is small, a higher first quality level is assigned to the first content; while when the distance between different clusters is large, a lower first quality level is assigned to the first content.

[0104] In the embodiments provided in this application, by clustering at least one window content and its corresponding second quality level, the content features of the at least one window content can be automatically classified, the distribution structure of the content features and isolated content features can be determined, thereby determining whether the at least one window content contains an abnormal response, and thus determining the first quality level corresponding to the first content. In this way, the first quality level can be determined using multi-dimensional content features, making the first quality level more accurate and more interpretable.

[0105] In some implementations, the method further includes the step S1224: Step S1224: For each window content, determine at least one keyword in the window content.

[0106] Here, keywords are information used to express the core characteristics of the window content.

[0107] In some implementations, when the first content is a token sequence, the token sequence can be decoded to determine at least one keyword from the decoding result.

[0108] In some implementations, when the window content is natural language text, keyword detection or analysis can be performed on the natural language to identify at least one keyword.

[0109] In some implementations, a topic summary can be performed on the window content to obtain the topic terms corresponding to the window content, and then the topic terms can be used as keywords.

[0110] In some implementations, the window content can be categorized according to multiple preset content tags to obtain the target content tags corresponding to the window content, and then the target content tags can be used as keywords.

[0111] Thus, clustering is performed on the semantic information and second quality level corresponding to at least one window content to obtain the clustering result. That is, the above step S1222 can be implemented as the following steps S12221: Step S12221: Perform clustering processing on the semantic information, second quality level and keywords corresponding to at least one window content to obtain clustering results.

[0112] Here, when performing clustering, the semantic information, second quality level, and keywords corresponding to each window content in at least one window content are all used as the content features of that window content for clustering.

[0113] In the embodiments provided in this application, by also using the keywords corresponding to each window content as content features of the window content and participating in clustering processing, it is possible to perform clustering analysis on more feature information in at least one window content, thereby improving the accuracy of clustering results and thus improving the accuracy of quality evaluation of the first content (i.e., improving the accuracy of the first quality level).

[0114] In some implementations, the clustering results include multiple clusters.

[0115] Here, after performing clustering processing on the semantic information and second quality level corresponding to at least one window content, or the semantic information, second quality level and keywords, multiple clusters are obtained.

[0116] Thus, based on the clustering results, the first quality level is determined, that is, the above step S1223 can be implemented as the following step S12231: Step S12231: Determine the first quality level based on the distance between different clusters.

[0117] Here, the distance between different clusters in the clustering results is calculated to obtain at least one inter-cluster distance, and then the first quality level is determined based on the at least one inter-cluster distance.

[0118] Because a larger inter-cluster distance indicates lower similarity between clusters, the content of at least one window participating in the clustering is less relevant to the target task, deviates more from the predetermined topic trajectory of the target task, and exhibits more redundancy or ambiguity avoidance. Therefore, a larger inter-cluster distance results in a lower quality level.

[0119] Thus, when determining the first quality level based on at least one inter-cluster distance corresponding to different clusters, the larger the at least one inter-cluster distance, the lower the first quality level; conversely, the smaller the at least one inter-cluster distance, the higher the first quality level.

[0120] In some implementations, a first quality level is determined based on the largest inter-cluster distance among at least one inter-cluster distance. Thus, the larger the largest inter-cluster distance, the lower the first quality level; conversely, the smaller the largest inter-cluster distance, the higher the first quality level.

[0121] In some implementations, a first quality level is determined based on the average inter-cluster distance corresponding to at least one inter-cluster distance. Thus, the larger the average inter-cluster distance, the lower the first quality level; conversely, the smaller the average inter-cluster distance, the higher the first quality level.

[0122] Thus, before determining the third agent capable of performing the target task from among multiple agents in response to the first quality level meeting the specified conditions, i.e., before step S13 above, the method further includes the following step S15: Step S15: When the first quality level is less than the level threshold, determine that the first quality level meets the specified conditions.

[0123] The quality threshold is the minimum quality level used to determine whether generated content meets quality requirements.

[0124] Therefore, when the first quality level is not less than the level threshold, the generation quality of at least one window content participating in the clustering meets the preset quality requirements, and it is not necessary to use the first intelligent agent to take over the target task. Therefore, the first quality level does not meet the specified conditions.

[0125] Conversely, when the first quality level is less than the level threshold, the generation quality of at least one window content representing the clustering does not meet the preset quality requirements. Therefore, it is necessary to use the first intelligent agent to take over the target task and ensure that the first quality level meets the specified conditions.

[0126] In the embodiments provided in this application, the first quality level corresponding to the first content is determined based on the distance between different clusters, so that the first quality level can accurately reflect whether the content of at least one window has deviated from the target task, whether it has deviated from the established topic, whether there is duplicate content, empty output or fuzzy rejection, etc., thus providing a reliable basis for the first intelligent agent to determine whether to take over the target task.

[0127] The first content is generated in a streaming manner during the execution of the target task by the second intelligent agent, and each window content in the first content has a corresponding timestamp. Therefore, in some implementations, the first quality level is cumulatively calculated starting from the first window content in the first content according to the timestamp corresponding to the window content. For example, when calculating the first quality level for the first time, clustering is performed on the first window content and the second window content (i.e., the window content generated after the first window content), and the corresponding first quality level is determined based on the clustering result; when calculating the first quality level for the second time, clustering is performed on the first window content, the second window content, and the third window content (i.e., the window content generated after the second window content), and the corresponding first quality level is determined based on the clustering result; and so on. In this way, the first quality level can be calculated in real time and cumulatively using the window content generated in real time from the first content, and the first intelligent agent can determine in real time whether to take over the target task based on whether the first quality level meets the specified conditions.

[0128] Understandably, during the cumulative calculation process, if the number of abnormal window contents (i.e., abnormal responses) in the first content gradually increases, the first quality level will gradually decrease until the first quality level meets the specified conditions. In this way, through the cumulative calculation method, the first agent can take over the target task only when the first content contains a large number of abnormal window contents, avoiding the problem of the first agent taking over the target task when the first content contains only a small number of abnormal window contents, which would lead to premature interruption and reduced processing efficiency of the target task.

[0129] In some implementations, after determining a third agent capable of performing the target task from among multiple agents, that is, after step S13 above, the following steps S16 to S17 are further included: Step S16: Generate target prompt information based on the target task and the third content; the third content includes at least the first content and the generated content related to the target task that was output before the first content. Step S17: Send the target prompt information to the third intelligent agent so that the third intelligent agent can generate the second content.

[0130] Here, when using a third agent to perform a target task, in order to make the content generated by the third agent more accurate and have better coherence with the output content (e.g., the content that has been output to the user), third content related to the target task is sent to the third agent.

[0131] The third content includes the first content generated by the second agent. Thus, sending the first content to the third agent allows the third agent to reflect on the first content and, based on the reflection results and the content characteristics of the first content, execute the target task.

[0132] The third content includes generated content related to the target task that was output before the first content. This allows the third agent to obtain more context related to the target task, thereby improving the accuracy of the second content generated by the third agent and its coherence with the already output content. For example, when the target task is the user task corresponding to the current round of a multi-turn dialogue, the dialogue content of the N completed rounds of dialogue can be sent to the third agent; where N≥1.

[0133] Thus, based on the target task and the third content, the first agent constructs target prompt information and sends the target prompt information to the third agent, so that the third agent can generate the second content based on the target prompt information.

[0134] In the embodiments provided in this application, constructing target prompt information based on the target task and the third content enables the third agent to obtain context related to the target task, thereby making the second content generated by the third agent semantically coherent with the output generated content.

[0135] In some implementations, the method further includes the following step S18: Step S18: If the first quality level meets the specified conditions and there is no working agent among the multiple agents capable of performing the target task, determine the real-time output content for the target task based on the fourth content generated by the first agent; wherein, the fourth content represents the content generated based on the first content and the target task.

[0136] In some implementations, the absence of a working agent capable of performing the target task among the multiple agents means that, apart from the second agent, the multiple agents do not include an agent capable of processing the target task; or, apart from the second agent, all agents capable of processing the target task among the multiple agents are in an occupied state.

[0137] Thus, if the first quality level meets the specified conditions and there is no working agent among the multiple agents capable of performing the target task, the first agent determines to perform the target task itself and generates the fourth content.

[0138] When the first agent generates the fourth content, it generates a prompt message based on the first content and the target task, and then generates the fourth content based on the generated prompt message.

[0139] In some implementations, the first intelligent agent generates target prompt information based on the target task, the first content, and generated content related to the target task that is output before the first content; based on the target prompt information, the first intelligent agent generates fourth content.

[0140] After generating the fourth content, the specific implementation method of the real-time output content corresponding to the target task is determined based on the fourth content. This is the same as the specific implementation method of determining the real-time output content corresponding to the target task based on the second content, and will not be repeated here.

[0141] In the embodiments provided in this application, after the first intelligent agent takes over the target task, if there is no working intelligent agent among the multiple intelligent agents that can execute the target task, the first intelligent agent itself will execute the target task. In this way, the execution efficiency and completion rate of the target task can be improved.

[0142] In some implementations, the method further includes the following step S19 before outputting the real-time output content corresponding to the target task: Step S19: Generate and output the fifth content in real time; the fifth content represents the clarified content.

[0143] In the case where the first content is generated and output in a streaming manner, if the first agent determines that the first quality level meets the specified conditions during the dynamic monitoring of the first content, since part of the first content has already been output to the user, the first agent needs to clarify the output content and continue to output the real-time output content corresponding to the target task after clarification, so as to improve the naturalness and fluency of the dialogue.

[0144] In some implementations, the first agent may use a clarification template to generate the fifth content. For example, if the first agent determines that the output has deviated from the intended topic, the fifth content could be, "The previous content seems to have gone off-topic; let's get back to your question...". Or, if the first agent determines that the output contains repetitive content, the fifth content could be, "The previous content seems a bit wordy; let's simplify it...".

[0145] Below, in conjunction with Figure 2 This application will describe one embodiment; wherein the first intelligent agent is implemented as a supervisory intelligent agent, the second intelligent agent is implemented as a first working intelligent agent, and the third intelligent agent is implemented as a second working intelligent agent. Figure 2 As shown, this embodiment includes the following steps S21 to S27: Step S21: Based on user input, the supervisory agent determines the target task; then, step S22 is executed. Here, after receiving user input, the supervisory agent judges the user's intent and determines the target task.

[0146] Step S22: The supervisory agent distributes the target task to the first working agent, so that the first working agent generates the sixth content; then, step S23 is executed. Here, the first working agent streams and outputs the sixth content.

[0147] Step S23: The supervisory agent obtains the sixth content and determines the third quality level corresponding to the sixth content based on the target task; then, step S24 is executed. Here, the regulatory agent acquires the generated sixth content in real time and performs dynamic asynchronous monitoring.

[0148] In some implementations, the regulatory agent can use sliding window technology to determine the window content corresponding to the sixth content, and then determine the third quality level corresponding to multiple window contents of the sixth content through cumulative calculation. In some implementations, the specific method for determining the third quality level is the same as the specific method for determining the first quality level, so please refer to the detailed description above, which will not be repeated here.

[0149] In step S24, in response to the third quality level being less than the level threshold, the supervisory agent stops outputting the sixth content; then, steps S25 and S26 are executed. In some implementations, the supervisory agent sends an End of Sequence (EOS) marker to the first working agent to stop the first working agent from continuing to generate the sixth content.

[0150] In some implementations, the supervisory agent controls the front end to block the sixth content in order to stop the output of the sixth content.

[0151] Step S25: The supervisory agent generates and outputs clarification content; then, step S27 is executed. Step S26: Based on the target task, the sixth content, and the prior dialogue content related to the target task, the supervisory agent generates a prompt message and sends the prompt message to the second working agent; then, step S27 is executed. Step S27: Based on the prompt information sent by the supervisory agent, the second working agent generates the seventh content.

[0152] Here, the second working agent can stream and output the seventh content. During the streaming output of the seventh content, the supervisory agent can continue to acquire the seventh content to achieve dynamic asynchronous monitoring of the seventh content.

[0153] As can be seen from the above, in the intelligent agent supervision method provided in this application, on the one hand, since the supervisory intelligent agent is responsible for judging user intent and distributing requests, it has more complete and comprehensive contextual information related to the dialogue. Therefore, using the supervisory intelligent agent as an intelligent agent to dynamically monitor the quality of generated content gives it global perception capabilities, which can improve the accuracy of the supervisory intelligent agent in monitoring the quality of generated content. On the other hand, the supervisory intelligent agent continuously and asynchronously listens to the generated content of the working intelligent agent, and combines sliding window technology to perform fine-grained semantic and quality monitoring of the generated content of the working intelligent agent, realizing the dynamic capture and takeover judgment of abnormal responses, thereby improving the quality and coherence of the dialogue. Furthermore, by accumulating and calculating quality levels and level thresholds, the timing of the supervisory intelligent agent's takeover of the target task can be precisely controlled, achieving the effect of not interrupting prematurely and not allowing erroneous output.

[0154] Based on the foregoing embodiments, this application provides a multi-agent cooperative system. For example... Figure 3 As shown, the multi-agent cooperative system 300 includes: a first agent 310, a second agent 320, and a third agent 330; the first agent 310 represents an agent with supervisory functions in the multi-agent cooperative system; the second agent 320 and the third agent 330 represent working agents in the multi-agent cooperative system; wherein, The second intelligent agent 320 executes the target task assigned by the first intelligent agent 310 to generate the first content; First agent 310 determines a first quality level corresponding to the first content based on the first content and the target task; in response to the first quality level meeting specified conditions, it determines a third agent 330 from the multi-agent cooperative system that can perform the target task. The third intelligent agent 330 generates the second content based on the first content and the target task; The first intelligent agent 310 also determines the real-time output content corresponding to the target task based on the second content.

[0155] In some implementations, the first content includes at least one window content; First intelligent agent 310, used for: For each window's content, determine the corresponding second quality level based on the target task; The first quality level is determined based on at least one second quality level and at least one window content.

[0156] In some implementations, the first intelligent agent 310 is used for: Determine the relevance level of the window content to the target task; Determine the invalid response level corresponding to the window content; the invalid response level is determined based on the invalid response content contained in the window content; Based on the relevant level and the invalid response level, a second quality level is determined.

[0157] In some implementations, the first intelligent agent 310 is used for: For each window's content, determine the semantic information corresponding to that content; Clustering is performed on the semantic information and second quality level corresponding to at least one window content to obtain the clustering result; Based on the clustering results, the first quality level is determined.

[0158] In some implementations, the first intelligent agent 310 is used for: For each window's content, identify at least one keyword within the window's content; Clustering is performed on the semantic information, second quality level, and keywords corresponding to at least one window content to obtain the clustering results.

[0159] In some implementations, the clustering results include multiple clusters; First intelligent agent 310, used for: The first quality level is determined based on the distance between different clusters; When the first quality level is less than the level threshold, the first quality level is determined to meet the specified conditions.

[0160] In some implementations, the first intelligent agent 310 is further configured to: Based on the target task and the third content, generate target prompt information; the third content includes at least the first content, as well as generated content related to the target task that is output before the first content; The target prompt information is sent to a third intelligent agent so that the third intelligent agent can generate the second content.

[0161] In some implementations, the first intelligent agent 310 is further configured to: If the first quality level meets the specified conditions and there is no working agent among the multiple agents that can perform the target task, determine the real-time output content for the target task based on the fourth content generated by the first agent. The fourth content represents the content generated based on the first content and the target task.

[0162] In some implementations, the first intelligent agent 310 generates and outputs the fifth content in real time; the fifth content represents the clarified content.

[0163] The descriptions of the system embodiments above are similar to those of the method embodiments above, and have similar beneficial effects. In some embodiments, the functions or modules included in the system provided by this application can be used to execute the methods described in the method embodiments above. For technical details not disclosed in the system embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0164] It should be noted that, in the embodiments of this application, if the above-described intelligent agent supervision method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the related technology, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware, software, or firmware, or any combination of hardware, software, and firmware.

[0165] It should be noted that the description of the various embodiments above tends to emphasize the differences between the various embodiments, while the similarities or similarities can be referred to each other.

[0166] It should be noted that, Figure 4 This is a schematic diagram of a hardware entity of the electronic device in this application, such as... Figure 4 As shown, the hardware entity of the electronic device 400 includes: a processor 401, a communication interface 402, and a memory 403, wherein: The processor 401 typically controls the overall operation of the electronic device 400.

[0167] Communication interface 402 enables electronic devices to communicate with other terminals or servers via a network.

[0168] The memory 403 is configured to store instructions and applications executable by the processor 401, and can also cache data to be processed or already processed (e.g., image data, audio data, voice communication data, and video communication data) in the processor 401 and various modules in the electronic device 400. It can be implemented using flash memory or random access memory (RAM). Data transfer between the processor 401, the communication interface 402, and the memory 403 can be performed via bus 404.

[0169] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above steps / processes do not imply a sequential order of execution; the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above embodiments of this application are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0170] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0171] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0172] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0173] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0174] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0175] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence or the part that contributes to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0176] The above are merely embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for supervising intelligent agents, comprising: Obtain the first content generated by the second agent; The first content represents the content generated when the second agent executes the target task assigned by the first agent; Based on the first content and the target task, determine the first quality level corresponding to the first content; In response to the first quality level meeting specified conditions, a third intelligent agent capable of performing the target task is determined from among multiple intelligent agents; Based on the second content generated by the third intelligent agent, determine the real-time output content corresponding to the target task; The second content represents content generated based on the first content and the target task; Wherein, the first intelligent agent represents the intelligent agent with supervisory function among the plurality of intelligent agents; the second intelligent agent and the third intelligent agent represent the working intelligent agents among the plurality of intelligent agents.

2. The method according to claim 1, wherein the first content includes at least one window content; The step of determining the first quality level corresponding to the first content based on the first content and the target task includes: For each window content, a second quality level is determined according to the target task; The first quality level is determined based on at least one second quality level and the content of the at least one window.

3. The method according to claim 2, wherein determining the second quality level corresponding to the window content based on the target task includes: Determine the relevance level of the window content to the target task; Determine the invalid response level corresponding to the window content; The invalid response level is determined based on the invalid response content contained in the window content; The second quality level is determined based on the relevant level and the invalid response level.

4. The method according to claim 2, wherein determining the first quality level based on at least one second quality level and the at least one window content comprises: For each window's content, determine the semantic information corresponding to that content; Clustering is performed on the semantic information and second quality level corresponding to the content of at least one window to obtain the clustering result; Based on the clustering results, the first quality level is determined.

5. The method according to claim 4, further comprising: For each of the window contents, determine at least one keyword in the window content; The step of performing clustering processing on the semantic information and second quality level corresponding to the content of at least one window to obtain clustering results includes: Clustering is performed on the semantic information, second quality level, and keywords corresponding to the content of at least one window to obtain the clustering result.

6. The method according to claim 4 or 5, wherein the clustering result comprises multiple clusters; Determining the first quality level based on the clustering results includes: determining the first quality level based on the distance between different clusters; The method further includes: when the first quality level is less than the level threshold, determining that the first quality level meets a specified condition.

7. The method according to any one of claims 1 to 6, further comprising, after determining the third agent capable of performing the target task from among a plurality of agents: Based on the target task and the third content, generate target prompt information; The third content includes at least the first content, and generated content related to the target task that is output before the first content; The target prompt information is sent to the third intelligent agent so that the third intelligent agent can generate the second content.

8. The method according to any one of claims 1 to 6, further comprising: If the first quality level meets the specified conditions and the plurality of intelligent agents does not include a working intelligent agent capable of performing the target task, the real-time output content for the target task is determined based on the fourth content generated by the first intelligent agent. The fourth content represents content generated based on the first content and the target task.

9. The method according to any one of claims 1 to 6, further comprising, before outputting the real-time output content corresponding to the target task: Generate and output the fifth content in real time; The fifth content represents the clarification content.

10. A multi-agent cooperative system, comprising: First intelligent agent, second intelligent agent, and third intelligent agent; The first intelligent agent represents the intelligent agent with supervisory function in the multi-agent cooperative system; The second agent and the third agent represent the working agents in the multi-agent cooperative system; wherein, The second intelligent agent executes the target task assigned by the first intelligent agent to generate the first content; The first agent determines a first quality level corresponding to the first content based on the first content and the target task; in response to the first quality level meeting a specified condition, it determines a third agent from the multi-agent collaborative system that can perform the target task. The third intelligent agent generates second content based on the first content and the target task; The first intelligent agent also determines the real-time output content corresponding to the target task based on the second content.

Images