Agent training method, data processing method and data processing system

Abstract

The application discloses an agent training method, a data processing method and a data processing system, and relates to the fields of artificial intelligence and large model technology. The method comprises the following steps: determining an exploration direction based on existing knowledge of an agent system, wherein the exploration direction is used for planning and exploring a target task to be generated, the agent system comprises a plurality of agents with different capability levels; controlling the plurality of agents to execute an adversarial walk strategy according to the exploration direction, and generating test data and training data; performing capability boundary evaluation on the plurality of agents by using the test data, and obtaining an evaluation result; and training a to-be-evolved agent in the plurality of agents according to the evaluation result and the training data, and obtaining a target agent. The application solves the technical problem of low robustness of an agent in the related art because an agent training and evaluation method relies on a relatively static test data set.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence and large model technology, in particular to an agent training method, a data processing method and a data processing system. BACKGROUND

[0002] With the rapid development of the field of artificial intelligence (AI), agents (Agents) centered on large language models (LLM) are leading the new trend of artificial intelligence. Agents can perform complex tasks based on understanding natural language. However, due to the limitations of static and fixed training and evaluation methods, agents have shown obvious performance bottlenecks when facing challenges that require multi-tool collaboration and deep logical reasoning.

[0003] Currently, the agent training and evaluation method relies on relatively static test data sets. These test data sets are mainly designed around information retrieval or single tool invocation, and fail to fully reflect the collaborative tool use ability and deep reasoning skills of LLM in complex task execution, resulting in a lack of authenticity and diversity of training data, which limits the adaptability and robustness of agents when facing complex problems.

[0004] In view of the above problems, no effective solutions have been proposed so far. SUMMARY

[0005] The embodiments of the present application provide an agent training method, a data processing method and a data processing system to at least solve the technical problem of low robustness of agents in related technologies due to the dependence of agent training and evaluation methods on relatively static test data sets.

[0006] According to an aspect of an embodiment of the present application, an agent training method is provided, comprising: determining an exploration direction based on existing knowledge of an agent system, wherein the exploration direction is used to plan and explore a target task to be generated, and the agent system comprises: a plurality of agents with different capability levels; controlling the plurality of agents to execute an adversarial walk strategy according to the exploration direction to generate test data and training data; performing capability boundary evaluation on the plurality of agents using the test data to obtain an evaluation result; and training a to-be-evolved agent in the plurality of agents according to the evaluation result and the training data to obtain a target agent.

[0007] According to another aspect of an embodiment of the present application, a data processing method is also provided, comprising: obtaining problem data; and performing answer generation processing on the problem data using a target agent to obtain a target reply; wherein the target agent is generated according to the agent training method of any one of the above.

[0008] According to another aspect of the embodiments of the present application, a data processing method is further provided, comprising: obtaining intelligent customer service consultation question data; and generating answers to the intelligent customer service consultation question data by using an intelligent customer service agent to obtain an intelligent customer service consultation reply; wherein the intelligent customer service agent is generated according to the agent training method in any one of the preceding embodiments.

[0009] According to another aspect of the embodiments of the present application, a data processing method is further provided, comprising: obtaining data processing requests through a first application programming interface, wherein the request data carried in the data processing requests comprises question data; and returning data processing responses through a second application programming interface, wherein the response data carried in the data processing responses comprises a target reply, which is obtained by generating answers to the question data by using a target agent, and the target agent is generated according to the agent training method in any one of the preceding embodiments.

[0010] According to another aspect of the embodiments of the present application, a data processing method is further provided, comprising: obtaining a current input data processing dialogue request, wherein the request data carried in the data processing dialogue request comprises question data; and returning a data processing dialogue reply in response to the data processing dialogue request, wherein the information carried in the data processing dialogue reply comprises a target reply, which is obtained by generating answers to the question data by using a target agent, and the target agent is generated according to the agent training method in any one of the preceding embodiments; and displaying the target reply in a graphical user interface.

[0011] According to another aspect of the embodiments of the present application, a data processing method is further provided, comprising: displaying question data on an operation interface in response to an input instruction acting on the operation interface; and displaying a target reply on the operation interface in response to a processing instruction acting on the operation interface; wherein the target reply is obtained by generating answers to the question data by using a target agent, and the target agent is generated according to the agent training method in any one of the preceding embodiments.

[0012] According to another aspect of the embodiments of the present application, a data processing system is further provided, comprising: a client configured to send question data; a server connected to the client and configured to generate answers to the question data by using a target agent to obtain a target reply, wherein the target agent is generated according to the agent training method in any one of the preceding embodiments; and the client is further configured to output the target reply.

[0013] According to another aspect of the embodiments of the present application, an electronic device is further provided, comprising: a memory storing an executable program; and a processor connected to the memory through a bus and configured to run the program, wherein the program performs the agent training method or the data processing method in any one of the preceding embodiments when running.

[0014] According to another aspect of the embodiments of the present application, a computer readable storage medium is also provided, which includes a stored executable program, wherein the executable program controls the device where the computer readable storage medium is located to perform any of the above-mentioned agent training method or data processing method when the executable program is executed.

[0015] According to another aspect of the embodiments of the present application, a computer program product is also provided, which includes a computer program, the computer program implements any of the above-mentioned agent training method or data processing method when executed by a processor.

[0016] In the embodiments of the present application, the exploration direction is determined based on the existing knowledge of the agent system, wherein the exploration direction is used to plan and explore the target task to be generated, and the agent system includes a plurality of agents with different capability levels. Then the plurality of agents are controlled to perform the adversarial walk strategy according to the exploration direction, and the test data and the training data are generated. After that, the capability boundary of the plurality of agents is evaluated by using the test data, and the evaluation result is obtained. Finally, the to-be-evolved agent in the plurality of agents is trained according to the evaluation result and the training data, and the target agent is obtained. Thus, the agent with high robustness is trained, thereby realizing the dynamic data construction, the capability boundary evaluation and the iterative training, breaking the limitation of the static test data set, and significantly improving the robustness and flexibility of the agent in processing complex tasks, thereby solving the technical problem that the agent robustness is low due to the dependence of the agent training and evaluation method on the relatively static test data set in the related art.

[0017] It is easy to note that the above general description and the following detailed description are only for exemplifying and explaining the present application, and do not constitute a limitation on the present application. BRIEF DESCRIPTION OF DRAWINGS

[0018] The drawings described herein are used to provide further understanding of the present application, constitute a part of the present application, and the illustrative embodiments of the present application and the description thereof are used to explain the present application, and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 is an application scenario schematic diagram of an agent training method according to an embodiment of the present application;

[0020] Figure 2 is a flowchart of an agent training method according to an embodiment of the present application;

[0021] Figure 3 is a flowchart of data construction according to an embodiment of the present application;

[0022] Figure 4 is a flowchart of a data processing method according to an embodiment of the present application;

[0023] Figure 5 is a flowchart of another data processing method according to an embodiment of the application;

[0024] Figure 6 is a flowchart of another data processing method according to an embodiment of the application;

[0025] Figure 7 is a flowchart of another data processing method according to an embodiment of the application;

[0026] Figure 8 is a flowchart of another data processing method according to an embodiment of the application;

[0027] Figure 9 is a structural schematic diagram of a data processing system according to an embodiment of the application;

[0028] Figure 10 is a structural schematic diagram of an intelligent agent training apparatus according to an embodiment of the application;

[0029] Figure 11 is a structural schematic diagram of another data processing apparatus according to an embodiment of the application;

[0030] Figure 12 is a structural schematic diagram of another data processing apparatus according to an embodiment of the application;

[0031] Figure 13 is a structural schematic diagram of another data processing apparatus according to an embodiment of the application;

[0032] Figure 14 is a structural schematic diagram of another data processing apparatus according to an embodiment of the application;

[0033] Figure 15 is a structural schematic diagram of another data processing apparatus according to an embodiment of the application;

[0034] Figure 16 is a structural block diagram of a computing device according to an embodiment of the application;

[0035] Figure 17 is a structural block diagram of an electronic device according to an embodiment of the application. DETAILED DESCRIPTION

[0036] In the following, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application, so that those skilled in the art can better understand the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should fall within the scope of protection of the present application.

[0037] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0038] The technical solutions provided by the present application are mainly implemented by using large model technology. Here, the large model refers to a deep learning model with large-scale model parameters, which can typically include hundreds of millions, billions, tens of billions, hundreds of billions or even tens of billions of model parameters. The large model can also be referred to as a foundation model. Through large-scale unlabeled corpus pre-training, a pre-trained model with hundreds of millions of parameters is produced. This model can adapt to a wide range of downstream tasks and has good generalization ability. For example, large language models (LLM) and multi-modal pre-training models.

[0039] It should be noted that in actual application, the large model can be fine-tuned by a small amount of samples on the pre-trained model, so that the large model can be applied to different tasks. For example, the large model can be widely applied to the fields of natural language processing (NLP), computer vision, speech processing, etc., and can be applied to computer vision field tasks such as visual question answering (VQA), image captioning (IC), image generation, etc., and can be widely applied to natural language processing field tasks such as text-based sentiment classification, text summary generation, machine translation, etc. Therefore, the main application scenarios of the large model include but are not limited to digital assistants, intelligent robots, search, online education, office software, e-commerce, intelligent design, etc. In the embodiments of the present application, the target agent trained by the agent training method proposed in the present application is taken as an example to explain and describe the data processing in the data processing scenario.

[0040] First, some of the nouns or terms that appear in the description of the embodiments of the present application are applicable to the following explanations:

[0041] Large language model (LLM): that is, the large model, is an advanced machine learning model trained by large-scale text data. The large language model can understand and generate natural language, and has strong text generation, semantic understanding, knowledge reasoning and cross-domain transfer ability.

[0042] Agent: refers to a large language model as the core, which can independently perceive, make decisions, and interact with the environment by calling external tools (such as application programming interface (API), database) to complete specific goals.

[0043] Tool chain: an ordered set of multiple tools and technologies, designed to work together to complete complex tasks. By calling multiple tools (API, functions, models, etc.) in a specific logical order by the agent, problems that cannot be solved independently by a single tool are solved.

[0044] Trace: in the embodiments of the present application, it refers to a record of a series of decisions, actions and results of the agent in the process of executing tasks. The trace describes the entire process of the agent from receiving instructions to completing tasks, including each decision point, called tools, received feedback and final results, and is a key basis for evaluating the behavior logic and efficiency of the agent.

[0045] Large language models are leading a new wave of artificial intelligence revolution. Language models themselves are like a powerful "brain", but they are trapped in a "digital black box" that cannot interact directly with the real world. The introduction of tools is the key to breaking this limitation and releasing the full potential of intelligent agents. By calling external APIs, databases, and various applications, intelligent agents can overcome their inherent three bottlenecks:

[0046] Bottleneck 1: Limitations of knowledge. The knowledge of intelligent agents is "frozen" after training and cannot learn about recent events. By accessing search engines or professional databases, intelligent agents can obtain real-time and accurate external information, effectively avoiding outdated knowledge and content "hallucinations".

[0047] Bottleneck 2: Limitations of ability. Intelligent agents alone cannot process precise structured data, understand spatial relationships, or perform actual actions. By calling map APIs, intelligent agents can plan precise routes. By calling online automation tools, intelligent agents can connect thousands of applications to perform real-world workflows such as sending emails and updating calendars. By accessing professional models, intelligent agents can even extend their ability from text generation to multi-modal creation of images and 3D models.

[0048] Bottleneck 3: Limitations of domain. General models lack "professional ability" in vertical fields. Compared to general web search, directly calling domain-specific APIs (such as querying game performance platforms or obtaining website technology stack platforms) can obtain more in-depth, structured, and machine-readable professional data, allowing for more in-depth analysis and decision-making.

[0049] However, with the explosive growth of the tool ecosystem, the ability of intelligent agents is evolving at an unprecedented rate, but the current evaluation and training methods to measure and improve their tool usage ability are severely lagging behind.

[0050] Current mainstream evaluation benchmarks are mostly designed around simple information retrieval or isolated API calls, and their task construction methods often rely on "multi-hop" queries of a single concept. These outdated paradigms cannot effectively measure the real ability of intelligent agents in solving problems that require multi-tool collaboration, deep logical reasoning, and creative problem-solving. At the same time, the corresponding training data construction strategy is also becoming rigid, making it difficult to generate task trajectories that truly reflect the complexity of the real world.

[0051] Specifically, the current evaluation benchmark mainly aims to "measure and evaluate" the tool usage ability of large language models in different scenarios. Through the design of a comprehensive test data set and an automatic evaluation system, the strengths and weaknesses of existing large language models are diagnosed. The test data set is relatively static, and the Agent scenario relies on expert manual construction, which limits the diversity and scalability of the scenario.

[0052] Therefore, the industry urgently needs a new methodology to get out of the current "let the model use new tools to solve old problems" dilemma. In this context, the present application aims to solve the core contradiction between "capability evolution" and "evaluation lag" in the current agent field and promote the evolution of agents from passive "information carriers" to active "complex problem solvers".

[0053] The agent training and evaluation method in the related art has the following defects.

[0054] Defect 1: Evaluation benchmark lags behind. The current evaluation standard mostly focuses on simple information retrieval or isolated tool invocation, and fails to effectively measure the real ability of the model in solving complex problems that require deep logical reasoning and multi-tool collaboration.

[0055] Defect 2: Task construction method is limited. The Agent scenario relying on expert manual construction and the relatively static data set limit the diversity and scalability of the task scenario, making it difficult to dynamically adapt to the rapid growth of the tool library and the evolution of the agent's ability.

[0056] Defect 3: Training data is limited. The training data construction strategy is fixed, making it difficult to generate truly complex and challenging task trajectories that reflect the complexity and challenges of the real world, limiting the training quality and robustness of the model.

[0057] Defect 4: Insufficient exploration of capability boundaries. There is a lack of systematic methods to explore and define the model's limit ability in tool usage, i.e., the so-called "capability boundary", leading to possible blind spots in the model's performance in complex task execution.

[0058] Defect 5: Lack of adaptability and foresight. It is difficult to automatically adapt to the addition of new tools or changes in the agent's ability, leading to rapid obsolescence of the evaluation and training framework, which cannot continuously promote the development of the agent system.

[0059] No effective solution has been proposed to address the above defects prior to the present application.

[0060] According to an embodiment of the present application, an agent training method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0061] Considering that the model parameter quantity of the large model is large, and the operation resource of the mobile terminal is limited, the above method provided by the embodiments of the present application can be applied to the application scenarios as shown in Figure 1 , but is not limited thereto. In the application scenarios as shown in Figure 1 , the large model is deployed in the server 10, and the server 10 can be connected to one or more client devices 20 through a local area network connection, a wide area network connection, an Internet connection, or other types of data networks. The client device 20 can include, but is not limited to, a smartphone, a tablet computer, a notebook computer, a palm computer, a personal computer, a smart home device, a vehicle-mounted device, etc. The client device 20 can interact with the user through a graphical user interface to realize the calling of the large model, and thus realize the method provided by the embodiments of the present application.

[0062] In the embodiments of the present application, the system composed of the client device and the server can perform the following steps: the client device performs the steps of sending a training instruction to the server, and the server performs the steps of training an intelligent agent based on the training instruction, specifically including the steps of determining an exploration direction based on the existing knowledge of an intelligent agent system, wherein the exploration direction is used to plan and explore a target task to be generated, and the intelligent agent system includes: a plurality of intelligent agents with different capability levels; controlling the plurality of intelligent agents to perform an adversarial walk strategy according to the exploration direction to generate test data and training data; performing capability boundary evaluation on the plurality of intelligent agents by using the test data to obtain an evaluation result; training a to-be-evolved intelligent agent in the plurality of intelligent agents according to the evaluation result and the training data to obtain a target intelligent agent, and returning the target intelligent agent to the client device, and the like. It should be noted that in the case that the running resource of the client device can meet the deployment and running conditions of the large model, the embodiments of the present application can be performed in the client device.

[0063] It should be noted that with the rapid development of high-performance computing units, the above method provided by the embodiments of the present application can also be applied to a model all-in-one machine in other application scenarios. In an optional embodiment, the model all-in-one machine is internally provided with a plurality of models, and a user can select a model to adjust according to needs to obtain a model of the user, so that the high-performance computing unit internally provided in the model all-in-one machine can directly call the adjusted model to execute the above method provided by the embodiments of the present application. In another optional embodiment, the large model all-in-one machine is internally provided with a trained model, so that the high-performance computing unit internally provided in the model all-in-one machine can directly call the model to execute the above method provided by the embodiments of the present application.

[0064] Further, when the user needs to train his own model, he can also upload his own data set through the client, which is sent to the server, so that the server can adjust the pre-trained model with the data set to obtain the user's own model, and then deploy it to the production environment. In order to facilitate the user's adjustment requirements of the model, the server can provide complete adjustment tools, development frameworks and processes, and can support multiple adjustment strategies, so that the adjusted model can better adapt to different field applications and realize high customization.

[0065] In the above running environment, the application provides an agent training method as shown in Figure 2 Figure 2 is a flowchart of an agent training method according to an embodiment of the application. As shown in Figure 2 , the method can include the following steps:

[0066] Step S21, determining an exploration direction based on the existing knowledge of the agent system, wherein the exploration direction is used to plan and explore the target task to be generated, and the agent system includes multiple agents with different capability levels.

[0067] Step S22, controlling the multiple agents to execute the adversarial walk strategy according to the exploration direction, and generating test data and training data.

[0068] Step S23, performing capability boundary evaluation on the multiple agents using the test data, and obtaining an evaluation result.

[0069] Step S24, training the to-be-evolved agent in the multiple agents according to the evaluation result and the training data, and obtaining a target agent.

[0070] In the embodiment of the application, in the agent training process, first, the exploration direction is determined based on the existing knowledge of the agent system.

[0071] Among them, the agent system refers to a comprehensive system composed of one or more agents, which can cooperate with each other to execute various tasks. Each agent in the agent system has the ability to process specific tasks or problems, and can interact with the outside world and solve problems by calling external tools (such as API, database, etc.).

[0072] ​Furthermore, an intelligent agent system can include multiple agents with different capability levels, differing in tool usage, reasoning abilities, and other aspects. For example, these agents at different capability levels can include strong toolchain agents and weak toolchain agents. A strong toolchain agent can be understood as an "expert-level" agent capable of using all tools; for instance, when asked to plan a route including real-time traffic conditions, a strong toolchain agent can directly call a map API. A weak toolchain agent can be understood as a "novice-level" agent restricted to using only basic tools, such as a common search engine. By comparing the performance of different agents, the capability boundaries of a large model can be identified.

[0073] Existing knowledge refers to all the information and experience that an agent in an intelligent agent system acquires through historical training, task execution, and other information channels, including but not limited to domain knowledge, tool usage experience, and task execution history. This existing knowledge constitutes the basis for the agent's understanding of the world.

[0074] Exploration directions refer to the types of research or tasks that the intelligent agent system should focus on in the next step. These directions are derived from the analysis of existing knowledge and are used to plan and explore the target tasks to be generated. The aim of exploration directions is to improve the capabilities of the intelligent agent and to discover new capability boundaries of large models, that is, to discover potential capability deficiencies and room for improvement in large models.

[0075] As can be seen, this application first analyzes existing knowledge about intelligent agent systems to understand which domains or task types the intelligent agent system has limitations or performs well in. Based on the analysis results, it determines the next direction of exploration, that is, the domains or task types that need further development and optimization. Therefore, it can be seen that the direction of exploration is determined by observing the performance differences of intelligent agents with different ability levels when facing the same type of task. This means that the determination of the direction of exploration is not random, but targeted, aiming to guide the intelligent agent system towards more complex and challenging task areas.

[0076] For example, if a strong toolchain agent succeeds in solving a task that requires combining real-time financial data and geolocation information, while a weak toolchain agent fails, then such a task may be set as the next direction for exploration, with the aim of narrowing the capability gap through training.

[0077] Thus, it can be ensured that subsequent task generation and agent training are targeted and focused on improving the performance of the agent in a specific domain or task type, rather than blindly averaging efforts in all directions. Moreover, by analyzing the performance differences between strong and weak agents, the ability boundaries of the current large model can be more accurately located, providing a clear target for training. In addition, determining the exploration direction based on existing knowledge can avoid repeated exploration of mastered knowledge domains, saving valuable time and computing resources, and improving the overall training and data generation efficiency.

[0078] Then, the multiple agents are controlled to execute the adversarial walk strategy according to the exploration direction, to generate test data and training data.

[0079] Among them, the adversarial walk strategy refers to solving the same task by agents with different ability levels. Since agents with different ability levels will adopt different strategies and approaches to solve the task, the adversarial walk strategy can automatically identify and generate complex tasks that can distinguish the ability differences of different agents by comparing the performance of different agents, especially focusing on task cases that strong tool chain agents can successfully solve while weak tool chain agents cannot, as the basis for task generation and evaluation.

[0080] During the execution of the adversarial walk strategy by multiple agents, each step of operation, decision logic and final result (whether successful or failed) of each agent are recorded in detail, which constitutes the test data and training data reflecting the real task execution process. It can be understood that the data generated by the agent executing the adversarial walk strategy is used as both test data and training data, but the use scenarios and stages of test data and training data are different.

[0081] Test data is mainly used to evaluate the current ability boundary of the agent. The test data reflects the performance of the agent in solving a specific task, including successful paths and failure cases. Training data is used to train and optimize the agent so that it can learn and improve its performance on a specific task. Test data and training data contain effective practices and failure cases of the agent in solving complex problems, and are important materials for subsequent ability evaluation and model training.

[0082] It can be seen that after determining the exploration direction, the application will control multiple agents to execute the confrontation wandering strategy according to the exploration direction, and generate test data and training data through the confrontation exploration between the strong tool chain agent and the weak tool chain agent. In this stage, each agent is assigned a task goal, but since the tools and abilities of agents of different ability levels to solve problems are different, the application will track and record each decision and action step of agents of different ability levels, and by comparing the performance of agents of different ability levels in executing the same task, especially the differences in difficulties and solutions, a series of task cases reflecting the complexity of the real world, i.e. test data and training data, can be automatically generated. Illustratively, these task cases contain both successful problem solving instances and failure or partial success samples, which will be used for subsequent testing and training to improve the performance of agents in similar tasks.

[0083] Illustratively, agents of different ability levels can be allowed to try to solve tasks of the same type but gradually increasing difficulty, record the reasoning track and decision of each agent, and compare the solutions and execution processes of agents of different ability levels, thereby forming high-quality test data and training data.

[0084] Thus, the test data and training data generated by the confrontation wandering strategy contain multi-dimensional decision paths and results, which can more comprehensively reflect the behavior patterns and difficulty points of agents in solving complex problems, improve the quality and diversity of training data, i.e. generate high-value data. Moreover, the confrontation wandering strategy can be automatically adjusted with the update and expansion of the agent system tool library, thereby ensuring that the test data and training data always reflect new technical capabilities and task requirements. In addition, the confrontation wandering strategy can accurately identify which tools and strategies are more critical to improving complex task processing capabilities by comparing the execution results of agents of different ability levels, thereby optimizing resource allocation and training processes in a targeted manner.

[0085] Then, the test data is used to evaluate the ability boundary of the multiple agents, and an evaluation result is obtained.

[0086] The ability boundary evaluation refers to determining the ability limit of the agent in processing a specific task or problem, and the limitations of the agent in tool use, logical reasoning, cross-domain knowledge integration, etc. through analysis of the test data, i.e. the ability boundary of the large model can be accurately located through the ability boundary evaluation.

[0087] The evaluation result refers to the quantitative analysis result of the capabilities and limitations of different agents in the agent system in processing complex tasks obtained through the ability boundary evaluation. The evaluation result can indicate which agents perform poorly in which aspects and need further training and optimization.

[0088] As can be seen, by analyzing the aforementioned generated test data, this application can evaluate the actual performance of each agent in handling complex tasks and obtain evaluation results. The evaluation results will reveal the strengths and weaknesses of the agents in task execution, thereby obtaining the capability boundaries of each agent and providing a clear direction for subsequent training and optimization.

[0089] Therefore, capability boundary assessment can clearly identify in which specific tasks or domains an agent performs poorly, accurately pinpointing capability deficiencies and providing precise guidance for subsequent model optimization and training. Furthermore, capability boundary assessment makes the measurement of agent performance more objective, helping to understand the impact of toolchains on agent performance and how to optimize toolchains to improve overall capabilities.

[0090] Finally, based on the evaluation results and training data, the agent to be evolved among multiple agents is trained to obtain the target agent.

[0091] Among them, agents to be evolved are those that, based on the evaluation results, perform poorly in specific aspects or domains and require further training to enhance their capabilities. In other words, these are agents that show weak capabilities or room for improvement in the capability boundary assessment. For example, agents to be evolved are typically weak toolchain agents, or agents that, according to the evaluation results, require further optimization.

[0092] A target agent is an agent that, after training and optimization, has reached or exceeded predetermined capability boundaries, enabling it to perform tasks more efficiently in complex multi-agent systems. The target agent possesses enhanced tool usage capabilities, reasoning skills, and problem-solving efficiency.

[0093] As can be seen, after determining the capability boundaries of the agent, this application utilizes the aforementioned generated training data to conduct targeted training on the agent to be evolved. For example, training the agent to be evolved typically involves reinforcement learning techniques, using both positive and negative training data to guide the agent in learning better decision-making strategies and tool-calling methods. By simulating the successful strategies of a strong toolchain agent, a weak toolchain agent can obtain "guidance" on how to overcome existing obstacles and improve its ability to handle complex tasks, ultimately evolving into the target agent.

[0094] Therefore, by conducting targeted training on the agent to be evolved, the capabilities of the trained agent in specific tasks and domains are significantly improved, enabling it to handle complex problems that were previously impossible or difficult to solve. Furthermore, the agent system can self-iterate and continuously evolve to meet higher-level task challenges, improving efficiency and stability in multi-agent collaborative environments, thereby enhancing the robustness and adaptability of the overall agent system.

[0095] In summary, the application can automatically generate training data reflecting the complexity of the real world through the adversarial wandering strategy, avoiding the limitations brought by relying on static test data sets, making the training process closer to real challenges, and enhancing the robustness and adaptability of the agent. At the same time, the application uses the generated test data to evaluate the ability boundary, which can accurately identify the ability boundary and difficulty point of the agent when dealing with complex tasks, providing clear guidance for subsequent targeted training. In addition, the application trains the agent according to the evaluation results and training data, which can accelerate the evolution process of large models, significantly improve the agent's ability in multi-tool collaboration, deep reasoning, and other aspects, so as to better cope with complex tasks. Therefore, through dynamic data construction, ability boundary evaluation and iterative training, the application breaks the limitations of static test data sets and significantly improves the robustness and flexibility of the agent in dealing with complex tasks.

[0096] The above agent training method provided by the embodiments of the application can be applied in, but is not limited to, application scenarios involving agent training in fields such as e-commerce services, education services, legal services, medical services, conference services, social network services, financial product services, logistics services, and navigation services, for example: e-commerce service related agent training, education service related agent training, legal service related agent training, and the like, which are not limited here.

[0097] By adopting the embodiments of the application, the exploration direction is determined based on the existing knowledge of the agent system, wherein the exploration direction is used to plan and explore the target task to be generated, and the agent system includes multiple agents with different capability levels. Then, the multiple agents are controlled to execute the adversarial wandering strategy according to the exploration direction to generate test data and training data. After that, the multiple agents are evaluated for the ability boundary using the test data to obtain evaluation results. Finally, the to-be-evolved agent in the multiple agents is trained according to the evaluation results and the training data to obtain the target agent. Thus, the purpose of training an agent with high robustness is achieved, thereby realizing the technical effect of breaking the limitations of static test data sets through dynamic data construction, ability boundary evaluation and iterative training, significantly improving the robustness and flexibility of the agent in dealing with complex tasks, and further solving the technical problem of low robustness of the agent in the related art due to the dependence of the agent training and evaluation method on the relatively static test data set.

[0098] In an optional embodiment, in step S21, the exploration direction is determined based on the existing knowledge of the agent system, including the following method steps:

[0099] In step S211, the known task type, tool coverage range, model existing capability, and / or current service demand are determined based on the existing knowledge of the agent system.

[0100] Step S212, determining the exploration direction by using the known task type, tool coverage, model existing capability and / or current service demand.

[0101] In the embodiments of the present application, when determining the exploration direction based on the existing knowledge of the agent system, the known task type, tool coverage, model existing capability and / or current service demand can be determined based on the existing knowledge of the agent system.

[0102] The existing knowledge is a set of information and capabilities that the agent system has mastered before starting a new data construction and training process, and mainly includes the following four dimensions: known task type, tool coverage, model existing capability and / or current service demand.

[0103] The known task type refers to the types of tasks that the agent has encountered or successfully executed before, such as information retrieval, route planning, data analysis, etc., thereby helping the agent system to understand which fields it has certain processing capabilities.

[0104] The tool coverage refers to the types and functions of tools available in the current agent system, such as various API interfaces, database access permissions, external program calling capabilities, etc., and the tool coverage determines the means and scope of problem solving by the agents in the agent system.

[0105] The model existing capability refers to the current performance level of the agent model, such as including language understanding, logical reasoning, knowledge integration, etc.

[0106] The current service demand refers to the user requests or task scenarios recently received by the agent system, and the current service demand will affect the exploration direction, prompting the agent system to preferentially improve the agent capabilities in certain specific fields.

[0107] As can be seen, the present application will summarize and summarize the existing knowledge of the agent system, and clarify the known task type, tool coverage, model current capability level and / or task demand faced. Exemplarily, the ability status of the agent and the actual demand of the service object can be comprehensively understood through analysis of historical data, evaluation of model performance, etc.

[0108] Then, the exploration direction is determined by using the known task type, tool coverage, model existing capability and / or current service demand.

[0109] As can be seen, the present application will plan a development strategy that can maximize the performance improvement of the agent and meet the user demand based on the known task type, tool coverage, model existing capability and / or current service demand determined as described above, thereby determining the future exploration direction.

[0110] Thus, through the above steps, the direction of the agent capability improvement can be determined, blind training can be avoided, and the comprehensive capability of the agent can be improved more efficiently. Moreover, the exploration direction determination fully considers the task requirements and service scenarios, so that the evolution of the agent is closer to the actual application scenarios, and the agent can better solve user problems and meet market needs.

[0111] In an optional embodiment, the known task type is used to determine the task type currently known by the agent system, the tool coverage range is used to determine the capability coverage range of the multifunctional tool library, the model existing capability is used to determine the model capability currently possessed by the agent system, and the current service demand is used to determine the demand of the agent system for the current service scenario.

[0112] In the embodiments of the present application, the known task type is used to determine the task type currently known by the agent system, for example, including searching information, planning a trip, data analysis, emotion understanding, etc.

[0113] The tool coverage range is used to determine the capability coverage range of the multifunctional tool library, and the multifunctional tool library is a tool library pre-equipped for the agent system for calling and using by the agent system to solve specific or various complex tasks. The multifunctional tool library includes various APIs, data sources, algorithm models, etc.

[0114] The model existing capability is used to determine the model capability currently possessed by the agent system, for example, including language understanding, knowledge integration, logical reasoning, etc.

[0115] The current service demand is used to determine the demand of the agent system for the current service scenario, for example, including consultation of a specific industry, emergency response, new function request, etc.

[0116] In an optional embodiment, in step S22, the plurality of agents are controlled to execute the adversarial walk strategy according to the exploration direction to generate the test data and the training data, including the following method steps:

[0117] Step S221, the plurality of agents are controlled to execute the adversarial walk strategy according to the exploration direction to form a plurality of task trajectories.

[0118] Step S222, the test data and the training data are generated based on the plurality of task trajectories.

[0119] In the embodiments of the present application, when the plurality of agents are controlled to execute the adversarial walk strategy according to the exploration direction to generate the test data and the training data, the plurality of agents can be first controlled to execute the adversarial walk strategy according to the exploration direction to form a plurality of task trajectories.

[0120] The task trajectory is a behavior record of the agent in the process of performing the task, for example, including the decision of each step, tool calling, and final result, and the task trajectory can provide the behavior details of the agent when facing a complex task.

[0121] It can be seen that the application will control multiple agents to perform the adversarial wandering strategy according to the determined exploration direction. Since each agent has a different tool chain (i.e., a combination of tools and calling permissions), by letting each agent independently solve the same or a series of similar tasks, it can be able to produce differences in the execution process, so that some agents can find better solutions through higher-level tool chains, while some agents appear to be less efficient or fail due to tool or capability limitations. At the same time, during the execution of the adversarial wandering strategy, the decision path, tool usage, and final result of each agent are recorded, thereby forming multiple task trajectories.

[0122] Then, test data and training data are generated based on the multiple task trajectories. It can be seen that the application will generate test data and training data according to the multiple task trajectories formed, that is, test data and training data will be constructed according to the data of the multiple task trajectories. Exemplarily, test data and training data can be further generated by analyzing the action sequence, decision logic, and tool calling information in the task trajectory.

[0123] Thus, the task trajectory generated by the adversarial wandering strategy can analyze the behavior and ability of the agent in a complex task in detail, and provide a basis for subsequent evaluation and training strategies. Moreover, the upper limit and bottleneck of the agent's ability are automatically identified during the execution of the task, so that the training process can be designed accordingly to improve the comprehensive ability of the agent.

[0124] In an optional embodiment, in step S221, the multiple agents are controlled to perform the adversarial wandering strategy according to the exploration direction, and multiple task trajectories are formed, including the following method steps:

[0125] Step S2211, generating a task target based on the exploration direction.

[0126] Step S2212, controlling the multiple agents to perform the adversarial wandering strategy according to the task target, and forming multiple task trajectories.

[0127] In the embodiment of the application, when the multiple agents are controlled to perform the adversarial wandering strategy according to the exploration direction, and multiple task trajectories are formed, the task target can be generated based on the exploration direction first.

[0128] The task target can be understood as a specific task set under a given exploration direction, which aims to test the ability of the agent to solve a specific type of problem, or to guide the agent to learn new skills and strategies.

[0129] It can be seen that the present application will design a series of task targets according to the determined exploration direction, the design of the task targets needs to ensure the complexity and the diversity of performance cover multiple dimensions of the agent's ability, and the task targets should be a challenge to the existing ability of the agent, which can test the performance of the agent in a specific field or tool use, and also should consider the future service demand that the agent may face.

[0130] Then, the multiple agents are controlled to execute the adversarial wandering strategy according to the task target, and multiple task trajectories are formed.

[0131] It can be seen that the present application will control multiple agents to execute the adversarial wandering strategy according to the set task target, the agent will make decisions according to the task target, call different tools, and try to solve the task. In the execution process, the decision-making process, tool usage, intermediate calculation results and final output of each agent are recorded. Whether the task is successful or not, one or more task trajectories will be formed, which contains all the details of the agent solving the task, and provides rich information for subsequent analysis and learning.

[0132] Therefore, by allowing agents with different ability levels to try to solve the same task, the impact of ability difference on task solving can be intuitively seen, so that the existing ability of the agent can be more accurately evaluated. And the adversarial wandering strategy can actively explore the ability boundary of the agent, by comparing the performance of the agent under different tool chains, it can identify which tools and abilities are the key to the success of the agent, and indicate the direction for subsequent targeted improvement.

[0133] In an optional embodiment, in step S2212, the multiple agents are controlled to execute the adversarial wandering strategy according to the task target, and multiple task trajectories are formed, including the following method steps:

[0134] Step S22121, according to the task target, respectively control the multiple agents to execute the adversarial wandering strategy on the corresponding tool chain in the multiple tool chains, and form multiple task trajectories, wherein the multiple tool chains include: multiple first tool chains and multiple second tool chains, the multiple first tool chains are authorized to use the first category of tools in the multifunctional tool library, and the multiple second tool chains are authorized to use the second category of tools in the multifunctional tool library, the first category of tools has professional functional attributes, and the second category of tools has ordinary functional attributes.

[0135] In the present application, when the multiple agents are controlled to execute the adversarial wandering strategy according to the task target, and multiple task trajectories are formed, the multiple agents can be controlled to execute the adversarial wandering strategy on the corresponding tool chain in the multiple tool chains according to the task target, and multiple task trajectories are formed.

[0136] The multi-functional tool library can be understood as a repository of tools that includes both first category tools with specialized functional attributes and second category tools with ordinary functional attributes, aiming to provide the diversity of resources required for agents to perform complex tasks.

[0137] The plurality of tool chains includes a plurality of first tool chains and a plurality of second tool chains. The plurality of first tool chains can be understood as a plurality of strong tool chains, which are authorized to use the first category tools in the multi-functional tool library, and the first category tools have specialized functional attributes. That is, the plurality of first tool chains can be understood as consisting of a group of tools with specialized functional attributes, which often provide deeper data access, more accurate decision support, or more complex operational functions, such as financial data APIs, professional database queries, etc.

[0138] The plurality of second tool chains can be understood as a plurality of weak tool chains, which are authorized to use the second category tools in the multi-functional tool library, and the second category tools have ordinary functional attributes. That is, the plurality of second tool chains can be understood as consisting of a group of tools with ordinary functional attributes, which usually provide basic data retrieval and simple operational support, such as general search engines, basic file management tools, etc.

[0139] It can be seen that, for a specific task target generated, multiple agents will be deployed to execute an adversarial wandering strategy. Typically, the multiple agents will be divided into two groups, such as a first group of agents and a second group of agents, which use the first tool chain and the second tool chain respectively. Specifically, the first group of agents is authorized to use the first category tools in the multi-functional tool library, and attempts to complete the task target by calling the first category tools, and the behavior path of the first group of agents constitutes part of the task trajectory. The second group of agents is only allowed to use the second category tools in the multi-functional tool library, and also attempts to complete the task target by calling the second category tools, and the behavior path of the second group of agents constitutes another group of task trajectories. In the entire execution of the adversarial wandering strategy, the multiple agents execute their respective authorized tool chains, form their respective adversarial wanderings, and record each decision of the agent, each tool call and its result until the task is completed or a certain predetermined cutoff point is reached, thereby forming a plurality of detailed task trajectories. These task trajectories contain the complete process from the beginning to the end of the task, and are the key to evaluating the capabilities of the agents and generating training data.

[0140] Thus, by comparing between different groups of agents (i.e., the first group of agents and the second group of agents), the performance difference of the agents supported by different toolchains (i.e., the first toolchain and the second toolchain) can be intuitively observed, so as to evaluate the actual ability of the agents in handling complex tasks and the influence of tool selection on the performance of the agents. Moreover, the task trajectories provide complex scenarios and solutions encountered in the real world for model training, and especially when the first group of agents succeeds while the second group of agents fails, the differences contained in these task trajectories provide valuable cases for the model to learn how to use tools more effectively, and generate high-quality training data.

[0141] In an optional embodiment, in step S23, the capability boundary evaluation of the plurality of agents is performed by using the test data, and an evaluation result is obtained, including the following method steps:

[0142] In step S231, the plurality of agents are controlled to execute the test data, and an execution result is obtained, wherein the execution result is used to determine whether the plurality of agents successfully execute the toolchain test process.

[0143] In step S232, the capability boundary of the plurality of agents is evaluated according to the execution result, and an evaluation result is obtained.

[0144] In the embodiments of the present application, when the capability boundary evaluation of the plurality of agents is performed by using the test data, and an evaluation result is obtained, the plurality of agents can be controlled to execute the test data first, and an execution result is obtained, wherein the execution result is used to determine whether the plurality of agents successfully execute the toolchain test process.

[0145] The execution result refers to the output of the agent after completing the task in the test data, including a success result or a failure result, and the quality and efficiency of task completion in the case of success, etc., and the execution result is used to determine whether the plurality of agents successfully execute the toolchain test process.

[0146] As can be seen, the present application will control the plurality of agents to execute the generated test data, and the test data contains tasks designed based on various real challenges that may be encountered in a complex multi-agent system. Each agent will try to solve the task according to the toolchain and capability level it has, so as to record the detailed behavior and result of the agent when handling each task, and obtain an execution result.

[0147] For example, the recorded content includes whether the agent can understand and analyze the task requirements, the solution strategy and the toolchain selected by the agent, the challenges and errors encountered in the execution process, whether the task is finally successfully completed, and the speed and quality of completing the task, etc.

[0148] Thus, the execution result provides a direct basis for the capability evaluation of the agent, and the execution result not only reflects the performance of the agent on a specific task, but also reflects the strategy and logic of the agent in solving the problem, which is the key information for evaluating the capability boundary of the agent.

[0149] Then, the capability boundaries of the plurality of agents are evaluated according to the execution result to obtain an evaluation result. It can be understood that after collecting the execution results of all the agents, the capability boundary evaluation stage is entered, and the capability boundaries of the plurality of agents are evaluated according to the execution result. Exemplarily, the evaluation process includes, for example, determining the success rate of each agent in completing the task, identifying the agent and tool chain combination with a higher success rate and the agent and tool chain combination with a lower success rate. The evaluation process also includes, for example, analyzing the divergence point in the execution process, especially the point at which the strong tool chain agent can succeed while the weak tool chain agent fails, and the divergence point can reveal the capability boundary of the model. The evaluation process also includes, for example, quantifying the degree of dependence of the agent on a specific type of task or tool chain, and understanding which tool chain or capability is indispensable for completing a high-difficulty task. The evaluation process also includes, for example, evaluating the robustness of the agent when dealing with unknown or edge tasks, i.e., the reaction and recovery ability of the agent when facing challenges beyond the conventional range.

[0150] The evaluation result provides a quantitative indicator of the capability of the agent, and also reflects the pattern of capability distribution in the agent system, which can effectively distinguish high-capability models from low-capability models, and provide guidance for further optimizing the design of the agent and the configuration of the tool chain, and formulating more effective training strategies.

[0151] Thus, through the execution of the test data and the analysis of the results, the capability of the agent in dealing with complex tasks can be accurately evaluated, the adaptability of the agent to different tool chains and task types can be distinguished, and the real capability boundary of the agent can be revealed.

[0152] In an optional embodiment, in step S24, the to-be-evolved agent in the plurality of agents is trained according to the evaluation result and the training data to obtain a target agent, including the following method steps:

[0153] Step S241, adjusting the exploration direction according to the evaluation result to obtain an adjusted exploration direction.

[0154] Step S242, labeling the process reward for the training data according to the evaluation result to obtain labeled data.

[0155] Step S243, based on the adjusted exploration direction and the labeled data, performing process-guided reinforcement learning training on the to-be-evolved agent in the plurality of agents to obtain a target agent.

[0156] In the embodiments of the present application, when the to-be-evolved agent in the plurality of agents is trained according to the evaluation result and the training data to obtain the target agent, the exploration direction can be adjusted according to the evaluation result to obtain an adjusted exploration direction, and the process reward is labeled for the training data according to the evaluation result to obtain labeled data.

[0157] In reinforcement learning, the process reward can be understood as positive feedback given to the key decisions of the agent in the task execution process in addition to the achievement of the final goal, to encourage the agent to take more effective and reasonable action paths.

[0158] As can be seen, the exploration direction is adjusted according to the aforementioned evaluation result to ensure that the training and evolution of the agent are more focused on key areas, and an adjusted exploration direction is obtained, that is, the exploration direction is dynamically adjusted according to subsequent discoveries. Illustratively, the evaluation result can identify task areas with challenges and distinctions. For example, if the evaluation result indicates that the agent performs poorly in processing a certain type of task, the adjusted exploration direction will focus more on generating trajectories of such tasks to fill the gaps in the model's capabilities.

[0159] At the same time, the process reward is labeled for the training data according to the evaluation result to obtain labeled data. Illustratively, the training data can be analyzed in depth, especially focusing on decision points in the task solving process of the agent, and process reward labels are added to these decision points to generate labeled data. The introduction of process rewards can not only focus on the final result of the task in the training process of the agent, but also encourage the agent to take more effective and reasonable decision paths in the task solving process, thereby improving its overall ability to handle complex tasks.

[0160] Then, the to-be-evolved agent in the plurality of agents is trained by process-guided reinforcement learning based on the adjusted exploration direction and the labeled data to obtain the target agent.

[0161] As can be seen, the to-be-evolved agent in the plurality of agents is trained by process-guided reinforcement learning based on the adjusted exploration direction and the labeled data with process reward labels, that is, the key decision points are supervised and guided during execution, so that the agent not only learns how to use the tool chain to solve the task, but also learns the optimal decision in the decision-making process. By continuously receiving process rewards, the agent can gradually accumulate experience, optimize its decision logic, and improve its intelligence level in complex environments. After sufficient training, the agent reaches the expected ability standard and obtains the target agent, so as to more efficiently and accurately execute complex tasks.

[0162] Thus, by adjusting the exploration direction, the training process focuses more on the weaknesses and limitations of the agent, ensuring the pertinence and effectiveness of the ability improvement. Moreover, the introduction of process rewards enables the reinforcement learning training strategy to dynamically adjust according to the actual situation of task execution, encouraging the agent to make better decisions and accelerating its learning process, thereby enhancing the adaptability and stability of the agent in the face of uncertainty and complex environments.

[0163] In an optional embodiment, in step S222, the training data is generated based on the plurality of task trajectories, including the following method steps:

[0164] In step S2221, positive training samples are generated based on the successful execution results of the plurality of task trajectories on the plurality of first tool chains, and negative training samples are generated based on the failure execution results of the plurality of task trajectories on the plurality of second tool chains.

[0165] In step S2222, the training data is determined according to the positive training samples and the negative training samples.

[0166] In the embodiments of the present application, when generating training data based on a plurality of task trajectories, positive training samples can be generated based on the successful execution results of the plurality of task trajectories on the plurality of first tool chains, and negative training samples can be generated based on the failure execution results of the plurality of task trajectories on the plurality of second tool chains.

[0167] The positive training samples are examples extracted from the trajectories of successfully executing tasks on the first tool chains, and the positive training samples are used to represent the patterns of the agent correctly using the tool chains to complete the tasks, which helps the model learn the correct tool chain coordination and complex task solving strategy. The negative training samples are examples extracted from the trajectories of failing to successfully execute tasks on the second tool chains, and the negative training samples are used to represent common errors and challenges of the agent when facing complex tasks, which helps the model learn to avoid errors and thus improve the robustness of the model. For example, the negative training samples include failure execution results that fail to successfully execute tasks, i.e., completely incorrect execution of tasks, and failure execution results that generate ambiguous, unstructured, and poor quality answers, i.e., a special kind of failure execution result, i.e., fail to meet the expected quality standards and structured level, thus failing to meet the specific requirements of the task or demonstrating the advanced skills of the agent.

[0168] As can be seen, the present application generates positive and negative training samples based on the results of the previous agent performing multiple task trajectories. Further, for the successful execution results of the agent on the first tool chain (such as the strong tool chain agent), the tool calls, decision paths and final achievements thereof are extracted as positive training samples, providing the model with examples of learning correct tool usage and efficient problem solving strategies. For the failure execution results of the agent on the second tool chain (such as the weak tool chain agent), negative training samples are generated based on the failure execution results, providing the model with training materials for learning how to avoid the same errors and how to find alternative solutions under limited resources.

[0169] Then, the training data is determined according to the positive training samples and the negative training samples. As can be seen, the present application determines the final training data by integrating the positive training samples and the negative training samples. Exemplarily, when determining the training data, not only the balance of the training data is ensured, i.e. the number of positive training samples and negative training samples is appropriate, but also the format and structure suitable for model learning are designed according to the information in the training samples, so as to improve the effect of model training.

[0170] Thus, the positive training samples provide the model with efficient problem solving demonstrations, which helps the agent learn how to reasonably call and coordinate tool chains in complex environments, and improves the success rate and efficiency of task execution. The negative training samples reflect the common failure modes and limitations of tool usage when the agent processes complex tasks, which helps the model learn how to avoid these errors and improve robustness when facing challenges. By integrating positive training samples and negative training samples, the present application can construct more balanced and comprehensive training data, covering the full spectrum from successful experience to failure lessons, thereby helping the agent achieve comprehensive capability improvement.

[0171] In an optional embodiment, the agent training method further includes the following method steps:

[0172] The functional attributes of the existing tools in the multifunctional tool library are expanded, and / or the tool categories not covered in the multifunctional tool library are expanded.

[0173] In the embodiments of the present application, the functional attributes of the existing tools in the multifunctional tool library can also be expanded. By expanding the functional attributes of the existing tools, the performance, functions or application scope of the existing tools can be improved.

[0174] Exemplarily, the expansion of the functional attributes of the existing tools can include adding parameter options of the existing tools, improving data filtering and data processing capabilities, expanding data sources, etc. Specifically, the decision support capability of the tool can be improved by introducing more complex algorithms or models, such as deep learning technology, etc. The timeliness and accuracy of information can also be improved by adding connections with external data sources, using real-time updated database or API query results as inputs of the tool. In addition, the interaction interface of the tool can also be improved to make it easier for the agent to understand and call.

[0175] And / or, the expansion of the tool categories not covered in the multifunctional tool library, by introducing new tool categories in the multifunctional tool library, to fill the service areas or functional needs that the existing tools cannot cover. For example, if the current multifunctional tool library lacks tools for image processing or natural language understanding, corresponding APIs or software interfaces can be added.

[0176] Exemplarily, when expanding the multifunctional tool library, image recognition and analysis tools can be expanded to process visual information. Speech recognition and synthesis tools can also be expanded to enhance the voice interaction capability of the agent. Professional knowledge base interfaces, such as medical and legal databases, can also be expanded to provide more in-depth answers to professional problems. Automated process tools can also be expanded to integrate multiple services or operations, thereby improving the efficiency of the agent in handling consecutive tasks. The specific expansion content is determined according to the actual situation and is not limited here.

[0177] Thus, by expanding the functional attributes of the existing tools, the agent can call more powerful and flexible tools, thereby processing more extensive and complex tasks in the agent system and enhancing its competitiveness. By expanding the multifunctional tool library, the expanded multifunctional tool library can support more diverse task requirements, and the application range of the agent is expanded, which can better serve the actual scene.

[0178] Figure 3 is a flowchart of a data construction according to an embodiment of the present application, as Figure 3As shown, first, a complex agent system is built, the agent system is initialized, and a rich and diverse tool library is constructed for the agent system, which contains both general basic tools and a series of powerful professional APIs. Next, instead of blindly starting to generate tasks, the agent system will plan a general exploration direction based on existing knowledge (such as known task types, tool coverage, existing capabilities of models, etc.), such as focusing on generating complex tasks that require multi-step reasoning combining real-time financial data and geographic location information. The process of planning the exploration direction itself is also an iterative planning and exploration process, and the exploration direction will be dynamically adjusted with subsequent discoveries.

[0179] Then, an adversarial walk strategy is executed, that is, the agent system will generate a high-order task target based on the exploration direction, and then instantiate multiple "players" with different capabilities to solve this target. Further, the "players" include multiple strong tool chain agents, that is, a series of "expert-level" agents, which are authorized to use efficient and professional tools in the tool library, that is, expert tool chains, in addition to general tool chains and weak tool chains. For example, when asked to plan a route that includes real-time traffic, the strong tool chain agent can directly call the map API. The "players" also include multiple weak tool chain agents, that is, a series of "novice-level" agents, which are limited to using some basic and general tools, that is, general tool chains and weak tool chains, such as only using ordinary search engines. After instantiating multiple "players" with different capabilities, the multiple "players" will start an adversarial walk, that is, they will try to complete the same task with the tools they are authorized to use.

[0180] After that, the divergence and the generation of difficult tasks are found, that is, the behavior and results of the above two groups of agents are closely monitored during the confrontation process, and key signals from the divergence are monitored and identified. The divergence means that the strong tool chain agent has obtained a more difficult answer. Here, the divergence refers to the fact that the strong tool chain agent successfully obtains an accurate, structured, and high-quality answer (for example, through the "expert tool chain" path) using its professional tools, while the weak tool chain agent is limited by tools and either fails completely (i.e. Figure 3 the "X" on the middle path), or can only give a vague, unstructured, and poor-quality answer. Therefore, when such a divergence is identified, it is determined that a high-quality, valuable difficult task has been successfully "captured", because the task can effectively distinguish high-capability models from low-capability models.

[0181] Finally, the data track is formed and iteratively optimized, forming a closed loop. That is, the characteristics of the newly discovered difficult tasks are fed back to the step of planning the exploration direction. For example, if it is found that "combining API query results with geographic information" is a task type that is easy to produce divergences, then in the next round of iteration, more emphasis will be placed on generating and exploring variants of this task type, so as to continuously explore the capability boundary of the model and continuously output high-quality and high-complexity training data and test data.

[0182] As can be seen from the above, the present application can dynamically and automatically generate tasks, and the automation of the present application does not depend on a static data set, but generates and executes complex task tracks through the antagonistic strategy of "strong and weak tool chains". The method of the present application can actively explore and discover the "capability boundary" of the model, and continuously generate new and high-quality evaluation data, rather than being limited to a fixed question bank.

[0183] The present application provides a dynamically adaptable and expandable framework. When the tool library is continuously expanded, new tools can be seamlessly integrated into the antagonistic process according to the automatic generation strategy, so as to generate more challenging task chains that match new capabilities, ensuring that the difficulty of evaluation and training can grow in synchronization with the model capability, maintaining the forefront.

[0184] The method of the present application not only can evaluate the model, but also can train the model. Through the antagonistic mechanism, the method of the present application can continuously generate challenging difficult samples. These difficult samples (i.e. task tracks that can be solved by strong tool chain agents but failed by weak tool chain agents) help to train more robust and powerful models, aiming to promote the evolution of model capabilities, rather than just measuring the current level.

[0185] Specifically, the present application discards the traditional method of manually designing or generating tasks based on templates, and sets multiple strong tool chain agents (i.e. expert-level agents that can call all professional APIs) and weak tool chain agents (i.e. novice-level agents that can only use basic tools), so that the two groups of agents perform antagonistic problem-solving exploration for the same vague goal. When the strong tool chain agent succeeds and the weak tool chain agent fails, the system automatically excavates a high-quality difficult task. The mechanism of the present application can accurately locate scenarios that can reflect the advanced capabilities of the model (such as deep reasoning and multi-tool collaboration), thereby systematically discovering the capability boundary of the model.

[0186] The method of the present application is not an isolated tool call, but focuses on the complete tool chain execution process, and the finally generated data is a complete trajectory containing the target, high-level planning, multi-step reasoning, input and output of each tool call, and the final result. Such trajectory data not only contains "how to do it right", but also records "why it is wrong" through the failure of the weak tool chain, providing a large amount of learning samples containing positive and negative examples for model training, which is much more than the simple "question-answer" pair in traditional benchmarks.

[0187] And for the problem of sparse final reward and unstable training in complex tasks, the present application proposes an improved reinforcement learning method, which not only gives rewards when the task is successful, but more importantly, supervises and guides at key decision points in the execution process. Through the reward of the execution process, the learning efficiency and stability of the model in long-chain and complex tasks can be significantly improved, helping the model to learn the correct "thinking process" rather than just "guessing" the final answer, solving the core pain point of complex agent training.

[0188] It is easy to understand that the beneficial effects of the agent training method provided by the present application include the following points.

[0189] Beneficial effect (1), the present application generates task trajectories automatically, covering complex problems that require deep logical reasoning and multi-tool collaboration. These dynamic task scenarios driven by tool chains can more realistically reflect the model's ability to solve complex problems, thereby providing a more advanced evaluation framework that effectively overcomes the lagging defects of traditional evaluation benchmarks.

[0190] Beneficial effect (2), the present application automatically generates challenging and diverse task scenarios through the "strong-weak tool chain confrontation" strategy. This method overcomes the limitations of manual construction by experts and can dynamically generate new tasks as the tool library updates and the agent's ability evolves, increasing the diversity and scalability of task scenarios, thereby overcoming the limitations of the traditional task construction method.

[0191] Beneficial effect (3), the data construction process of the present application dynamically generates high-quality training data containing positive and negative examples based on the real behavior of agents solving complex tasks. These data not only contain successful cases, but also record failure patterns, thereby providing the model with more comprehensive and realistic learning materials, improving the complexity and diversity of training data, and enhancing the effectiveness and robustness of model training, thereby overcoming the limitations of traditional training data.

[0192] Beneficial effect (4): Through a continuous adversarial walk strategy, the agent actively explores and identifies its own capability boundaries during the execution of tasks. When there is a divergence between strong and weak toolchains, it can automatically capture and analyze these divergence points, generate targeted difficult samples, help the agent break through the limitations of its existing capabilities, and gradually expand its capability boundaries, thereby overcoming the shortcomings of insufficient capability boundary exploration in traditional solutions.

[0193] Beneficial Effects (5): The data construction and training process of this application has good adaptability and foresight. As new tools are continuously added to the tool library, the task construction strategy can be automatically adjusted to generate complex tasks that match the new tools. At the same time, through continuous evaluation and training, the agent can continuously evolve to adapt to the updates of the tool library and the development of its own capabilities, ensuring the continuous innovation of the evaluation and training framework and the continuous progress of the agent system, thereby overcoming the shortcomings of lack of adaptability and foresight in traditional solutions.

[0194] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0195] Furthermore, it should be noted that, for the sake of simplicity, the aforementioned method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Secondly, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0196] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, they can also be implemented by hardware. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0197] According to embodiments of this application, the following are also provided: Figure 4The data processing method shown. Figure 4 is a flowchart of a data processing method according to an embodiment of the present application. As shown in the figure, Figure 4 The method can include the following steps:

[0198] Step S41, obtaining problem data.

[0199] Step S42, using a target agent to perform answer generation processing on the problem data to obtain a target reply. The target agent is generated according to the agent training method of any one of the above.

[0200] In the embodiment of the present application, after the target agent is trained, it can be applied in the data processing scene. First, the problem data is obtained, wherein the problem data can be a specific problem or request submitted by the user to the agent system for solution. For example, the problem data can be about information query, task execution, decision support, etc.

[0201] Then, using a target agent to perform answer generation processing on the problem data to obtain a target reply. The target agent is generated by the above-mentioned agent training method, and has high ability to solve complex problems and robustness.

[0202] Through the processing of the problem data by the target agent, the target agent can deeply understand the problem data, identify the key elements and potential needs of the problem data, and then obtain more accurate target replies for the user, greatly improving the user experience and service satisfaction.

[0203] For specific description, please refer to the description of the above-mentioned embodiments.

[0204] The above-mentioned data processing method provided by the embodiment of the present application can be applied in the application scene involving agent training in the field of e-commerce service, education service, legal service, medical service, conference service, social network service, financial product service, logistics service and navigation service, etc., such as agent training related to e-commerce service, agent training related to education service, agent training related to legal service, etc., which is not limited here.

[0205] By using the embodiment of the present application, the problem data is obtained, and the target agent generated according to the agent training method of any one of the above is used to perform answer generation processing on the problem data to obtain a target reply, thereby achieving the purpose of providing more convenient and accurate solutions for users, thereby realizing the technical effect of breaking through the limitation of static test data set through dynamic data construction, capability boundary evaluation and iterative training, significantly improving the robustness and flexibility of the agent in processing complex tasks, and further solving the technical problem of low robustness of the agent in the related art due to the dependence of the agent training and evaluation method on the relatively static test data set.

[0206] According to the embodiments of the present application, the data processing method shown in the above is also provided. Figure 5 Figure 5 is a flow chart of another data processing method according to the embodiments of the present application. As shown in the above, the method can include the following steps: Figure 5

[0207] Step S51, obtaining intelligent customer service consultation question data.

[0208] Step S52, using an intelligent customer service agent to perform answer generation processing on the intelligent customer service consultation question data to obtain an intelligent customer service consultation reply. The intelligent customer service agent is generated according to the agent training method of any one of the above.

[0209] In the embodiments of the present application, after the target agent is trained, it can be applied in the customer service consultation scene. First, intelligent customer service consultation question data is obtained, wherein the intelligent customer service consultation question data can be a question or a problem to be solved raised by a user through an intelligent customer service system. For example, the intelligent customer service consultation question data can involve product information, order status, troubleshooting, operation guide and other types.

[0210] Then, using an intelligent customer service agent to perform answer generation processing on the intelligent customer service consultation question data to obtain an intelligent customer service consultation reply. The intelligent customer service agent is generated by the above-mentioned agent training method, and is an agent specially used for processing customer consultation service. The intelligent customer service agent can call appropriate service tools to generate accurate replies to solve user problems.

[0211] For specific description, refer to the foregoing description of the embodiments.

[0212] The above-mentioned data processing method provided by the embodiments of the present application can be applied in, but not limited to, the application scenarios involving agent training in the fields of e-commerce service, education service, legal service, medical service, conference service, social network service, financial product service, logistics service and navigation service, such as: e-commerce service related agent training, education service related agent training, legal service related agent training, etc., which are not limited herein.

[0213] ​​By adopting the embodiment of the application, the intelligent customer service consultation question data is acquired, the intelligent customer service agent generated by the intelligent agent training method in any one of the above is used to perform answer generation processing on the intelligent customer service consultation question data, and the intelligent customer service consultation reply is obtained, thereby achieving the purpose of providing a more convenient and accurate solution for the user, so as to realize the technical effects of breaking the limitation of the static test data set through dynamic data construction, capability boundary evaluation and iterative training, significantly improving the robustness and flexibility of the intelligent agent in processing complex tasks, and further solving the technical problem of low robustness of the intelligent agent in the related art due to the dependence of the intelligent agent training and evaluation method on the relatively static test data set.

[0214] According to the embodiments of the application, a data processing method is also provided, as shown in the data processing method. Figure 6 Figure 6 is a flowchart of another data processing method according to the embodiments of the application. As shown in the data processing method. Figure 6

[0215] In step S61, the data processing request is acquired through the first application programming interface, wherein the request data carried in the data processing request includes the question data.

[0216] In step S62, the data processing response is returned through the second application programming interface, wherein the response data carried in the data processing response includes the target reply, the target reply is obtained by performing answer generation processing on the question data by the target intelligent agent, and the target intelligent agent is generated according to the intelligent agent training method in any one of the above.

[0217] In the embodiments of the application, the first application programming interface (Application Programming Interface, API) and the second application programming interface can be the same application programming interface or different application programming interfaces. In an optional embodiment, the interface parameters in the above first application programming interface and second application programming interface can include but are not limited to: interface global identifier, interface signature key, interface timestamp, interface request identifier, system call credential identifier, etc. The above first application programming interface can use GET or POST as the interface request method to acquire the file processing request. The above second application programming interface can use the JSON format to feed back the file processing response.

[0218] The data processing request can be a request sent by an external system or application to the intelligent agent system through the first application programming interface, requesting the intelligent agent system to process the question data carried in the data processing request.

[0219] ​​The data processing response is a processing result returned by the agent system to the requester through the second application programming interface, and the target reply is carried in the data processing response. The target reply is an answer or solution generated by the target agent for the problem data, and is intended to meet the needs of the external requester.

[0220] The target agent is an agent generated according to the agent training method described above, and has the ability to process complex problems and generate high-quality replies.

[0221] For specific descriptions, refer to the descriptions of the embodiments described above.

[0222] The data processing method provided in the embodiments of the present application can be applied in, but is not limited to, application scenarios involving agent training in the fields of e-commerce services, education services, legal services, medical services, conference services, social network services, financial product services, logistics services, and navigation services, such as agent training related to e-commerce services, agent training related to education services, agent training related to legal services, and the like, without limitation.

[0223] By adopting the embodiments of the present application, the data processing request is obtained through the first application programming interface, wherein the request data carried in the data processing request includes problem data, and the data processing response is returned through the second application programming interface, wherein the response data carried in the data processing response includes a target reply. The target reply is obtained by performing answer generation processing on the problem data by the target agent. The target agent is generated according to the agent training method of any one of the above embodiments. Thus, the purpose of providing a more convenient and accurate solution for users is achieved, thereby realizing dynamic data construction, capability boundary evaluation, and iterative training, breaking the limitation of the static test data set, and significantly improving the robustness and flexibility of the agent in processing complex tasks, thereby solving the technical problem of low robustness of the agent in the related art due to the dependence of the agent training and evaluation method on the relatively static test data set.

[0224] According to the embodiments of the present application, a data processing method is also provided as shown in Figure 7 . Figure 7 is a flowchart of another data processing method according to the embodiments of the present application. As shown in Figure 7 , the method can include the following steps:

[0225] In step S71, a current input data processing dialogue request is obtained, wherein the request data carried in the data processing dialogue request includes problem data.

[0226] Step S72, in response to the data processing dialogue request, a data processing dialogue reply is returned, wherein the information carried in the data processing dialogue reply includes: a target reply, the target reply is obtained after the target intelligent agent processes the question data, and the target intelligent agent is generated according to the intelligent agent training method in any one of the above.

[0227] Step S73, the target reply is displayed in the graphical user interface.

[0228] In the embodiment of the application, the data processing dialogue request can be a request issued by a user or an external system through an intelligent customer service platform, requiring an intelligent customer service intelligent agent to process the question data contained in the request to generate an answer or a solution.

[0229] The data processing dialogue reply is a reply made by the intelligent customer service intelligent agent to the data processing dialogue request, and contains the target reply, which is a direct response to the external request.

[0230] The graphical user interface is a user interaction interface for displaying the reply of the intelligent customer service intelligent agent, and the user can view, understand and interact through this interface.

[0231] For specific description, refer to the foregoing description of the embodiment.

[0232] The above data processing method provided by the embodiment of the application can be applied in, but is not limited to, application scenarios involving intelligent agent training in the fields of e-commerce services, education services, legal services, medical services, conference services, social network services, financial product services, logistics services and navigation services, such as: e-commerce service related intelligent agent training, education service related intelligent agent training, legal service related intelligent agent training, etc., which are not limited here.

[0233] By adopting the embodiment of the application, the current input data processing dialogue request is obtained, wherein the request data carried in the data processing dialogue request includes question data, and then in response to the data processing dialogue request, a data processing dialogue reply is returned, wherein the information carried in the data processing dialogue reply includes: a target reply, the target reply is obtained after the target intelligent agent processes the question data, and the target intelligent agent is generated according to the intelligent agent training method in any one of the above, and finally the target reply is displayed in the graphical user interface, thereby achieving the purpose of providing more convenient and accurate solutions for users, thereby realizing the technical effects of breaking through the limitation of static test data set through dynamic data construction, capability boundary evaluation and iterative training, significantly improving the robustness and flexibility of intelligent agents in processing complex tasks, and further solving the technical problem of low robustness of intelligent agents in related technologies due to the dependence of intelligent agent training and evaluation method on relatively static test data set.

[0234] According to the embodiment of the present application, the data processing method shown in the above is also provided. Figure 8 Figure 8 is a flow chart of another data processing method according to the embodiment of the present application. As shown in the above, the method can include the following steps: Figure 8

[0235] Step S81, in response to the input instruction acting on the operation interface, display the question data on the operation interface.

[0236] Step S82, in response to the processing instruction acting on the operation interface, display the target reply on the operation interface. The target reply is obtained by the target intelligent agent after the answer generation processing of the question data, and the target intelligent agent is generated according to the intelligent agent training method of any one of the above.

[0237] In the embodiment of the present application, the operation interface is the interface for the user to interact with the intelligent agent system. Through the operation interface, the user can input instructions, query questions, receive replies, etc., which has high operation flexibility and responsiveness.

[0238] The input instruction can be an instruction initiated by the user on the operation interface, requiring the intelligent agent system to display or pay attention to a certain question data. The input instruction can include clicking, touching, voice command, etc., according to the specific needs and operation habits of the user.

[0239] The processing instruction can be an instruction for the user to require the intelligent agent system to process the question data displayed before, to generate the corresponding reply. The processing instruction can also be issued in various ways, and the intelligent agent system generates the answer after receiving the instruction.

[0240] For specific description, please refer to the description of the above embodiment.

[0241] The above data processing method provided by the embodiment of the present application can be applied in the application scenarios related to intelligent agent training in the fields of e-commerce service, education service, legal service, medical service, conference service, social network service, financial product service, logistics service and navigation service, etc., such as: e-commerce service related intelligent agent training, education service related intelligent agent training, legal service related intelligent agent training, etc., which is not limited here.

[0242] ​​By responding to the input instruction acting on the operation interface, the problem data is displayed on the operation interface, and then by responding to the processing instruction acting on the operation interface, the target reply is displayed on the operation interface. The target reply is obtained by the target intelligent agent performing answer generation processing on the problem data, and the target intelligent agent is generated according to the intelligent agent training method in any one of the above. Thus, the purpose of providing a more convenient and accurate solution for the user is achieved, thereby realizing the technical effects of breaking the limitation of the static test data set through dynamic data construction, capability boundary evaluation and iterative training, significantly improving the robustness and flexibility of the intelligent agent in processing complex tasks, and further solving the technical problem in the related art that the intelligent agent training and evaluation method depends on the relatively static test data set, resulting in low robustness of the intelligent agent.

[0243] According to the embodiments of the present application, a data processing system as shown in Figure 9 is also provided. Figure 9 is a structural schematic diagram of a data processing system according to an embodiment of the present application. As shown in Figure 9 , the system can include:

[0244] The client is configured to send problem data.

[0245] The server is connected with the client, and is configured to perform answer generation processing on the problem data by using a target intelligent agent to obtain a target reply, wherein the target intelligent agent is generated according to the intelligent agent training method in any one of the above.

[0246] The client is further configured to output the target reply.

[0247] In the embodiments of the present application, the data processing system is used to execute the above-mentioned data processing method, and the specific description is referred to the description of the above-mentioned embodiments.

[0248] The above-mentioned data processing system provided by the embodiments of the present application can be applied in, but not limited to, the application scenarios related to intelligent agent training in the fields of e-commerce service, education service, legal service, medical service, conference service, social network service, financial product service, logistics service and navigation service, for example: e-commerce service related intelligent agent training, education service related intelligent agent training, legal service related intelligent agent training, etc., which are not limited herein.

[0249] By adopting the embodiment of the present application, the data processing method is executed by the data processing system, so that the purpose of providing a more convenient and accurate solution for the user is achieved, thereby realizing the technical effects of breaking the limitation of the static test data set through dynamic data construction, capability boundary evaluation and iterative training, and significantly improving the robustness and flexibility of the agent in processing complex tasks, thereby solving the technical problem of low robustness of the agent in the related art due to the dependence of the agent training and evaluation method on the relatively static test data set.

[0250] It should be noted that the preferred embodiments of the present embodiment can refer to the related description in the embodiments, which will not be repeated here.

[0251] According to the embodiments of the present application, an apparatus embodiment for implementing the above-mentioned agent training method is also provided. Figure 10 is a structural schematic diagram of an agent training apparatus according to an embodiment of the present application, as shown in Figure 10 The apparatus comprises:

[0252] A determination module 1001 is configured to determine an exploration direction based on existing knowledge of an agent system, wherein the exploration direction is used to plan and explore a target task to be generated, and the agent system comprises a plurality of agents with different capability levels.

[0253] A generation module 1002 is configured to control the plurality of agents to execute an adversarial walk strategy according to the exploration direction, and generate test data and training data.

[0254] An evaluation module 1003 is configured to perform capability boundary evaluation on the plurality of agents by using the test data, and obtain an evaluation result.

[0255] A training module 1004 is configured to train a to-be-evolved agent in the plurality of agents according to the evaluation result and the training data, and obtain a target agent.

[0256] Optionally, the determination module 1001 is further configured to determine a known task type, a tool coverage range, a model existing capability and / or a current service demand based on the existing knowledge of the agent system, and determine the exploration direction by using the known task type, the tool coverage range, the model existing capability and / or the current service demand.

[0257] Optionally, the known task type is used to determine a task type currently known by the agent system, the tool coverage range is used to determine a capability coverage range of a multifunctional tool library, the multifunctional tool library is a tool library pre-equipped for the agent system, the model existing capability is used to determine a model capability currently possessed by the agent system, and the current service demand is used to determine a demand of the agent system for a current service scenario.

[0258] Optionally, the generation module 1002 is further configured to control the plurality of agents to perform the adversarial wandering strategy according to the exploration direction, to form a plurality of task trajectories; and generate the test data and the training data based on the plurality of task trajectories.

[0259] Optionally, the generation module 1002 is further configured to generate a task target based on the exploration direction; and control the plurality of agents to perform the adversarial wandering strategy according to the task target, to form a plurality of task trajectories.

[0260] Optionally, the generation module 1002 is further configured to control the plurality of agents to perform the adversarial wandering strategy on corresponding tool chains in a plurality of tool chains according to the task target, to form a plurality of task trajectories, wherein the plurality of tool chains include a plurality of first tool chains and a plurality of second tool chains, the plurality of first tool chains are authorized to use a first category of tools in a multifunctional tool library, the plurality of second tool chains are authorized to use a second category of tools in the multifunctional tool library, the first category of tools have professional functional attributes, and the second category of tools have ordinary functional attributes.

[0261] Optionally, the evaluation module 1003 is further configured to control the plurality of agents to perform the test data, to obtain an execution result, wherein the execution result is used to determine whether the plurality of agents successfully perform the tool chain test process; and evaluate the capability boundary of the plurality of agents according to the execution result, to obtain an evaluation result.

[0262] Optionally, the training module 1004 is further configured to adjust the exploration direction according to the evaluation result, to obtain an adjusted exploration direction; label a process reward for the training data according to the evaluation result, to obtain labeled data; and perform process-guided reinforcement learning training on a to-be-evolved agent in the plurality of agents based on the adjusted exploration direction and the labeled data, to obtain a target agent.

[0263] Optionally, the generation module 1002 is further configured to generate positive example training samples based on successful execution results of the plurality of task trajectories on the plurality of first tool chains, and generate negative example training samples based on failure execution results of the plurality of task trajectories on the plurality of second tool chains; and determine the training data according to the positive example training samples and the negative example training samples.

[0264] Optionally, the apparatus further includes an expansion module configured to expand the functional attributes of existing tools in the multifunctional tool library, and / or expand tool categories not covered in the multifunctional tool library.

[0265] By adopting the embodiment of the application, the exploration direction is determined based on the existing knowledge of the agent system, wherein the exploration direction is used to plan and explore the target task to be generated, and the agent system includes multiple agents with different capability levels. Then, the multiple agents are controlled to perform the adversarial walking strategy according to the exploration direction, and the test data and the training data are generated. After that, the capability boundary of the multiple agents is evaluated by using the test data, and the evaluation result is obtained. Finally, the to-be-evolved agent in the multiple agents is trained according to the evaluation result and the training data, and the target agent is obtained. Thus, the purpose of training the agent with high robustness is achieved, thereby realizing the dynamic data construction, the capability boundary evaluation and the iterative training, breaking the limitation of the static test data set, and significantly improving the robustness and flexibility of the agent in processing complex tasks, and further solving the technical problem that the agent robustness is low due to the dependence of the agent training and evaluation method on the relatively static test data set in the related art.

[0266] It should be noted that the determination module 1001, the generation module 1002, the evaluation module 1003 and the training module 1004 correspond to steps S21 to S24 in the embodiment, and the four modules have the same instances and application scenarios as the corresponding steps, but are not limited to the contents disclosed in the above embodiment. It should be noted that the above modules or units can be hardware components or software components stored in the memory and processed by one or more processors, and the above modules can also run in the server 10 provided in the embodiment.

[0267] According to the embodiment of the application, a device embodiment for implementing the above data processing method is also provided. Figure 11 is a structural schematic diagram of another data processing device according to the embodiment of the application, as shown in Figure 11 The device includes:

[0268] The first acquisition module 1101 is configured to acquire problem data.

[0269] The second generation module 1102 is configured to perform answer generation processing on the problem data by using the target agent, and obtain a target reply; wherein the target agent is generated according to the agent training method in any one of the above embodiments.

[0270] By adopting the embodiment of the present application, the question data is acquired, the target agent generated by the agent training method in any one of the above is used to perform answer generation processing on the question data, and a target reply is obtained, thereby achieving the purpose of providing a more convenient and accurate solution for the user, so as to realize the technical effects of breaking the limitation of the static test data set through dynamic data construction, capability boundary evaluation and iterative training, significantly improving the robustness and flexibility of the agent in processing complex tasks, and further solving the technical problem of low robustness of the agent in the related art due to the dependence of the agent training and evaluation method on the relatively static test data set.

[0271] It should be noted that the first acquisition module 1101 and the second generation module 1102 correspond to steps S41 and S42 in the embodiment, and the two modules have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in the above embodiment. It should be noted that the above modules or units can be hardware components or software components stored in the memory and processed by one or more processors, and the above modules can also run in the server 10 provided in the embodiment.

[0272] According to the embodiment of the present application, another device embodiment for implementing the above data processing method is also provided. Figure 12 is a structural schematic diagram of another data processing device according to the embodiment of the present application, as Figure 12 shown, the device comprises:

[0273] The second acquisition module 1201 is configured to acquire intelligent customer service consultation question data.

[0274] The third generation module 1202 is configured to perform answer generation processing on the intelligent customer service consultation question data by using an intelligent customer service agent, and obtain an intelligent customer service consultation reply. The intelligent customer service agent is generated according to the agent training method in any one of the above.

[0275] By adopting the embodiment of the present application, the intelligent customer service consultation question data is acquired, the intelligent customer service agent generated by the agent training method in any one of the above is used to perform answer generation processing on the intelligent customer service consultation question data, and the intelligent customer service consultation reply is obtained, thereby achieving the purpose of providing a more convenient and accurate solution for the user, so as to realize the technical effects of breaking the limitation of the static test data set through dynamic data construction, capability boundary evaluation and iterative training, significantly improving the robustness and flexibility of the agent in processing complex tasks, and further solving the technical problem of low robustness of the agent in the related art due to the dependence of the agent training and evaluation method on the relatively static test data set.

[0276] It should be noted that the second obtaining module 1201 and the third generating module 1202 correspond to steps S51 and S52 in the embodiments, and the two modules have the same instances and application scenarios as the corresponding steps, but are not limited to the above-mentioned embodiments. It should be noted that the above-mentioned modules or units can be hardware components or software components stored in the memory and processed by one or more processors, and the above-mentioned modules can also run in the server 10 provided in the embodiments.

[0277] According to the embodiments of the present application, another device embodiment for implementing the above-mentioned data processing method is also provided. Figure 13 is a structural schematic diagram of another data processing device according to the embodiments of the present application, as shown in Figure 13 The device comprises:

[0278] The third obtaining module 1301 is configured to obtain a data processing request through a first application programming interface, wherein the request data carried in the data processing request comprises problem data.

[0279] The first returning module 1302 is configured to return a data processing response through a second application programming interface, wherein the response data carried in the data processing response comprises a target reply, the target reply is obtained by performing answer generation processing on the problem data by a target agent, and the target agent is generated according to the agent training method in any one of the above-mentioned embodiments.

[0280] According to the embodiments of the present application, the data processing request is obtained through the first application programming interface, wherein the request data carried in the data processing request comprises problem data, and then the data processing response is returned through the second application programming interface, wherein the response data carried in the data processing response comprises a target reply, the target reply is obtained by performing answer generation processing on the problem data by a target agent, and the target agent is generated according to the agent training method in any one of the above-mentioned embodiments. Thus, the purpose of providing a more convenient and accurate solution for users is achieved, thereby realizing the technical effects of breaking the limitation of static test data set through dynamic data construction, capability boundary evaluation and iterative training, significantly improving the robustness and flexibility of the agent in processing complex tasks, and further solving the technical problem that the agent robustness is low due to the dependence of the agent training and evaluation method on the relatively static test data set in the related art.

[0281] It should be noted that the third obtaining module 1301 and the first returning module 1302 correspond to steps S61 and S62 in the embodiments, and the two modules have the same instances and application scenarios as the corresponding steps, but are not limited to the above-mentioned embodiments. It should be noted that the above-mentioned modules or units can be hardware components or software components stored in the memory and processed by one or more processors, and the above-mentioned modules can also run in the server 10 provided in the embodiments.

[0282] According to the embodiments of the present application, another device embodiment for implementing the above-mentioned data processing method is also provided. Figure 14 is a structural schematic diagram of another data processing device according to the embodiments of the present application, as shown in Figure 14 , the device comprises:

[0283] The fourth obtaining module 1401 is configured to obtain a current input data processing dialogue request, wherein the request data carried in the data processing dialogue request comprises question data.

[0284] The second returning module 1402 is configured to return a data processing dialogue reply in response to the data processing dialogue request, wherein the information carried in the data processing dialogue reply comprises a target reply, the target reply is obtained by performing answer generation processing on the question data by a target agent, and the target agent is generated according to the agent training method in any one of the above-mentioned embodiments.

[0285] The display module 1403 is configured to display the target reply in the graphical user interface.

[0286] By adopting the embodiments of the present application, the current input data processing dialogue request is obtained, wherein the request data carried in the data processing dialogue request comprises question data, then a data processing dialogue reply is returned in response to the data processing dialogue request, wherein the information carried in the data processing dialogue reply comprises a target reply, the target reply is obtained by performing answer generation processing on the question data by a target agent, and the target agent is generated according to the agent training method in any one of the above-mentioned embodiments, and finally the target reply is displayed in the graphical user interface, thereby achieving the purpose of providing a more convenient and accurate solution for the user, thereby realizing the technical effects of breaking through the limitation of static test data set through dynamic data construction, capability boundary evaluation and iterative training, significantly improving the robustness and flexibility of the agent in processing complex tasks, and further solving the technical problem of low robustness of the agent in the related art due to the dependence of the agent training and evaluation method on the relatively static test data set.

[0287] It should be noted that the fourth obtaining module 1401, the second returning module 1402 and the display module 1403 correspond to steps S71 to S73 in the embodiment, and the three modules have the same instances and application scenarios as the corresponding steps, but are not limited to the contents disclosed in the above embodiment. It should be noted that the above modules or units can be hardware components or software components stored in the memory and processed by one or more processors, and the above modules can also run in the server 10 provided in the embodiment.

[0288] According to the embodiment of the present application, another device embodiment for implementing the above data processing method is also provided. Figure 15 is a structural schematic diagram of another data processing device according to the embodiment of the present application, as shown in Figure 15 The device comprises:

[0289] The first display module 1501 is configured to display the problem data on the operation interface in response to the input instruction acting on the operation interface.

[0290] The second display module 1502 is configured to display the target reply on the operation interface in response to the processing instruction acting on the operation interface, wherein the target reply is obtained by the target agent performing answer generation processing on the problem data, and the target agent is generated according to the agent training method in any one of the above.

[0291] According to the embodiment of the present application, the problem data is displayed on the operation interface in response to the input instruction acting on the operation interface, and then the target reply is displayed on the operation interface in response to the processing instruction acting on the operation interface. The target reply is obtained by the target agent performing answer generation processing on the problem data, and the target agent is generated according to the agent training method in any one of the above. Thus, the purpose of providing a more convenient and accurate solution for users is achieved, thereby realizing the technical effects of breaking the limitation of static test data set through dynamic data construction, capability boundary evaluation and iterative training, significantly improving the robustness and flexibility of the agent in processing complex tasks, and further solving the technical problem of low robustness of the agent in the related art due to the dependence of the agent training and evaluation method on the relatively static test data set.

[0292] It should be noted that the first display module 1501 and the second display module 1502 correspond to steps S81 and S82 in the embodiment, and the two modules have the same instances and application scenarios as the corresponding steps, but are not limited to the contents disclosed in the above embodiment. It should be noted that the above modules or units can be hardware components or software components stored in the memory and processed by one or more processors, and the above modules can also run in the server 10 provided in the embodiment.

[0293] It should be noted that the preferred embodiments involved in the above embodiments of the present application are the same as the schemes provided by the embodiments, and the application scenarios and implementation processes, but are not limited to the schemes provided by the embodiments.

[0294] Embodiments of the present application can provide a computing device. Figure 16 is a structural block diagram of a computing device according to an embodiment of the present application. As shown in the figure, the computing device A can include one or more (only one is shown in the figure) processors 1602, a memory 1604, a storage controller, and a peripheral interface, wherein the peripheral interface can be connected to a radio frequency module, an audio module, and a display screen, etc., which is not limited here. Figure 16 Figure 16 The above computing device A can be understood as an integrated intelligent terminal, including but not limited to servers, desktop computers, PC (Personal Computer, personal computer), model integrated machines, etc., and the computing device can be pre-installed with the model described in the above embodiments of the present application.

[0295] Specifically, the computing device A can preinstall multiple types of models, including but not limited to natural language processing, visual processing, speech processing, code processing, multi-modal task processing, etc. to provide diversified model selection. In different product forms, the computing device A can support one or more model usage modes, including but not limited to model training, model calling, model fine-tuning, model deployment, model inference and application, etc. In some product forms, the computing device A also supports model management, including but not limited to multi-type model management (supporting discriminative, generative, etc. Management of multiple types of models), model version control (supporting control of different model versions), model evaluation (based on model evaluation tools, evaluating the performance and effect of the model) and the like. In another product form, the computing device A can also create an application based on the model, provide API calling capability, and call the model into the created application through the API interface, while providing application management tools to realize the management and monitoring of the application.

[0296] Further, the computing device A can also include data management (supporting the creation and management of model optimization data sets), training center (providing rich training resources to help users learn and master AI technology), basic control capability (providing enterprise-level basic control capability to ensure the security and efficient operation of the system), through the above functions, a comprehensive and integrated AI development, training, deployment and application device is provided.

[0297] Further, the computing device A can also include data management (supporting the creation and management of model optimization data sets), training center (providing rich training resources to help users learn and master AI technology), basic control capability (providing enterprise-level basic control capability to ensure the security and efficient operation of the system), through the above functions, a comprehensive and integrated AI development, training, deployment and application device is provided.

[0298] ​The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and apparatus in the embodiments of this application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the methods in the above embodiments. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0299] The processor can invoke an executable program stored in memory via a transmission device to execute the method described in any of the above embodiments.

[0300] Those skilled in the art will understand that, Figure 16 The structure shown is for illustrative purposes only; computing device A can also be a smartphone, tablet, PDA, mobile internet device (MID), PAD, or other terminal device. Figure 16 This does not limit the structure of the aforementioned computing device. For example, computing device A may also include components that are more advanced than those described above. Figure 16 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same as the Figure 16 The different configurations shown.

[0301] Embodiments of this application may provide an electronic device. Figure 17 This is a structural block diagram of an electronic device according to an embodiment of this application. Figure 17 As shown, the electronic device may include: an input / output device 172; a memory 174; and a processor 176, wherein the processor 176 is connected to the input / output device 172 and the memory 174 via a bus 178.

[0302] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and apparatus in the embodiments of this application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the methods in the above embodiments. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0303] The processor can invoke the executable program stored in the memory through the transmission device to execute the method described in any one of the above embodiments.

[0304] Those skilled in the art can understand that all or part of the steps in the above-mentioned embodiments can be instructed by a program to the hardware related to the terminal device, and the program can be stored in a computer readable storage medium, which can include a flash disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0305] The embodiments of the present application also provide a computer readable storage medium. Optionally, in the embodiments, the computer readable storage medium can be used to save the program codes executed by the agent training method or the data processing method provided by the above embodiments.

[0306] Optionally, in the embodiments, the computer readable storage medium can be located in any one of the computer terminals in the computer terminal group in the computer network, or in any one of the mobile terminals in the mobile terminal group.

[0307] The embodiments of the present application also provide a computer program product, which comprises a computer program, and the computer program implements the agent training method or the data processing method described in any one of the above embodiments when executed by a processor.

[0308] In the above embodiments of the present application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0309] In several embodiments provided by the present application, it should be understood that the disclosed technology can be implemented in other ways. For example, the above-mentioned device embodiments are only schematic. The division of the units is only a logical function division. There can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, units or modules, and can be electrical or other forms.

[0310] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0311] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0312] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or the part that contributes to the prior art, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to make a computer device (which can be a personal computer, a server or a network device, etc.) execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic or optical disk and various program code storage media.

[0313] The above is only the preferred embodiment of the present application, it should be noted that for those skilled in the art, without departing from the principles of the present application, can make a number of improvements and refinements, these improvements and refinements should be considered as the protection scope of the present application.

Claims

1. A data processing method, characterized in that, include: Obtain the problem data; The target intelligent agent is used to process the question data to generate an answer, thus obtaining the target response; The target agent is trained according to the following steps: The exploration direction is determined based on the existing knowledge of the intelligent agent system, wherein the exploration direction is used to plan and explore the target task to be generated, and the intelligent agent system includes: multiple intelligent agents with different capability levels; The multiple agents are controlled to execute an adversarial walk strategy according to the exploration direction, forming multiple task trajectories. Test data and training data are generated based on the multiple task trajectories. The adversarial walk strategy obtains tasks that distinguish the differences in the capabilities of the multiple agents by comparing the performance of the multiple agents in solving the same task. The task trajectory is used to represent the operation, decision-making logic and result of the agent when solving the task. The capability boundaries of the multiple intelligent agents are evaluated using the test data to obtain the evaluation results. Based on the evaluation results and the training data, the agent to be evolved among the plurality of agents is trained to obtain the target agent.

2. The data processing method according to claim 1, characterized in that, Determining the exploration direction based on the existing knowledge of the intelligent agent system includes: Based on the existing knowledge of the intelligent agent system, the known task types, tool coverage, existing model capabilities, and / or current service requirements are determined. The exploration direction is determined by utilizing the known task types, the tool coverage, the existing capabilities of the model, and / or the current service requirements.

3. The data processing method according to claim 2, characterized in that, The known task types are used to determine the currently known task types of the intelligent agent system; the tool coverage is used to determine the capability coverage of the multi-functional tool library, which is a tool library pre-equipped for the intelligent agent system; the existing model capabilities are used to determine the model capabilities currently possessed by the intelligent agent system; and the current service requirements are used to determine the requirements of the intelligent agent system for the current service scenario.

4. The data processing method according to claim 1, characterized in that, Controlling the multiple agents to execute the adversarial walk strategy according to the exploration direction, forming the multiple task trajectories includes: Generate task objectives based on the described exploration direction; According to the stated task objective, the multiple intelligent agents are controlled to execute the adversarial walk strategy, thereby forming the multiple task trajectories.

5. The data processing method according to claim 4, characterized in that, According to the task objective, the multiple agents are controlled to execute the adversarial walk strategy, forming the multiple task trajectories, including: According to the task objective, the multiple agents are controlled to execute the adversarial walk strategy on the corresponding toolchains in the multiple toolchains to form the multiple task trajectories. The multiple toolchains include: multiple first toolchains and multiple second toolchains. The multiple first toolchains are authorized to use first category tools in the multi-functional tool library, and the multiple second toolchains are authorized to use second category tools in the multi-functional tool library. The first category tools have professional functional attributes, and the second category tools have ordinary functional attributes.

6. The data processing method according to claim 1, characterized in that, The capability boundaries of the multiple agents are evaluated using the test data, and the evaluation results include: The multiple intelligent agents are controlled to execute the test data to obtain execution results, wherein the execution results are used to determine whether the multiple intelligent agents have successfully executed the toolchain test process; The capability boundaries of the multiple agents are evaluated based on the execution results to obtain the evaluation results.

7. The data processing method according to claim 1, characterized in that, Based on the evaluation results and the training data, the agent to be evolved among the plurality of agents is trained to obtain the target agent, which includes: The exploration direction is adjusted based on the evaluation results to obtain the adjusted exploration direction; The training data annotation process is rewarded based on the evaluation results, and the annotated data is obtained. Based on the adjusted exploration direction and the labeled data, the agent to be evolved among the multiple agents is subjected to process-guided reinforcement learning training to obtain the target agent.

8. The data processing method according to claim 1, characterized in that, The training data generated based on the multiple task trajectories includes: Based on the successful execution results of the multiple task trajectories on multiple first toolchains, positive training samples are generated; and based on the failed execution results of the multiple task trajectories on multiple second toolchains, negative training samples are generated. The training data is determined based on the positive training samples and the negative training samples.

9. The data processing method according to claim 3, characterized in that, The data processing method further includes: Expand the functional attributes of existing tools in the multi-functional tool library, and / or expand tool categories not covered in the multi-functional tool library.

10. A data processing method, characterized in that, include: Obtain data on customer service inquiries from intelligent customer service systems; An intelligent customer service agent is used to process the intelligent customer service inquiry data to generate answers and obtain intelligent customer service responses. The intelligent customer service agent is trained according to the following steps: First, an exploration direction is determined based on the existing knowledge of the agent system. This exploration direction is used to plan and explore the target task to be generated. The agent system includes multiple agents with different capability levels. Second, the multiple agents are controlled to execute an adversarial walk strategy according to the exploration direction, forming multiple task trajectories. Test data and training data are generated based on these task trajectories. The adversarial walk strategy compares the performance of the multiple agents in solving the same task to identify tasks that differentiate the capabilities of the agents. The task trajectories represent the operations, decision-making logic, and results of the agents in solving the tasks. Third, the capability boundaries of the multiple agents are evaluated using the test data to obtain evaluation results. Finally, the agents to be evolved among the multiple agents are trained based on the evaluation results and the training data to obtain the intelligent customer service agent.

11. A data processing method, characterized in that, include: A data processing request is obtained through a first application programming interface, wherein the request data carried in the data processing request includes: problem data; A data processing response is returned through a second application programming interface (API). The response data includes a target response, which is obtained by a target agent generating an answer to the question data. The target agent is trained according to the following steps: determining an exploration direction based on the existing knowledge of the agent system, wherein the exploration direction is used to plan and explore the target task to be generated; the agent system includes multiple agents with different capability levels; controlling the multiple agents to execute an adversarial walk strategy according to the exploration direction to form multiple task trajectories, and generating test data and training data based on the multiple task trajectories; wherein the adversarial walk strategy compares the performance of the multiple agents in solving the same task to obtain tasks that distinguish the capabilities of the multiple agents; the task trajectory represents the operation, decision-making logic, and result of the agent in solving the task; using the test data to evaluate the capability boundaries of the multiple agents to obtain evaluation results; and training the agent to be evolved among the multiple agents based on the evaluation results and the training data to obtain the target agent.

12. A data processing method, characterized in that, include: Obtain the currently input data processing dialogue request, wherein the request data carried in the data processing dialogue request includes: question data; In response to the data processing dialogue request, a data processing dialogue reply is returned. The information carried in the data processing dialogue reply includes: a target reply, which is obtained by a target agent generating an answer to the question data. The target agent is trained according to the following steps: determining an exploration direction based on the existing knowledge of the agent system, wherein the exploration direction is used to plan and explore the target task to be generated; the agent system includes multiple agents with different ability levels; controlling the multiple agents to execute an adversarial walk strategy according to the exploration direction, forming multiple task trajectories, and generating test data and training data based on the multiple task trajectories; wherein the adversarial walk strategy obtains tasks that distinguish the differences in ability among the multiple agents by comparing their performance in solving the same task; the task trajectory is used to represent the operation, decision logic, and result of the agent in solving the task; using the test data to evaluate the capability boundaries of the multiple agents, obtaining evaluation results; and training the agent to be evolved among the multiple agents based on the evaluation results and the training data to obtain the target agent. The target response is displayed within the graphical user interface.

13. A data processing method, characterized in that, include: In response to input commands applied to the user interface, the system displays problem data on the user interface. In response to the processing instructions applied to the operation interface, the target response is displayed on the operation interface; The target response is obtained by a target agent processing the question data to generate an answer. The target agent is trained according to the following steps: determining an exploration direction based on the existing knowledge of the agent system, wherein the exploration direction is used to plan and explore the target task to be generated; the agent system includes multiple agents with different ability levels; controlling the multiple agents to execute an adversarial walk strategy according to the exploration direction to form multiple task trajectories, and generating test data and training data based on the multiple task trajectories; wherein the adversarial walk strategy obtains tasks that distinguish the ability differences of the multiple agents by comparing the performance of the multiple agents in solving the same task; the task trajectory is used to represent the operation, decision logic and result of the agent in solving the task; using the test data to evaluate the capability boundaries of the multiple agents to obtain the evaluation result; and training the agent to be evolved among the multiple agents according to the evaluation result and the training data to obtain the target agent.

14. A data processing system, characterized in that, include: The client is used to send the problematic data; A server, connected to the client, is used to process the question data using a target agent to generate a target response. The target agent is trained according to the following steps: determining an exploration direction based on the existing knowledge of the agent system, wherein the exploration direction is used to plan and explore the target task to be generated; the agent system includes multiple agents with different ability levels; controlling the multiple agents to execute an adversarial walk strategy according to the exploration direction, forming multiple task trajectories, and generating test data and training data based on the multiple task trajectories; wherein the adversarial walk strategy compares the performance of the multiple agents in solving the same task to obtain tasks that distinguish the differences in ability among the multiple agents; the task trajectory represents the operation, decision logic, and result of the agent in solving the task; using the test data to evaluate the capability boundaries of the multiple agents to obtain evaluation results; and training the agent to be evolved among the multiple agents based on the evaluation results and the training data to obtain the target agent. The client is also used to output the target response.

15. An electronic device, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, performs the data processing method according to any one of claims 1 to 13.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein, when the executable program is executed, it controls the device on which the computer-readable storage medium is located to perform the data processing method according to any one of claims 1 to 13.

17. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the data processing method according to any one of claims 1 to 13.

Images