Intelligent agent training method, data processing method and data processing system

Abstract

The invention discloses an agent training method, a data processing method and a data processing system, and relates to the technical field of artificial intelligence and large models. The method comprises the steps that an exploration direction is determined based on existing knowledge of an intelligent agent system, the exploration direction is used for planning and exploring a to-be-generated target task, and the intelligent agent system comprises multiple intelligent agents of different capability levels; according to the exploration direction, the multiple agents are controlled to execute a migration resisting strategy, and test data and training data are generated; performing capability boundary evaluation on the plurality of agents by using the test data to obtain an evaluation result; and training the to-be-evolved intelligent agent in the plurality of intelligent agents according to the evaluation result and the training data to obtain a target intelligent agent. According to the method and the device, the technical problem of relatively low robustness of the intelligent agent due to the fact that the intelligent agent training and evaluation method depends on a relatively static test data set in related technologies is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence and large model technology, in particular to an agent training method, a data processing method and a data processing system. BACKGROUND

[0002] With the rapid development of the field of artificial intelligence (AI), agents (Agents) centered on large language models (LLM) are leading the new trend of artificial intelligence. Agents can perform complex tasks based on understanding natural language. However, due to the limitations of static and fixed training and evaluation methods, agents exhibit obvious performance bottlenecks when facing challenges that require multi-tool collaboration and deep logical reasoning.

[0003] Currently, the agent training and evaluation method relies on relatively static test data sets. These test data sets are mainly designed around information retrieval or single tool invocation, and fail to fully reflect the collaborative tool use ability and deep reasoning skills of LLM in complex task execution, resulting in a lack of authenticity and diversity of training data, which limits the adaptability and robustness of agents when facing complex problems.

[0004] In view of the above problems, no effective solutions have been proposed so far. SUMMARY

[0005] The embodiments of the present application provide an agent training method, a data processing method and a data processing system to at least solve the technical problem of low robustness of agents in related technologies due to the dependence of agent training and evaluation methods on relatively static test data sets.

[0006] According to an aspect of an embodiment of the present application, an agent training method is provided, comprising: determining an exploration direction based on existing knowledge of an agent system, wherein the exploration direction is used to plan and explore a target task to be generated, and the agent system comprises: a plurality of agents with different capability levels; controlling the plurality of agents to execute an adversarial walk strategy according to the exploration direction to generate test data and training data; performing capability boundary evaluation on the plurality of agents using the test data to obtain an evaluation result; and training a to-be-evolved agent in the plurality of agents according to the evaluation result and the training data to obtain a target agent.

[0007] According to another aspect of an embodiment of the present application, a data processing method is also provided, comprising: obtaining problem data; and performing answer generation processing on the problem data using a target agent to obtain a target reply; wherein the target agent is generated according to the agent training method of any one of the above.

[0008] According to another aspect of the embodiments of the present application, a data processing method is further provided, comprising: obtaining intelligent customer service consultation question data; and generating answers to the intelligent customer service consultation question data by using an intelligent customer service agent to obtain an intelligent customer service consultation reply; wherein the intelligent customer service agent is generated according to the agent training method in any one of the preceding embodiments.

[0009] According to another aspect of the embodiments of the present application, a data processing method is further provided, comprising: obtaining data processing requests through a first application programming interface, wherein the request data carried in the data processing requests comprises question data; and returning data processing responses through a second application programming interface, wherein the response data carried in the data processing responses comprises a target reply, which is obtained by generating answers to the question data by using a target agent, and the target agent is generated according to the agent training method in any one of the preceding embodiments.

[0010] According to another aspect of the embodiments of the present application, a data processing method is further provided, comprising: obtaining a current input data processing dialogue request, wherein the request data carried in the data processing dialogue request comprises question data; and returning a data processing dialogue reply in response to the data processing dialogue request, wherein the information carried in the data processing dialogue reply comprises a target reply, which is obtained by generating answers to the question data by using a target agent, and the target agent is generated according to the agent training method in any one of the preceding embodiments; and displaying the target reply in a graphical user interface.

[0011] According to another aspect of the embodiments of the present application, a data processing method is further provided, comprising: displaying question data on an operation interface in response to an input instruction acting on the operation interface; and displaying a target reply on the operation interface in response to a processing instruction acting on the operation interface; wherein the target reply is obtained by generating answers to the question data by using a target agent, and the target agent is generated according to the agent training method in any one of the preceding embodiments.

[0012] According to another aspect of the embodiments of the present application, a data processing system is further provided, comprising: a client configured to send question data; a server connected to the client and configured to generate answers to the question data by using a target agent to obtain a target reply, wherein the target agent is generated according to the agent training method in any one of the preceding embodiments; and the client is further configured to output the target reply.

[0013] According to another aspect of the embodiments of the present application, an electronic device is further provided, comprising: a memory storing an executable program; and a processor connected to the memory through a bus and configured to run the program, wherein the program performs the agent training method or the data processing method in any one of the preceding embodiments when running.

[0014] According to another aspect of the embodiments of the present application, a computer readable storage medium is also provided, which includes a stored executable program, wherein the executable program controls the device where the computer readable storage medium is located to perform any of the above-mentioned agent training method or data processing method when the executable program is executed.

[0015] According to another aspect of the embodiments of the present application, a computer program product is also provided, which includes a computer program, the computer program implements any of the above-mentioned agent training method or data processing method when executed by a processor.

[0016] In the embodiments of the present application, the exploration direction is determined based on the existing knowledge of the agent system, wherein the exploration direction is used to plan and explore the target task to be generated, and the agent system includes a plurality of agents with different capability levels. Then the plurality of agents are controlled to perform the adversarial walk strategy according to the exploration direction, and the test data and the training data are generated. After that, the capability boundary of the plurality of agents is evaluated by using the test data, and the evaluation result is obtained. Finally, the to-be-evolved agent in the plurality of agents is trained according to the evaluation result and the training data, and the target agent is obtained. Thus, the agent with high robustness is trained, thereby realizing the dynamic data construction, the capability boundary evaluation and the iterative training, breaking the limitation of the static test data set, and significantly improving the robustness and flexibility of the agent in processing complex tasks, thereby solving the technical problem that the agent robustness is low due to the dependence of the agent training and evaluation method on the relatively static test data set in the related art.

[0017] It is easy to note that the above general description and the following detailed description are only for exemplifying and explaining the present application, and do not constitute a limitation on the present application. BRIEF DESCRIPTION OF DRAWINGS

[0018] The drawings described herein are used to provide further understanding of the present application, constitute a part of the present application, and the illustrative embodiments of the present application and the description thereof are used to explain the present application, and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 is an application scenario schematic diagram of an agent training method according to an embodiment of the present application;

[0020] Figure 2 is a flowchart of an agent training method according to an embodiment of the present application;

[0021] Figure 3 is a flowchart of data construction according to an embodiment of the present application;

[0022] Figure 4 is a flowchart of a data processing method according to an embodiment of the present application;

[0023] Figure 5 is a flowchart of another data processing method according to an embodiment of the application;

[0024] Figure 6 is a flowchart of another data processing method according to an embodiment of the application;

[0025] Figure 7 is a flowchart of another data processing method according to an embodiment of the application;

[0026] Figure 8 is a flowchart of another data processing method according to an embodiment of the application;

[0027] Figure 9 is a structural schematic diagram of a data processing system according to an embodiment of the application;

[0028] Figure 10 is a structural schematic diagram of an intelligent agent training apparatus according to an embodiment of the application;

[0029] Figure 11 is a structural schematic diagram of another data processing apparatus according to an embodiment of the application;

[0030] Figure 12 is a structural schematic diagram of another data processing apparatus according to an embodiment of the application;

[0031] Figure 13 is a structural schematic diagram of another data processing apparatus according to an embodiment of the application;

[0032] Figure 14 is a structural schematic diagram of another data processing apparatus according to an embodiment of the application;

[0033] Figure 15 is a structural schematic diagram of another data processing apparatus according to an embodiment of the application;

[0034] Figure 16 is a structural block diagram of a computing device according to an embodiment of the application;

[0035] Figure 17 is a structural block diagram of an electronic device according to an embodiment of the application. DETAILED DESCRIPTION

[0036] In the following, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application, so that those skilled in the art can better understand the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should fall within the scope of protection of the present application.

[0037] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0038] The technical solutions provided by the present application are mainly implemented by using large model technology. Here, the large model refers to a deep learning model with large-scale model parameters, which can typically include hundreds of millions, billions, tens of billions, hundreds of billions or even tens of billions of model parameters. The large model can also be referred to as a foundation model. Through large-scale unlabeled corpus pre-training, a pre-trained model with hundreds of millions of parameters is produced. This model can adapt to a wide range of downstream tasks and has good generalization ability. For example, large language models (LLM) and multi-modal pre-training models.

[0039] It should be noted that in actual application, the large model can be fine-tuned by a small amount of samples on the pre-trained model, so that the large model can be applied to different tasks. For example, the large model can be widely applied to the fields of natural language processing (NLP), computer vision, speech processing, etc., and can be applied to computer vision field tasks such as visual question answering (VQA), image captioning (IC), image generation, etc., and can be widely applied to natural language processing field tasks such as text-based sentiment classification, text summary generation, machine translation, etc. Therefore, the main application scenarios of the large model include but are not limited to digital assistants, intelligent robots, search, online education, office software, e-commerce, intelligent design, etc. In the embodiments of the present application, the target agent trained by the agent training method proposed in the present application is taken as an example to explain and describe the data processing in the data processing scenario.

[0040] First, some of the nouns or terms that appear in the description of the embodiments of the present application are applicable to the following explanations:

[0041] Large language model (LLM): that is, the large model, is an advanced machine learning model trained by large-scale text data. The large language model can understand and generate natural language, and has strong text generation, semantic understanding, knowledge reasoning and cross-domain transfer ability.

[0042] Agent: refers to a large language model as the core, which can independently perceive, make decisions, and interact with the environment by calling external tools (such as application programming interface (API), database) to complete specific goals.

[0043] Tool chain: an ordered set of multiple tools and technologies, designed to work together to complete complex tasks. By calling multiple tools (API, functions, models, etc.) in a specific logical order by the agent, problems that cannot be solved independently by a single tool are solved.

[0044] Trace: in the embodiments of the present application, it refers to a record of a series of decisions, actions and results of the agent in the process of executing tasks. The trace describes the entire process of the agent from receiving instructions to completing tasks, including each decision point, called tools, received feedback and final results, and is a key basis for evaluating the behavior logic and efficiency of the agent.

[0045] Large language models are leading a new wave of artificial intelligence revolution. Language models themselves are like a powerful "brain", but they are trapped in a "digital black box" that cannot interact directly with the real world. The introduction of tools is the key to breaking this limitation and releasing the full potential of intelligent agents. By calling external APIs, databases, and various applications, intelligent agents can overcome their inherent three bottlenecks:

[0046] Bottleneck 1: Limitations of knowledge. The knowledge of intelligent agents is "frozen" after training and cannot learn about recent events. By accessing search engines or professional databases, intelligent agents can obtain real-time and accurate external information, effectively avoiding outdated knowledge and content "hallucinations".

[0047] Bottleneck 2: Limitations of ability. Intelligent agents alone cannot process precise structured data, understand spatial relationships, or perform actual actions. By calling map APIs, intelligent agents can plan precise routes. By calling online automation tools, intelligent agents can connect thousands of applications to perform real-world workflows such as sending emails and updating calendars. By accessing professional models, intelligent agents can even extend their ability from text generation to multi-modal creation of images and 3D models.

[0048] Bottleneck 3: Limitations of domain. General models lack "professional ability" in vertical fields. Compared to general web search, directly calling domain-specific APIs (such as querying game performance platforms or obtaining website technology stack platforms) can obtain more in-depth, structured, and machine-readable professional data, allowing for more in-depth analysis and decision-making.

[0049] However, with the explosive growth of the tool ecosystem, the ability of intelligent agents is evolving at an unprecedented rate, but the current evaluation and training methods to measure and improve their tool usage ability are severely lagging behind.

[0050] Current mainstream evaluation benchmarks are mostly designed around simple information retrieval or isolated API calls, and their task construction methods often rely on "multi-hop" queries of a single concept. These outdated paradigms cannot effectively measure the real ability of intelligent agents in solving problems that require multi-tool collaboration, deep logical reasoning, and creative problem-solving. At the same time, the corresponding training data construction strategies are also becoming rigid, making it difficult to generate task trajectories that truly reflect the complexity of the real world.

[0051] Specifically, the current evaluation benchmark mainly aims to "measure and evaluate" the tool usage ability of large language models in different scenarios. Through the design of a comprehensive test data set and an automatic evaluation system, the strengths and weaknesses of existing large language models are diagnosed. The test data set is relatively static, and the Agent scenario relies on expert manual construction, which limits the diversity and scalability of the scenario.

[0052] Therefore, the industry urgently needs a new methodology to get out of the current "let the model use new tools to solve old problems" dilemma. In this context, the present application aims to solve the core contradiction between "capability evolution" and "evaluation lag" in the current agent field and promote the evolution of agents from passive "information carriers" to active "complex problem solvers".

[0053] The agent training and evaluation method in the related art has the following defects.

[0054] Defect 1: Evaluation benchmark lags behind. The current evaluation standard mostly focuses on simple information retrieval or isolated tool invocation, and fails to effectively measure the real ability of the model in solving complex problems that require deep logical reasoning and multi-tool collaboration.

[0055] Defect 2: Task construction method is limited. The Agent scenario relying on expert manual construction and the relatively static data set limit the diversity and scalability of the task scenario, making it difficult to dynamically adapt to the rapid growth of the tool library and the evolution of the agent's ability.

[0056] Defect 3: Training data is limited. The training data construction strategy is fixed, making it difficult to generate truly complex and challenging task trajectories that reflect the complexity and challenges of the real world, limiting the training quality and robustness of the model.

[0057] Defect 4: Insufficient exploration of capability boundaries. There is a lack of systematic methods to explore and define the model's limit ability in tool usage, i.e., the so-called "capability boundary", leading to possible blind spots in the model's performance in complex task execution.

[0058] Defect 5: Lack of adaptability and foresight. It is difficult to automatically adapt to the addition of new tools or changes in the agent's ability, leading to rapid obsolescence of the evaluation and training framework, which cannot continuously promote the development of the agent system.

[0059] No effective solution has been proposed to address the above defects prior to the present application.

[0060] According to an embodiment of the present application, an agent training method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0061] Considering that the model parameter quantity of the large model is large, and the operation resource of the mobile terminal is limited, the above method provided by the embodiments of the present application can be applied to the application scenarios as shown in Figure 1 , but is not limited thereto. In the application scenarios as shown in Figure 1 , the large model is deployed in the server 10, and the server 10 can be connected to one or more client devices 20 through a local area network connection, a wide area network connection, an Internet connection, or other types of data networks. The client device 20 can include, but is not limited to, a smartphone, a tablet computer, a notebook computer, a palm computer, a personal computer, a smart home device, a vehicle-mounted device, etc. The client device 20 can interact with the user through a graphical user interface to realize the calling of the large model, and thus realize the method provided by the embodiments of the present application.

[0062] In the embodiments of the present application, the system composed of the client device and the server can perform the following steps: the client device performs the steps of sending a training instruction to the server, etc., and the server performs the steps of training the agent based on the training instruction, specifically including determining an exploration direction based on the existing knowledge of the agent system, wherein the exploration direction is used to plan and explore the target task to be generated, and the agent system includes: a plurality of agents with different capability levels; controlling the plurality of agents to perform an adversarial walk strategy according to the exploration direction to generate test data and training data; performing capability boundary evaluation on the plurality of agents using the test data to obtain an evaluation result; training a to-be-evolved agent in the plurality of agents according to the evaluation result and the training data to obtain a target agent, and returning the target agent to the client device, etc. It should be noted that in the case that the running resource of the client device can meet the deployment and running conditions of the large model, the embodiments of the present application can be performed in the client device.

[0063] It should be noted that with the rapid development of high-performance computing units, the above method provided by the embodiments of the present application can also be applied to a model all-in-one machine in other application scenarios. In an optional embodiment, the model all-in-one machine is built-in with a plurality of models, and a user can select a model to adjust according to needs to obtain a user's own model, so that the high-performance computing unit built-in in the model all-in-one machine can directly call the adjusted model to execute the above method provided by the embodiments of the present application. In another optional embodiment, the large model all-in-one machine is built-in with a trained model, so that the high-performance computing unit built-in in the model all-in-one machine can directly call the model to execute the above method provided by the embodiments of the present application.

[0064] Further, when the user needs to train his own model, he can also upload his own data set through the client, which is sent to the server, so that the server can adjust the pre-trained model with the data set to obtain the user's own model, and then deploy it to the production environment. In order to facilitate the user's adjustment requirements of the model, the server can provide complete adjustment tools, development frameworks and processes, and can support multiple adjustment strategies, so that the adjusted model can better adapt to different field applications and realize high customization.

[0065] In the above running environment, the application provides an agent training method as shown in Figure 2 Figure 2 is a flowchart of an agent training method according to an embodiment of the application. As shown in Figure 2 , the method can include the following steps:

[0066] Step S21, determining an exploration direction based on the existing knowledge of the agent system, wherein the exploration direction is used to plan and explore the target task to be generated, and the agent system includes a plurality of agents with different capability levels.

[0067] Step S22, controlling the plurality of agents to execute the adversarial walk strategy according to the exploration direction, and generating test data and training data.

[0068] Step S23, performing capability boundary evaluation on the plurality of agents using the test data, and obtaining an evaluation result.

[0069] Step S24, training the to-be-evolved agent in the plurality of agents according to the evaluation result and the training data, and obtaining a target agent.

[0070] In the embodiment of the application, in the agent training process, first, the exploration direction is determined based on the existing knowledge of the agent system.

[0071] Among them, the agent system refers to a comprehensive system composed of one or more agents, which can cooperate with each other to execute various tasks. Each agent in the agent system has the ability to process specific tasks or problems, and can interact with the outside world and solve problems by calling external tools (such as API, database, etc.).

[0072] ​Furthermore, an intelligent agent system can include multiple agents with different capability levels, differing in tool usage, reasoning abilities, and other aspects. For example, these agents at different capability levels can include strong toolchain agents and weak toolchain agents. A strong toolchain agent can be understood as an "expert-level" agent capable of using all tools; for instance, when asked to plan a route including real-time traffic conditions, a strong toolchain agent can directly call a map API. A weak toolchain agent can be understood as a "novice-level" agent restricted to using only basic tools, such as a common search engine. By comparing the performance of different agents, the capability boundaries of a large model can be identified.

[0073] Existing knowledge refers to all the information and experience that an agent in an intelligent agent system acquires through historical training, task execution, and other information channels, including but not limited to domain knowledge, tool usage experience, and task execution history. This existing knowledge constitutes the basis for the agent's understanding of the world.

[0074] Exploration directions refer to the types of research or tasks that the intelligent agent system should focus on in the next step. These directions are derived from the analysis of existing knowledge and are used to plan and explore the target tasks to be generated. The aim of exploration directions is to improve the capabilities of the intelligent agent and to discover new capability boundaries of large models, that is, to discover potential capability deficiencies and room for improvement in large models.

[0075] As can be seen, this application first analyzes existing knowledge about intelligent agent systems to understand which domains or task types the intelligent agent system has limitations or performs well in. Based on the analysis results, it determines the next direction of exploration, that is, the domains or task types that need further development and optimization. Therefore, it can be seen that the direction of exploration is determined by observing the performance differences of intelligent agents with different ability levels when facing the same type of task. This means that the determination of the direction of exploration is not random, but targeted, aiming to guide the intelligent agent system towards more complex and challenging task areas.

[0076] For example, if a strong toolchain agent succeeds in solving a task that requires combining real-time financial data and geolocation information, while a weak toolchain agent fails, then such a task may be set as the next direction for exploration, with the aim of narrowing the capability gap through training.

[0077] Thus, it can be ensured that subsequent task generation and agent training are targeted and focused on improving the performance of the agent in a specific domain or task type, rather than blindly averaging efforts in all directions. Moreover, by analyzing the performance differences between strong and weak agents, the ability boundaries of the current large model can be more accurately located, providing a clear target for training. In addition, determining the exploration direction based on existing knowledge can avoid repeated exploration of mastered knowledge domains, saving valuable time and computing resources, and improving the overall training and data generation efficiency.

[0078] Then, the multiple agents are controlled to execute the adversarial walk strategy according to the exploration direction, to generate test data and training data.

[0079] Among them, the adversarial walk strategy refers to solving the same task by agents with different ability levels. Since agents with different ability levels will adopt different strategies and approaches to solve the task, the adversarial walk strategy can automatically identify and generate complex tasks that can distinguish the ability differences of different agents by comparing the performance of different agents, especially focusing on task cases that strong tool chain agents can successfully solve while weak tool chain agents cannot, as the basis for task generation and evaluation.

[0080] During the execution of the adversarial walk strategy by multiple agents, each step of operation, decision logic and final result (whether successful or failed) of each agent are recorded in detail, which constitutes the test data and training data reflecting the real task execution process. It can be understood that the data generated by the agent executing the adversarial walk strategy is used as both test data and training data, but the use scenarios and stages of test data and training data are different.

[0081] Test data is mainly used to evaluate the current ability boundary of the agent. The test data reflects the performance of the agent in solving a specific task, including successful paths and failure cases. Training data is used to train and optimize the agent so that it can learn and improve its performance on a specific task. Test data and training data contain effective practices and failure cases of the agent in solving complex problems, and are important materials for subsequent ability evaluation and model training.

[0082] It can be seen that after determining the exploration direction, the application will control multiple agents to execute the confrontation wandering strategy according to the exploration direction, and generate test data and training data through the confrontation exploration between the strong tool chain agent and the weak tool chain agent. In this stage, each agent is assigned a task goal, but since the tools and abilities of agents of different ability levels to solve problems are different, the application will track and record each decision and action step of agents of different ability levels, and by comparing the performance of agents of different ability levels in executing the same task, especially the differences in difficulties and solutions, a series of task cases reflecting the complexity of the real world, i.e. test data and training data, can be automatically generated. Illustratively, these task cases contain both successful problem solving instances and failure or partial success samples, which will be used for subsequent testing and training to improve the performance of agents in similar tasks.

[0083] Illustratively, agents of different ability levels can be allowed to try to solve tasks of the same type but gradually increasing difficulty, record the reasoning track and decision of each agent, and compare the solutions and execution processes of agents of different ability levels, thereby forming high-quality test data and training data.

[0084] Thus, the test data and training data generated by the confrontation wandering strategy contain multi-dimensional decision paths and results, which can more comprehensively reflect the behavior patterns and difficulty points of agents in solving complex problems, improve the quality and diversity of training data, i.e. generate high-value data. Moreover, the confrontation wandering strategy can be automatically adjusted with the update and expansion of the agent system tool library, thereby ensuring that the test data and training data always reflect new technical capabilities and task requirements. In addition, the confrontation wandering strategy can accurately identify which tools and strategies are more critical to improving complex task processing capabilities by comparing the execution results of agents of different ability levels, thereby optimizing resource allocation and training processes in a targeted manner.

[0085] Then, the test data is used to evaluate the ability boundary of the multiple agents, and an evaluation result is obtained.

[0086] The ability boundary evaluation refers to determining the ability limit of the agent in processing a specific task or problem, and the limitations of the agent in tool use, logical reasoning, cross-domain knowledge integration, etc. through analysis of the test data, i.e. the ability boundary of the large model can be accurately located through the ability boundary evaluation.

[0087] The evaluation result refers to the quantitative analysis result of the capabilities and limitations of different agents in the agent system in processing complex tasks obtained through the ability boundary evaluation. The evaluation result can indicate which agents perform poorly in which aspects and need further training and optimization.

[0088] As can be seen, by analyzing the aforementioned generated test data, this application can evaluate the actual performance of each agent in handling complex tasks and obtain evaluation results. The evaluation results will reveal the strengths and weaknesses of the agents in task execution, thereby obtaining the capability boundaries of each agent and providing a clear direction for subsequent training and optimization.

[0089] Therefore, capability boundary assessment can clearly identify in which specific tasks or domains an agent performs poorly, accurately pinpointing capability deficiencies and providing precise guidance for subsequent model optimization and training. Furthermore, capability boundary assessment makes the measurement of agent performance more objective, helping to understand the impact of toolchains on agent performance and how to optimize toolchains to improve overall capabilities.

[0090] Finally, based on the evaluation results and training data, the agent to be evolved among multiple agents is trained to obtain the target agent.

[0091] Among them, agents to be evolved are those that, based on the evaluation results, perform poorly in specific aspects or domains and require further training to enhance their capabilities. In other words, these are agents that show weak capabilities or room for improvement in the capability boundary assessment. For example, agents to be evolved are typically weak toolchain agents, or agents that, according to the evaluation results, require further optimization.

[0092] A target agent is an agent that, after training and optimization, has reached or exceeded predetermined capability boundaries, enabling it to perform tasks more efficiently in complex multi-agent systems. The target agent possesses enhanced tool usage capabilities, reasoning skills, and problem-solving efficiency.

[0093] As can be seen, after determining the capability boundaries of the agent, this application utilizes the aforementioned generated training data to conduct targeted training on the agent to be evolved. For example, training the agent to be evolved typically involves reinforcement learning techniques, using both positive and negative training data to guide the agent in learning better decision-making strategies and tool-calling methods. By simulating the successful strategies of a strong toolchain agent, a weak toolchain agent can obtain "guidance" on how to overcome existing obstacles and improve its ability to handle complex tasks, ultimately evolving into the target agent.

[0094] Therefore, by conducting targeted training on the agent to be evolved, the capabilities of the trained agent in specific tasks and domains are significantly improved, enabling it to handle complex problems that were previously impossible or difficult to solve. Furthermore, the agent system can self-iterate and continuously evolve to meet higher-level task challenges, improving efficiency and stability in multi-agent collaborative environments, thereby enhancing the robustness and adaptability of the overall agent system.

[0095] In summary, the application can automatically generate training data reflecting the complexity of the real world through the adversarial wandering strategy, avoiding the limitations brought by relying on static test data sets, making the training process closer to real challenges, and enhancing the robustness and adaptability of the agent. At the same time, the application uses the generated test data to evaluate the ability boundary, which can accurately identify the ability boundary and difficulty point of the agent when dealing with complex tasks, providing clear guidance for subsequent targeted training. In addition, the application trains the agent according to the evaluation results and training data, which can accelerate the evolution process of large models, significantly improve the agent's ability in multi-tool collaboration, deep reasoning, and other aspects, so as to better cope with complex tasks. Therefore, through dynamic data construction, ability boundary evaluation and iterative training, the application breaks the limitations of static test data sets and significantly improves the robustness and flexibility of the agent in dealing with complex tasks.

[0096] The above agent training method provided by the embodiments of the application can be applied in, but is not limited to, application scenarios involving agent training in fields such as e-commerce services, education services, legal services, medical services, conference services, social network services, financial product services, logistics services, and navigation services, for example: e-commerce service related agent training, education service related agent training, legal service related agent training, and the like, which are not limited here.

[0097] By adopting the embodiments of the application, the exploration direction is determined based on the existing knowledge of the agent system, wherein the exploration direction is used to plan and explore the target task to be generated, and the agent system includes multiple agents with different capability levels. Then, the multiple agents are controlled to execute the adversarial wandering strategy according to the exploration direction to generate test data and training data. After that, the multiple agents are evaluated for the ability boundary using the test data to obtain evaluation results. Finally, the to-be-evolved agent in the multiple agents is trained according to the evaluation results and the training data to obtain the target agent. Thus, the purpose of training an agent with high robustness is achieved, thereby realizing the technical effect of breaking the limitations of static test data sets through dynamic data construction, ability boundary evaluation and iterative training, significantly improving the robustness and flexibility of the agent in dealing with complex tasks, and further solving the technical problem of low robustness of the agent in the related art due to the dependence of the agent training and evaluation method on the relatively static test data set.

[0098] In an optional embodiment, in step S21, the exploration direction is determined based on the existing knowledge of the agent system, including the following method steps:

[0099] In step S211, the known task type, tool coverage range, model existing capability, and / or current service demand are determined based on the existing knowledge of the agent system.

[0100] Step S212, determining the exploration direction by using the known task type, tool coverage, model existing capability and / or current service demand.

[0101] In the embodiments of the present application, when determining the exploration direction based on the existing knowledge of the agent system, the known task type, tool coverage, model existing capability and / or current service demand can be determined based on the existing knowledge of the agent system.

[0102] The existing knowledge is a set of information and capabilities that the agent system has mastered before starting a new data construction and training process, and mainly includes the following four dimensions: known task type, tool coverage, model existing capability and / or current service demand.

[0103] The known task type refers to the types of tasks that the agent has encountered or successfully executed before, such as information retrieval, route planning, data analysis, etc., thereby helping the agent system to understand which fields it has certain processing capabilities.

[0104] The tool coverage refers to the types and functions of tools available in the current agent system, such as various API interfaces, database access permissions, external program calling capabilities, etc., and the tool coverage determines the means and scope of problem solving by the agents in the agent system.

[0105] The model existing capability refers to the current performance level of the agent model, such as including language understanding, logical reasoning, knowledge integration, etc.

[0106] The current service demand refers to the user requests or task scenarios recently received by the agent system, and the current service demand will affect the exploration direction, prompting the agent system to preferentially improve the agent capabilities in certain specific fields.

[0107] As can be seen, the present application will summarize and summarize the existing knowledge of the agent system, and clarify the known task type, tool coverage, model current capability level and / or task demand faced. Exemplarily, the ability status of the agent and the actual demand of the service object can be comprehensively understood through analysis of historical data, evaluation of model performance, etc.

[0108] Then, the exploration direction is determined by using the known task type, tool coverage, model existing capability and / or current service demand.

[0109] As can be seen, the present application will plan a development strategy that can maximize the performance improvement of the agent and meet the user demand based on the known task type, tool coverage, model existing capability and / or current service demand determined as described above, thereby determining the future exploration direction.

[0110] Thus, through the above steps, the direction of the agent capability improvement can be determined, blind training can be avoided, and the comprehensive capability of the agent can be improved more efficiently. Moreover, the exploration direction determination fully considers the task requirements and service scenarios, so that the evolution of the agent is closer to the actual application scenarios, and the agent can better solve user problems and meet market needs.

[0111] In an optional embodiment, the known task type is used to determine the task type currently known by the agent system, the tool coverage range is used to determine the capability coverage range of the multifunctional tool library, the model existing capability is used to determine the model capability currently possessed by the agent system, and the current service demand is used to determine the demand of the agent system for the current service scenario.

[0112] In the embodiments of the present application, the known task type is used to determine the task type currently known by the agent system, for example, including searching information, planning a trip, data analysis, emotion understanding, etc.

[0113] The tool coverage range is used to determine the capability coverage range of the multifunctional tool library, and the multifunctional tool library is a tool library pre-equipped for the agent system for calling and using by the agent system to solve specific or various complex tasks. The multifunctional tool library includes various APIs, data sources, algorithm models, etc.

[0114] The model existing capability is used to determine the model capability currently possessed by the agent system, for example, including language understanding, knowledge integration, logical reasoning, etc.

[0115] The current service demand is used to determine the demand of the agent system for the current service scenario, for example, including consultation of a specific industry, emergency response, new function request, etc.

[0116] In an optional embodiment, in step S22, the plurality of agents are controlled to execute the adversarial walk strategy according to the exploration direction to generate the test data and the training data, including the following method steps:

[0117] Step S221, the plurality of agents are controlled to execute the adversarial walk strategy according to the exploration direction to form a plurality of task trajectories.

[0118] Step S222, the test data and the training data are generated based on the plurality of task trajectories.

[0119] In the embodiments of the present application, when the plurality of agents are controlled to execute the adversarial walk strategy according to the exploration direction to generate the test data and the training data, the plurality of agents can be first controlled to execute the adversarial walk strategy according to the exploration direction to form a plurality of task trajectories.

[0120] The task trajectory is a behavior record of the agent in the process of performing the task, for example, including the decision of each step, tool calling, and final result, and the task trajectory can provide the behavior details of the agent when facing a complex task.

[0121] It can be seen that the application will control multiple agents to perform the adversarial wandering strategy according to the determined exploration direction. Since each agent has a different tool chain (i.e., a combination of tools and calling permissions), by letting each agent independently solve the same or a series of similar tasks, it can be able to produce differences in the execution process, so that some agents can find better solutions through higher-level tool chains, while some agents appear to be less efficient or fail due to tool or capability limitations. At the same time, during the execution of the adversarial wandering strategy, the decision path, tool usage, and final result of each agent are recorded, thereby forming multiple task trajectories.

[0122] Then, test data and training data are generated based on the multiple task trajectories. It can be seen that the application will generate test data and training data according to the multiple task trajectories formed, that is, test data and training data will be constructed according to the data of the multiple task trajectories. Exemplarily, test data and training data can be further generated by analyzing the action sequence, decision logic, and tool calling information in the task trajectory.

[0123] Thus, the task trajectory generated by the adversarial wandering strategy can analyze the behavior and ability of the agent in a complex task in detail, and provide a basis for subsequent evaluation and training strategies. Moreover, the upper limit and bottleneck of the agent's ability are automatically identified during the execution of the task, so that the training process can be designed accordingly to improve the comprehensive ability of the agent.

[0124] In an optional embodiment, in step S221, the multiple agents are controlled to perform the adversarial wandering strategy according to the exploration direction, and multiple task trajectories are formed, including the following method steps:

[0125] Step S2211, generating a task target based on the exploration direction.

[0126] Step S2212, controlling the multiple agents to perform the adversarial wandering strategy according to the task target, and forming multiple task trajectories.

[0127] In the embodiment of the application, when the multiple agents are controlled to perform the adversarial wandering strategy according to the exploration direction, and multiple task trajectories are formed, the task target can be generated based on the exploration direction first.

[0128] The task target can be understood as a specific task set under a given exploration direction, which aims to test the ability of the agent to solve a specific type of problem, or to guide the agent to learn new skills and strategies.

[0129] It can be seen that the present application will design a series of task targets according to the determined exploration direction, the design of the task targets needs to ensure the complexity and the diversity of performance cover multiple dimensions of the agent's ability, and the task targets should be a challenge to the existing ability of the agent, which can test the performance of the agent in a specific field or tool use, and also should consider the future service demand that the agent may face.

[0130] Then, the multiple agents are controlled to execute the adversarial wandering strategy according to the task target, and multiple task trajectories are formed.

[0131] It can be seen that the present application will control multiple agents to execute the adversarial wandering strategy according to the set task target, the agent will make decisions according to the task target, call different tools, and try to solve the task. In the execution process, the decision-making process, tool usage, intermediate calculation results and final output of each agent are recorded. Whether the task is successful or not, one or more task trajectories will be formed, which contains all the details of the agent solving the task, and provides rich information for subsequent analysis and learning.

[0132] Therefore, by allowing agents with different ability levels to try to solve the same task, the impact of ability difference on task solving can be intuitively seen, so that the existing ability of the agent can be more accurately evaluated. And the adversarial wandering strategy can actively explore the ability boundary of the agent, by comparing the performance of the agent under different tool chains, it can identify which tools and abilities are the key to the success of the agent, and indicate the direction for subsequent targeted improvement.

[0133] In an optional embodiment, in step S2212, the multiple agents are controlled to execute the adversarial wandering strategy according to the task target, and multiple task trajectories are formed, including the following method steps:

[0134] Step S22121, according to the task target, respectively control the multiple agents to execute the adversarial wandering strategy on the corresponding tool chain in the multiple tool chains, and form multiple task trajectories, wherein the multiple tool chains include: multiple first tool chains and multiple second tool chains, the multiple first tool chains are authorized to use the first category of tools in the multifunctional tool library, and the multiple second tool chains are authorized to use the second category of tools in the multifunctional tool library, the first category of tools has professional functional attributes, and the second category of tools has ordinary functional attributes.

[0135] In the present application, when the multiple agents are controlled to execute the adversarial wandering strategy according to the task target, and multiple task trajectories are formed, the multiple agents can be controlled to execute the adversarial wandering strategy on the corresponding tool chain in the multiple tool chains according to the task target, and multiple task trajectories are formed.

[0136] The multi-functional tool library can be understood as a repository of tools that includes both first category tools with specialized functional attributes and second category tools with ordinary functional attributes, aiming to provide the diversity of resources required for agents to perform complex tasks.

[0137] The plurality of tool chains includes a plurality of first tool chains and a plurality of second tool chains. The plurality of first tool chains can be understood as a plurality of strong tool chains, which are authorized to use the first category tools in the multi-functional tool library, and the first category tools have specialized functional attributes. That is, the plurality of first tool chains can be understood as consisting of a group of tools with specialized functional attributes, which often provide deeper data access, more accurate decision support, or more complex operational functions, such as financial data APIs, professional database queries, etc.

[0138] The plurality of second tool chains can be understood as a plurality of weak tool chains, which are authorized to use the second category tools in the multi-functional tool library, and the second category tools have ordinary functional attributes. That is, the plurality of second tool chains can be understood as consisting of a group of tools with ordinary functional attributes, which usually provide basic data retrieval and simple operational support, such as general search engines, basic file management tools, etc.

[0139] It can be seen that, for a specific task target generated, multiple agents will be deployed to execute an adversarial wandering strategy. Typically, the multiple agents will be divided into two groups, such as a first group of agents and a second group of agents, which use the first tool chain and the second tool chain respectively. Specifically, the first group of agents is authorized to use the first category tools in the multi-functional tool library, and attempts to complete the task target by calling the first category tools, and the behavior path of the first group of agents constitutes part of the task trajectory. The second group of agents is only allowed to use the second category tools in the multi-functional tool library, and also attempts to complete the task target by calling the second category tools, and the behavior path of the second group of agents constitutes another group of task trajectories. In the entire execution of the adversarial wandering strategy, the multiple agents execute their respective authorized tool chains, form their respective adversarial wanderings, and record each decision of the agent, each tool call and its result until the task is completed or a certain predetermined cutoff point is reached, thereby forming a plurality of detailed task trajectories. These task trajectories contain the complete process from the beginning to the end of the task, and are the key to evaluating the capabilities of the agents and generating training data.

[0140] Thus, by comparing between different groups of agents (i.e., the first group of agents and the second group of agents), the performance difference of the agents supported by different toolchains (i.e., the first toolchain and the second toolchain) can be intuitively observed, so as to evaluate the actual ability of the agents in handling complex tasks and the influence of tool selection on the performance of the agents. Moreover, the task trajectories provide complex scenarios and solutions encountered in the real world for model training, and especially when the first group of agents succeeds while the second group of agents fails, the differences contained in these task trajectories provide valuable cases for the model to learn how to use tools more effectively, and generate high-quality training data.

[0141] In an optional embodiment, in step S23, the capability boundary evaluation of the plurality of agents is performed by using the test data, and an evaluation result is obtained, including the following method steps:

[0142] In step S231, the plurality of agents are controlled to execute the test data, and an execution result is obtained, wherein the execution result is used to determine whether the plurality of agents successfully execute the toolchain test process.

[0143] In step S232, the capability boundary of the plurality of agents is evaluated according to the execution result, and an evaluation result is obtained.

[0144] In the embodiments of the present application, when the capability boundary evaluation of the plurality of agents is performed by using the test data, and an evaluation result is obtained, the plurality of agents can be controlled to execute the test data first, and an execution result is obtained, wherein the execution result is used to determine whether the plurality of agents successfully execute the toolchain test process.

[0145] The execution result refers to the output of the agent after completing the task in the test data, including a success result or a failure result, and the quality and efficiency of task completion in the case of success, etc., and the execution result is used to determine whether the plurality of agents successfully execute the toolchain test process.

[0146] As can be seen, the present application will control the plurality of agents to execute the generated test data, and the test data contains tasks designed based on various real challenges that may be encountered in a complex multi-agent system. Each agent will try to solve the task according to the toolchain and capability level it has, so as to record the detailed behavior and result of the agent when handling each task, and obtain an execution result.

[0147] For example, the recorded content includes whether the agent can understand and analyze the task requirements, the solution strategy and the toolchain selected by the agent, the challenges and errors encountered in the execution process, whether the task is finally successfully completed, and the speed and quality of completing the task, etc.

[0148] Thus, the execution result provides a direct basis for the capability evaluation of the agent, and the execution result not only reflects the performance of the agent on a specific task, but also reflects the strategy and logic of the agent in solving the problem, which is the key information for evaluating the capability boundary of the agent.

[0149] Then, the capability boundaries of the plurality of agents are evaluated according to the execution result to obtain an evaluation result. It can be understood that after collecting the execution results of all the agents, the capability boundary evaluation stage is entered, and the capability boundaries of the plurality of agents are evaluated according to the execution result. Exemplarily, the evaluation process includes, for example, determining the success rate of each agent in completing the task, identifying the agent and tool chain combination with a higher success rate and the agent and tool chain combination with a lower success rate. The evaluation process also includes, for example, analyzing the divergence point in the execution process, especially the point at which the strong tool chain agent can succeed while the weak tool chain agent fails, and the divergence point can reveal the capability boundary of the model. The evaluation process also includes, for example, quantifying the degree of dependence of the agent on a specific type of task or tool chain, and understanding which tool chain or capability is indispensable for completing a high-difficulty task. The evaluation process also includes, for example, evaluating the robustness of the agent when dealing with unknown or edge tasks, i.e., the reaction and recovery ability of the agent when facing challenges beyond the conventional range.

[0150] The evaluation result provides a quantitative indicator of the capability of the agent, and also reflects the pattern of capability distribution in the agent system, which can effectively distinguish high-capability models from low-capability models, and provide guidance for further optimizing the design of the agent and the configuration of the tool chain, and formulating more effective training strategies.

[0151] Thus, through the execution of the test data and the analysis of the results, the capability of the agent in dealing with complex tasks can be accurately evaluated, the adaptability of the agent to different tool chains and task types can be distinguished, and the real capability boundary of the agent can be revealed.

[0152] In an optional embodiment, in step S24, the to-be-evolved agent in the plurality of agents is trained according to the evaluation result and the training data to obtain a target agent, including the following method steps:

[0153] Step S241, adjusting the exploration direction according to the evaluation result to obtain an adjusted exploration direction.

[0154] Step S242, labeling the process reward for the training data according to the evaluation result to obtain labeled data.

[0155] Step S243, based on the adjusted exploration direction and the labeled data, performing process-guided reinforcement learning training on the to-be-evolved agent in the plurality of agents to obtain a target agent.

[0156] In the embodiments of the present application, when the to-be-evolved agent in the plurality of agents is trained according to the evaluation result and the training data to obtain the target agent, the exploration direction can be adjusted according to the evaluation result to obtain an adjusted exploration direction, and the process reward is labeled for the training data according to the evaluation result to obtain labeled data.

[0157] In reinforcement learning, the process reward can be understood as positive feedback given to the key decisions of the agent in the task execution process in addition to the achievement of the final goal, to encourage the agent to take more effective and reasonable action paths.

[0158] As can be seen, the exploration direction is adjusted according to the aforementioned evaluation result to ensure that the training and evolution of the agent are more focused on key areas, and an adjusted exploration direction is obtained, that is, the exploration direction is dynamically adjusted according to subsequent discoveries. For example, according to the evaluation result, a task area with challenge and distinction can be identified. For example, if the evaluation result indicates that the agent performs poorly in processing a certain type of task, the adjusted exploration direction will focus more on generating trajectories of such tasks to fill the gaps in the model's capabilities.

[0159] At the same time, the process reward is labeled for the training data according to the evaluation result to obtain labeled data. For example, by conducting in-depth analysis of the training data, especially focusing on the decision points of the agent in the task solving process, process reward labels can be added to these decision points to generate labeled data. The introduction of process rewards can not only focus on the final result of the task during the training of the agent, but also encourage the agent to take more effective and reasonable decision paths during the task solving process, thereby improving its overall ability to handle complex tasks.

[0160] Then, based on the adjusted exploration direction and the labeled data, the to-be-evolved agent in the plurality of agents is trained by process-guided reinforcement learning to obtain the target agent.

[0161] As can be seen, the to-be-evolved agent in the plurality of agents is trained by process-guided reinforcement learning based on the adjusted exploration direction and the training data with process reward labels, that is, the key decision points are supervised and guided during execution, so that the agent not only learns how to use the tool chain to solve the task, but also learns the optimal decision in the decision-making process. By continuously receiving process rewards, the agent can gradually accumulate experience, optimize its decision logic, and improve its intelligence level in complex environments. After sufficient training, the agent reaches the expected ability standard and obtains the target agent, so as to more efficiently and accurately execute complex tasks.

[0162] Therefore, by adjusting the direction of exploration, the training process becomes more focused on the agent's weaknesses and limitations, ensuring the targeted and effective improvement of capabilities. Furthermore, the introduction of process rewards allows the reinforcement learning training strategy to be dynamically adjusted according to the actual situation of task execution, encouraging the agent to make better decisions, accelerating its learning process, and thus enhancing the agent's adaptability and stability in the face of uncertainty and complex environments.

[0163] In an optional embodiment, step S222 involves generating training data based on multiple task trajectories, including the following method steps:

[0164] Step S2221: Based on the successful execution results of multiple task trajectories on multiple first toolchains, generate positive training samples, and based on the failed execution results of multiple task trajectories on multiple second toolchains, generate negative training samples.

[0165] Step S2222: Determine training data based on positive and negative training samples.

[0166] In this embodiment of the application, when generating training data based on multiple task trajectories, positive training samples can be generated first based on the successful execution results of multiple task trajectories on multiple first toolchains, and negative training samples can be generated based on the failed execution results of multiple task trajectories on multiple second toolchains.

[0167] Positive training samples are examples extracted from the trajectories of successful task execution using the first toolchain. These positive samples represent patterns in which the agent correctly uses the toolchain to complete tasks, helping the model learn correct toolchain coordination and complex task-solving strategies. Negative training samples are examples extracted from the trajectories of unsuccessful task execution using the second toolchain. These negative samples represent common errors and challenges encountered by the agent when facing complex tasks, helping the model learn to avoid errors and thus improve its robustness. For example, negative training samples include failed execution results (i.e., completely erroneous task execution) as well as failed execution results that generate ambiguous, unstructured, and poor-quality answers (i.e., a special type of failed execution result), which fail to meet the expected quality standards and level of structure, thus failing to meet the specific requirements of the task or demonstrate the agent's advanced skills.

[0168] As can be seen, this application generates positive and negative training samples based on the results of the agent executing multiple task trajectories. Furthermore, for successful execution results of agents in the first toolchain (e.g., strong toolchain agents), their tool calls, decision paths, and final results are extracted as positive training samples, providing the model with examples for learning correct tool usage and efficient problem-solving strategies. For failed execution results of agents in the second toolchain (e.g., weak toolchain agents), negative training samples are generated based on the failed execution results, providing the model with training material to learn how to avoid the same errors and how to find alternative solutions under limited resource conditions.

[0169] Then, the training data is determined based on the positive and negative training samples. It can be seen that this application combines the positive and negative training samples to determine the final training data. For example, when determining the training data, it is necessary not only to ensure the balance of the training data, i.e., the appropriate number of positive and negative training samples, but also to design a format and structure suitable for model learning based on the information in the training samples, in order to improve the model training effect.

[0170] Therefore, positive training samples provide the model with efficient problem-solving demonstrations, helping the agent learn how to rationally invoke and coordinate toolchains in complex environments, thereby improving the success rate and efficiency of task execution. Negative training samples reflect common failure modes and limitations in tool use when the agent handles complex tasks, helping the model learn how to avoid these errors and improve its robustness in the face of challenges. By integrating positive and negative training samples, this application can construct more balanced and comprehensive training data, covering the full spectrum from successful experiences to lessons learned from failures, thereby helping the agent achieve comprehensive capability improvement.

[0171] In one optional embodiment, the agent training method further includes the following method steps:

[0172] Expand the functional attributes of existing tools in the multi-functional tool library, and / or expand tool categories not covered in the multi-functional tool library.

[0173] In this embodiment of the application, the functional attributes of existing tools in the multi-functional tool library can also be expanded. By expanding the functional attributes of existing tools, the performance, functions or scope of application of existing tools can be improved.

[0174] For example, expanding the functional attributes of existing tools can include adding parameter options, improving data filtering and processing capabilities, and expanding data sources. Specifically, this can be achieved by introducing more complex algorithms or models, such as deep learning techniques, to enhance the tool's decision support capabilities. Furthermore, increasing connections to external data sources, using real-time updated database or API query results as input, can improve the timeliness and accuracy of information. In addition, the tool's user interface can be improved to make it easier for intelligent agents to understand and use.

[0175] And / or, expand the tool categories not covered in the multi-functional tool library by introducing new tool categories to fill service areas or functional needs that existing tools cannot cover. For example, if the current multi-functional tool library lacks tools for image processing or natural language understanding, corresponding APIs or software interfaces can be added.

[0176] For example, when expanding the multi-functional toolkit, image recognition and analysis tools can be added to process visual information. Speech recognition and synthesis tools can also be added to enhance the agent's voice interaction capabilities. Domain-specific knowledge base interfaces, such as databases in the medical and legal fields, can be added to provide more in-depth answers to specialized questions. Automated process tools can also be added to integrate multiple services or operations, thereby improving the agent's efficiency in handling sequential tasks. The specific expansion content depends on the actual situation and is not limited here.

[0177] Therefore, by expanding the functional attributes of existing tools, intelligent agents can invoke more powerful and flexible tools, thereby handling a wider range of more complex tasks within the intelligent agent system and enhancing its competitiveness. Expanding the multi-functional tool library enables it to support more diverse task requirements, thus broadening the applicability of intelligent agents and better serving real-world scenarios.

[0178] Figure 3 This is a flowchart of a data construction according to an embodiment of this application, such as... Figure 3As shown, firstly, a complex intelligent agent system is built and initialized. A rich and diverse tool library is constructed for the intelligent agent system, containing both general-purpose basic tools and a series of powerful specialized APIs. Next, the intelligent agent system does not blindly begin generating tasks. Instead, it plans a general exploration direction based on existing knowledge (such as known task types, tool coverage, and the model's current capabilities). For example, it might focus on generating complex tasks that require multi-step reasoning combining real-time financial data and geographic location information. The process of planning the exploration direction is itself an iterative planning and exploration process, and the exploration direction will be dynamically adjusted based on subsequent discoveries.

[0179] Then, an adversarial walk strategy is implemented. Based on the exploration direction, the agent system generates a high-order task objective and instantiates multiple "players" with different capabilities to solve this objective. Further, the "players" include multiple strong toolchain agents, i.e., a series of "expert" agents. Strong toolchain agents are authorized to use efficient and specialized tools from the tool library, i.e., expert toolchains. In addition, strong toolchain agents can also use general toolchains and weak toolchains. For example, when asked to plan a route including real-time traffic conditions, a strong toolchain agent can directly call a map API. The "players" also include multiple weak toolchain agents, i.e., a series of "novice" agents. Weak toolchain agents are restricted to using only some basic and general tools, i.e., general toolchains and weak toolchains, such as only being able to use a basic search engine. After instantiating multiple "players" with different capabilities, the multiple "players" will begin an adversarial walk, i.e., simultaneously attempting to complete the same task using their respective authorized tools.

[0180] Next, the divergence and the generation of difficult tasks are identified. This involves closely monitoring the behavior and outcomes of the two groups of agents during the adversarial process, identifying key signals arising from the divergence. The appearance of a divergence indicates that the strong toolchain agent has obtained a more difficult answer. Here, divergence refers to the strong toolchain agent successfully deriving a precise, structured, and high-quality answer using its specialized tools (e.g., through an "expert toolchain" path), while the weak toolchain agent, due to tool limitations, either completely fails the task (i.e.,...). Figure 3 The "X" on the middle path can only provide a vague, unstructured, and poor-quality answer. Therefore, when such a divergence is identified, it is considered a successful "capture" of a high-quality, valuable difficult task, because the task effectively distinguishes high-capability models from low-capability models.

[0181] Finally, the data trajectory is formed and iteratively optimized, creating a closed loop. That is, the characteristics of newly discovered challenging tasks are fed back into the steps of planning the exploration direction. For example, if "combining API query results with geographic information" is found to be a task type that easily leads to divergence, the next iteration will focus more on generating and exploring variations of this task type, thereby continuously exploring the model's capability boundaries and consistently producing high-quality, highly complex training and testing data.

[0182] In summary, this application can dynamically and automatically generate tasks. Furthermore, this automation does not rely on static datasets but rather utilizes an adversarial strategy of "strong and weak toolchains" to automatically generate and execute complex task trajectories. The method in this application proactively explores and discovers the "capability boundaries" of the model, continuously generating new, high-quality evaluation data, rather than being limited to a fixed question bank.

[0183] This application provides a dynamically adaptable and scalable framework that can seamlessly incorporate new tools into the adversarial process as the tool library expands, based on an automated generation strategy. This generates more challenging task chains that match the new capabilities, ensuring that the difficulty of evaluation and training grows in sync with the model's capabilities and remains cutting-edge.

[0184] The method presented in this application can not only evaluate the model but also train it. Through an adversarial mechanism, the method can continuously generate challenging hard samples. These hard samples (i.e., task trajectories that strong toolchain agents can solve but weak toolchain agents can fail) help train a more robust and powerful model, aiming to drive the evolution of the model's capabilities, rather than simply measuring its current level.

[0185] Specifically, this application abandons the traditional methods of manually designing or generating tasks based on templates. Instead, it sets up multiple strong toolchain agents (expert-level agents that can call all professional APIs) and weak toolchain agents (novice-level agents that can only use basic tools), allowing the two groups of agents to engage in adversarial problem-solving explorations towards the same fuzzy objective. When the strong toolchain agent succeeds while the weak toolchain agent fails, the system automatically identifies a high-quality, difficult task. This mechanism of this application can accurately locate scenarios that demonstrate the model's advanced capabilities (such as deep reasoning and multi-tool collaboration), thereby systematically discovering the model's capability boundaries.

[0186] The method presented in this application does not involve isolated tool calls, but rather focuses on the complete toolchain execution process. The final generated data is a complete trajectory that includes the objective, high-level planning, multi-step reasoning, the inputs and outputs of each tool call, and the final result. This trajectory data not only contains "how to do it right" but also records "why it went wrong" through the failures of the weak toolchain, providing the model with highly informative learning samples containing both positive and negative examples, which far exceeds the simple "question-answer" pairs in traditional benchmarks.

[0187] Furthermore, addressing the issues of sparse final rewards and unstable training in complex tasks, this application proposes an improved reinforcement learning method. This method not only rewards successful task completion but, more importantly, supervises and guides key decision points during execution. By rewarding the execution process, the learning efficiency and stability of the model in long-chain, complex tasks can be significantly improved. This helps the model learn the correct "thinking process" rather than simply "guessing" the final answer, thus solving the core pain point of training complex intelligent agents.

[0188] It is easy to understand that the beneficial effects of the agent training method provided in this application include the following points.

[0189] Beneficial effects (1) This application automatically generates task trajectories, covering complex problems that require deep logical reasoning and multi-tool collaboration. These dynamic task scenarios driven by the toolchain can more realistically reflect the model's ability to solve complex problems, thus providing a more advanced evaluation framework and effectively overcoming the shortcomings of lagging evaluation benchmarks in traditional solutions.

[0190] Beneficial effect (2): This application automatically generates challenging and diverse task scenarios through the automated strategy of "strong and weak toolchain confrontation". This method gets rid of the limitations of manual construction by experts and can dynamically generate new tasks as the tool library is updated and the capabilities of the agent evolve, increasing the diversity and scalability of task scenarios, thereby overcoming the shortcomings of the limited task construction method in traditional schemes.

[0191] Beneficial effects (3) The data construction process of this application dynamically generates high-quality training data containing positive and negative examples based on the real behavior of the agent when solving complex tasks. These data not only contain successful cases, but also record the failure patterns, thus providing the model with more comprehensive and realistic learning materials, improving the complexity and diversity of training data, enhancing the effectiveness and robustness of model training, and overcoming the shortcomings of the limited training data in traditional schemes.

[0192] Beneficial effect (4): Through a continuous adversarial walk strategy, the agent actively explores and identifies its own capability boundaries during the execution of tasks. When there is a divergence between strong and weak toolchains, it can automatically capture and analyze these divergence points, generate targeted difficult samples, help the agent break through the limitations of its existing capabilities, and gradually expand its capability boundaries, thereby overcoming the shortcomings of insufficient capability boundary exploration in traditional solutions.

[0193] Beneficial Effects (5): The data construction and training process of this application has good adaptability and foresight. As new tools are continuously added to the tool library, the task construction strategy can be automatically adjusted to generate complex tasks that match the new tools. At the same time, through continuous evaluation and training, the agent can continuously evolve to adapt to the updates of the tool library and the development of its own capabilities, ensuring the continuous innovation of the evaluation and training framework and the continuous progress of the agent system, thereby overcoming the shortcomings of lack of adaptability and foresight in traditional solutions.

[0194] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0195] Furthermore, it should be noted that, for the sake of simplicity, the aforementioned method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Secondly, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0196] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, they can also be implemented by hardware. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0197] According to embodiments of this application, the following are also provided: Figure 4The data processing method shown. Figure 4 This is a flowchart of a data processing method according to an embodiment of this application. Figure 4 As shown, the method may include the following steps:

[0198] Step S41: Obtain problem data.

[0199] Step S42: The target agent processes the question data to generate an answer, thus obtaining the target response. The target agent is generated according to any of the agent training methods described above.

[0200] In this embodiment, after the target agent is trained, it can be applied in data processing scenarios. First, problem data is acquired. This problem data can be a specific problem or request submitted by a user to the agent system for resolution. For example, the problem data can be of various types, such as information query, task execution, and decision support.

[0201] Then, a target agent is used to process the question data to generate answers, resulting in the target response. The target agent is an agent generated using the aforementioned agent training method, possessing high capabilities and robustness in solving complex problems.

[0202] By processing the problem data through the target intelligent agent, the target intelligent agent gains a deeper understanding of the problem data, identifies the key elements and potential needs of the problem data, and can then provide more accurate target responses to users, greatly improving user experience and service satisfaction.

[0203] For a detailed description, please refer to the foregoing description of the embodiments.

[0204] The data processing method provided in this application embodiment can be applied, but is not limited to, to application scenarios involving agent training in fields such as e-commerce services, education services, legal services, medical services, conference services, social networking services, financial product services, logistics services, and navigation services. For example, agent training related to e-commerce services, agent training related to education services, agent training related to legal services, etc., are not limited here.

[0205] By adopting the embodiments of this application, problem data is obtained, and a target agent generated according to any of the above-mentioned agent training methods is used to generate answers to the problem data to obtain the target response. This achieves the goal of providing users with a more convenient and accurate solution. It also realizes the technical effect of breaking the limitation of static test datasets through dynamic data construction, capability boundary evaluation and iterative training, significantly improving the robustness and flexibility of agents in handling complex tasks. This solves the technical problem in related technologies where the robustness of agents is low because the agent training and evaluation methods rely on relatively static test datasets.

[0206] According to embodiments of this application, the following are also provided: Figure 5 The data processing method shown. Figure 5 This is a flowchart of another data processing method according to an embodiment of this application. For example... Figure 5 As shown, the method may include the following steps:

[0207] Step S51: Obtain intelligent customer service consultation data.

[0208] Step S52: The intelligent customer service agent processes the intelligent customer service inquiry data to generate answers, thus obtaining the intelligent customer service inquiry response. The intelligent customer service agent is generated according to any of the agent training methods described above.

[0209] In this embodiment, after the target intelligent agent is trained, it can be applied in customer service consultation scenarios. First, intelligent customer service consultation question data is acquired. This data can be inquiries or problems raised by users through the intelligent customer service system. For example, the intelligent customer service consultation question data can involve various types such as product information, order status, troubleshooting, and operation guides.

[0210] Then, an intelligent customer service agent is used to process the data of intelligent customer service inquiries and generate answers, resulting in intelligent customer service responses. The intelligent customer service agent is generated using the aforementioned agent training method and is specifically designed to handle customer inquiries. This agent can invoke appropriate service tools to generate accurate responses and resolve user issues.

[0211] For a detailed description, please refer to the foregoing description of the embodiments.

[0212] The data processing method provided in this application embodiment can be applied, but is not limited to, to application scenarios involving agent training in fields such as e-commerce services, education services, legal services, medical services, conference services, social networking services, financial product services, logistics services, and navigation services. For example, agent training related to e-commerce services, agent training related to education services, agent training related to legal services, etc., are not limited here.

[0213] By adopting the embodiments of this application, intelligent customer service consultation question data is obtained, and an intelligent customer service agent generated according to any of the above-mentioned agent training methods is used to generate answers to the intelligent customer service consultation question data, thereby obtaining intelligent customer service consultation responses. This achieves the goal of providing users with more convenient and accurate solutions. It also realizes the technical effect of breaking the limitation of static test datasets through dynamic data construction, capability boundary evaluation and iterative training, significantly improving the robustness and flexibility of the agent in handling complex tasks, and thus solving the technical problem in related technologies where the agent training and evaluation methods rely on relatively static test datasets, resulting in low agent robustness.

[0214] According to embodiments of this application, the following are also provided: Figure 6 The data processing method shown. Figure 6 This is a flowchart of another data processing method according to an embodiment of this application. For example... Figure 6 As shown, the method may include the following steps:

[0215] Step S61: Obtain a data processing request through the first application programming interface, wherein the request data carried in the data processing request includes: problem data.

[0216] Step S62: Return a data processing response through the second application programming interface. The response data carried in the data processing response includes: the target response, which is obtained by the target agent generating an answer to the question data. The target agent is generated according to any of the agent training methods mentioned above.

[0217] In this embodiment, the first Application Programming Interface (API) and the second API can be the same API or different APIs. In an optional embodiment, the interface parameters in the first and second APIs may include, but are not limited to: global interface identifier, interface signature key, interface timestamp, interface request identifier, system call credential identifier, etc. The first API can use GET or POST as the interface request method to obtain the file processing request. The second API can use JSON format to return the file processing response.

[0218] A data processing request can be a request sent by an external system or application to the intelligent agent system through a first application programming interface, requesting the intelligent agent system to process the problem data carried in the data processing request.

[0219] The data processing response is the processing result returned by the intelligent agent system to the requester through the second application programming interface. The data processing response carries the target response. The target response is the answer or solution generated by the target intelligent agent for the problem data, which aims to meet the needs of the external requester.

[0220] The target agent is the agent generated according to the above agent training method, which has the ability to handle complex problems and generate high-quality responses.

[0221] For a detailed description, please refer to the foregoing description of the embodiments.

[0222] The data processing method provided in this application embodiment can be applied, but is not limited to, to application scenarios involving agent training in fields such as e-commerce services, education services, legal services, medical services, conference services, social networking services, financial product services, logistics services, and navigation services. For example, agent training related to e-commerce services, agent training related to education services, agent training related to legal services, etc., are not limited here.

[0223] By employing the embodiments of this application, a data processing request is obtained through a first application programming interface (API), wherein the request data carried in the data processing request includes: question data. Then, a data processing response is returned through a second API, wherein the response data carried in the data processing response includes: a target response, which is obtained by a target agent generating an answer to the question data. The target agent is generated according to any of the above-mentioned agent training methods. This achieves the goal of providing users with a more convenient and accurate solution, thereby breaking the limitations of static test datasets through dynamic data construction, capability boundary assessment, and iterative training. This significantly improves the robustness and flexibility of the agent in handling complex tasks, thus solving the technical problem in related technologies where the agent training and evaluation methods rely on relatively static test datasets, resulting in low agent robustness.

[0224] According to embodiments of this application, the following are also provided: Figure 7 The data processing method shown. Figure 7 This is a flowchart of another data processing method according to an embodiment of this application. For example... Figure 7 As shown, the method may include the following steps:

[0225] Step S71: Obtain the currently input data processing dialogue request, wherein the request data carried in the data processing dialogue request includes: question data.

[0226] Step S72: In response to the data processing dialogue request, return a data processing dialogue response, wherein the information carried in the data processing dialogue response includes: target response, which is obtained by the target agent after processing the question data to generate an answer, and the target agent is generated according to any of the above agent training methods.

[0227] Step S73: Display the target response within the graphical user interface.

[0228] In this embodiment of the application, the data processing dialogue request can be a request issued by a user or an external system through an intelligent customer service platform, requesting the intelligent customer service agent to process the question data contained in the request in order to generate an answer or solution.

[0229] The data processing dialogue response is the reply made by the intelligent customer service agent to the data processing dialogue request. It includes the target response and is a direct response to the external request.

[0230] The graphical user interface (GUI) is the user interaction interface used to display the responses of the intelligent customer service agent. Users can view, understand, and interact with the responses through this interface.

[0231] For a detailed description, please refer to the foregoing description of the embodiments.

[0232] The data processing method provided in this application embodiment can be applied, but is not limited to, to application scenarios involving agent training in fields such as e-commerce services, education services, legal services, medical services, conference services, social networking services, financial product services, logistics services, and navigation services. For example, agent training related to e-commerce services, agent training related to education services, agent training related to legal services, etc., are not limited here.

[0233] By employing the embodiments of this application, a data processing dialogue request is obtained from the current input, wherein the request data carried in the data processing dialogue request includes: question data. Then, in response to the data processing dialogue request, a data processing dialogue response is returned, wherein the information carried in the data processing dialogue response includes: a target response, which is obtained by the target agent performing answer generation processing on the question data. The target agent is generated according to any of the above-mentioned agent training methods. Finally, the target response is displayed in the graphical user interface, thereby achieving the goal of providing users with a more convenient and accurate solution. This realizes the technical effect of breaking the limitation of static test datasets through dynamic data construction, capability boundary evaluation and iterative training, significantly improving the robustness and flexibility of agents in handling complex tasks, and thus solving the technical problem in related technologies where the agent training and evaluation methods rely on relatively static test datasets, resulting in low agent robustness.

[0234] According to embodiments of this application, the following are also provided: Figure 8 The data processing method shown. Figure 8 This is a flowchart of another data processing method according to an embodiment of this application. For example... Figure 8 As shown, the method may include the following steps:

[0235] Step S81: In response to the input command applied to the operation interface, display the problem data on the operation interface.

[0236] Step S82: In response to the processing command applied to the operation interface, the target response is displayed on the operation interface. The target response is obtained by the target agent processing the question data to generate an answer, and the target agent is generated according to any of the agent training methods described above.

[0237] In this embodiment of the application, the operation interface is the interface through which the user interacts with the intelligent agent system. Through this operation interface, the user can input commands, query questions, receive replies, etc., which has high operational flexibility and responsiveness.

[0238] Input commands can be initiated by the user on the user interface, requesting the intelligent agent system to display or pay attention to specific data. Input commands may include clicks, touches, voice commands, etc., depending on the user's specific needs and operating habits.

[0239] Processing instructions can be commands from users to instruct the intelligent agent system to process previously displayed question data and generate corresponding responses. These instructions can be issued in various ways, and the intelligent agent system will then process and generate the answer upon receiving the instruction.

[0240] For a detailed description, please refer to the foregoing description of the embodiments.

[0241] The data processing method provided in this application embodiment can be applied, but is not limited to, to application scenarios involving agent training in fields such as e-commerce services, education services, legal services, medical services, conference services, social networking services, financial product services, logistics services, and navigation services. For example, agent training related to e-commerce services, agent training related to education services, agent training related to legal services, etc., are not limited here.

[0242] In this embodiment, by responding to input commands applied to the operation interface, problem data is displayed on the operation interface, and then, in response to processing commands applied to the operation interface, the target response is displayed on the operation interface. The target response is obtained by a target agent processing the problem data to generate an answer. The target agent is generated according to any of the agent training methods described above. This achieves the goal of providing users with a more convenient and accurate solution. It breaks through the limitations of static test datasets through dynamic data construction, capability boundary assessment, and iterative training, significantly improving the robustness and flexibility of the agent in handling complex tasks. This solves the technical problem in related technologies where the agent training and evaluation methods rely on relatively static test datasets, resulting in low agent robustness.

[0243] According to embodiments of this application, the following are also provided: Figure 9 The data processing system shown. Figure 9 This is a schematic diagram of the structure of a data processing system according to an embodiment of this application. Figure 9 As shown, the system may include:

[0244] The client is used to send problem data.

[0245] The server connects to the client and is used to process the question data using a target agent to generate an answer and obtain the target response. The target agent is generated according to any of the agent training methods mentioned above.

[0246] The client is also used to output the target response.

[0247] In this embodiment, the data processing system is used to execute the data processing method described above, and for details, please refer to the foregoing description of the embodiments.

[0248] The data processing system provided in this application embodiment can be applied, but is not limited to, to application scenarios involving agent training in fields such as e-commerce services, education services, legal services, medical services, conference services, social networking services, financial product services, logistics services, and navigation services. For example, agent training related to e-commerce services, agent training related to education services, agent training related to legal services, etc., are not limited here.

[0249] By employing the embodiments of this application, a data processing method is executed through a data processing system, thereby achieving the goal of providing users with a more convenient and accurate solution. This achieves the technical effect of breaking the limitations of static test datasets through dynamic data construction, capability boundary assessment, and iterative training, significantly improving the robustness and flexibility of intelligent agents in handling complex tasks. In turn, it solves the technical problem in related technologies where the robustness of intelligent agents is low due to the reliance of intelligent agent training and evaluation methods on relatively static test datasets.

[0250] It should be noted that the preferred implementation of this embodiment can be found in the relevant descriptions in the embodiments, and will not be repeated here.

[0251] According to an embodiment of this application, an embodiment of an apparatus for implementing the above-described agent training method is also provided. Figure 10 This is a schematic diagram of the structure of an intelligent agent training device according to an embodiment of this application, as shown below. Figure 10 As shown, the device includes:

[0252] The determination module 1001 is used to determine the exploration direction based on the existing knowledge of the intelligent agent system. The exploration direction is used to plan and explore the target task to be generated. The intelligent agent system includes multiple intelligent agents with different capability levels.

[0253] The generation module 1002 is used to control multiple agents to execute adversarial walking strategies according to the exploration direction, and to generate test data and training data.

[0254] The evaluation module 1003 is used to evaluate the capability boundaries of multiple agents using test data and obtain evaluation results.

[0255] Training module 1004 is used to train the agent to be evolved among multiple agents based on the evaluation results and training data to obtain the target agent.

[0256] Optionally, the aforementioned determining module 1001 is further configured to: determine known task types, tool coverage, existing model capabilities, and / or current service requirements based on the existing knowledge of the intelligent agent system; and determine the exploration direction using the known task types, tool coverage, existing model capabilities, and / or current service requirements.

[0257] Optionally, the known task type is used to determine the currently known task types of the intelligent agent system, the tool coverage is used to determine the capability coverage of the multi-functional tool library, which is a tool library pre-equipped for the intelligent agent system, the existing model capabilities are used to determine the model capabilities currently possessed by the intelligent agent system, and the current service requirements are used to determine the requirements of the intelligent agent system for the current service scenario.

[0258] Optionally, the generation module 1002 is further configured to: control multiple agents to execute adversarial walking strategies according to the exploration direction, forming multiple task trajectories; and generate test data and training data based on the multiple task trajectories.

[0259] Optionally, the generation module 1002 is further configured to: generate task objectives based on the exploration direction; and control multiple agents to execute adversarial walking strategies according to the task objectives, thereby forming multiple task trajectories.

[0260] Optionally, the generation module 1002 is further configured to: control multiple agents to execute adversarial walk strategies on corresponding toolchains in multiple toolchains according to the task objective, thereby forming multiple task trajectories. The multiple toolchains include: multiple first toolchains and multiple second toolchains. The multiple first toolchains are authorized to use first-category tools in the multi-functional tool library, and the multiple second toolchains are authorized to use second-category tools in the multi-functional tool library. The first-category tools have professional functional attributes, and the second-category tools have ordinary functional attributes.

[0261] Optionally, the evaluation module 1003 is further configured to: control multiple agents to execute test data and obtain execution results, wherein the execution results are used to determine whether the multiple agents have successfully executed the toolchain testing process; and evaluate the capability boundaries of the multiple agents based on the execution results to obtain evaluation results.

[0262] Optionally, the training module 1004 is further configured to: adjust the exploration direction based on the evaluation results to obtain the adjusted exploration direction; reward the training data labeling process based on the evaluation results to obtain labeled data; and perform process-guided reinforcement learning training on the agent to be evolved among multiple agents based on the adjusted exploration direction and labeled data to obtain the target agent.

[0263] Optionally, the generation module 1002 is further configured to: generate positive training samples based on the successful execution results of multiple task trajectories on multiple first toolchains, and generate negative training samples based on the failed execution results of multiple task trajectories on multiple second toolchains; and determine training data based on the positive training samples and negative training samples.

[0264] Optionally, the device further includes an expansion module for: expanding the functional attributes of existing tools in the multi-function tool library, and / or expanding tool categories not covered in the multi-function tool library.

[0265] This application's embodiments determine the exploration direction based on existing knowledge of the agent system. The exploration direction is used to plan and explore the target task to be generated. The agent system includes multiple agents with different capability levels. Then, according to the exploration direction, the multiple agents are controlled to execute adversarial walk strategies, generating test data and training data. Next, the capability boundaries of the multiple agents are evaluated using the test data, obtaining evaluation results. Finally, based on the evaluation results and training data, the agent to be evolved among the multiple agents is trained to obtain the target agent. This achieves the goal of training a highly robust agent, thus breaking the limitations of static test datasets through dynamic data construction, capability boundary evaluation, and iterative training. This significantly improves the robustness and flexibility of the agent in handling complex tasks, thereby solving the technical problem in related technologies where agent training and evaluation methods rely on relatively static test datasets, resulting in low agent robustness.

[0266] It should be noted that the aforementioned determining module 1001, generating module 1002, evaluating module 1003, and training module 1004 correspond to steps S21 to S24 in the embodiments. The four modules and their corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the above embodiments. It should be noted that the above modules or units can be hardware or software components stored in memory and processed by one or more processors, and the above modules can also run in the server 10 provided in the embodiments.

[0267] According to an embodiment of this application, an apparatus embodiment for implementing the above-described data processing method is also provided. Figure 11 This is a schematic diagram of another data processing apparatus according to an embodiment of this application, such as... Figure 11 As shown, the device includes:

[0268] The first acquisition module 1101 is used to acquire problem data;

[0269] The second generation module 1102 is used to process the question data using a target agent to generate an answer and obtain a target response; wherein the target agent is generated according to any of the agent training methods mentioned above.

[0270] By adopting the embodiments of this application, problem data is obtained, and a target agent generated according to any of the above-mentioned agent training methods is used to generate answers to the problem data to obtain the target response. This achieves the goal of providing users with a more convenient and accurate solution. It also realizes the technical effect of breaking the limitation of static test datasets through dynamic data construction, capability boundary evaluation and iterative training, significantly improving the robustness and flexibility of agents in handling complex tasks. This solves the technical problem in related technologies where the robustness of agents is low because the agent training and evaluation methods rely on relatively static test datasets.

[0271] It should be noted that the first acquisition module 1101 and the second generation module 1102 mentioned above correspond to steps S41 and S42 in the embodiments. The two modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the above embodiments. It should be noted that the above modules or units can be hardware components or software components stored in memory and processed by one or more processors. The above modules can also run in the server 10 provided in the embodiments.

[0272] According to an embodiment of this application, another embodiment of an apparatus for implementing the above-described data processing method is also provided. Figure 12 This is a schematic diagram of another data processing apparatus according to an embodiment of this application, such as... Figure 12 As shown, the device includes:

[0273] The second acquisition module 1201 is used to acquire intelligent customer service consultation data;

[0274] The third generation module 1202 is used to process the intelligent customer service inquiry data using an intelligent customer service agent to generate answers and obtain intelligent customer service inquiry responses; wherein, the intelligent customer service agent is generated according to any of the agent training methods mentioned above.

[0275] By adopting the embodiments of this application, intelligent customer service consultation question data is obtained, and an intelligent customer service agent generated according to any of the above-mentioned agent training methods is used to generate answers to the intelligent customer service consultation question data, thereby obtaining intelligent customer service consultation responses. This achieves the goal of providing users with more convenient and accurate solutions. It also realizes the technical effect of breaking the limitation of static test datasets through dynamic data construction, capability boundary evaluation and iterative training, significantly improving the robustness and flexibility of the agent in handling complex tasks, and thus solving the technical problem in related technologies where the agent training and evaluation methods rely on relatively static test datasets, resulting in low agent robustness.

[0276] It should be noted that the second acquisition module 1201 and the third generation module 1202 mentioned above correspond to steps S51 and S52 in the embodiments. The two modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the above embodiments. It should be noted that the above modules or units can be hardware components or software components stored in memory and processed by one or more processors. The above modules can also run in the server 10 provided in the embodiments.

[0277] According to an embodiment of this application, another embodiment of an apparatus for implementing the above-described data processing method is also provided. Figure 13 This is a schematic diagram of another data processing apparatus according to an embodiment of this application, such as... Figure 13 As shown, the device includes:

[0278] The third acquisition module 1301 is used to acquire a data processing request through the first application programming interface, wherein the request data carried in the data processing request includes: problem data;

[0279] The first return module 1302 is used to return a data processing response through the second application programming interface. The response data carried in the data processing response includes: the target response, which is obtained by the target agent generating an answer to the question data. The target agent is generated according to any of the above agent training methods.

[0280] By employing the embodiments of this application, a data processing request is obtained through a first application programming interface (API), wherein the request data carried in the data processing request includes: question data. Then, a data processing response is returned through a second API, wherein the response data carried in the data processing response includes: a target response, which is obtained by a target agent generating an answer to the question data. The target agent is generated according to any of the above-mentioned agent training methods. This achieves the goal of providing users with a more convenient and accurate solution, thereby breaking the limitations of static test datasets through dynamic data construction, capability boundary assessment, and iterative training. This significantly improves the robustness and flexibility of the agent in handling complex tasks, thus solving the technical problem in related technologies where the agent training and evaluation methods rely on relatively static test datasets, resulting in low agent robustness.

[0281] It should be noted that the third acquisition module 1301 and the first return module 1302 mentioned above correspond to steps S61 and S62 in the embodiments. The two modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the above embodiments. It should be noted that the above modules or units can be hardware components or software components stored in memory and processed by one or more processors. The above modules can also run in the server 10 provided in the embodiments.

[0282] According to an embodiment of this application, another embodiment of an apparatus for implementing the above-described data processing method is also provided. Figure 14 This is a schematic diagram of another data processing apparatus according to an embodiment of this application, such as... Figure 14 As shown, the device includes:

[0283] The fourth acquisition module 1401 is used to acquire the currently input data processing dialogue request, wherein the request data carried in the data processing dialogue request includes: question data;

[0284] The second return module 1402 is used to respond to the data processing dialogue request and return a data processing dialogue response. The information carried in the data processing dialogue response includes: the target response, which is obtained by the target agent after processing the question data to generate an answer. The target agent is generated according to any of the above agent training methods.

[0285] Display module 1403 is used to display the target response within a graphical user interface.

[0286] By employing the embodiments of this application, a data processing dialogue request is obtained from the current input, wherein the request data carried in the data processing dialogue request includes: question data. Then, in response to the data processing dialogue request, a data processing dialogue response is returned, wherein the information carried in the data processing dialogue response includes: a target response, which is obtained by the target agent performing answer generation processing on the question data. The target agent is generated according to any of the above-mentioned agent training methods. Finally, the target response is displayed in the graphical user interface, thereby achieving the goal of providing users with a more convenient and accurate solution. This realizes the technical effect of breaking the limitation of static test datasets through dynamic data construction, capability boundary evaluation and iterative training, significantly improving the robustness and flexibility of agents in handling complex tasks, and thus solving the technical problem in related technologies where the agent training and evaluation methods rely on relatively static test datasets, resulting in low agent robustness.

[0287] It should be noted that the fourth acquisition module 1401, the second return module 1402, and the display module 1403 mentioned above correspond to steps S71 to S73 in the embodiments. The three modules and their corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the above embodiments. It should be noted that the above modules or units can be hardware or software components stored in memory and processed by one or more processors. The above modules can also run in the server 10 provided in the embodiments.

[0288] According to an embodiment of this application, another embodiment of an apparatus for implementing the above-described data processing method is also provided. Figure 15 This is a schematic diagram of another data processing apparatus according to an embodiment of this application, such as... Figure 15 As shown, the device includes:

[0289] The first display module 1501 is used to respond to input commands applied to the operation interface and display the problem data on the operation interface;

[0290] The second display module 1502 is used to respond to processing instructions applied to the operation interface and display the target response on the operation interface; wherein, the target response is obtained by the target agent generating an answer from the question data, and the target agent is generated according to any of the above agent training methods.

[0291] In this embodiment, by responding to input commands applied to the operation interface, problem data is displayed on the operation interface, and then, in response to processing commands applied to the operation interface, the target response is displayed on the operation interface. The target response is obtained by a target agent processing the problem data to generate an answer. The target agent is generated according to any of the agent training methods described above. This achieves the goal of providing users with a more convenient and accurate solution. It breaks through the limitations of static test datasets through dynamic data construction, capability boundary assessment, and iterative training, significantly improving the robustness and flexibility of the agent in handling complex tasks. This solves the technical problem in related technologies where the agent training and evaluation methods rely on relatively static test datasets, resulting in low agent robustness.

[0292] It should be noted that the first display module 1501 and the second display module 1502 mentioned above correspond to steps S81 and S82 in the embodiments. The two modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the above embodiments. It should be noted that the above modules or units can be hardware components or software components stored in memory and processed by one or more processors. The above modules can also run in the server 10 provided in the embodiments.

[0293] It should be noted that the preferred implementation schemes involved in the above embodiments of this application are the same as the schemes, application scenarios and implementation processes provided in the embodiments, but are not limited to the schemes provided in the embodiments.

[0294] Embodiments of this application may provide a computing device. Figure 16 This is a structural block diagram of a computing device according to an embodiment of this application. Figure 16 As shown, the computing device A may include one or more ( Figure 16 (Only one is shown) Processor 1602, memory 1604, memory controller, and peripheral interface, wherein the peripheral interface may connect to radio frequency module, audio module, display screen, etc., which are not limited here.

[0295] The aforementioned computing device A can be understood as an integrated smart terminal, including but not limited to servers, desktop computers, PCs (Personal Computers), all-in-one model machines, etc., and the computing device may have the model described in the above embodiments of this application pre-installed.

[0296] Specifically, computing device A can pre-install various types of models, including but not limited to models in natural language processing, visual processing, speech processing, code processing, and multimodal task processing, thus providing diverse model selection. In different product forms, computing device A can support one or more model usage methods, including but not limited to model training, model invocation, model fine-tuning, model deployment, model inference, and application. In some product forms, computing device A also supports model management, including but not limited to multi-type model management (supporting the management of discriminative, generative, and other model types), model version control (supporting the control of different model versions), and model evaluation (evaluating model performance and effectiveness based on model evaluation tools). In other product forms, computing device A can also create applications based on models, providing API calling capabilities, allowing models to be called into created applications through API interfaces, and providing application management tools for application management and monitoring.

[0297] Furthermore, the computing device A can also include data management (supporting the creation and management of model tuning datasets), a training center (providing abundant training resources to help users learn and master AI technology), and basic control capabilities (providing enterprise-level basic control capabilities to ensure the security and efficient operation of the system). Through the above functions, it provides a comprehensive and integrated device for AI development, training, deployment, and application.

[0298] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and apparatus in the embodiments of this application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the methods in the above embodiments. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0299] The processor can invoke an executable program stored in memory via a transmission device to execute the method described in any of the above embodiments.

[0300] Those skilled in the art will understand that, Figure 16 The structure shown is for illustrative purposes only; computing device A can also be a smartphone, tablet, PDA, mobile internet device (MID), PAD, or other terminal device. Figure 16 This does not limit the structure of the aforementioned computing device. For example, computing device A may also include components that are more advanced than those described above. Figure 16 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same as the Figure 16 The different configurations shown.

[0301] Embodiments of this application may provide an electronic device. Figure 17 This is a structural block diagram of an electronic device according to an embodiment of this application. Figure 17 As shown, the electronic device may include: an input / output device 172; a memory 174; and a processor 176, wherein the processor 176 is connected to the input / output device 172 and the memory 174 via a bus 178.

[0302] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and apparatus in the embodiments of this application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the methods in the above embodiments. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0303] The processor can invoke an executable program stored in memory via a transmission device to execute the method described in any of the above embodiments.

[0304] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0305] Embodiments of this application also provide a computer-readable storage medium. Optionally, in this embodiment, the aforementioned computer-readable storage medium can be used to store the program code executed by the agent training method or data processing method provided in the above embodiments.

[0306] Optionally, in this embodiment, the computer-readable storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0307] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements any of the above-described agent training methods or data processing methods.

[0308] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0309] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0310] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0311] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0312] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0313] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for training an intelligent agent, characterized in that, include: The exploration direction is determined based on the existing knowledge of the intelligent agent system, wherein the exploration direction is used to plan and explore the target task to be generated, and the intelligent agent system includes: multiple intelligent agents with different capability levels; According to the exploration direction, control the multiple intelligent agents to execute adversarial walk strategies to generate test data and training data; The capability boundaries of the multiple intelligent agents are evaluated using the test data to obtain the evaluation results. Based on the evaluation results and the training data, the agent to be evolved among the multiple agents is trained to obtain the target agent.

2. The agent training method according to claim 1, characterized in that, Determining the exploration direction based on the existing knowledge of the intelligent agent system includes: Based on the existing knowledge of the intelligent agent system, the known task types, tool coverage, existing model capabilities, and / or current service requirements are determined. The exploration direction is determined by utilizing the known task types, the tool coverage, the existing capabilities of the model, and / or the current service requirements.

3. The agent training method according to claim 2, characterized in that, The known task types are used to determine the currently known task types of the intelligent agent system; the tool coverage is used to determine the capability coverage of the multi-functional tool library, which is a tool library pre-equipped for the intelligent agent system; the existing model capabilities are used to determine the model capabilities currently possessed by the intelligent agent system; and the current service requirements are used to determine the requirements of the intelligent agent system for the current service scenario.

4. The agent training method according to claim 1, characterized in that, Controlling the multiple agents to execute the adversarial walk strategy according to the exploration direction, and generating the test data and training data includes: According to the exploration direction, the multiple intelligent agents are controlled to execute the adversarial wandering strategy to form multiple task trajectories; The test data and training data are generated based on the multiple task trajectories.

5. The agent training method according to claim 4, characterized in that, Controlling the multiple agents to execute the adversarial walk strategy according to the exploration direction, forming the multiple task trajectories includes: Generate task objectives based on the described exploration direction; According to the stated task objective, the multiple intelligent agents are controlled to execute the adversarial walk strategy, thereby forming the multiple task trajectories.

6. The agent training method according to claim 5, characterized in that, According to the task objective, the multiple agents are controlled to execute the adversarial walk strategy, forming the multiple task trajectories, including: According to the task objective, the multiple agents are controlled to execute the adversarial walk strategy on the corresponding toolchains in the multiple toolchains to form the multiple task trajectories. The multiple toolchains include: multiple first toolchains and multiple second toolchains. The multiple first toolchains are authorized to use first category tools in the multi-functional tool library, and the multiple second toolchains are authorized to use second category tools in the multi-functional tool library. The first category tools have professional functional attributes, and the second category tools have ordinary functional attributes.

7. The agent training method according to claim 4, characterized in that, The capability boundaries of the multiple agents are evaluated using the test data, and the evaluation results include: The multiple intelligent agents are controlled to execute the test data to obtain execution results, wherein the execution results are used to determine whether the multiple intelligent agents have successfully executed the toolchain test process; The capability boundaries of the multiple agents are evaluated based on the execution results to obtain the evaluation results.

8. The agent training method according to claim 1, characterized in that, Based on the evaluation results and the training data, the agent to be evolved among the plurality of agents is trained to obtain the target agent, which includes: The exploration direction is adjusted based on the evaluation results to obtain the adjusted exploration direction; The training data annotation process is rewarded based on the evaluation results, and the annotated data is obtained. Based on the adjusted exploration direction and the labeled data, the agent to be evolved among the multiple agents is subjected to process-guided reinforcement learning training to obtain the target agent.

9. The agent training method according to claim 4, characterized in that, The training data generated based on the multiple task trajectories includes: Based on the successful execution results of the multiple task trajectories on multiple first toolchains, positive training samples are generated; and based on the failed execution results of the multiple task trajectories on multiple second toolchains, negative training samples are generated. The training data is determined based on the positive training samples and the negative training samples.

10. The agent training method according to claim 3, characterized in that, The agent training method further includes: Expand the functional attributes of existing tools in the multi-functional tool library, and / or expand tool categories not covered in the multi-functional tool library.

11. A data processing method, characterized in that, include: Obtain the problem data; The target intelligent agent is used to process the question data to generate an answer, thus obtaining the target response; The target agent is generated according to the agent training method described in any one of claims 1 to 10.

12. A data processing method, characterized in that, include: Obtain data on customer service inquiries from intelligent customer service systems; An intelligent customer service agent is used to process the intelligent customer service inquiry data to generate answers and obtain intelligent customer service responses. The intelligent customer service agent is generated according to the agent training method described in any one of claims 1 to 10.

13. A data processing method, characterized in that, include: A data processing request is obtained through a first application programming interface, wherein the request data carried in the data processing request includes: problem data; The data processing response is returned through the second application programming interface, wherein the response data carried in the data processing response includes: a target response, which is obtained by a target agent generating an answer to the question data, and the target agent is generated according to the agent training method according to any one of claims 1 to 10.

14. A data processing method, characterized in that, include: Obtain the currently input data processing dialogue request, wherein the request data carried in the data processing dialogue request includes: question data; In response to the data processing dialogue request, a data processing dialogue response is returned, wherein the information carried in the data processing dialogue response includes: a target response, which is obtained by a target agent performing answer generation processing on the question data, and the target agent is generated according to the agent training method according to any one of claims 1 to 10; The target response is displayed within the graphical user interface.

15. A data processing method, characterized in that, include: In response to input commands applied to the user interface, the system displays problem data on the user interface. In response to the processing instructions applied to the operation interface, the target response is displayed on the operation interface; The target response is obtained by a target agent processing the question data to generate an answer, and the target agent is generated according to the agent training method described in any one of claims 1 to 10.

16. A data processing system, characterized in that, include: The client is used to send the problematic data; The server, connected to the client, is used to process the question data using a target intelligent agent to generate an answer and obtain a target response, wherein the target intelligent agent is generated according to the intelligent agent training method according to any one of claims 1 to 10; The client is also used to output the target response.

17. An electronic device, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, executes the agent training method according to any one of claims 1 to 10 or the data processing method according to any one of claims 11 to 15.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein, when the executable program is executed, it controls the device on which the computer-readable storage medium is located to perform the agent training method according to any one of claims 1 to 10 or the data processing method according to any one of claims 11 to 15.

19. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the agent training method according to any one of claims 1 to 10 or the data processing method according to any one of claims 11 to 15.

Images