Test method and device for intelligent agent and electronic equipment

Abstract

The invention relates to the technical field of large models, and discloses an intelligent agent testing method which comprises the following steps: establishing an initial data set for data accumulation; inputting the initial data set into an agent, and analyzing the output of the agent by using a general large model to obtain a preliminary analysis conclusion; according to the preliminary analysis conclusion, the general large model is optimized; and retesting the intelligent agent by using the optimized general large model to obtain a target analysis conclusion until the target analysis conclusion meets the test requirement. According to the method, the accuracy and efficiency of testing can be improved, so that the construction efficiency of the intelligent agent is effectively improved, and the construction quality of the intelligent agent is ensured. The invention further discloses a testing device for the intelligent agent and electronic equipment.

Description

Technical Field

[0001] This application relates to the field of large model technology, such as a testing method and apparatus for intelligent agents, and electronic devices. Background Technology

[0002] Currently, with the continuous development of AI large-scale model technology, the number of intelligent agents relying on AI large-scale models is growing rapidly. During the construction of intelligent agents, as new functions are added and existing functions are iterated, previously debugged functions may become unusable, and traditional manual smoke testing is simply insufficient to exhaustively test all possibilities. The more functions an intelligent agent has, the more severe the mutual interference becomes, and the more situations need to be classified and resolved, further reducing the accuracy of the intelligent agent.

[0003] To test intelligent agents, related technologies disclose an intelligent agent evaluation method, apparatus, electronic device, storage medium, and program. The method determines a test case set for a target intelligent agent based on the agent's role; generates test results for the target intelligent agent based on the test case set; and determines the evaluation result corresponding to each test result according to an intelligent agent evaluation model.

[0004] In the process of implementing the embodiments of this disclosure, at least the following problems were found in the related art: Test cases for related technologies are based on role definitions, resulting in low test coverage and reliance on manual testing and verification of each test case, leading to a large workload for manual annotation. This results in a decrease in the efficiency and accuracy of testing intelligent agents.

[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general commentary, nor is it intended to identify key / important components or describe the scope of protection of these embodiments, but rather as a prelude to the detailed description that follows.

[0007] This disclosure provides a testing method, apparatus, and electronic device for intelligent agents, which can improve the accuracy of testing.

[0008] In some embodiments, the testing method for the agent includes: establishing an initial dataset for data accumulation; inputting the initial dataset into the agent and analyzing the agent's output using a general large model to obtain preliminary analysis conclusions; optimizing the general large model based on the preliminary analysis conclusions; retesting the agent using the optimized general large model to obtain target analysis conclusions until the target analysis conclusions meet the testing requirements.

[0009] Optionally, the initial dataset includes user questions and reference answers.

[0010] Optionally, the initial dataset is input into the agent, and the agent's output is analyzed to obtain preliminary analysis conclusions, including: inputting the user's question from the initial dataset into the agent to obtain the process data output by the agent; and using the process data to analyze the agent to obtain preliminary analysis conclusions.

[0011] Optionally, the user problem of the initial dataset is input into the agent to obtain the process data output by the agent, including: after inputting the user problem of the initial dataset into the agent, obtaining multiple process data through the checkpoints of the agent's implementation path; and saving all the obtained process data.

[0012] Optionally, the process data includes the final answer. The process data is used to analyze the agent and obtain preliminary analysis conclusions, including: using a general large model to compare the reference answer and the final answer in multiple dimensions to obtain preliminary analysis conclusions.

[0013] Optionally, based on the preliminary analysis conclusions, the general large model is optimized, including: labeling the error data in the preliminary analysis conclusions, wherein the error data is used to characterize the data that caused errors when the general large model was analyzed; and optimizing the general large model using the error data.

[0014] Optionally, the testing method for intelligent agents further includes: acquiring process data corresponding to the target analysis conclusions that meet the testing requirements; and optimizing the general large model based on the process data.

[0015] In some embodiments, the testing apparatus for the agent includes: a data accumulation module configured to establish an initial dataset for data accumulation; a preliminary analysis module configured to input the initial dataset into the agent and analyze the agent's output using a general large model to obtain preliminary analysis conclusions; a model optimization module configured to optimize the general large model based on the preliminary analysis conclusions; and a retesting module configured to retest the agent using the optimized general large model to obtain target analysis conclusions until the target analysis conclusions meet the test requirements.

[0016] In some embodiments, the testing apparatus for an intelligent agent includes a processor and a memory storing program instructions, the processor being configured to execute the testing method for an intelligent agent as described above when the program instructions are executed.

[0017] In some embodiments, the electronic device includes: an electronic device body; and a testing device for an intelligent agent as described above, mounted on the electronic device body.

[0018] The testing method, apparatus, and electronic device for intelligent agents provided in this disclosure can achieve the following technical effects: In this embodiment, during the data accumulation phase, an initial dataset containing user questions and reference answers is constructed. In the automated testing phase, questions are input in batches into the target agent, and process data is captured in real-time at checkpoints. In the intelligent analysis phase, a general large model is used to compare the reference answers and process data across multiple dimensions, outputting preliminary analysis conclusions. The system uses these preliminary analysis conclusions to optimize the general large model and analysis algorithm, then restarts full-scale testing until the target analysis conclusions meet the testing requirements. After multiple iterations, validated process data is automatically included in the training set, continuously improving the large model's evaluation capability and reducing the amount of manual annotation. This embodiment achieves automated testing of the agent and, through a series of processes, ultimately provides a test evaluation through intelligent analysis, thereby improving the accuracy and efficiency of testing, effectively enhancing the agent's construction efficiency, and ensuring the quality of the agent's construction.

[0019] The above general description and the description below are exemplary and illustrative only and are not intended to limit this application. Attached Figure Description

[0020] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations and drawings do not constitute a limitation on the embodiments. Elements having the same reference numerals in the drawings are shown as similar elements. The drawings are not to be scaled. And wherein: Figure 1 This is a schematic diagram of the implementation environment of the testing method for intelligent agents provided in the embodiments of this disclosure; Figure 2 This is a schematic diagram of a testing method for an intelligent agent provided in an embodiment of this disclosure; Figure 3 This is a schematic diagram of another testing method for intelligent agents provided in an embodiment of this disclosure; Figure 4 This is a schematic diagram of a testing device for an intelligent agent provided in an embodiment of this disclosure; Figure 5 This is a schematic diagram of another testing device for intelligent agents provided in an embodiment of this disclosure. Detailed Implementation

[0021] To provide a more detailed understanding of the features and technical content of the embodiments of this disclosure, the implementation of the embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for illustrative purposes only and are not intended to limit the embodiments of this disclosure. In the following technical description, for ease of explanation, several details are used to provide a full understanding of the disclosed embodiments. However, one or more embodiments may still be implemented without these details. In other cases, well-known structures and devices may be simplified in their depiction to simplify the drawings.

[0022] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this disclosure described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0023] Unless otherwise stated, the term "multiple" means two or more.

[0024] In this embodiment of the disclosure, the character " / " indicates that the objects before and after it are in an "or" relationship. For example, A / B means: A or B.

[0025] The term "and / or" describes an association between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or A and B.

[0026] The term "correspondence" can refer to an association or binding relationship. The correspondence between A and B means that there is an association or binding relationship between A and B.

[0027] With the continuous development of AI large-scale model technology, the number of intelligent agents relying on this technology and their supported capabilities are growing rapidly. During the construction of intelligent agents, as new functions are added and existing functions are iterated, it is discovered that previously optimized functions may no longer work effectively, and traditional manual smoke testing is simply insufficient to exhaustively test all possibilities. The results produced by AI are often not static, and verification consumes a significant amount of time and effort. The more functions an intelligent agent has, the more severe the mutual influence becomes, and the more situations need to be classified and resolved, further reducing the accuracy of the intelligent agent.

[0028] Existing methods for testing intelligent agents have many problems, including: Relying on manual testing and verification of each item results in low coverage, high subjectivity, and low efficiency; traditional automated testing lacks a process verification mechanism for test results, leading to insufficient reliability of the results; existing technologies struggle to achieve intelligent comparison and analysis between test results and expected answers; and there is a lack of an effective closed-loop feedback mechanism between test data accumulation and model optimization.

[0029] Combination Figure 1 As shown, Figure 1 This is a schematic diagram of the implementation environment for the testing method for intelligent agents according to an embodiment of this disclosure. Figure 1 As shown, the implementation environment may include an agent 100, a general-purpose large model 200, and a processor 500.

[0030] This disclosure embodiment constructs an architecture based on automated testing of intelligent agents. The processor 500 utilizes the multi-dimensional analysis and comparison capabilities of the general large model 200 to analyze the output of the intelligent agent 100, effectively improving the efficiency of intelligent agent construction and ensuring the quality of intelligent agent construction.

[0031] Combination Figure 2 As shown, this disclosure provides a testing method for intelligent agents, including: S201, the processor builds an initial dataset for data accumulation.

[0032] S202, the processor inputs the initial dataset into the agent and uses a general large model to analyze the agent's output and obtain preliminary analysis conclusions.

[0033] S203, the processor optimizes the general large model based on the preliminary analysis results.

[0034] S204, the processor retests the agent using the optimized general large model to obtain target analysis conclusions until the target analysis conclusions meet the test requirements.

[0035] The testing method provided in this disclosure involves constructing an initial dataset containing user questions and reference answers during the data accumulation phase. In the automated testing phase, questions are input into the target agent in batches, and process data is captured in real-time at checkpoints. In the intelligent analysis phase, a general large model is used to compare the reference answers and process data across multiple dimensions, outputting preliminary analysis conclusions. The system uses these preliminary analysis conclusions to optimize the general large model and analysis algorithm, then restarts full-scale testing until the target analysis conclusions meet the testing requirements. After multiple iterations, validated process data is automatically included in the training set, continuously improving the large model's evaluation capability and reducing the amount of manual annotation. This disclosure implements automated testing of the agent and, through a series of processes, ultimately provides a test evaluation through intelligent analysis, thereby improving the accuracy and efficiency of testing, effectively enhancing the agent's construction efficiency, and ensuring the quality of the agent's construction.

[0036] Optionally, the initial dataset includes user questions and reference answers.

[0037] In this embodiment, an initial dataset containing user questions and reference answers is constructed for business scenarios and edge cases. First, domain experts identify high-frequency, critical, and error-prone tasks, forming a question list covering the entire functional chain. Then, edge cases are expanded using methods such as data augmentation, adversarial examples, and boundary value analysis. Subsequently, high-confidence reference answers are generated for each question, and after multiple rounds of cross-review and version freezing, structured "question-answer" pairs are formed and uniformly vectorized and stored in a database. During the testing phase, this initial dataset is injected in batches into the tested agents. Process data is captured through multiple checkpoints, and a large model analyzes the reference answers and agent outputs.

[0038] In practical applications, user questions and corresponding answers can relate to different home appliances. For example, if a user's question is that an "E1" message appears on the washing machine screen, the corresponding answer is that "E1" indicates a water inlet timeout; please check if the faucet is turned on, if the water pressure is normal, and if the inlet hose is bent. If a user's question is that the refrigerator light is not on, the corresponding answer is to ensure the door is closed tightly, unplug the power cord for 10 seconds, and then plug it back in.

[0039] Optionally, the initial dataset is input into the agent, and the agent's output is analyzed to obtain preliminary analysis conclusions, including: inputting the user's question from the initial dataset into the agent to obtain the process data output by the agent; and using the process data to analyze the agent to obtain preliminary analysis conclusions.

[0040] In this embodiment, all user questions from the initial dataset are injected into the tested agent using a concurrent strategy, triggering its complete inference chain. Configurable checkpoints are automatically inserted at key nodes such as retrieval, generation, and verification, and process data such as unrendered text, vector retrieval Top-K, and inference chain are captured in real time. The reference answer and all process data are packaged into a structured record and written to a high-speed cache. The general large model Prompt is called to perform a three-dimensional comparison of semantic, logical, and factual consistency between the reference answer and the process data, and outputs preliminary conclusions.

[0041] Optionally, the user problem of the initial dataset is input into the agent to obtain the process data output by the agent, including: after inputting the user problem of the initial dataset into the agent, obtaining multiple process data through the checkpoints of the agent's implementation path; and saving all the obtained process data.

[0042] In this embodiment, each user question from the initial dataset is sent in batches to the target agent and a complete inference chain is triggered; pluggable checkpoints are pre-deployed at key nodes to capture multimodal process data in real time; the captured data is immediately desensitized, compressed, and version-marked, and uniformly written into high-performance logs and object storage in a key-value structure to form a traceable and replayable complete data snapshot; all process data are aggregated by question dimension to establish a "question-process data-reference answer" triplet index for direct use in subsequent multidimensional analysis.

[0043] By collecting and storing data across the entire process, we achieve transparent and traceable management of the behavior process. This significantly reduces the complexity of subsequent problem reproduction and root cause investigation, and provides structured and reusable data assets for continuous optimization, thereby improving the efficiency, stability and scalability of testing and iteration.

[0044] Specifically, after inputting user questions from the initial dataset into the agent, the acquired process data includes user questions and multiple checkpoint data.

[0045] One user issue was that their washing machine displayed "E1". At checkpoint 1, the fault code was retrieved as: "E1 - Water ingress timeout - Fault manual section 3.2"; Checkpoint 2 performs image recognition: it identifies user-uploaded photos of faucets and uses AI to detect the angle of the valve handle; Checkpoint 3 provides the original response: "E1" indicates that the water inlet timed out and the tap has been detected as closed. Please turn on the tap and restart.

[0046] Checkpoint 4 recorded the time taken: 1.2 seconds for the response.

[0047] Optionally, the process data includes the final answer. The process data is used to analyze the agent and obtain preliminary analysis conclusions, including: using a general large model to compare the reference answer and the final answer in multiple dimensions to obtain preliminary analysis conclusions.

[0048] In this embodiment, reference answers and unrendered final answers are first extracted from the saved process data to construct paired samples. Then, a general large-scale model is invoked, and a pre-designed Prompt instruction is used to perform a multi-dimensional comparison between the two, outputting the difference type, confidence level, and preliminary conclusions. This embodiment can quickly complete large-scale answer quality screening without manual review of each answer, and the automatic classification of error types makes problem diagnosis more accurate. Furthermore, multi-dimensional comparison significantly reduces the risk of misjudgment and missed judgment, providing a reliable basis for subsequent Prompt optimization and model iteration, and improving overall testing efficiency and the credibility of conclusions.

[0049] In practical applications, if a user's problem is "the washing machine displays E1," the suggested answer is "E1 indicates a water inlet timeout; please check the faucet, water pressure, and inlet hose," while the final answer is "E1 may occur because the faucet is not turned on or the water pressure is insufficient." Let's compare semantic consistency and logical completeness separately: First, semantic consistency comparison can be used to compare core information such as "water pressure," as both the reference answer and the final answer contain "water pressure." Then, in the comparison of logical completeness, the reference answer included "the water inlet pipe was bent" for investigation, and the final answer was missing.

[0050] Thus, the preliminary analysis concludes that it is necessary to add the following: "Check whether the water inlet pipe is bent."

[0051] Optionally, based on the preliminary analysis conclusions, the general large model is optimized, including: labeling the error data in the preliminary analysis conclusions, wherein the error data is used to characterize the data that caused errors when the general large model was analyzed; and optimizing the general large model using the error data.

[0052] In this embodiment, based on the confidence level and difference type in the preliminary analysis conclusions, high-error samples are automatically selected and a visual difference report is generated. Subsequently, these samples are quickly confirmed or corrected, and the "reference answer-final answer-difference label" triplet is written into the error dataset. Next, using incremental learning techniques, the error data is back-injected into a general large model to adjust the comparison rules and scoring weights. Finally, regression testing is triggered to verify the optimization effect. This embodiment of the invention recycles error data, making the model more accurate with use, thus reducing the intensity of human intervention and achieving low-cost, sustainable improvement in the quality of agent testing.

[0053] Optionally, the testing method for intelligent agents further includes: acquiring process data corresponding to the target analysis conclusions that meet the testing requirements; and optimizing the general large model based on the process data.

[0054] Combination Figure 3 As shown, this disclosure provides another testing method for intelligent agents, including: S301, the processor builds an initial dataset for data accumulation.

[0055] S302, the processor inputs the initial dataset into the agent and uses a general large model to analyze the agent's output and obtain preliminary analysis conclusions.

[0056] The S303 processor optimizes the general large model based on preliminary analysis results.

[0057] S304, the processor acquires process data corresponding to the target analysis conclusions that meet the test requirements.

[0058] The S305 processor optimizes a general large model based on process data.

[0059] S306: The processor retests the agent using the optimized general large model to obtain target analysis conclusions until the target analysis conclusions meet the test requirements.

[0060] In this embodiment, multiple process data are acquired. All process data for which the reference answers and final answers are deemed to meet the requirements are generally considered to also meet the requirements. No manual annotation of other process data is necessary. After multiple rounds of fully automated testing, the process data from each round can better improve the accuracy of intelligent analysis, and the amount of data requiring manual annotation will decrease.

[0061] Combination Figure 4 As shown, this embodiment of the disclosure provides a testing device 40 for an intelligent agent, including a data accumulation module 401, a preliminary analysis module 402, a model optimization module 403, and a retesting module 404. The data accumulation module 401 is configured to establish an initial dataset for data accumulation; the preliminary analysis module 402 is configured to input the initial dataset into the intelligent agent and analyze the agent's output using a general large model to obtain preliminary analysis conclusions; the model optimization module 403 is configured to optimize the general large model based on the preliminary analysis conclusions; and the retesting module 404 is configured to retest the intelligent agent using the optimized general large model to obtain target analysis conclusions until the target analysis conclusions meet the testing requirements.

[0062] Using the apparatus 40 provided in this embodiment, the data accumulation module 401 constructs an initial dataset containing user questions and reference answers. The preliminary analysis module 402 inputs questions in batches into the target agent and captures process data in real time at checkpoints. A general large model is used to compare the reference answers and process data from multiple dimensions, outputting preliminary analysis conclusions. The model optimization module 403 optimizes the general large model and analysis algorithm using the preliminary analysis conclusions, and then the retesting module 404 restarts the full-scale test until the target analysis conclusions meet the test requirements. After multiple iterations, the verified process data is automatically included in the training set, continuously improving the large model's evaluation capability and reducing the amount of manual annotation. This embodiment of the present disclosure achieves automated testing of the agent and, through a series of processes, ultimately provides a test evaluation through intelligent analysis, thereby improving the accuracy and efficiency of testing, effectively enhancing the construction efficiency of the agent, and ensuring the quality of the agent's construction.

[0063] Combination Figure 5As shown, this disclosure provides a testing device 50 for intelligent agents, including a processor 500 and a memory 501. Optionally, the device 50 may further include a communication interface 502 and a bus 503. The processor 500, communication interface 502, and memory 501 can communicate with each other via the bus 503. The communication interface 502 can be used for information transmission. The processor 500 can call logical instructions in the memory 501 to execute the testing method for intelligent agents described in the above embodiments.

[0064] Furthermore, the logic instructions in the aforementioned memory 501 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0065] The memory 501, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as program instructions / modules corresponding to the methods in the embodiments of this disclosure. The processor 500 executes functional applications and data processing by running the program instructions / modules stored in the memory 501, thereby implementing the testing method for intelligent agents described in the above embodiments.

[0066] The memory 501 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 501 may include high-speed random access memory and may also include non-volatile memory.

[0067] This disclosure provides an electronic device, including: an electronic device body, and the aforementioned testing device for an intelligent agent. The testing device for the intelligent agent is mounted on the electronic device body. The mounting relationship described herein is not limited to placement within the electronic device body, but also includes mounting connections with other components of the electronic device, including but not limited to physical connections, electrical connections, or signal transmission connections. Those skilled in the art will understand that the testing device for the intelligent agent can be adapted to feasible electronic device bodies, thereby realizing other feasible embodiments.

[0068] The technical solutions of this disclosure can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes one or more instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in this disclosure. The aforementioned storage medium can be a non-transitory storage medium, such as a USB flash drive, external hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, etc., and other media capable of storing program code.

[0069] The foregoing description and accompanying drawings fully illustrate embodiments of this disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, procedural, and other changes. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operation may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. Moreover, the terminology used in this application is for describing embodiments only and is not intended to limit the claims. As used in the description of embodiments and claims, the singular forms “a,” “an,” and “the” are intended to equally include the plural forms unless the context clearly indicates otherwise. Similarly, the term “and / or” as used in this application means including one or more of the associated listed items and all possible combinations thereof. Additionally, when used in this application, the term "comprise" and its variations "comprises" and / or "comprising" refer to the presence of stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof. Without further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the process, method, or apparatus that includes said element. In this document, each embodiment may focus on the differences from other embodiments, and similar or identical parts between embodiments can be referred to mutually. For methods, products, etc., disclosed in the embodiments, if they correspond to the method section disclosed in the embodiments, the relevant parts can be referred to the description of the method section.

[0070] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this disclosure. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0071] The methods and products disclosed in the embodiments herein (including but not limited to devices and equipment) can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units may be merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to implement this embodiment according to actual needs. In addition, the functional units in the embodiments of this disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0072] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in a different order than disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. Each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

Claims

1. A testing method for an agent, characterized in that, Comprise: establishing an initial data set for data accumulation; inputting the initial data set into an agent, and analyzing the output of the agent by using a general large model to obtain a preliminary analysis conclusion; optimizing the general large model according to the preliminary analysis conclusion; retesting the agent by using the optimized general large model to obtain a target analysis conclusion until the target analysis conclusion meets the test requirements.

2. The method of claim 1, wherein, The initial data set includes user questions and reference answers.

3. The method of claim 2, wherein, Input the initial data set into the agent, and analyze the output of the agent to obtain a preliminary analysis conclusion, including: inputting the user questions of the initial data set into the agent to obtain process data of the agent output; analyzing the agent by using the process data to obtain a preliminary analysis conclusion.

4. The method of claim 3, wherein, Input the user questions of the initial data set into the agent to obtain process data of the agent output, including: after inputting the user questions of the initial data set into the agent, obtain a plurality of process data through the inspection points of the agent implementation path; save all the obtained process data.

5. The method of claim 3, wherein, The process data includes the final answer, and the agent is analyzed by using the process data to obtain a preliminary analysis conclusion, including: using a general large model to compare the reference answer and the final answer in multiple dimensions to obtain a preliminary analysis conclusion.

6. The method of claim 1, wherein, According to the preliminary analysis conclusion, the general large model is optimized, including: labeling error data in the preliminary analysis conclusion, wherein the error data is used to represent the data that the general large model analyzes and appears error; optimizing the general large model by using the error data.

7. The method according to any one of claims 3 to 6, characterized in that, Also include: obtaining process data corresponding to the target analysis conclusion that meets the test requirements; optimizing the general large model according to the process data.

8. A testing device for an agent, characterized in that Comprise: a data accumulation module configured to establish an initial data set for data accumulation; a preliminary analysis module configured to input the initial data set into an agent, and analyze the output of the agent by using a general large model to obtain a preliminary analysis conclusion; a model optimization module configured to optimize the general large model according to the preliminary analysis conclusion; a retesting module configured to retest the agent by using the optimized general large model to obtain a target analysis conclusion until the target analysis conclusion meets the test requirements.

9. A testing apparatus for an agent, comprising: A processor and a memory storing program instructions, characterized in that the processor is configured to execute the test method for the agent as claimed in any one of claims 1 to 7 when running the program instructions.

10. An electronic device, comprising: Comprise: an electronic device body; the test device for the agent as claimed in claim 8 or 9 is installed in the electronic device body.

Images