A model evaluation method, device, platform and computer storage medium

Abstract

This application proposes a model evaluation method, apparatus, platform, and computer storage medium. The method includes: a central agent configuring at least one role agent according to the evaluation dimensions of the test task; wherein the evaluation dimensions include evaluation content and / or evaluation requirements; different role agents are distinguished by their responsibilities, evaluation dimensions, and / or key technical implementation points; the central agent distributing the model outputs of several tested models to the role agents for evaluation; the role agents determining the evaluation information of the model output of each tested model according to their responsibilities and feeding it back to the central agent; and the central agent generating evaluation results for the several tested models based on the evaluation information from all role agents. Through this model evaluation method, role specialization, task decomposition, and collaborative evaluation are achieved, improving the comprehensiveness and accuracy of model evaluation.

Description

Technical Field

[0001] This application relates to the field of model testing technology, and in particular to a model evaluation method, a model evaluation device, a model evaluation platform, and a computer storage medium. Background Technology

[0002] With the rapid development of artificial intelligence technology, especially the widespread application of large language models (LLMs) and multimodal large models, the number of various AI models (such as text generation models, dialogue systems, code generation models, image generation models, etc.) has exploded. Different institutions, teams, and even open source communities continue to release models with varying performance and architectures. Faced with numerous models to choose from, users urgently need an efficient, objective, interpretable, and scenario-adaptable model performance evaluation mechanism to support model selection, optimization iteration, and compliant deployment.

[0003] However, traditional model evaluation methods have significant limitations: they are primarily based on automated evaluation using a single large-scale judge model (Single LLM-as-a-Judge), making them susceptible to biases, knowledge limitations, or the design of prompts within the judge model itself. Furthermore, they cannot provide professional and collaborative evaluation across different evaluation dimensions. Summary of the Invention

[0004] To address the aforementioned technical problems, this application proposes a model evaluation method, a model evaluation device, a model evaluation platform, and a computer storage medium.

[0005] To address the aforementioned technical problems, this application proposes a model evaluation method. This method is applied to a model evaluation platform, which includes a central agent and several role agents. The model evaluation method includes:

[0006] The central agent acquires the model outputs of several tested models executing test tasks;

[0007] The central agent configures at least one role agent according to the evaluation dimensions of the test task; wherein, the evaluation dimensions include evaluation content and / or evaluation requirements; different role agents are distinguished by their responsibilities, evaluation dimensions, and / or key technical implementation points;

[0008] The central agent distributes the model outputs of the several tested models to the role agents for evaluation;

[0009] The role-based intelligent agent determines the evaluation information of the model output of each tested model according to its responsibilities and feeds it back to the central intelligent agent.

[0010] The central agent generates evaluation results for the tested models based on the evaluation information from all role agents.

[0011] The evaluation information output by the intelligent agent includes evaluation dimensions, evaluation scores, evaluation confidence levels, and / or comments.

[0012] Before the central agent generates the evaluation results of the plurality of tested models based on the evaluation information of all role agents, the model evaluation method further includes:

[0013] The central agent traverses the evaluation information of all role agents and determines whether the evaluation information of each role agent meets the evaluation requirements.

[0014] If not, the central agent removes the evaluation information of role agents that do not meet the evaluation requirements, or returns an instruction to re-evaluate.

[0015] Before the central agent generates the evaluation results of the plurality of tested models based on the evaluation information of all role agents, the model evaluation method further includes:

[0016] The central agent traverses the evaluation information of all role agents and determines whether the evaluation confidence value of the evaluation information of each role agent is lower than a preset threshold.

[0017] If so, the central agent removes the evaluation information of the role agent whose evaluation confidence value is lower than a preset threshold, or returns an instruction to re-evaluate.

[0018] The central agent generates evaluation results for the tested models based on the evaluation information from all role agents, including:

[0019] The central agent, for each tested model, integrates the evaluation information from all role agents to generate the total evaluation score for the tested model.

[0020] The central agent determines the tested model with the highest total evaluation score as the winning model.

[0021] The central agent, for each tested model, integrates the evaluation information from all role agents to generate a total evaluation score for the tested model, including:

[0022] The central agent determines the evaluation score and evaluation content of each tested model in the evaluation information of all role agents;

[0023] The central intelligent agent determines the weight of each evaluation item according to the evaluation requirements;

[0024] The central agent performs a weighted fusion of the evaluation scores of each tested model across all role agents according to the weight of the evaluation content, to obtain the total evaluation score of each tested model.

[0025] The central agent generates evaluation results for the tested models based on the evaluation information from all role agents, including:

[0026] The central agent determines the evaluation scores of the tested models in each evaluation dimension based on the evaluation information of all role agents.

[0027] The central agent compares the evaluation scores of the several tested models in each evaluation dimension to determine the winning model in each evaluation dimension.

[0028] The central agent selects the tested model with the most wins as the final winning model.

[0029] To address the aforementioned technical problems, this application also proposes a model evaluation platform, which includes an intelligent agent comprising a central intelligent agent and role-specific intelligent agents, wherein the intelligent agent executes the aforementioned model evaluation method.

[0030] To address the aforementioned technical problems, this application also proposes a model evaluation device, which includes a memory and a processor coupled to the memory; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the model evaluation method described above.

[0031] To address the aforementioned technical problems, this application also proposes a computer storage medium for storing program data, which, when executed by a computer, is used to implement the aforementioned model evaluation method.

[0032] Compared with existing technologies, the beneficial effects of this application are as follows: the central agent configures role agents according to the test task, so that each model evaluation is configured with a role agent with corresponding evaluation function, which helps to improve the targeting of the evaluation; the central agent is responsible for distributing the evaluation task and assigning the referee function to the role agents with evaluation function, which can solve the capability limitation of a single fixed referee model; each role agent performs evaluation separately, and finally the central agent summarizes and verifies the results, realizing collaborative evaluation and improving the comprehensiveness and accuracy of model evaluation. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0034] Figure 1 This is a flowchart illustrating the automated evaluation scheme for a single referee large model provided in this application;

[0035] Figure 2 This is a flowchart illustrating the first embodiment of the model evaluation method provided in this application;

[0036] Figure 3 This is a flowchart illustrating the automated evaluation scheme for multi-agent collaboration provided in this application.

[0037] Figure 4 This is a flowchart illustrating the second embodiment of the model evaluation method provided in this application;

[0038] Figure 5 This is a flowchart illustrating the third embodiment of the model evaluation method provided in this application;

[0039] Figure 6 This is a flowchart illustrating the fourth embodiment of the model evaluation method provided in this application;

[0040] Figure 7 This is a flowchart illustrating the fifth embodiment of the model evaluation method provided in this application;

[0041] Figure 8 This is a schematic diagram of the structure of an embodiment of the model evaluation platform provided in this application;

[0042] Figure 9 This is a schematic diagram of the structure of an embodiment of the model evaluation device provided in this application;

[0043] Figure 10 This is a schematic diagram of the structure of an embodiment of the computer storage medium provided in this application. Detailed Implementation

[0044] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0045] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0046] For details on the current automated evaluation scheme based on a single referee model, please refer to [link / reference]. Figure 1 , Figure 1 This is a flowchart illustrating the automated evaluation scheme for a single referee large model provided in this application.

[0047] like Figure 1 As shown, in existing technologies, manual queries are sent to a large model or API calls are made to send pre-defined queries to the large model in batches to obtain the model's responses. Then, a powerful large language model (such as GPT-4) is used as a "referee" to compare the responses of the tested models pairwise to determine the winning model. Finally, the win rates of each model are calculated and the models are ranked.

[0048] Therefore, Figure 1 The automated evaluation scheme presented requires the development and maintenance of a referee model capable of evaluating multiple dimensions. Furthermore, the evaluation results are strongly correlated with the results of this referee model, raising concerns about opaque evaluation standards, inherent biases in the referee model itself, non-reproducible evaluation processes, and a lack of verification of the reliability of the evaluation results. Especially when dealing with sensitive content, value judgments, or specialized domain knowledge, the conclusions of a single referee model may be inaccurate. Therefore, a systematic, configurable, multi-agent collaborative automated evaluation architecture is urgently needed to improve the comprehensiveness, fairness, and reliability of the evaluation.

[0049] To address this issue, this application provides an automated evaluation architecture for multi-agent collaboration. Figure 1 The automated evaluation solution shown has issues. Please refer to the following for details. Figure 2 and Figure 3 , Figure 2 This is a flowchart illustrating the first embodiment of the model evaluation method provided in this application. Figure 3 This is a flowchart illustrating the automated evaluation scheme for multi-agent collaboration provided in this application.

[0050] The core idea of ​​the model evaluation method in this application is to decompose the evaluation task traditionally undertaken by a single referee model into multiple specialized sub-tasks, which are executed in parallel or iteratively by intelligent agents with different roles, and the results are integrated through a collaborative mechanism to form a highly reliable, multi-dimensional comprehensive evaluation conclusion.

[0051] The model evaluation method of this application is applied to a model evaluation platform, maintaining an automated evaluation architecture for multi-agent collaboration. This multi-agent architecture specifically includes a central agent and various role-based agents, combining the functions and usage methods of these two types of agents. Figure 2 and Figure 3 The steps shown will be explained in detail.

[0052] like Figure 2 As shown, the specific steps are as follows:

[0053] Step S11: The central agent obtains the model outputs of several tested models executing test tasks.

[0054] In the embodiments of this application, such as Figure 3 As shown, staff create evaluation tasks through the model evaluation platform, specifying information such as the test set and the model under test. Then, the central agent inputs the test data from the test set into each model under test to obtain the model output for each model.

[0055] Specifically, the model under test can be a large language model. In this case, the test data can be a number of user questions. The model under test provides relevant intelligent responses based on the user questions, which are used as the model output to evaluate the model's capabilities.

[0056] Step S12: The central agent configures at least one role agent according to the evaluation dimensions of the test task.

[0057] In the embodiments of this application, such as Figure 3 As shown, the central agent configures relevant role agents according to the evaluation task in step S11. Specifically, the model evaluation platform can maintain agents with different responsibilities, i.e., different role agents. In each model evaluation process, the central agent configures relevant role agents according to the needs of the evaluation task, which can reflect the agent responsibilities of different role agents and achieve targeted evaluation.

[0058] In one specific implementation, the agents maintained by the model evaluation platform include, but are not limited to, the following types of agents:

[0059] Table 1

[0060]

[0061] In other implementations, the role of the intelligent agent may have other names, other responsibilities, or other evaluation dimensions, such as a correlation analysis agent, a factual accuracy agent, a creativity and helpful agent, etc., which will not be listed here.

[0062] As can be seen from the above introduction of intelligent agents, intelligent agents with different roles can be distinguished by their responsibilities, evaluation dimensions, and key technical implementation points. Therefore, the central intelligent agent can analyze the evaluation dimensions or evaluation content required by the test task to configure the relevant intelligent agents with evaluation dimensions to participate in this model evaluation, thereby improving the relevance and accuracy of the evaluation.

[0063] Step S13: The central agent distributes the model outputs of several tested models to the role agents for evaluation.

[0064] In the embodiments of this application, such as Figure 3 As shown, the central agent distributes evaluation tasks to the role agents configured in step S12 and sends out the model outputs of each tested model, so that each role agent can act as a judge model to evaluate the model.

[0065] Step S14: The role agent determines the evaluation information of the model output of each tested model according to the agent's responsibilities and feeds it back to the central agent.

[0066] In this embodiment of the application, after receiving the evaluation task, each role-based intelligent agent uses its own professional knowledge to evaluate the evaluation model and feeds back the evaluation results to the central intelligent agent.

[0067] Specifically, each agent can evaluate the performance of the tested model's output across different evaluation dimensions according to its respective agent responsibility. The evaluation information output includes, but is not limited to: evaluation dimensions, evaluation scores, evaluation confidence, and comments, specifically represented as: {Dimension: "Logicality", Score: 4 / 5, Comment: "...", Confidence: 0.85}, or other forms of representation.

[0068] Step S15: The central agent generates evaluation results for several tested models based on the evaluation information of all role agents.

[0069] In this embodiment, the central agent summarizes and verifies the evaluation information of all role agents, thereby generating evaluation reports for several tested models. The evaluation reports include, but are not limited to, the following:

[0070] 1. Generate a scoring matrix for each role agent in each model.

[0071] 2. Generate a weighted scoring matrix for each model.

[0072] 3. Generate model rankings.

[0073] The model evaluation method proposed in this application upgrades the evaluation of large models from "single-point judge" to "multi-expert consultation" by specializing roles, decomposing tasks, and coordinating evaluation. This not only solves the inherent defects of single-judge models, but also builds a reliable, interpretable, and evolvable next-generation automated evaluation infrastructure.

[0074] based on Figure 2 In addition to the model evaluation method shown, this application also provides a verification scheme for the evaluation information. Please refer to the following for details. Figure 4 , Figure 4 This is a flowchart illustrating the second embodiment of the model evaluation method provided in this application.

[0075] like Figure 4 As shown, the specific steps are as follows:

[0076] Step S21: The central agent traverses the evaluation information of all role agents and determines whether the evaluation information of each role agent meets the evaluation requirements.

[0077] In this embodiment of the application, the central agent traverses the evaluation information of each role agent and determines whether the evaluation information of each role agent meets the evaluation requirements. If there is evaluation information that does not meet the evaluation requirements, then step S22 is executed.

[0078] Specifically, the evaluation requirements provided in this application include, but are not limited to, general evaluation requirements and role-based evaluation requirements. Among them, general evaluation requirements include, but are not limited to, the completeness and legality of the evaluation information. Completeness refers to whether the evaluation information should contain all the required information, and legality refers to whether the evaluation information is output in the preset format.

[0079] Role evaluation requirements refer to whether the role agent outputs relevant comments according to its own responsibilities and evaluation dimensions, whether the confidence score and evaluation score output by the role agent are within the preset score range of the role agent, and whether the evaluation dimensions output by the role agent are consistent with the evaluation dimensions it has recorded.

[0080] Step S22: The central agent removes the evaluation information of role agents that do not meet the evaluation requirements, or returns an instruction to re-evaluate.

[0081] In this embodiment, the central agent verifies and removes evaluation information that does not meet the evaluation requirements to ensure the accuracy of the evaluation report. Additionally, the central agent can interactively notify the role agent to re-evaluate and guide the role agent to output evaluation information that meets the requirements.

[0082] based on Figure 2 In addition to the model evaluation method shown, this application also provides another verification scheme for evaluation information. Please refer to the following for details. Figure 5 , Figure 5 This is a flowchart illustrating the third embodiment of the model evaluation method provided in this application.

[0083] like Figure 5 As shown, the specific steps are as follows:

[0084] Step S31: The central agent traverses the evaluation information of all role agents and determines whether the evaluation confidence value of the evaluation information of each role agent is lower than the preset threshold.

[0085] In this embodiment, the central agent extracts the evaluation confidence value from each evaluation information and determines whether the confidence value of the evaluation information is lower than a preset threshold, such as whether it is lower than 0.5. If so, the evaluation information is considered a low-confidence result, and the central agent performs step S32.

[0086] Step S32: The central agent removes the evaluation information of the role agent whose evaluation confidence value is lower than the preset threshold, or returns the instruction to re-evaluate.

[0087] In this embodiment, for evaluation information with a confidence value lower than a preset threshold, the central agent verifies and removes it to ensure the accuracy of the evaluation report. Additionally, the central agent can interactively notify the role agent to re-evaluate and guide the role agent to output evaluation information with higher confidence values.

[0088] based on Figure 2 In addition to the model evaluation method shown, this application also provides a model selection scheme. Please refer to the following for details. Figure 6 , Figure 6 This is a flowchart illustrating the fourth embodiment of the model evaluation method provided in this application.

[0089] like Figure 6 As shown, the specific steps are as follows:

[0090] Step S41: The central agent, for each tested model, integrates the evaluation information of all role agents to generate the total evaluation score of the tested model.

[0091] In this embodiment, the central agent extracts the evaluation scores and evaluation confidence from each evaluation information, and then merges the evaluation scores of all role agents for the same tested model to generate the total evaluation score of the tested model.

[0092] For example, the central agent can sum the evaluation scores output by all role agents, and the resulting score is the total evaluation score of the model being tested.

[0093] For example, the central agent can also weight the evaluation scores output by all role agents according to the evaluation confidence level, and then add them together to increase the influence of high confidence evaluation scores on the total evaluation score. The weighted sum is the total evaluation score of the model being tested.

[0094] For example, the central agent can also weight the evaluation scores output by all role agents according to the importance of the evaluation content to the evaluation requirements, and then add them together to increase the influence of high-confidence evaluation scores on the total evaluation score. The weighted score is the total evaluation score of the model being tested.

[0095] Step S42: The central agent determines the tested model with the highest total evaluation score as the winning model.

[0096] In this embodiment, the central agent compares the total evaluation scores of all tested models and outputs the tested model with the highest total score as the final winning model.

[0097] based on Figure 2 In addition to the model evaluation method shown, this application also provides a model selection scheme. Please refer to the following for details. Figure 7 , Figure 7 This is a flowchart illustrating the fifth embodiment of the model evaluation method provided in this application.

[0098] like Figure 7 As shown, the specific steps are as follows:

[0099] Step S51: The central agent determines the evaluation scores of several tested models in each evaluation dimension based on the evaluation information of all role agents.

[0100] Step S52: The central agent compares the evaluation scores of several tested models in each evaluation dimension to determine the winning model in each evaluation dimension.

[0101] In this embodiment of the application, the central agent can also compare the tested models for each evaluation dimension to determine the winning model for each evaluation dimension.

[0102] Specifically, the central agent can display the scores of model A and model B side-by-side in the form of a table or radar chart across various dimensions, which visually demonstrates their respective strengths and weaknesses:

[0103] Table 2

[0104]

[0105] In other specific embodiments, the central agent may also perform weighted summation of the evaluation scores of the tested model in each evaluation dimension to determine the total evaluation dimension score of the tested model, and take the tested model with the highest total evaluation dimension score as the final generated model.

[0106] In other specific embodiments, the central agent may also perform a weighted average of the evaluation scores of the tested model in each evaluation dimension to determine the comprehensive average score of the tested model in each evaluation dimension, and take the tested model with the highest comprehensive average score in each evaluation dimension as the final generated model.

[0107] It should be noted that the weights mentioned above refer to the fact that the weights for each evaluation dimension can be the same or different. When the weights for evaluation dimensions are different, the weight of each evaluation dimension is determined by the preset emphasis of the role's intelligent agent. For example, the evaluation dimensions of factual consistency and knowledge correctness output by the authenticity verification agent have a higher weight in the calculation compared to other evaluation dimensions.

[0108] Step S53: The central agent selects the tested model with the most wins as the final winning model.

[0109] In this embodiment, the central agent can output the tested model with the most wins as the final winning model.

[0110] Furthermore, the central agent can also output the ranking of the tested model, and the ranking criteria can be one or more of the following reference factors: the number of wins, the average score of the evaluation dimensions, and the total score of the evaluation dimensions.

[0111] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0112] To implement the above model evaluation method, this application also proposes a model evaluation platform, which can be found in the following details. Figure 8 , Figure 8 This is a schematic diagram of the structure of an embodiment of the model evaluation platform provided in this application.

[0113] The model evaluation platform 800 in this embodiment includes intelligent agents, specifically including a central intelligent agent 81 and various role intelligent agents 82. The central intelligent agent 81 is responsible for coordinating and calling various role intelligent agents 82 to implement the model evaluation method in the above embodiment. The implementation process of the method will not be described in detail here.

[0114] To implement the above model evaluation method, this application also proposes a model evaluation device, which can be found in the following details. Figure 9 , Figure 9 This is a schematic diagram of an embodiment of the model evaluation device provided in this application.

[0115] The model evaluation device 700 in this embodiment includes a processor 71, a memory 72, an input / output device 73, and a bus 74.

[0116] The processor 71, memory 72, and input / output device 73 are connected to the bus 74. The memory 72 stores program data, and the processor 71 is used to execute the program data to implement the model evaluation method described in the above embodiments.

[0117] In this embodiment, processor 71 can also be referred to as a CPU (Central Processing Unit). Processor 71 may be an integrated circuit chip with signal processing capabilities. Processor 71 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor, or processor 71 can be any conventional processor.

[0118] This application also provides a computer storage medium; please refer to the following: Figure 10 , Figure 10 This is a schematic diagram of a computer storage medium according to an embodiment of the present application. The computer storage medium 600 stores a computer program 61, which, when executed by a processor, is used to implement the model evaluation method of the above embodiment.

[0119] When the embodiments of this application are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0120] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A model evaluation method, characterized in that, The model evaluation method is applied to a model evaluation platform, which includes a central agent and several role agents; the model evaluation method includes: The central agent acquires the model outputs of several tested models executing test tasks; The central agent configures at least one role agent according to the evaluation dimensions of the test task; wherein, the evaluation dimensions include evaluation content and / or evaluation requirements; different role agents are distinguished by their responsibilities, evaluation dimensions, and / or key technical implementation points; The central agent distributes the model outputs of the several tested models to the role agents for evaluation; The role-based intelligent agent determines the evaluation information of the model output of each tested model according to its responsibilities and feeds it back to the central intelligent agent. The central agent generates evaluation results for the tested models based on the evaluation information from all role agents.

2. The model evaluation method according to claim 1, characterized in that, The evaluation information output by the role-based intelligent agent includes evaluation dimensions, evaluation scores, evaluation confidence levels, and / or comments.

3. The model evaluation method according to claim 2, characterized in that, Before the central agent generates the evaluation results of the tested models based on the evaluation information of all role agents, the model evaluation method further includes: The central agent traverses the evaluation information of all role agents and determines whether the evaluation information of each role agent meets the evaluation requirements. If not, the central agent removes the evaluation information of role agents that do not meet the evaluation requirements, or returns an instruction to re-evaluate.

4. The model evaluation method according to claim 2, characterized in that, Before the central agent generates the evaluation results of the tested models based on the evaluation information of all role agents, the model evaluation method further includes: The central agent traverses the evaluation information of all role agents and determines whether the evaluation confidence value of the evaluation information of each role agent is lower than a preset threshold. If so, the central agent removes the evaluation information of the role agent whose evaluation confidence value is lower than a preset threshold, or returns an instruction to re-evaluate.

5. The model evaluation method according to claim 1, characterized in that, The central agent generates evaluation results for the tested models based on the evaluation information from all role agents, including: The central agent, for each tested model, integrates the evaluation information from all role agents to generate the total evaluation score for the tested model. The central agent determines the tested model with the highest total evaluation score as the winning model.

6. The model evaluation method according to claim 5, characterized in that, The central agent, for each tested model, integrates the evaluation information from all role agents to generate a total evaluation score for that model, including: The central agent determines the evaluation score and evaluation content of each tested model in the evaluation information of all role agents; The central intelligent agent determines the weight of each evaluation item according to the evaluation requirements; The central agent performs a weighted fusion of the evaluation scores of each tested model across all role agents according to the weight of the evaluation content, to obtain the total evaluation score of each tested model.

7. The model evaluation method according to claim 1, characterized in that, The central agent generates evaluation results for the tested models based on the evaluation information from all role agents, including: The central agent determines the evaluation scores of the tested models in each evaluation dimension based on the evaluation information of all role agents. The central agent compares the evaluation scores of the several tested models in each evaluation dimension to determine the winning model in each evaluation dimension. The central agent selects the tested model with the most wins as the final winning model.

8. A model evaluation platform, characterized in that, The model evaluation platform includes an intelligent agent, which includes a central intelligent agent and role-based intelligent agents, and the intelligent agent executes the model evaluation method according to any one of claims 1 to 7.

9. A model evaluation device, characterized in that, The model evaluation device includes a memory and a processor coupled to the memory; The memory is used to store program data, and the processor is used to execute the program data to implement the model evaluation method as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that, The computer storage medium is used to store program data, which, when executed by the computer, is used to implement the model evaluation method as described in any one of claims 1 to 7.

Images