Model evaluation method and apparatus, and electronic device, storage medium and program product

Abstract

The present disclosure relates to the technical field of computers and relates to a model evaluation method and apparatus, and an electronic device, a storage medium and a program product. The method of the present disclosure comprises: on the basis of prompt information of a model to be evaluated, using an interactive robot to interact with said model; acquiring content generated by said model; and on the basis of the prompt information of said model and the content generated by said model, using an evaluation robot to evaluate said model to obtain an evaluation result.

Description

Model evaluation method, device, electronic device, storage medium and program product

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is based on the application with CN application number 202410216795.3 and application date February 27, 2024, and claims its priority. The disclosed content of the CN application is hereby introduced as a whole into this application. Technical Field

[0003] The present disclosure relates to the field of computer technology, and in particular to a model evaluation method, device, electronic device, storage medium, and program product. Background Art

[0004] Prompt Engineering is a technique in natural language processing that uses the design, optimization, and evaluation of input prompts to guide large language models to produce desired outputs. For example, the large language model is a Generative Pre-Trained Transformer (GPT) model.

[0005] A key requirement in prompt engineering is determining whether prompt information is appropriate. Currently, this is mainly done manually by evaluating the output of the model to determine whether the input prompt information is appropriate.

[0006] Summary of the Invention

[0007] This summary is provided to briefly introduce concepts that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0008] According to some embodiments of the present disclosure, a model evaluation method is provided, including: based on the prompt information of the model to be evaluated, using an interactive robot to interact with the model to be evaluated; obtaining content generated by the model to be evaluated; based on the prompt information and generated content of the model to be evaluated, using an evaluation robot to evaluate the model to be evaluated to obtain an evaluation result.

[0009] According to other embodiments of the present disclosure, a model evaluation device is provided, including: an interaction module, configured to interact with the model to be evaluated by using an interactive robot based on prompt information of the model to be evaluated; an acquisition module, configured to acquire content generated by the model to be evaluated; and an evaluation module, configured to evaluate the model to be evaluated by using an evaluation robot based on the prompt information and generated content of the model to be evaluated to obtain an evaluation result.

[0010] According to some further embodiments of the present disclosure, there is provided an electronic device comprising: a memory; and a processor coupled to the memory, wherein the processor is configured to execute the model evaluation method of any embodiment of the present disclosure based on instructions stored in the memory.

[0011] According to some further embodiments of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the model evaluation method of any embodiment of the present disclosure is performed.

[0012] According to some further embodiments of the present disclosure, a computer program product is provided, comprising: instructions, wherein when the instructions are executed by a processor, the model evaluation method of any embodiment of the present disclosure is implemented.

[0013] According to some further embodiments of the present disclosure, a computer program is provided, comprising: instructions, wherein when the instructions are executed by a processor, the model evaluation method of any embodiment of the present disclosure is implemented.

[0014] Other features, aspects and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The preferred embodiments of the present disclosure are described below with reference to the accompanying drawings. The drawings described herein are used to provide a further understanding of the present disclosure. Each of the drawings, together with the following detailed description, is included in this specification and forms a part of the specification to explain the present disclosure. It should be understood that the drawings described below only relate to some embodiments of the present disclosure and do not constitute a limitation of the present disclosure. In the drawings:

[0016] FIG1 is a flow chart showing a method for evaluating a model according to some embodiments of the present disclosure;

[0017] FIG2 is a flow chart showing a method for evaluating a model according to other embodiments of the present disclosure;

[0018] FIG3 is a flow chart showing a method for evaluating a model according to some other embodiments of the present disclosure;

[0019] FIG4 is a schematic diagram showing the structure of a model evaluation device according to some embodiments of the present disclosure;

[0020] FIG5 is a schematic structural diagram of an electronic device according to some embodiments of the present disclosure;

[0021] FIG6 shows a schematic structural diagram of a computer system according to some embodiments of the present disclosure.

[0022] It should be understood that, for ease of description, the dimensions of the various parts shown in the drawings are not necessarily drawn to scale. The same or similar reference numerals are used throughout the drawings to indicate the same or similar parts. Therefore, once an item is defined in one drawing, it may not be discussed further in subsequent drawings. DETAILED DESCRIPTION

[0023] The following will be combined with the accompanying drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. However, it is obvious that the embodiments described are only some embodiments of the present disclosure, rather than all embodiments. The following description of the embodiments is actually only illustrative and is in no way intended to limit the present disclosure and its application or use. It should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein.

[0024] It should be understood that the various steps described in the method embodiments of the present disclosure can be performed in different orders and / or performed in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect. Unless otherwise specifically stated, the relative arrangement, numerical expressions and numerical values ​​of the parts and steps set forth in these embodiments should be interpreted as being merely exemplary and do not limit the scope of the present disclosure.

[0025] As used in this disclosure, the term "include" and its variations are intended to be open-ended terms that include at least the following elements / features but do not exclude other elements / features, i.e., "including but not limited to." Furthermore, the term "comprise" and its variations are intended to be open-ended terms that include at least the following elements / features but do not exclude other elements / features, i.e., "including but not limited to." Therefore, "include" and "include" are synonymous. The term "based on" means "based, at least in part, on."

[0026] Reference throughout this specification to "one embodiment," "some embodiments," or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. For example, the term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Furthermore, the appearances of the phrases "in one embodiment," "in some embodiments," or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment, but may.

[0027] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules, or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules, or units. Unless otherwise specified, concepts such as "first" and "second" are not intended to imply that the objects described in such a manner must be in a given order in time, space, ranking, or any other manner.

[0028] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0029] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0030] The following detailed description of the embodiments of the present disclosure is provided in conjunction with the accompanying drawings, but the present disclosure is not limited to these specific embodiments. The following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. In addition, in one or more embodiments, specific features, structures, or characteristics may be combined in any suitable manner that will be apparent to those skilled in the art from this disclosure.

[0031] Manual evaluation to determine whether prompt information is appropriate and whether the model's output meets expectations is inefficient. In view of this, the present disclosure proposes a model evaluation method, which is described below in conjunction with Figures 1 to 3. The model to be evaluated, the interactive robot, and the evaluation robot in the present disclosure can all be agents.

[0032] Figure 1 is a flow chart of some embodiments of the evaluation method of the disclosed model. As shown in Figure 1 , the method of this embodiment includes: steps S102 to S106.

[0033] In step S102, based on the prompt information of the model to be evaluated, the interactive robot is used to interact with the model to be evaluated.

[0034] The model to be evaluated is generated based on the prompt information (prompt) of the model to be evaluated. For example, for a language model (or a large language model), the prompt information is input into the language model, and the language model adjusts its parameters based on the prompt information to output content that meets the requirements of the prompt information. The language model with the adjusted parameters based on the input prompt information can be used as the model to be evaluated. The model to be evaluated can be, for example, a translation model, a question-answering model, etc., but is not limited to these examples.

[0035] For example, the prompt information of the model to be evaluated includes: the role of the model to be evaluated (or the perspective during interaction), the description of the task, the description of the input, the description of the output, etc. The description of the input includes, for example, the type and format of the input, and the description of the output includes, for example, the expected standard of the output. For example, the prompt information of the model to be evaluated is: You are a translation assistant, responsible for translating English words into Chinese words, and comply with the translation word count requirements, do not output punctuation marks, and the words should be as colloquial as possible, not written. The role of the model to be evaluated is a translation assistant, and its task is to translate English words into Chinese words. The input is English words, the output is Chinese words, and the translation word count requirements are complied with, do not output punctuation marks, and the words should be as colloquial as possible, not written.

[0036] Based on the prompt information of the model to be evaluated, the interactive robot can automatically interact with the model to be evaluated. For example, the interactive robot can generate input information that matches the prompt information of the model to be evaluated and input it into the model to be evaluated. The interaction can be one or more times.

[0037] In step S104, the content generated by the model to be evaluated is obtained.

[0038] After the interactive robot inputs information into the model to be evaluated, it can obtain the content generated by the model to be evaluated. For example, if the model to be evaluated is a translation model, the interactive robot generates Chinese input into the model to be evaluated, and then obtains the English output generated by the model to be evaluated. For another example, if the model to be evaluated is a question-and-answer model, the interactive robot generates questions and then inputs them into the model to be evaluated, and then obtains the answers generated by the model to be evaluated.

[0039] In step S106, based on the prompt information and generated content of the model to be evaluated, the model to be evaluated is evaluated by using an evaluation robot to obtain an evaluation result.

[0040] Based on the prompt information and generated content of the model to be evaluated, the evaluation robot can automatically evaluate whether the content generated by the model to be evaluated matches its prompt information and whether it meets the requirements, thereby obtaining the evaluation results and determining whether the prompt information of the model to be evaluated is appropriate.

[0041] The method of the above embodiment can utilize an interactive robot to automatically interact with the model to be evaluated based on the prompt information of the model to be evaluated, eliminating the need for manual input of information into the model to be evaluated. After obtaining the content generated by the model to be evaluated, the evaluation robot automatically evaluates the model to be evaluated based on the prompt information and the generated content of the model to be evaluated, eliminating the need for manual judgment and evaluation. The overall evaluation of the model to be evaluated is completed automatically without any human intervention, improving the efficiency of evaluating the model to be evaluated and its prompt information, and enabling a more rapid and accurate determination of the appropriateness of the prompt information of the model to be evaluated.

[0042] The present disclosure provides some embodiments for a method of generating an interactive robot and an evaluation robot, a method of interacting an interactive robot with a model to be evaluated, and a method of evaluating a model to be evaluated by an evaluation robot, which are described below.

[0043] In some embodiments, dedicated prompt information of the interactive robot is generated based on the prompt information of the model to be evaluated; an interactive robot is generated based on the dedicated prompt information of the interactive robot; and the interactive robot is used to interact with the model to be evaluated.

[0044] The interactive robot's dedicated prompts are generated based on the prompts of the model being evaluated. Therefore, the interactive robot can be customized to generate interactive input information specifically for the model being evaluated. For example, if the model being evaluated is a translation model, the interactive robot can directly generate sentences to be translated. If the model being evaluated is a question-and-answer model, the interactive robot can directly generate questions.

[0045] In some embodiments, based on the type of the model to be evaluated, the database corresponding to the model to be evaluated is determined, wherein different types of models to be evaluated correspond to different databases; the input information of the first interaction is obtained from the database corresponding to the model to be evaluated and input into the model to be evaluated; in each interaction after the first interaction, the interactive robot is used to generate the input information of this interaction based on the content generated by the model to be evaluated in the previous interaction, and input into the model to be evaluated.

[0046] For example, the model to be evaluated may be a translation model, a question-and-answer model, or a chat model, each corresponding to a different database or storage location. The database can store exemplary input information for the model to be evaluated, allowing the interactive robot to further learn how to interact with the model to be evaluated and improve interaction accuracy. The interactive robot can retrieve the input information from the database during its first interaction with the model to be evaluated. For example, if the model to be evaluated is a chat model, the interactive robot can retrieve the input information from the database for the first interaction as "What's the weather like today?"

[0047] After the first interaction, the interactive robot can generate input information for this interaction based on the content generated by the model to be evaluated last time. The interactive robot can have multiple rounds of dialogue with the model to be evaluated.

[0048] In some embodiments, a first prompt generation model is generated based on the prompt information of the first prompt generation model; and the first prompt generation model is used to generate special prompt information for the interactive robot based on the prompt information of the model to be evaluated.

[0049] For example, the prompt information of the first prompt generation model is input into a machine learning model (for example, a large language model), and the machine learning model adjusts parameters according to the prompt information to obtain the first prompt generation model.

[0050] The first prompt generation model can automatically generate special prompt information for the interactive robot based on the prompt information of the model to be evaluated, thereby further improving the generation efficiency of the interactive robot.

[0051] In some embodiments, the prompt information of the first prompt generation model is used to prompt the first prompt generation model to receive the prompt information of the first model and generate prompt information of the second model that interacts with the first model based on characteristics of the first model.

[0052] The prompt information of the first prompt generation model may include: descriptions of the inputs, outputs, and tasks of the first prompt generation model. For example, the input description may be prompt information for receiving the first model, the output description may be prompt information for generating the second model, and the task description may be prompt information for generating the second model that interacts with the first model based on the characteristics of the first model. The prompt information of the first prompt generation model may also include examples to assist the first prompt generation model in understanding the tasks, inputs, outputs, etc. to be performed, thereby improving the accuracy of the specialized prompt information generated by the first prompt generation model for the interactive robot.

[0053] For example, the prompt information of the first prompt generation model is: Now you are provided with the prompt content of a model AI No. 1. You need to generate a new model AI No. 2 based on the prompt content of AI No. 1, and require AI No. 2 to be able to communicate according to the characteristics of AI No. 1. For example: The basic role of AI No. 1 is set as a math teacher who helps students solve problems, then AI No. 2 should be a student responsible for asking math questions to the teacher. The prompt of AI No. 2 can be set as "You are a student, actively asking math questions to the teacher". You just need to provide the corresponding prompt content for AI No. 2.

[0054] In some embodiments, the prompt information of the model to be evaluated is input into the first prompt generation model as the prompt information of the first model; the first prompt generation model is used to determine the characteristics of the first model based on the prompt information of the model to be evaluated; the first prompt generation model is used to generate prompt information for the second model that interacts with the first model based on the characteristics of the first model, as special prompt information for the interactive robot.

[0055] The model to be evaluated is taken as the first model, and the interactive robot is taken as the second model. The characteristics of the first model are the characteristics of the model to be evaluated, such as the type of information received by the model to be evaluated (the type of input information) and the role of the model to be evaluated when interacting.

[0056] In some embodiments, the prompt information of the model to be evaluated is parsed, and the type of information received by the model to be evaluated and the role of the model to be evaluated when interacting are extracted from the prompt information of the model to be evaluated as characteristics of the first model; based on the type of information received by the model to be evaluated and the role of the model to be evaluated when interacting, the type of information generated by the second model and the role of the second model when interacting are determined; based on the type of information generated by the second model and the role of the second model when interacting, prompt information of the second model is generated.

[0057] The first prompt generation model can perform semantic understanding and analysis on the prompt information of the model to be evaluated, extract the characteristics of the model to be evaluated, and then determine the characteristics of the interactive robot to generate prompt information of the interactive robot.

[0058] For example, the prompt message for the model under evaluation might read: "You are an online doctor. Please answer medical questions from users. Do not answer questions that are not medical." The type of information received by the model under evaluation is medical-related inquiries, and the user's role during the interaction is that of an online doctor. For example, the prompt message for the generated interactive robot might read: "You are a patient seeking consultation. Please ask the online doctor your medical questions." The type of information generated by the interactive robot is medical-related questions, and the user's role during the interaction is that of a patient.

[0059] In some embodiments, an interactive robot is used to generate interactive input information according to the type of information generated by the interactive robot and the role of the interactive robot during interaction, and the information is input into the model to be evaluated.

[0060] For example, the prompt message for the model to be evaluated is: You are an online doctor. Please respond to users' medical inquiries. Do not answer non-medical questions. The prompt message for the interactive robot is: You are a patient in need of consultation. Please ask the online doctor your medical questions. The actual dialogue between the interactive robot and the model to be evaluated can be: Interactive robot: I have been feeling tired, dizzy, have no appetite, and have a mild headache recently. What should I do? What kind of illness might this be? Model to be evaluated: The symptoms you describe are quite common and may be related to many diseases. Common causes may include cardiovascular disease, anemia, hypothyroidism, malnutrition, emotional problems, etc. Therefore, it is recommended that you seek medical attention as soon as possible and have a professional doctor perform a detailed physical examination and evaluation to understand the pathological cause.

[0061] In the above embodiment, prompt information of the interactive robot is automatically generated based on the prompt information of the model to be evaluated, and then the interactive robot is automatically generated without human participation. The interactive robot can interact with the model to be evaluated based on the characteristics of the model to be evaluated, thereby improving the efficiency and accuracy of the generation of the interactive robot and further improving the efficiency and accuracy of the evaluation.

[0062] The above embodiments describe a method for generating a customized interactive robot and a method for a customized interactive robot to interact with a robot to be evaluated. The following describes a method for generating a general interactive robot and a method for a general interactive robot to interact with a robot to be evaluated in combination with some embodiments.

[0063] In some embodiments, an interactive robot is generated based on the general prompt information of the interactive robot; the prompt information of the model to be evaluated is input into the interactive robot, and the interactive robot is used to interact with the model to be evaluated.

[0064] Unlike the dedicated prompt information of the interactive robot, the general prompt information of the interactive robot does not need to be generated based on the prompt information of the model to be evaluated. Instead, when the interactive robot interacts with the model to be evaluated, the interactive robot generates the input information of the interaction based on the prompt information of the model to be evaluated.

[0065] In some embodiments, based on the type of the model to be evaluated, a database corresponding to the model to be evaluated is determined, wherein different types of models to be evaluated correspond to different databases; the input information of the first interaction is obtained from the database corresponding to the model to be evaluated and input into the model to be evaluated; in each interaction after the first interaction, an interactive robot is used to generate input information for this interaction based on the content generated by the model to be evaluated in the previous interaction and the prompt information of the model to be evaluated, and input into the model to be evaluated.

[0066] The interactive robot can interact with the model to be evaluated multiple times based on the prompt information of the model to be evaluated.

[0067] In some embodiments, the general prompt information of the interactive robot is used to prompt the interactive robot to receive the prompt information of the third model and interact with the third model based on the characteristics of the third model.

[0068] The interactive robot's general prompt information may include descriptions of the robot's inputs, outputs, and tasks. For example, the input description may include prompts for receiving third-party model information, the output description may include interacting with the third model, and the task description may include interacting with the third model based on its characteristics. General prompt information may also include examples to help the interactive robot understand the task, inputs, and outputs to be performed, thereby improving the accuracy of the generated interactive robot.

[0069] For example, the general prompt message of the interactive robot is: Now we provide you with the prompt content of a large language model AI, and you need to communicate with it based on the characteristics of the AI. For example: the basic role of the AI ​​is set as a math teacher who helps students solve problems, then you should be a student responsible for asking math questions to the teacher.

[0070] In some embodiments, the prompt information of the model to be evaluated is input into the interactive robot as the prompt information of the third model; the interactive robot is used to determine the characteristics of the third model based on the prompt information of the model to be evaluated; and the interactive robot is used to generate interactive input information based on the characteristics of the third model.

[0071] The model to be evaluated is regarded as the third model, and the characteristics of the third model are the characteristics of the model to be evaluated, such as the type of information received by the model to be evaluated (the type of input information) and the role of the model to be evaluated during interaction.

[0072] For example, the general prompt information of the interactive robot is input into the language model (or large language model), and the language model adjusts the parameters according to the prompt information to obtain the interactive robot.

[0073] In some embodiments, the prompt information of the model to be evaluated is parsed, and the type of information received by the model to be evaluated and the role of the model to be evaluated when interacting are extracted from the prompt information of the model to be evaluated as characteristics of the third model; based on the type of information received by the model to be evaluated and the role of the model to be evaluated when interacting, the type of information generated by the interactive robot and the role of the interactive robot when interacting are determined; based on the type of information generated by the interactive robot and the role of the interactive robot when interacting, the input information of the interaction is generated.

[0074] The interactive robot can semantically understand and parse the prompt information of the model to be evaluated, extract the characteristics of the model to be evaluated, and then interact with the model to be evaluated based on the characteristics of the model to be evaluated.

[0075] For example, the prompt message for the model under evaluation reads: "You are an online doctor. Please respond to users' medical inquiries. Do not answer non-medical questions." After entering this prompt message into the interactive robot, the interactive robot will understand and parse it. The actual conversation with the model under evaluation might be: Interactive robot: "I've been feeling tired, dizzy, have no appetite, and have a mild headache lately. What should I do? What could be the cause of this?" The model under evaluation responds: "The symptoms you describe are quite common and may be related to a wide range of conditions." Common causes may include cardiovascular disease, anemia, hypothyroidism, malnutrition, and emotional issues. Therefore, it is recommended that you seek medical attention as soon as possible for a professional doctor to conduct a detailed physical examination and evaluation to understand the underlying cause.

[0076] In the above embodiment, an interactive robot is automatically generated based on the general prompt information of the interactive robot. The interactive robot can interact with the model to be evaluated based on the prompt information of the model to be evaluated, thereby improving the efficiency and accuracy of the generation of the interactive robot and further improving the efficiency and accuracy of the interaction and evaluation.

[0077] The above embodiments describe how to generate an interactive robot and how the interactive robot interacts with the model to be evaluated. The following describes how to generate an evaluation robot and how to evaluate the model to be evaluated in conjunction with some embodiments.

[0078] In some embodiments, special prompt information for an evaluation robot is generated based on prompt information of the model to be evaluated; an evaluation robot is generated based on the special prompt information of the evaluation robot; and the evaluation robot is used to evaluate the model to be evaluated based on the generated content.

[0079] The dedicated prompt information of the evaluation robot is generated based on the prompt information of the model to be evaluated. Therefore, the evaluation robot can be used exclusively for evaluating the model to be evaluated as a customized evaluation robot.

[0080] In some embodiments, a second prompt generation model is generated based on the prompt information of the second prompt generation model; and the second prompt generation model is used to generate special prompt information for the evaluation robot based on the prompt information of the model to be evaluated.

[0081] For example, the prompt information of the second prompt generation model is input into a machine learning model (e.g., a large language model), and the machine learning model adjusts parameters according to the prompt information to obtain the second prompt generation model.

[0082] The second prompt generation model can automatically generate special prompt information for the evaluation robot based on the prompt information of the model to be evaluated, thereby further improving the generation efficiency of the evaluation robot.

[0083] In some embodiments, the prompt information of the second prompt generation model is used to prompt the second prompt generation model to receive the prompt information of the fourth model, generate the prompt information of the fifth model, and the fifth model receives the input information and output information of the fourth model, determines whether the output information of the fourth model meets the expected standards based on the prompt information, input information and output information of the fourth model, and outputs the result.

[0084] The prompt information of the second prompt generation model may include: a description of the input of the second prompt generation model, a description of the output, a description of the task, and other information. For example, the description of the input is to receive the prompt information of the fourth model, the description of the output is to generate the prompt information of the fifth model, the description of the task is to generate the prompt information of the fifth model based on the prompt information of the fourth model, and the fifth model receives the input information and output information of the fourth model, determines whether the output information of the fourth model meets the expected standards based on the prompt information, input information and output information of the fourth model, and outputs the results. The prompt information of the second prompt generation model can also include examples, which can assist the second prompt generation model in understanding the tasks, inputs, outputs, etc. to be performed, and improve the accuracy of the special prompt information of the evaluation robot generated by the second prompt generation model.

[0085] In some embodiments, the dedicated prompt information of the evaluation robot includes first format information for indicating receiving input information of the model to be evaluated and second format information for receiving content generated by the model to be evaluated.

[0086] The first and second format information are used to create placeholders for received information. For example, the prompt information for the second prompt generation model is as follows: You are now provided with prompt content A for model AI 1. You need to generate a new model AI 2 based on A. This model is required to receive the following information: B: user input content for AI 1 {{user input placeholder}}, C: AI 1 output content {{model output placeholder}}, and based on information A, B, and C, determine whether AI 1's output meets the expected standards. If so, output true; otherwise, output false. You only need to provide the prompt content corresponding to AI 2.

[0087] In some embodiments, the prompt information of the model to be evaluated is input into the second prompt generation model as the prompt information of the fourth model; the second prompt generation model is used to determine the type of input information received by the fifth model from the fourth model, the type of output information, and the expected standard based on the prompt information of the model to be evaluated; the second prompt generation model is used to generate the prompt information of the fifth model based on the type of input information received by the fifth model from the fourth model, the type of output information, and the expected standard as special prompt information for the evaluation robot.

[0088] The model to be evaluated is considered the fourth model, and the evaluation robot is considered the fifth model. The fourth model's input information is the input generated by the interactive robot, and the fourth model's output information is the content generated by the model to be evaluated. Expected standards describe the requirements and standards that the content generated by the model to be evaluated must meet.

[0089] In some embodiments, the prompt information of the model to be evaluated is parsed, and the type of information received and the type of content generated by the model to be evaluated are extracted from the prompt information of the model to be evaluated as the type of input information and the type of output information of the fourth model, and the standards that the content generated by the model to be evaluated must meet are extracted as expected standards.

[0090] The second prompt generation model can semantically understand and parse the prompt information of the model to be evaluated, and extract the type of information received by the model to be evaluated, the type of content generated, and the standards that the content generated by the model to be evaluated must meet.

[0091] For example, the prompt message for the model to be evaluated is: You are a translation assistant, responsible for translating English words into Chinese. You must adhere to the word count requirement, avoid punctuation, and use as much colloquial language as possible, avoiding formal language. The model receives information in English and generates content in Chinese. The standards required for the content generated by the model to be evaluated are: adhere to the word count requirement, avoid punctuation, and use as much colloquial language as possible, avoiding formal language. For example, the prompt message for the generated evaluation robot is: You are a quality inspection system responsible for supervising a foreign language translation model. You need to receive the following information: B: English input from a human user to the translation model, i.e., the English words the user wishes to translate {{user input placeholder}}; C: Chinese output from the translation model, i.e., the translation result the model believes {{model output placeholder}}; Your task is to evaluate whether the translation model's output meets the following standards: 1. The output must adhere to the word count requirement and cannot contain overflows or omissions; 2. Punctuation must be absent; 3. The translated words must be as colloquial as possible, avoiding formal language. If these conditions are met, you output true; otherwise, you output false and explain in detail which rule the model output violated.

[0092] An evaluation robot is generated based on the prompt information of the evaluation robot, and the evaluation results can be output.

[0093] In some embodiments, an evaluation robot is used to receive input information and generated content of the model to be evaluated, wherein the input information of the model to be evaluated is input information generated by the interactive robot for interacting with the model to be evaluated; based on the input information and generated content of the model to be evaluated, it is determined whether the content generated by the model to be evaluated meets the expected standards.

[0094] The evaluation robot can perform scoring (evaluation results) in different dimensions according to needs. It only needs to control the output content of the evaluation robot, for example, use json format to describe the scoring results of different dimensions. In the case where the interactive robot interacts with the model to be evaluated multiple times, it can be evaluated for each interaction or for multiple interactions based on the content generated by the model to be evaluated. The specific evaluation can be determined according to the type of the model to be evaluated. For example, if the model to be evaluated is a translation model, it can be evaluated for each interaction based on the content generated by the model to be evaluated. For another example, if the model to be evaluated is a chat model, multiple interactions belonging to the same topic can be determined based on the context of the interaction, and multiple interactions can be evaluated based on the content generated by the model to be evaluated.

[0095] For example, the prompt for the model being evaluated reads: "You are a translation assistant, responsible for translating English words into Chinese. You must adhere to the word count requirement, avoid punctuation, and use colloquial language as much as possible, avoiding formal language." The evaluation process using an evaluation robot is as follows: The input to the evaluation robot is: B: dog, C: dog. The evaluation robot outputs: false. The translated word is not as colloquial as possible; a more common translation would be "dog."

[0096] In the above embodiment, prompt information of the evaluation robot is automatically generated based on the prompt information of the model to be evaluated, and then the evaluation robot is automatically generated without human participation. The evaluation robot can evaluate the machine prompt information of the model to be evaluated, thereby improving the efficiency and accuracy of the evaluation robot generation and further improving the efficiency and accuracy of the evaluation.

[0097] The above embodiments describe a method for generating a customized evaluation robot and a method for a customized evaluation robot to perform evaluation. The following describes a method for generating a general evaluation robot and a method for a general evaluation robot to perform evaluation in conjunction with some embodiments.

[0098] In some embodiments, an evaluation robot is generated based on the general prompt information of the evaluation robot; the prompt information of the model to be evaluated and the generated content are input into the evaluation robot, and the evaluation robot is used to evaluate the model to be evaluated.

[0099] The general prompt information of the evaluation robot does not need to be generated according to the model to be evaluated. The evaluation robot needs to perform evaluation according to the prompt information of the model to be evaluated.

[0100] In some embodiments, the general prompt information is used to prompt the evaluation robot to receive the prompt information and output information of the sixth model, determine whether the output information of the sixth model meets the expected standards based on the prompt information and output information of the sixth model, and output the results.

[0101] The general prompt information for the evaluation robot may include: descriptions of the evaluation robot's inputs, outputs, and tasks. For example, the input description may be the result of receiving the prompt information of the sixth model, the output description may be the result of whether the output information of the sixth model meets the expected standards, and the task description may be the result of determining whether the output information of the sixth model meets the expected standards based on the prompt information and output information of the sixth model and outputting the result. The general prompt information may also include examples to help the evaluation robot understand the tasks, inputs, outputs, etc. to be performed, thereby improving the accuracy of the generated evaluation robot.

[0102] For example, the general prompt message of the evaluation robot is: You are an evaluation system. Now we provide you with the prompt content of a model and the output content of the model to help you judge whether the output content of the model meets the expected standards. If it does, output true; if it does not, output false, and give your reasons.

[0103] In some embodiments, the prompt information, input information and generated content of the model to be evaluated are input into the evaluation robot as the prompt information, input information and output information of the sixth model, wherein the input information of the model to be evaluated is the input information generated by the interactive robot for interacting with the model to be evaluated; the evaluation robot is used to determine the expected standards based on the prompt information of the model to be evaluated; the evaluation robot is used to determine whether the output information of the model to be evaluated meets the expected standards based on the input information and generated content of the model to be evaluated, and an evaluation result is obtained.

[0104] The model to be evaluated is taken as the sixth model. The input information of the sixth model is the input information generated by the interactive robot for interacting with the model to be evaluated. The output information of the sixth model is the content generated by the model to be evaluated.

[0105] For example, the general prompt information of the evaluation robot is input into a machine learning model (e.g., a large language model), and the machine learning model adjusts parameters according to the prompt information to obtain the evaluation robot.

[0106] In some embodiments, the evaluation robot understands and parses the expected standards, understands and parses the generated content, matches the generated content with the expected standards, and obtains evaluation results.

[0107] For example, the prompt information of the model to be evaluated is: You are a translation assistant responsible for translating English words into Chinese, complying with the translation word count requirements, not outputting punctuation marks, and making the words as colloquial as possible to avoid being written. Input the prompt information of the model to be evaluated, the input information generated by the interactive robot for interacting with the model to be evaluated (translate this word: dog, word count requirement: 1), and the content generated by the model to be evaluated (dog) into the evaluation robot. The information output by the evaluation robot is: false, reason: Although "dog" is indeed the translation of "dog", in Chinese, "dog" is more of a written term and not colloquial enough. In spoken language, we more commonly use "dog" to represent "dog".

[0108] In the above embodiments, according to the general prompt information of the evaluation robot, an evaluation robot is automatically generated. The evaluation robot can evaluate the prompt information of the model to be evaluated according to the prompt information of the model to be evaluated, improving the efficiency and accuracy of the evaluation.

[0109] Some embodiments of evaluating the model to be evaluated using a customized interactive robot and an evaluation robot are described below in conjunction with Figure 2.

[0110] Figure 2 is a flowchart of some other embodiments of the evaluation method of the present disclosure model. As shown in Figure 2, the method of this embodiment includes: steps S202 to S216.

[0111] In step S202, input the prompt information (prompt) to be evaluated into the large model (machine learning model) to determine the model to be evaluated.

[0112] In step S204, input the prompt information of the model to be evaluated (i.e., the prompt information to be evaluated) into the first prompt generation model and the second prompt generation model.

[0113] The first prompt generation model is used to generate the dedicated prompt information of the customized interactive robot, and the second prompt generation model is used to generate the dedicated prompt information of the customized evaluation robot.

[0114] In step S206, the first prompt generation model automatically generates the prompt information (dedicated prompt information) of the customized interactive robot.

[0115] In step S208, the second prompt generation model automatically generates the prompt information of the customized evaluation robot.

[0116] In step S210, generate a customized interactive robot according to the prompt information of the customized interactive robot.

[0117] The prompt information of the customized interactive robot can be proofread manually and then the customized interactive robot can be generated.

[0118] In step S212, a customized evaluation robot is generated according to the prompt information of the customized evaluation robot.

[0119] The prompt information of the customized evaluation robot can be manually proofread and then the evaluation robot can be generated.

[0120] In step S214 , the customized interactive robot automatically generates input information for interacting with the model to be evaluated.

[0121] In step S216, the customized evaluation robot automatically determines the evaluation result based on the input information and generated content of the model to be evaluated.

[0122] The following describes some embodiments of using a general interactive robot and a general evaluation robot to evaluate a model to be evaluated in conjunction with FIG3 .

[0123] Figure 3 is a flow chart of some other embodiments of the evaluation method of the disclosed model. As shown in Figure 3, the method of this embodiment includes: steps S302 to S314.

[0124] In step S302, the prompt information of the general interactive robot (general prompt information) is input into the macro model to generate the general interactive robot.

[0125] In step S304, the prompt information of the general evaluation robot is input into the large model to generate the general evaluation robot.

[0126] In step S306, prompt information to be evaluated is input into the large model to determine the model to be evaluated.

[0127] Step S306 may be performed before steps S302 and S304 or in parallel with steps S302 and S304.

[0128] In step S308, the prompt information of the model to be evaluated is input into the general interactive robot.

[0129] In step S310, the prompt information of the model to be evaluated is input into the general evaluation robot.

[0130] In step S312, the general interactive robot automatically generates input information for interacting with the model to be evaluated based on the prompt information of the model to be evaluated.

[0131] In step S314, the general evaluation robot automatically determines the evaluation result according to the prompt information of the model to be evaluated, the input information of the model to be evaluated, and the generated content.

[0132] Based on the principle of large-scale model confrontation, this method introduces an interactive robot that automatically interacts with the model to be evaluated, replacing traditional human-computer dialogue, improving efficiency and reducing costs. The method also introduces an evaluation robot that automatically evaluates the content generated by the model to be evaluated, replacing manual evaluation, improving efficiency and reducing costs.

[0133] Customized interactive robots can also be used together with general evaluation robots, or general interactive robots can also be used together with customized evaluation robots.

[0134] The present disclosure also provides a model evaluation device, which is described below in conjunction with FIG4 .

[0135] FIG4 is a structural diagram of some embodiments of the model evaluation device disclosed herein. As shown in FIG4 , the model evaluation method device 40 of this embodiment includes: an interaction module 410 , an acquisition module 420 , and an evaluation module 430 .

[0136] The interaction module 410 is configured to utilize an interactive robot to interact with the model to be evaluated based on the prompt information of the model to be evaluated.

[0137] The acquisition module 420 is configured to acquire the content generated by the model to be evaluated.

[0138] The evaluation module 430 is configured to evaluate the model to be evaluated using an evaluation robot based on the prompt information and generated content of the model to be evaluated to obtain an evaluation result.

[0139] In some embodiments, the interaction module 410 is configured to generate special prompt information for the interactive robot based on the prompt information of the model to be evaluated; generate an interactive robot based on the special prompt information of the interactive robot; and use the interactive robot to interact with the model to be evaluated.

[0140] In some embodiments, the interaction module 410 is configured to determine the database corresponding to the model to be evaluated based on the type of the model to be evaluated, where different types of models to be evaluated correspond to different databases; obtain the input information of the first interaction from the database corresponding to the model to be evaluated, and input it into the model to be evaluated; in each interaction after the first interaction, use the interactive robot to generate the input information of this interaction based on the content generated by the model to be evaluated in the previous interaction, and input it into the model to be evaluated.

[0141] In some embodiments, the interaction module 410 is configured to generate a first prompt generation model based on the prompt information of the first prompt generation model; and use the first prompt generation model to generate special prompt information for the interactive robot based on the prompt information of the model to be evaluated.

[0142] In some embodiments, the prompt information of the first prompt generation model is used to prompt the first prompt generation model to receive the prompt information of the first model and generate prompt information of the second model that interacts with the first model based on the characteristics of the first model. The interaction module 410 is configured to input the prompt information of the model to be evaluated into the first prompt generation model as the prompt information of the first model; use the first prompt generation model to determine the characteristics of the first model according to the prompt information of the model to be evaluated; use the first prompt generation model to generate prompt information of the second model that interacts with the first model according to the characteristics of the first model as special prompt information for the interactive robot.

[0143] In some embodiments, the interaction module 410 is configured to parse the prompt information of the model to be evaluated, extract the type of information received by the model to be evaluated and the role of the model to be evaluated when interacting from the prompt information of the model to be evaluated as the characteristics of the first model; determine the type of information generated by the second model and the role of the second model when interacting according to the type of information received by the model to be evaluated and the role of the model to be evaluated when interacting; generate prompt information for the second model according to the type of information generated by the second model and the role of the second model when interacting.

[0144] In some embodiments, the interaction module 410 is configured to utilize the interactive robot to generate interactive input information according to the type of information generated by the interactive robot and the role of the interactive robot during interaction, and input the information into the model to be evaluated.

[0145] In some embodiments, the interaction module 410 is configured to generate an interactive robot based on the general prompt information of the interactive robot; input the prompt information of the model to be evaluated into the interactive robot, and use the interactive robot to interact with the model to be evaluated.

[0146] In some embodiments, the interaction module 410 is configured to determine the database corresponding to the model to be evaluated based on the type of the model to be evaluated, where different types of models to be evaluated correspond to different databases; obtain the input information of the first interaction from the database corresponding to the model to be evaluated, and input it into the model to be evaluated; in each interaction after the first interaction, use the interactive robot to generate the input information of this interaction based on the content generated by the model to be evaluated in the previous interaction and the prompt information of the model to be evaluated, and input it into the model to be evaluated.

[0147] In some embodiments, the general prompt information of the interactive robot is used to prompt the interactive robot to receive prompt information of the third model, and interact with the third model based on the characteristics of the third model. The interaction module 410 is configured to input the prompt information of the model to be evaluated into the interactive robot as the prompt information of the third model; use the interactive robot to determine the characteristics of the third model according to the prompt information of the model to be evaluated; use the interactive robot to generate interactive input information according to the characteristics of the third model.

[0148] In some embodiments, the interaction module 410 is configured to parse the prompt information of the model to be evaluated, extract the type of information received by the model to be evaluated and the role of the model to be evaluated when interacting from the prompt information of the model to be evaluated as the characteristics of the third model; determine the type of information generated by the interactive robot and the role of the interactive robot when interacting according to the type of information received by the model to be evaluated and the role of the model to be evaluated when interacting; generate input information for the interaction according to the type of information generated by the interactive robot and the role of the interactive robot when interacting.

[0149] In some embodiments, the evaluation module 430 is configured to generate special prompt information for the evaluation robot based on the prompt information of the model to be evaluated; generate an evaluation robot based on the special prompt information of the evaluation robot; and use the evaluation robot to evaluate the model to be evaluated based on the generated content.

[0150] In some embodiments, the evaluation module 430 is configured to generate a second prompt generation model based on the prompt information of the second prompt generation model; and use the second prompt generation model to generate special prompt information for the evaluation robot based on the prompt information of the model to be evaluated.

[0151] In some embodiments, the prompt information of the second prompt generation model is used to prompt the second prompt generation model to receive the prompt information of the fourth model and generate the prompt information of the fifth model, and the fifth model receives the input information and output information of the fourth model, determines whether the output information of the fourth model meets the expected standards and outputs the results based on the prompt information, input information and output information of the fourth model. The evaluation module 430 is configured to input the prompt information of the model to be evaluated into the second prompt generation model as the prompt information of the fourth model; uses the second prompt generation model to determine the type of input information, type of output information and expected standards received by the fifth model according to the prompt information of the model to be evaluated; uses the second prompt generation model to generate the prompt information of the fifth model according to the type of input information, type of output information and expected standards received by the fifth model according to the fourth model, as special prompt information for the evaluation robot.

[0152] In some embodiments, the evaluation module 430 is configured to parse the prompt information of the model to be evaluated, extract the type of information received and the type of content generated by the model to be evaluated from the prompt information of the model to be evaluated as the type of input information and the type of output information of the fourth model, and extract the standards that the content generated by the model to be evaluated must meet as the expected standards.

[0153] In some embodiments, the dedicated prompt information of the evaluation robot includes first format information for indicating receiving input information of the model to be evaluated and second format information for receiving content generated by the model to be evaluated.

[0154] In some embodiments, the evaluation module 430 is configured to utilize an evaluation robot to receive input information and generated content of the model to be evaluated, wherein the input information of the model to be evaluated is input information generated by the interactive robot for interacting with the model to be evaluated; and determine whether the content generated by the model to be evaluated meets the expected standards based on the input information and generated content of the model to be evaluated.

[0155] In some embodiments, the evaluation module 430 is configured to generate an evaluation robot based on the general prompt information of the evaluation robot; input the prompt information of the model to be evaluated and the generated content into the evaluation robot, and use the evaluation robot to evaluate the model to be evaluated.

[0156] In some embodiments, the general prompt information is used to prompt the evaluation robot to receive the prompt information and output information of the sixth model, determine whether the output information of the sixth model meets the expected standards based on the prompt information and output information of the sixth model, and output the results. The evaluation module 430 is configured to input the prompt information, input information and generated content of the model to be evaluated into the evaluation robot as the prompt information, input information and output information of the sixth model, wherein the input information of the model to be evaluated is the input information generated by the interactive robot to interact with the model to be evaluated; use the evaluation robot to determine the expected standards based on the prompt information of the model to be evaluated; use the evaluation robot to determine whether the output information of the model to be evaluated meets the expected standards based on the input information and generated content of the model to be evaluated, and obtain the evaluation results.

[0157] In some embodiments, the model to be evaluated is generated based on prompt information of the model to be evaluated.

[0158] It should be noted that the aforementioned units (modules) are merely logical modules divided according to the specific functions they implement, and are not intended to limit specific implementation methods. For example, they may be implemented in software, hardware, or a combination of software and hardware. In actual implementation, the aforementioned units may be implemented as independent physical entities, or they may be implemented by a single entity (e.g., a processor (CPU or DSP, etc.), an integrated circuit, etc.). In addition, the aforementioned units are illustrated with dashed lines in the drawings to indicate that these units may not actually exist, and that the operations / functions they implement may be implemented by the processing circuit itself.

[0159] In addition, although not shown, the device may also include a memory that can store various information generated by the device and the various units contained in the device during operation, programs and data used for operation, data to be sent by the communication unit, etc. The memory can be volatile memory and / or non-volatile memory. For example, the memory can include but is not limited to random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), read-only memory (ROM), and flash memory. Of course, the memory can also be located outside the device. Optionally, although not shown, the device may also include a communication unit that can be used to communicate with other devices. In one example, the communication unit can be implemented in an appropriate manner known in the art, for example, including communication components such as an antenna array and / or a radio frequency link, various types of interfaces, communication units, etc. This will not be described in detail here. In addition, the device may also include other components not shown, such as a radio frequency link, a baseband processing unit, a network interface, a processor, a controller, etc. This will not be described in detail here.

[0160] Some embodiments of the present disclosure also provide an electronic device. Figure 5 shows a block diagram of some embodiments of the electronic device of the present disclosure. For example, in some embodiments, the electronic device 50 can be various types of devices, for example, including but not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. For example, the electronic device 50 may include a display panel for displaying data and / or execution results utilized in the scheme of the present disclosure. For example, the display panel can be of various shapes, such as a rectangular panel, an elliptical panel, or a polygonal panel. In addition, the display panel can be not only a flat panel, but also a curved panel or even a spherical panel.

[0161] As shown in FIG5 , the electronic device 50 of this embodiment includes a memory 51 and a processor 52 coupled to the memory 51. It should be noted that the components of the electronic device 50 shown in FIG5 are merely exemplary and non-limiting. The electronic device 50 may also include other components as required by actual applications. The processor 52 may control the other components in the electronic device 50 to perform desired functions.

[0162] In some embodiments, the memory 51 is configured to store one or more computer-readable instructions. When the processor 52 is configured to execute the computer-readable instructions, the computer-readable instructions, when executed by the processor 52, implement a method according to any of the above-described embodiments. The specific implementation and related explanations of each step of the method can be found in the above-described embodiments, and any repetitive details are omitted here.

[0163] For example, the processor 52 and the memory 51 may communicate with each other directly or indirectly. For example, the processor 52 and the memory 51 may communicate with each other via a network. The network may include a wireless network, a wired network, and / or any combination of wireless networks and wired networks. The processor 52 and the memory 51 may also communicate with each other via a system bus, which is not limited in this disclosure.

[0164] For example, the processor 52 can be embodied as various appropriate processors, processing devices, etc., such as a central processing unit (CPU), a graphics processing unit (GPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The central processing unit (CPU) can be an X86 or ARM architecture, etc. For example, the memory 51 can include any combination of various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The memory 51 can include, for example, a system memory, which stores, for example, an operating system, an application, a boot loader (Boot Loader), a database, and other programs. Various applications and various data can also be stored in the storage medium.

[0165] In addition, according to some embodiments of the present disclosure, when various operations / processes according to the present disclosure are implemented through software and / or firmware, the programs constituting the software can be installed from a storage medium or a network to a computer system having a dedicated hardware structure, such as the computer system (or electronic device) 60 shown in Figure 6. When the various programs are installed, the computer system can perform various functions, including functions such as those described above. Figure 6 is a block diagram illustrating an example structure of a computer system that can be used in embodiments of the present disclosure.

[0166] In Figure 6, a central processing unit (CPU) 601 performs various processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 to a random access memory (RAM) 603. In the RAM 603, data required when the CPU 601 performs various processes, etc., is also stored as needed. The central processing unit is merely exemplary and may also be other types of processors, such as the various processors described above. The ROM 602, RAM 603, and storage part 608 may be various forms of computer-readable storage media, as described below. It should be noted that although ROM 602, RAM 603, and storage device 608 are shown separately in Figure 6, one or more of them may be combined or located in the same or different memory or storage modules.

[0167] The CPU 601, the ROM 602, and the RAM 603 are connected to one another via a bus 604. An input / output interface 605 is also connected to the bus 604.

[0168] The following components are connected to the input / output interface 605: an input portion 606, such as a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; an output portion 607, including a display, such as a cathode ray tube (CRT), liquid crystal display (LCD), speaker, vibrator, etc.; a storage portion 608, including a hard disk, magnetic tape, etc.; and a communication portion 609, including a network interface card, such as a LAN card, modem, etc. The communication portion 609 allows communication processing to be performed via a network, such as the Internet. It will be readily understood that although FIG6 shows that the various devices or modules in the computer system 60 communicate via the bus 604, they may also communicate via a network or other means, where the network may include a wireless network, a wired network, and / or any combination of wireless and wired networks.

[0169] A drive 610 is also connected to the input / output interface 605 as needed. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is mounted on the drive 610 as needed so that a computer program read therefrom is installed in the storage section 608 as needed.

[0170] In the case of realizing the above-described series of processing by software, a program constituting the software can be installed from a network such as the Internet or a storage medium such as the removable medium 611 .

[0171] According to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network via the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the CPU 601, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0172] It should be noted that in the context of the present disclosure, a computer-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that may be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0173] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0174] In some embodiments, a computer program is further provided, comprising: instructions, which, when executed by a processor, cause the processor to perform any of the methods of the above embodiments. For example, the instructions may be embodied as computer program codes.

[0175] In embodiments of the present disclosure, computer program code for performing the operations of the present disclosure can be written in one or more programming languages ​​or combinations thereof, including but not limited to object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In situations involving a remote computer, the remote computer can be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)), or can be connected to an external computer (e.g., using an Internet service provider to connect via the Internet).

[0176] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0177] The modules, components, or units described in the embodiments of the present disclosure may be implemented in software or hardware. The names of the modules, components, or units do not necessarily limit the modules, components, or units themselves.

[0178] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, and without limitation, exemplary hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0179] According to some embodiments of the present disclosure, a model evaluation method is provided, including: based on the prompt information of the model to be evaluated, using an interactive robot to interact with the model to be evaluated; obtaining content generated by the model to be evaluated; based on the prompt information and generated content of the model to be evaluated, using an evaluation robot to evaluate the model to be evaluated to obtain an evaluation result.

[0180] In some embodiments, based on the prompt information of the model to be evaluated, using an interactive robot to interact with the model to be evaluated includes: generating special prompt information for the interactive robot according to the prompt information of the model to be evaluated; generating an interactive robot according to the special prompt information of the interactive robot; and using the interactive robot to interact with the model to be evaluated.

[0181] In some embodiments, using an interactive robot to interact with the model to be evaluated includes: determining a database corresponding to the model to be evaluated based on the type of the model to be evaluated, wherein different types of models to be evaluated correspond to different databases; obtaining input information of the first interaction from the database corresponding to the model to be evaluated, and inputting it into the model to be evaluated; in each interaction after the first interaction, using the interactive robot to generate input information for this interaction based on the content generated by the model to be evaluated in the previous interaction, and inputting it into the model to be evaluated.

[0182] In some embodiments, generating special prompt information for the interactive robot based on the prompt information of the model to be evaluated includes: generating a first prompt generation model based on the prompt information of the first prompt generation model; using the first prompt generation model to generate special prompt information for the interactive robot based on the prompt information of the model to be evaluated.

[0183] In some embodiments, the prompt information of the first prompt generation model is used to prompt the first prompt generation model to receive the prompt information of the first model and generate prompt information of the second model that interacts with the first model based on the characteristics of the first model. Using the first prompt generation model to generate special prompt information for the interactive robot according to the prompt information of the model to be evaluated includes: inputting the prompt information of the model to be evaluated into the first prompt generation model as the prompt information of the first model; using the first prompt generation model to determine the characteristics of the first model according to the prompt information of the model to be evaluated; using the first prompt generation model to generate prompt information for the second model that interacts with the first model according to the characteristics of the first model as special prompt information for the interactive robot.

[0184] In some embodiments, determining the characteristics of the first model based on the prompt information of the model to be evaluated includes: parsing the prompt information of the model to be evaluated, and extracting the type of information received by the model to be evaluated and the role of the model to be evaluated when interacting from the prompt information of the model to be evaluated as the characteristics of the first model; generating prompt information of the second model that interacts with the first model based on the characteristics of the first model includes: determining the type of information generated by the second model and the role of the second model when interacting based on the type of information received by the model to be evaluated and the role of the model to be evaluated when interacting; generating prompt information for the second model based on the type of information generated by the second model and the role of the second model when interacting.

[0185] In some embodiments, using an interactive robot to interact with the model to be evaluated includes: using the interactive robot to generate interactive input information according to the type of information generated by the interactive robot and the role of the interactive robot during interaction, and inputting the information into the model to be evaluated.

[0186] In some embodiments, based on the prompt information of the model to be evaluated, using an interactive robot to interact with the model to be evaluated includes: generating an interactive robot according to the general prompt information of the interactive robot; inputting the prompt information of the model to be evaluated into the interactive robot, and using the interactive robot to interact with the model to be evaluated.

[0187] In some embodiments, using an interactive robot to interact with the model to be evaluated includes: determining a database corresponding to the model to be evaluated based on the type of the model to be evaluated, wherein different types of models to be evaluated correspond to different databases; obtaining input information for the first interaction from the database corresponding to the model to be evaluated, and inputting it into the model to be evaluated; in each interaction after the first interaction, using the interactive robot to generate input information for this interaction based on the content generated by the model to be evaluated in the previous interaction and the prompt information of the model to be evaluated, and inputting it into the model to be evaluated.

[0188] In some embodiments, the general prompt information of the interactive robot is used to prompt the interactive robot to receive prompt information of the third model, interact with the third model based on the characteristics of the third model, and input the prompt information of the model to be evaluated into the interactive robot. Using the interactive robot to interact with the model to be evaluated includes: inputting the prompt information of the model to be evaluated into the interactive robot as the prompt information of the third model; using the interactive robot to determine the characteristics of the third model based on the prompt information of the model to be evaluated; and using the interactive robot to generate interactive input information based on the characteristics of the third model.

[0189] In some embodiments, determining the characteristics of the third model based on the prompt information of the model to be evaluated includes: parsing the prompt information of the model to be evaluated, and extracting the type of information received by the model to be evaluated and the role of the model to be evaluated when interacting from the prompt information of the model to be evaluated as the characteristics of the third model; generating the input information of the interaction based on the characteristics of the third model includes: determining the type of information generated by the interactive robot and the role of the interactive robot when interacting based on the type of information received by the model to be evaluated and the role of the model to be evaluated when interacting; generating the input information of the interaction based on the type of information generated by the interactive robot and the role of the interactive robot when interacting.

[0190] In some embodiments, based on the prompt information and generated content of the model to be evaluated, using an evaluation robot to evaluate the model to be evaluated includes: generating special prompt information for the evaluation robot according to the prompt information of the model to be evaluated; generating an evaluation robot according to the special prompt information of the evaluation robot; and using the evaluation robot to evaluate the model to be evaluated according to the generated content.

[0191] In some embodiments, generating special prompt information for the evaluation robot based on the prompt information of the model to be evaluated includes: generating a second prompt generation model based on the prompt information of the second prompt generation model; using the second prompt generation model to generate special prompt information for the evaluation robot based on the prompt information of the model to be evaluated.

[0192] In some embodiments, the prompt information of the second prompt generation model is used to prompt the second prompt generation model to receive the prompt information of the fourth model, generate the prompt information of the fifth model, and the fifth model receives the input information and output information of the fourth model, determines whether the output information of the fourth model meets the expected standards and outputs the results based on the prompt information, input information and output information of the fourth model, and uses the second prompt generation model to generate special prompt information for the evaluation robot according to the prompt information of the model to be evaluated, including: inputting the prompt information of the model to be evaluated into the second prompt generation model as the prompt information of the fourth model; using the second prompt generation model to determine the type of input information, type of output information and expected standards of the fourth model received by the fifth model according to the prompt information of the model to be evaluated; using the second prompt generation model to generate prompt information of the fifth model as the special prompt information for the evaluation robot according to the type of input information, type of output information and expected standards of the fourth model received by the fifth model.

[0193] In some embodiments, based on the prompt information of the model to be evaluated, determining the type of input information and the type of output information received by the fifth model from the fourth model and the expected standard includes: parsing the prompt information of the model to be evaluated, extracting the type of information received by the model to be evaluated and the type of content generated from the prompt information of the model to be evaluated as the type of input information and the type of output information of the fourth model, and extracting the standard that the content generated by the model to be evaluated needs to meet as the expected standard.

[0194] In some embodiments, the dedicated prompt information of the evaluation robot includes first format information for indicating receiving input information of the model to be evaluated and second format information for receiving content generated by the model to be evaluated.

[0195] In some embodiments, using an evaluation robot to evaluate the model to be evaluated based on the generated content includes: using the evaluation robot to receive input information and generated content of the model to be evaluated, wherein the input information of the model to be evaluated is input information generated by the interactive robot for interacting with the model to be evaluated; based on the input information and generated content of the model to be evaluated, determining whether the content generated by the model to be evaluated meets the expected standards.

[0196] In some embodiments, based on the prompt information and generated content of the model to be evaluated, using an evaluation robot to evaluate the model to be evaluated includes: generating an evaluation robot according to the general prompt information of the evaluation robot; inputting the prompt information and generated content of the model to be evaluated into the evaluation robot, and using the evaluation robot to evaluate the model to be evaluated.

[0197] In some embodiments, the general prompt information is used to prompt the evaluation robot to receive the prompt information and output information of the sixth model, determine whether the output information of the sixth model meets the expected standards and output the results based on the prompt information and output information of the sixth model, input the prompt information and generated content of the model to be evaluated into the evaluation robot, and use the evaluation robot to evaluate the model to be evaluated, including: inputting the prompt information, input information and generated content of the model to be evaluated into the evaluation robot as the prompt information, input information and output information of the sixth model, wherein the input information of the model to be evaluated is the input information generated by the interactive robot to interact with the model to be evaluated; using the evaluation robot to determine the expected standards based on the prompt information of the model to be evaluated; using the evaluation robot to determine whether the output information of the model to be evaluated meets the expected standards based on the input information and generated content of the model to be evaluated, and obtain the evaluation results.

[0198] In some embodiments, the model to be evaluated is generated based on prompt information of the model to be evaluated.

[0199] According to other embodiments of the present disclosure, a model evaluation device is provided, including: an interaction module, configured to interact with the model to be evaluated by using an interactive robot based on prompt information of the model to be evaluated; an acquisition module, configured to acquire content generated by the model to be evaluated; and an evaluation module, configured to evaluate the model to be evaluated by using an evaluation robot based on the prompt information and generated content of the model to be evaluated to obtain an evaluation result.

[0200] According to some further embodiments of the present disclosure, an electronic device is provided, comprising: a memory; and a processor coupled to the memory, wherein the processor is configured to execute a model evaluation method as in any embodiment of the present disclosure based on instructions stored in the memory.

[0201] According to some further embodiments of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the model evaluation method of any embodiment of the present disclosure is implemented.

[0202] According to some further embodiments of the present disclosure, a computer program is provided, comprising: instructions, which, when executed by a processor, cause the processor to execute the model evaluation method of any embodiment of the present disclosure.

[0203] According to some embodiments of the present disclosure, a computer program product is provided, comprising instructions, which, when executed by a processor, implement the model evaluation method of any embodiment of the present disclosure.

[0204] According to some embodiments of the present disclosure, a computer program is provided, comprising instructions, which, when executed by a processor, implement the model evaluation method of any embodiment of the present disclosure.

[0205] The above descriptions are merely some embodiments of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in the present disclosure.

[0206] In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the present invention may be practiced without these specific details. In other cases, well-known methods, structures, and techniques are not presented in detail in order not to obscure the understanding of the description.

[0207] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0208] Although some specific embodiments of the present disclosure have been described in detail by way of examples, those skilled in the art will appreciate that the above examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Those skilled in the art will appreciate that modifications may be made to the above embodiments without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

1. A model evaluation method, comprising: Based on the prompt information of the model to be evaluated, using the interactive robot to interact with the model to be evaluated; Obtaining content generated by the model to be evaluated; Based on the prompt information of the model to be evaluated and the generated content, the model to be evaluated is evaluated by using an evaluation robot to obtain an evaluation result.

2. The evaluation method according to claim 1, wherein: The interactive robot interacting with the model to be evaluated based on the prompt information of the model to be evaluated includes: Generating dedicated prompt information for the interactive robot according to the prompt information of the model to be evaluated; generating the interactive robot according to the dedicated prompt information of the interactive robot; The interactive robot is used to interact with the model to be evaluated.

3. The evaluation method according to claim 2, wherein: The utilizing the interactive robot to interact with the model to be evaluated includes: Determining a database corresponding to the model to be evaluated according to the type of the model to be evaluated, wherein different types of models to be evaluated correspond to different databases; Obtaining input information of the first interaction from a database corresponding to the model to be evaluated, and inputting the information into the model to be evaluated; In each interaction after the first interaction, the interactive robot is used to generate input information for this interaction based on the content generated by the model to be evaluated in the previous interaction, and the information is input into the model to be evaluated.

4. The evaluation method according to claim 2 or 3, wherein: Generating the dedicated prompt information of the interactive robot according to the prompt information of the model to be evaluated includes: generating the first prompt generation model according to the prompt information of the first prompt generation model; The first prompt generation model is used to generate dedicated prompt information for the interactive robot according to the prompt information of the model to be evaluated.

5. The evaluation method according to claim 4, wherein: The prompt information of the first prompt generation model is used to prompt the first prompt generation model to receive the prompt information of the first model and generate prompt information of a second model that interacts with the first model based on the characteristics of the first model. The generating of dedicated prompt information for the interactive robot using the first prompt generation model and the prompt information of the model to be evaluated includes: Inputting the prompt information of the model to be evaluated into the first prompt generation model as prompt information of the first model; generating a model using the first prompt, and determining characteristics of the first model according to the prompt information of the model to be evaluated; The first prompt generation model is used to generate prompt information of a second model that interacts with the first model according to the characteristics of the first model, as dedicated prompt information for the interactive robot.

6. The evaluation method according to claim 5, wherein: Determining the characteristics of the first model according to the prompt information of the model to be evaluated includes: Parsing the prompt information of the model to be evaluated, and extracting the type of information received by the model to be evaluated and the role of the model to be evaluated during interaction from the prompt information of the model to be evaluated as the characteristics of the first model; Generating prompt information of a second model interacting with the first model according to the characteristics of the first model includes: Determining the type of information generated by the second model and the role of the second model when interacting, based on the type of information received by the model to be evaluated and the role of the model to be evaluated when interacting; Prompt information of the second model is generated according to the type of information generated by the second model and the role when the second model interacts.

7. The evaluation method according to claim 6, wherein: The utilizing the interactive robot to interact with the model to be evaluated includes: The interactive robot is used to generate interactive input information according to the type of information generated by the interactive robot and the role of the interactive robot during interaction, and the information is input into the model to be evaluated.

8. The evaluation method according to any one of claims 1 to 7, wherein: The interactive robot interacting with the model to be evaluated based on the prompt information of the model to be evaluated includes: generating the interactive robot according to the general prompt information of the interactive robot; The prompt information of the model to be evaluated is input into the interactive robot, and the interactive robot is used to interact with the model to be evaluated.

9. The evaluation method according to claim 8, wherein: The utilizing the interactive robot to interact with the model to be evaluated includes: Determining a database corresponding to the model to be evaluated according to the type of the model to be evaluated, wherein different types of models to be evaluated correspond to different databases; Obtaining input information of the first interaction from a database corresponding to the model to be evaluated, and inputting the information into the model to be evaluated; In each interaction after the first interaction, the interactive robot is used to generate input information for this interaction based on the content generated by the model to be evaluated in the previous interaction and the prompt information of the model to be evaluated, and input the information into the model to be evaluated.

10. The evaluation method according to claim 8 or 9, wherein: The general prompt information of the interactive robot is used to prompt the interactive robot to receive prompt information of the third model and interact with the third model based on the characteristics of the third model. The inputting the prompt information of the model to be evaluated into the interactive robot and using the interactive robot to interact with the model to be evaluated includes: inputting the prompt information of the model to be evaluated into the interactive robot as prompt information of the third model; Using the interactive robot, determining the characteristics of the third model according to the prompt information of the model to be evaluated; The interactive robot is used to generate interactive input information according to the characteristics of the third model.

11. The evaluation method according to claim 10, wherein: Determining the characteristics of the third model according to the prompt information of the model to be evaluated includes: Parsing the prompt information of the model to be evaluated, and extracting the type of information received by the model to be evaluated and the role of the model to be evaluated during interaction from the prompt information of the model to be evaluated as the characteristics of the third model; Generating interactive input information according to the characteristics of the third model includes: Determining the type of information generated by the interactive robot and the role of the interactive robot during interaction based on the type of information received by the model to be evaluated and the role of the model to be evaluated during interaction; Interaction input information is generated according to the type of information generated by the interactive robot and the role of the interactive robot during interaction.

12. The evaluation method according to any one of claims 1 to 11, wherein: The evaluating the model to be evaluated by using an evaluation robot based on the prompt information of the model to be evaluated and the generated content includes: Generate dedicated prompt information for the evaluation robot according to the prompt information of the model to be evaluated; generating the evaluation robot according to the dedicated prompt information of the evaluation robot; The evaluation robot is used to evaluate the model to be evaluated based on the generated content.

13. The evaluation method according to claim 12, wherein: Generating the dedicated prompt information of the evaluation robot according to the prompt information of the model to be evaluated includes: generating the second prompt generation model according to the prompt information of the second prompt generation model; The second prompt generation model is used to generate dedicated prompt information for the evaluation robot according to the prompt information of the model to be evaluated.

14. The evaluation method according to claim 13, wherein: The prompt information of the second prompt generation model is used to prompt the second prompt generation model to receive the prompt information of the fourth model and generate prompt information of the fifth model, and the fifth model receives the input information and output information of the fourth model, determines whether the output information of the fourth model meets the expected standard based on the prompt information, input information and output information of the fourth model, and outputs the result. The use of the second prompt generation model to generate the dedicated prompt information of the evaluation robot based on the prompt information of the model to be evaluated includes: Inputting the prompt information of the model to be evaluated into the second prompt generation model as the prompt information of the fourth model; Determining, using the second prompt generation model, based on the prompt information of the model to be evaluated, the type of input information received by the fifth model from the fourth model, the type of output information, and the expected standard; The second prompt generation model is used to generate prompt information of the fifth model according to the type of input information, the type of output information and the expected standard received by the fifth model from the fourth model, as special prompt information for the evaluation robot.

15. The evaluation method according to claim 14, wherein: The determining, based on the prompt information of the model to be evaluated, the type of input information received by the fifth model from the fourth model, the type of output information, and the expected standard includes: Parse the prompt information of the model to be evaluated, extract the type of information received and the type of content generated by the model to be evaluated from the prompt information of the model to be evaluated as the type of input information and the type of output information of the fourth model, and extract the standards that the content generated by the model to be evaluated needs to meet as the expected standards.

16. The evaluation method according to any one of claims 12 to 15, wherein: The dedicated prompt information of the evaluation robot includes first format information for indicating receiving input information of the model to be evaluated and second format information for receiving content generated by the model to be evaluated.

17. The evaluation method according to claim 15, wherein: The evaluating the model to be evaluated using the evaluation robot according to the generated content includes: The evaluation robot receives the input information and generated content of the model to be evaluated, wherein the The input information of the model to be evaluated is input information generated by the interactive robot for interacting with the model to be evaluated; According to the input information and generated content of the model to be evaluated, it is determined whether the content generated by the model to be evaluated meets the expected standard.

18. The evaluation method according to any one of claims 1 to 17, wherein: The evaluating the model to be evaluated by using an evaluation robot based on the prompt information of the model to be evaluated and the generated content includes: Generating the evaluation robot according to the general prompt information of the evaluation robot; The prompt information of the model to be evaluated and the generated content are input into the evaluation robot, and the evaluation robot is used to evaluate the model to be evaluated.

19. The evaluation method according to claim 18, wherein: The general prompt information is used to prompt the evaluation robot to receive the prompt information and output information of the sixth model, determine whether the output information of the sixth model meets the expected standard based on the prompt information and output information of the sixth model, and output the result. The prompt information of the model to be evaluated and the generated content are input into the evaluation robot, and the evaluation of the model to be evaluated by the evaluation robot includes: Inputting the prompt information, input information, and generated content of the model to be evaluated into the evaluation robot as the prompt information, input information, and output information of the sixth model, wherein the input information of the model to be evaluated is the input information generated by the interactive robot for interacting with the model to be evaluated; Using the evaluation robot, determining the expected standard according to the prompt information of the model to be evaluated; The evaluation robot is used to determine whether the output information of the model to be evaluated meets the expected standards based on the input information and generated content of the model to be evaluated, and obtain an evaluation result.

20. The evaluation method according to any one of claims 1 to 19, wherein: The model to be evaluated is generated according to the prompt information of the model to be evaluated.

21. A model evaluation device comprising: An interaction module is configured to interact with the model to be evaluated using an interactive robot based on prompt information of the model to be evaluated; An acquisition module, configured to acquire the content generated by the model to be evaluated; The evaluation module is configured to evaluate the model to be evaluated by using an evaluation robot based on the prompt information of the model to be evaluated and the generated content to obtain an evaluation result.

22. An electronic device comprising: processor; as well as A memory coupled to the processor, for storing instructions, wherein when the instructions are executed by the processor, the processor executes the model evaluation method according to any one of claims 1 to 20.

23. A computer-readable storage medium having a computer program stored thereon, wherein: When the program is executed by a processor, the steps of the model evaluation method described in any one of claims 1 to 20 are implemented.

24. A computer program product comprising: Instructions, wherein when the instructions are executed by a processor, the steps of the model evaluation method described in any one of claims 1-20 are implemented.

25. A computer program comprising: Instructions, wherein when the instructions are executed by a processor, the steps of the model evaluation method described in any one of claims 1-20 are implemented.

Images