A large model intelligent agent evaluation method and device

Abstract

The embodiment of the present invention discloses a large-model intelligent agent evaluation method and device, which relates to the field of artificial intelligence technology. It is invented for relatively accurate evaluation of the stylization of AI simulation of people. The method includes: generating an AI intelligent agent of a preset character; obtaining the past speech background information and speech content information of the preset character; wherein the speech content information includes the answer content of the preset character to a given question; generating an evaluation set according to the speech background information and speech content information; testing the AI ​​intelligent agent according to the evaluation set to obtain the answer content of the AI ​​intelligent agent to the given question; comparing the answer content of the preset character and the AI ​​intelligent agent to the given question, and scoring the answer terms and answer style in the answer content of the AI ​​intelligent agent to identify the similarity of the AI ​​intelligent agent and the preset character in the answer terms and answer style. The present invention is applicable to intelligent agent evaluation scenarios.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a large-model intelligent agent evaluation method and device. Background Art

[0002] The rise of AI (Artificial Intelligence) has led to the development of new technologies that are changing the way we live and work. Conversational robots are computer programs that mimic human conversations. They are powered by artificial intelligence and can understand and respond to user questions, usually in natural language. Users can communicate with AI through conversations. AI can simulate a person through professionalism and stylization. Professionalism refers to skills or skill combinations that can objectively indicate right and wrong, such as mathematics, English, and logic. Stylization refers to a person's personalized expression, such as speaking style, personality, and speaking habits. Through professionalism and stylization, different styles of AI agents can be generated. When faced with different problems, AI agents can generate feedback on problems and communicate with users based on their own professionalism and stylization, complete the functions of answering questions, emotional companionship, etc., and meet user needs.

[0003] When using AI agents to simulate humans, it is necessary to evaluate whether they are "like" the real person. As for professionalism, it is relatively easy to use corresponding questions to judge whether a certain level has been reached based on the right or wrong answers, which is similar to an exam. However, in terms of style, real people will express themselves differently depending on the environment, the people they face, the context, and other different conditions. It is difficult to enumerate these changes in the evaluation, and it is also impossible to find every real person to judge whether the answer style or wording is accurate. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a large-model intelligent agent evaluation method and device, which is convenient for relatively accurate evaluation of the AI ​​agent's simulation of human stylization.

[0005] In a first aspect, an embodiment of the present invention provides a large-model intelligent agent evaluation method, comprising the steps of: generating an AI agent of a preset character; obtaining past speech background information and speech content information of the preset character; wherein the speech content information includes the preset character's answer to a given question; generating an evaluation set based on the speech background information and speech content information; testing the AI ​​agent based on the evaluation set to obtain the AI ​​agent's answer to the given question; comparing the answer contents of the preset character and the AI ​​agent to the given question, and scoring the answer terms and answer style in the answer content of the AI ​​agent to identify the similarity between the AI ​​agent and the preset character in terms of answer terms and answer style.

[0006] Optionally, generating an AI agent of a preset character includes: obtaining objective background information and speech information of the preset character; wherein the objective background information includes the subject level information of the preset character, and the speech information includes the speech background information and the speech content information; generating an AI agent of the preset character based on the objective background information and the speech information.

[0007] Optionally, the speech background information includes a given question, a type of the given question, an object of the given question, a time when the given question is asked, and an occasion when the given question is asked.

[0008] Optionally, testing the AI ​​agent according to the evaluation set to obtain the answer content of the AI ​​agent to a given question includes: classifying the given questions according to the type of the given question, the question object of the given question, the question time of the given question and the question occasion of the given question; asking questions to the AI ​​agent according to the given questions under different categories to obtain the answer content of the AI ​​agent to the given questions under the different categories.

[0009] Optionally, the type of the given question includes a decision-making given question; testing the AI ​​agent according to the evaluation set to obtain the answer content of the AI ​​agent to the given question includes: asking the AI ​​agent according to the decision-making given question to obtain the answer content of the AI ​​agent to the decision-making given question; comparing the answer contents of the preset character and the AI ​​agent to the given question, and scoring the answer terms and answer style in the answer content of the AI ​​agent to identify the similarity between the AI ​​agent and the preset character in terms of answer terms and answer style, includes: comparing the answer contents of the preset character and the AI ​​agent to the decision-making given question, and scoring the answer terms and answer style in the answer content of the AI ​​agent to identify the similarity between the AI ​​agent and the preset character in terms of answer terms and answer style.

[0010] Optionally, after generating an AI agent of a preset character, the method further includes: obtaining objective background information of the preset character; and testing the professionalism of the AI ​​agent based on the objective background information.

[0011] Optionally, after comparing the answers of the preset character and the AI ​​agent to a given question, and scoring the answer terms and answer style in the answer contents of the AI ​​agent to identify the similarity between the AI ​​agent and the preset character in terms of answer terms and answer style, the method also includes: prompting corresponding data that needs to be supplemented based on the similarity between the AI ​​agent and the preset character in terms of answer terms and answer style; prompting corresponding data that needs to be supplemented based on the similarity between the AI ​​agent and the preset character in terms of answer terms and answer style includes: when the similarity between the AI ​​agent and the preset character in terms of answer terms or answer style is lower than a first preset threshold, prompting to supplement the corresponding data of the answer terms or answer style.

[0012] Optionally, after prompting the corresponding data that needs to be supplemented based on the similarity between the AI ​​agent and the preset character in terms of answer terms and answer styles, the method also includes: under different given question classifications, when the similarity between the AI ​​agent and the preset character in terms of answer terms and answer styles reaches a second preset threshold, predicting the answer content of the AI ​​agent to the preset given question; continuously comparing the answer content of the preset character to the preset given question online with the predicted answer content of the AI ​​agent to the preset given question, and scoring the answer terms and answer style in the answer content of the AI ​​agent to identify the similarity between the AI ​​agent and the preset character in terms of answer terms and answer style.

[0013] In the second aspect, the large-model intelligent agent evaluation device provided by the embodiment of the present invention includes: a first generation unit, used to generate an AI intelligent agent of a preset character; a first acquisition unit, used to acquire the past speech background information and speech content information of the preset character; wherein the speech content information includes the answer content of the preset character to a given question; a second generation unit, used to generate an evaluation set according to the speech background information and speech content information; an evaluation unit, used to test the AI ​​intelligent agent according to the evaluation set to obtain the answer content of the AI ​​intelligent agent to the given question; a first scoring unit, used to compare the answer content of the preset character and the AI ​​intelligent agent to the given question, and score the answer terms and answer style in the answer content of the AI ​​intelligent agent, so as to identify the similarity between the AI ​​intelligent agent and the preset character in the answer terms and answer style.

[0014] Optionally, the device also includes: a prompting unit, used to prompt corresponding data that needs to be supplemented based on the similarity between the AI ​​agent and the preset character in answer terms and answer styles; the prompting unit is specifically used to prompt to supplement the corresponding data of the answer terms or answer style when the similarity between the AI ​​agent and the preset character in answer terms or answer styles is lower than a first preset threshold.

[0015] The large-model intelligent agent evaluation method and device provided by the embodiment of the present invention can generate an AI intelligent agent of a preset character; obtain the past speech background information and speech content information of the preset character; wherein the speech content information includes the answer content of the preset character to a given question; generate an evaluation set according to the speech background information and speech content information; test the AI ​​intelligent agent according to the evaluation set to obtain the answer content of the AI ​​intelligent agent to the given question; compare the answer content of the preset character and the AI ​​intelligent agent to the given question, and score the answer terms and answer style in the answer content of the AI ​​intelligent agent to identify the similarity of the AI ​​intelligent agent and the preset character in the answer terms and answer style. In this way, it is convenient to conduct relatively accurate evaluation in terms of the stylization of AI intelligent agents simulating people. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 This is a flow chart of a large model intelligent agent evaluation method according to an embodiment of the present invention;

[0018] Figure 2 This is a schematic diagram of the architecture of an embodiment of a large model intelligent agent evaluation device of the present invention;

[0019] Figure 3 The present invention is a schematic block diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0021] It should be clear that the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Embodiment 1

[0022] See also Figure 1 As shown, Figure 1 The following is a flow chart of a large model agent evaluation method according to an embodiment of the present invention. The method includes:

[0023] S1. Generate an AI agent of a preset character.

[0024] Specifically, a large amount of information such as data and speeches related to the preset character can be found to generate a basic AI agent of the preset character. It is understandable that the preset character is a real person, and the AI ​​agent is an AI role corresponding to the real person. The AI ​​agent of the preset character can be generated using the UGC (User Generated Content) capability. It is understandable that the user can ask questions to the generated AI agent of the preset character.

[0025] S2. Obtaining the background information and content information of the past speeches of the preset person; wherein the speech content information includes the answer of the preset person to a given question.

[0026] Specifically, the speech background information may be information related to a given question, such as the occasion when the given question is asked.

[0027] In some embodiments, the speech background information includes a given question, a type of the given question, an object of the given question, a time when the given question is asked, and an occasion when the given question is asked.

[0028] S3. Generate an evaluation set based on the speech background information and speech content information.

[0029] Specifically, the evaluation set may include past speech information of a preset person.

[0030] S4. Test the AI ​​agent according to the evaluation set to obtain the answer of the AI ​​agent to the given question.

[0031] In some embodiments, testing the AI ​​agent according to the evaluation set to obtain the answer of the AI ​​agent to a given question includes: classifying the given question according to the type of the given question, the question object of the given question, the question time of the given question, and the question occasion of the given question;

[0032] Questions are asked to the AI ​​agent according to given questions under different categories to obtain answers from the AI ​​agent to the given questions under different categories.

[0033] It is understandable that when the AI ​​agent is asked questions according to given questions under different categories in the evaluation set, the answer content of the AI ​​agent to the given questions under the different categories can be obtained. The answer content of the AI ​​agent can include two parts: the words used in the answer and the style of the answer. The style of the answer can be humorous, calm, rigorous, concise and implicit, etc.

[0034] In some embodiments, the type of the given question includes a decision-making given question; and testing the AI ​​agent according to the evaluation set to obtain the answer of the AI ​​agent to the given question includes:

[0035] The AI ​​agent is asked questions according to the given decision-making questions to obtain the answer content of the AI ​​agent to the given decision-making questions; the answer contents of the preset character and the AI ​​agent to the given questions are compared, and the answer terms and answer style in the answer content of the AI ​​agent are scored to identify the similarity between the AI ​​agent and the preset character in the answer terms and answer style, including: comparing the answer contents of the preset character and the AI ​​agent to the given decision-making questions, and scoring the answer terms and answer style in the answer content of the AI ​​agent to identify the similarity between the AI ​​agent and the preset character in the answer terms and answer style.

[0036] It can be understood that when the type of given question is a decision-making type given question, the answer of the AI ​​agent can be a deterministic answer such as agree, disagree, refuse to answer, like, etc.

[0037] S5. Compare the answers of the preset character and the AI ​​agent to a given question, and score the answer terms and answer style in the answer content of the AI ​​agent to identify the similarity between the AI ​​agent and the preset character in the answer terms and answer style.

[0038] Specifically, under different classifications of given problems, the similarity between the AI ​​agent and the preset character may be different.

[0039] It is understandable that the user can ask questions to the AI ​​agent of the preset character. When asking questions, the user can get relatively accurate answers according to the different similarities under the classification of different given questions, or according to the AI ​​agents with similarities in different directions. Among them, the similarities in different directions can be, for example, the similarities in the direction of answer terms, the similarities in the direction of answer styles, or the similarities in the directions of different professional levels, and these similarities in different directions can be different. The present invention can judge whether the generated AI agent is similar to a real person by comparing with the opinions already expressed by real people, and give a similarity method to judge whether the set AI agent is like a real person, or whether a part is like a real person, such as the answer language part, the answer style part, and the professional level part. In this way, the similarity of the AI ​​role can be more objectively calibrated to give users more accurate answers.

[0040] The large-model intelligent agent evaluation method provided by the embodiment of the present invention can generate an AI intelligent agent of a preset character; obtain the past speech background information and speech content information of the preset character; wherein the speech content information includes the answer content of the preset character to a given question; generate an evaluation set according to the speech background information and speech content information; test the AI ​​intelligent agent according to the evaluation set to obtain the answer content of the AI ​​intelligent agent to the given question; compare the answer content of the preset character and the AI ​​intelligent agent to the given question, and score the answer terms and answer style in the answer content of the AI ​​intelligent agent to identify the similarity of the AI ​​intelligent agent and the preset character in the answer terms and answer style. In this way, it is convenient to conduct relatively accurate evaluation in terms of the stylization of AI intelligent agent simulation of people.

[0041] In some embodiments, generating an AI agent of a preset character includes: obtaining objective background information and speech information of the preset character; wherein the objective background information includes the subject level information of the preset character, and the speech information includes the speech background information and the speech content information; generating an AI agent of the preset character based on the objective background information and the speech information.

[0042] Specifically, the subject level information may be, for example, mathematics level information, English level information or other professional level information of a preset person.

[0043] In some embodiments, after generating an AI agent of a preset character, the method further includes: obtaining objective background information of the preset character; and testing the professionalism of the AI ​​agent based on the objective background information.

[0044] Specifically, the professionalism of the AI ​​agent corresponding to the preset character can be tested based on some objective background information of the preset character, and the AI ​​agent can be asked questions to obtain the answer content of the AI ​​agent, and then the answers of the preset character and the AI ​​agent to the given questions can be compared, and the answer content of the AI ​​agent can be scored to identify the similarity between the professional level of the AI ​​agent and the preset character, so as to ensure that the AI ​​agent can be consistent with the real person in terms of professional level. Objective background information, such as the math level, English level or other professional level information of the preset character.

[0045] It is understandable that under different professional level classifications, the similarities of different professional levels may be different. According to the similarities between the AI ​​agent and the preset character at different professional levels, a prompt may be given to supplement the corresponding data to improve the similarity. The similarities at different professional levels may be the similarities between the AI ​​agent and the preset character in math level, English level or other professional level. When the similarity at a certain professional level is lower than a first preset threshold, a prompt may be given to supplement the corresponding data of the required professional level to improve the similarity.

[0046] Moreover, under the classification of different professional levels, when the similarity at a certain professional level reaches a second preset threshold, the speech of the AI ​​agent at the professional level can be predicted and continuously fitted and compared with the speech and decision of real people online, and the similarity and the data on the professional level that need to be supplemented can be dynamically changed to ensure that the AI ​​character and the real person grow synchronously.

[0047] In some embodiments, after comparing the answers of the preset character and the AI ​​agent to a given question, and scoring the answer terms and answer styles in the answer contents of the AI ​​agent to identify the similarity between the AI ​​agent and the preset character in terms of answer terms and answer styles, the method further includes: prompting corresponding data that needs to be supplemented based on the similarity between the AI ​​agent and the preset character in terms of answer terms and answer styles; prompting corresponding data that needs to be supplemented based on the similarity between the AI ​​agent and the preset character in terms of answer terms and answer styles includes: when the similarity between the AI ​​agent and the preset character in terms of answer terms or answer styles is lower than a first preset threshold, prompting to supplement the corresponding data of the answer terms or answer styles.

[0048] In some embodiments, after prompting the corresponding data that needs to be supplemented based on the similarity between the AI ​​agent and the preset character in terms of answer terms and answer styles, the method also includes: under different given question categories, when the similarity between the AI ​​agent and the preset character in terms of answer terms and answer styles reaches a second preset threshold, predicting the answer content of the AI ​​agent to the preset given question; continuously comparing the answer content of the preset character to the preset given question online with the predicted answer content of the AI ​​agent to the preset given question, and scoring the answer terms and answer style in the answer content of the AI ​​agent to identify the similarity between the AI ​​agent and the preset character in terms of answer terms and answer style.

[0049] Specifically, when the similarity in the answer terms and answer styles of the AI ​​agent and the preset character reaches a certain threshold, for example, when it reaches a second preset threshold, the character's speech can be predicted and continuously fitted and compared with the speech and decision of the real person online, and the similarity in the answer terms and answer styles of the AI ​​agent and the preset character, as well as the data on the answer terms and answer styles that need to be supplemented, are dynamically changed to ensure that the AI ​​character and the real person grow synchronously. Embodiment 2

[0050] See also Figure 2 As shown, Figure 2 This is a schematic diagram of the architecture of an embodiment of a large model intelligent agent evaluation device of the present invention, including:

[0051] A first generating unit 21, used to generate an AI agent of a preset character;

[0052] The first acquisition unit 22 is used to acquire the background information and speech content information of the preset person's past speeches; wherein the speech content information includes the answer content of the preset person to a given question;

[0053] A second generating unit 23, configured to generate an evaluation set according to the speech background information and the speech content information;

[0054] An evaluation unit 24, configured to test the AI ​​agent according to the evaluation set to obtain the answer content of the AI ​​agent to a given question;

[0055] The first scoring unit 25 is used to compare the answers of the preset character and the AI ​​agent to a given question, and to score the answer terms and answer style in the answer content of the AI ​​agent, so as to identify the similarity between the AI ​​agent and the preset character in the answer terms and answer style.

[0056] The device of this embodiment can realize Figure 1The method embodiment shown, the specific implementation process and beneficial effects are similar to Figure 1 The method embodiments shown are basically the same and will not be described in detail here.

[0057] In some embodiments, the device also includes: a prompting unit, used to prompt the corresponding data that needs to be supplemented based on the similarity between the AI ​​agent and the preset character in the answer terms and answer style; the prompting unit is specifically used to prompt to supplement the corresponding data of the answer terms or answer style when the similarity between the AI ​​agent and the preset character in the answer terms or answer style is lower than a first preset threshold.

[0058] In some embodiments, the first generation unit 21 is specifically used to obtain objective background information and speech information of a preset character; wherein the objective background information includes the subject level information of the preset character, and the speech information includes the speech background information and the speech content information; and an AI agent of the preset character is generated based on the objective background information and the speech information.

[0059] In some embodiments, the speech background information includes a given question, a type of the given question, an object of the given question, a time when the given question is asked, and an occasion when the given question is asked.

[0060] In some embodiments, the evaluation unit 24 is specifically used to classify the given questions according to the type of the given questions, the question object of the given questions, the question time of the given questions and the question occasion of the given questions; ask questions to the AI ​​agent according to the given questions under different categories, and obtain the answers of the AI ​​agent to the given questions under the different categories.

[0061] In some embodiments, the type of the given question includes a decision-making given question; the evaluation unit 24 is further used to ask the AI ​​agent questions based on the decision-making given question to obtain the answer content of the AI ​​agent to the decision-making given question; the first scoring unit 25 is used to compare the answer content of the preset character and the AI ​​agent to the decision-making given question, and score the answer terms and answer style in the answer content of the AI ​​agent to identify the similarity between the AI ​​agent and the preset character in the answer terms and answer style.

[0062] In some embodiments, the device further includes: a second acquisition unit, used to acquire objective background information of the preset character; and a professional testing unit, used to test the professionalism of the AI ​​agent based on the objective background information.

[0063] In some embodiments, the device also includes: a prediction unit, which is used to predict the answer content of the AI ​​agent to a preset given question under different given question classifications when the similarity in the answer terms and answer style of the AI ​​agent and the preset character reaches a second preset threshold; a second scoring unit, which is used to continuously compare the answer content of the preset character to the preset given question online and the predicted answer content of the AI ​​agent to the preset given question, and score the answer terms and answer style in the answer content of the AI ​​agent to identify the similarity in the answer terms and answer style between the AI ​​agent and the preset character. Embodiment 3

[0064] Figure 3 This is a schematic block diagram of the architecture of an embodiment of an electronic device of the present invention; based on the technical concept basically the same as that of the aforementioned embodiment 1, the electronic device provided by the embodiment of the present invention, as shown in 3, can implement the step flow of any embodiment method described in embodiment 1 of the present invention.

[0065] The above-mentioned electronic device may include: a shell 41, a processor 42, a memory 43, a circuit board 44 and a power supply circuit 45, wherein the circuit board 44 is arranged inside the space enclosed by the shell 41, and the processor 42 and the memory 43 are arranged on the circuit board 44; the power supply circuit 45 is used to supply power to various circuits or devices of the above-mentioned electronic device; the memory 43 is used to store executable program codes; the processor 42 runs the program corresponding to the executable program code by reading the executable program code stored in the memory 43, so as to execute the large model intelligent agent evaluation method described in any of the aforementioned embodiments one.

[0066] The specific execution process of the above steps by the processor 42 and the steps further executed by the processor 42 by running the executable program code can be found in the description of the first embodiment of the present invention, which will not be repeated here.

[0067] An embodiment of the present invention also provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the large-model intelligent agent evaluation method described in any of the above-mentioned embodiments.

[0068] The electronic device exists in many forms, including but not limited to:

[0069] (1) Mobile communication devices: These devices are characterized by their mobile communication functions and their main purpose is to provide voice and data communications. These terminals include: smart phones (such as iPhone), multimedia phones, feature phones, and low-end phones.

[0070] (2) Ultra-mobile personal computer devices: These devices fall into the category of personal computers, have computing and processing capabilities, and generally also have mobile Internet access features. These terminals include: PDA, MID and UMPC devices, such as iPad.

[0071] (3) Portable entertainment devices: These devices can display and play multimedia content. They include audio and video players (such as iPods), handheld game consoles, e-books, smart toys, and portable car navigation devices.

[0072] (4) Server: A device that provides computing services. The server consists of a processor, hard disk, memory, system bus, etc. The server is similar to a general-purpose computer architecture, but because it needs to provide highly reliable services, it has higher requirements in terms of processing power, stability, reliability, security, scalability, and manageability.

[0073] (5) Other electronic devices with data interaction functions.

[0074] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0075] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0076] For the convenience of description, if it involves a system, server, etc., it may be described separately by dividing the functions into various units / modules. Of course, when implementing the present invention, the functions of each unit / module can be implemented in the same or multiple software and / or hardware.

[0077] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0078] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A large model agent evaluation method, characterized in that: The method comprises: Generate AI agents of preset characters; Obtaining the background information and content information of the past speeches of the preset person; wherein the speech content information includes the answer of the preset person to a given question; Generate an evaluation set according to the speech background information and speech content information; Testing the AI ​​agent according to the evaluation set to obtain the answer of the AI ​​agent to a given question; The step of testing the AI ​​agent according to the evaluation set to obtain the answer of the AI ​​agent to a given question includes: Classify the given questions according to the type of the given questions, the object of the given questions, the time when the given questions are asked, and the occasion when the given questions are asked; Asking questions to the AI ​​agent according to given questions under different categories to obtain answers to the given questions under different categories by the AI ​​agent, wherein the answers include answer terms and answer styles; Compare the answers of the preset character and the AI ​​agent to a given question, and score the answer terms and answer style in the answer content of the AI ​​agent to identify the similarity between the AI ​​agent and the preset character in the answer terms and answer style.

2. The large model agent evaluation method according to claim 1 is characterized in that: The method of generating an AI agent of a preset character includes: Obtain objective background information and speech information of a preset person; wherein the objective background information includes the subject level information of the preset person, and the speech information includes the speech background information and the speech content information; An AI agent of a preset character is generated based on the objective background information and speech information.

3. The large model agent evaluation method according to claim 1 or 2, characterized in that: The speech background information includes a given question, a type of the given question, an object of the given question, a time when the given question is asked, and an occasion when the given question is asked.

4. The large model agent evaluation method according to claim 1, characterized in that: The types of the given problems include decision-making given problems; The step of testing the AI ​​agent according to the evaluation set to obtain the answer of the AI ​​agent to a given question includes: Ask the AI ​​agent questions based on the decision-making given question to obtain the AI ​​agent's answer to the decision-making given question; The comparing the answers of the preset character and the AI ​​agent to a given question, and scoring the answer terms and answer style in the answer of the AI ​​agent to identify the similarity between the AI ​​agent and the preset character in the answer terms and answer style, includes: Compare the answers of the preset character and the AI ​​agent to the given decision-making question, and score the answer terms and answer style in the answer content of the AI ​​agent to identify the similarity between the AI ​​agent and the preset character in the answer terms and answer style.

5. The large model agent evaluation method according to claim 1, characterized in that: After generating the AI ​​agent of the preset character, the method further includes: Obtaining objective background information of the preset character; The professionalism of the AI ​​agent is tested based on the objective background information.

6. The large model agent evaluation method according to claim 1, characterized in that: After comparing the answers of the preset character and the AI ​​agent to a given question, and scoring the answer terms and answer style in the answer content of the AI ​​agent to identify the similarity between the AI ​​agent and the preset character in the answer terms and answer style, the method further includes: Prompt corresponding data that needs to be supplemented based on the similarity between the AI ​​agent and the preset character in terms of answer terms and answer styles; The corresponding data to be supplemented according to the similarity between the AI ​​agent and the preset character in terms of answer terms and answer styles includes: When the similarity between the AI ​​agent and the preset character in terms of answer terms or answer styles is lower than a first preset threshold, a prompt is provided to supplement corresponding data of the answer terms or answer styles.

7. The large model agent evaluation method according to claim 6, characterized in that: After prompting the corresponding data to be supplemented according to the similarity between the AI ​​agent and the preset character in terms of answer terms and answer styles, the method further includes: Under different given question categories, when the similarity between the answer terms and answer styles of the AI ​​agent and the preset character reaches a second preset threshold, predict the answer content of the AI ​​agent to the preset given question; Continuously compare the online answer content of the preset character to the preset given question and the predicted answer content of the AI ​​agent to the preset given question, and score the answer terms and answer style in the answer content of the AI ​​agent to identify the similarity between the AI ​​agent and the preset character in the answer terms and answer style.

8. A large-model intelligent agent evaluation device, characterized in that: The device comprises: A first generating unit, used to generate an AI agent of a preset character; A first acquisition unit is used to acquire the background information and speech content information of the preset person's past speeches; wherein the speech content information includes the answer content of the preset person to a given question; A second generating unit, configured to generate an evaluation set according to the speech background information and the speech content information; An evaluation unit, configured to test the AI ​​agent according to the evaluation set to obtain the answer of the AI ​​agent to a given question; A first scoring unit is used to compare the answers of the preset character and the AI ​​agent to a given question, and score the answer terms and answer style in the answer content of the AI ​​agent to identify the similarity between the AI ​​agent and the preset character in the answer terms and answer style; The evaluation unit is specifically used to classify the given questions according to the type of the given questions, the question object of the given questions, the question time of the given questions and the question occasion of the given questions; Questions are asked to the AI ​​agent according to given questions under different categories to obtain answers of the AI ​​agent to the given questions under the different categories; the answers include answer terms and answer styles.

9. The large model agent evaluation device according to claim 8, characterized in that: The device also includes: A prompting unit, used for prompting corresponding data that needs to be supplemented according to the similarity between the AI ​​agent and the preset character in terms of answer terms and answer styles; The prompt unit is specifically used to prompt to supplement the corresponding data of the answer terms or answer style when the similarity between the AI ​​agent and the preset character in the answer terms or answer style is lower than a first preset threshold.

Images