Intelligent agent-based dialogue data generation method, evaluation method, training method and related device

Abstract

The invention provides an agent-based dialogue data generation method, and relates to the technical field of artificial intelligence. The specific implementation scheme is as follows: in response to a received dialogue data generation request, obtaining a plurality of first agents configured with respective object portraits and a plurality of second agents configured with respective entity portraits; a target first agent in the multiple first agents and a target second agent in the multiple second agents are selected to carry out dialogue to obtain dialogue data, the target first agent outputs first dialogue content according to a configured target object portrait, the first dialogue content has dialogue behavior characteristics and language style characteristics of a target object, and the target object portrait outputs the first dialogue content according to the configured target object portrait. And the target second agent outputs the second dialogue content according to the configured entity portrait. The invention further provides an agent evaluation method, an agent training method, an agent, a device, electronic equipment, a storage medium and a program product.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to large language models, intelligent agents, human-computer interaction and other technical fields, and can be applied to content generation, intelligent agent customer service, intelligent agent evaluation, question answering, marketing, advertising retrieval and distribution and other scenarios. More specifically, the present disclosure provides an agent-based dialogue data generation method, evaluation method, training method, intelligent agent, device, electronic device, storage medium and program product. Background Art

[0002] Intelligent agent is an important concept in the field of artificial intelligence. It refers to a system or entity that can perceive the environment, make decisions, and perform actions. With the rise of big model technology, intelligent agents have begun to play a role in various scenarios such as question-answering and marketing. For example, they can simulate human conversations and have text or voice conversations with users. However, it takes a long time to accumulate conversation data for intelligent agents in actual scenarios, and there is a lack of sufficient and real conversation samples, which is not conducive to the rapid iteration and optimization of intelligent agents. Summary of the invention

[0003] The present disclosure provides an agent-based dialogue data generation method, an evaluation method, a training method, an agent, an apparatus, an electronic device, a storage medium, and a program product.

[0004] According to one aspect of the present disclosure, there is provided an agent-based method for generating conversation data, the method comprising: in response to a received conversation data generation request, obtaining a plurality of first agents each configured with a respective object portrait, and a plurality of second agents each configured with a respective entity portrait; selecting a target first agent among the plurality of first agents to conduct a conversation with a target second agent among the plurality of second agents to obtain conversation data, wherein the target first agent outputs a first conversation content according to the configured target object portrait, the first conversation content having the conversation behavior characteristics and language style characteristics of the target object, and the target second agent outputs a second conversation content according to the configured entity portrait.

[0005] According to another aspect of the present disclosure, there is provided an agent evaluation method, the method comprising: in response to an evaluation request for a target second agent, obtaining conversation data between a target first agent and a target second agent, the conversation data being obtained according to the agent-based conversation data generation method as described above; inputting the conversation data into at least one evaluation large model, and obtaining an evaluation result of the at least one evaluation large model on the target second agent.

[0006] According to another aspect of the present disclosure, a method for training an intelligent agent is provided, the method comprising: obtaining a training sample set based on conversation data between a target first intelligent agent and a target second intelligent agent, the conversation data being obtained according to the above-mentioned method for generating conversation data based on intelligent agents; inputting conversation samples in the training sample set into a reward model to obtain a reward score output by the reward model; and performing reinforcement learning training on the target second intelligent agent based on the reward score to adjust the parameters of the base large model of the target second intelligent agent.

[0007] According to another aspect of the present disclosure, an intelligent agent is provided, which is configured to be obtained according to the training method of the intelligent agent provided in the embodiment of the present disclosure.

[0008] According to another aspect of the present disclosure, there is provided an agent-based dialogue data generation device, which includes: a first acquisition unit, used to acquire multiple first agents configured with respective object portraits, and multiple second agents configured with respective entity portraits in response to a received dialogue data generation request; a first dialogue unit, used to select a target first agent from the multiple first agents to dialogue with a target second agent from the multiple second agents to obtain dialogue data, wherein the target first agent outputs first dialogue content according to the configured target object portrait, the first dialogue content has the dialogue behavior characteristics and language style characteristics of the target object, and the target second agent outputs second dialogue content according to the configured entity portrait.

[0009] According to another aspect of the present disclosure, there is provided an agent evaluation device, which includes: a second acquisition unit, used to obtain conversation data between a target first agent and a target second agent in response to an evaluation request for a target second agent, wherein the conversation data is obtained based on an agent-based conversation data generation device; and an agent evaluation unit, used to input the conversation data into at least one evaluation large model to obtain an evaluation result of the at least one evaluation large model on the target second agent.

[0010] According to another aspect of the present disclosure, there is provided a training device for an intelligent agent, the device comprising: a training sample unit, for acquiring a training sample set based on dialogue data between a target first intelligent agent and a target second intelligent agent, the dialogue data being obtained according to an agent-based dialogue data generating device; a reward score unit, for inputting dialogue samples in the training sample set into a reward model to obtain a reward score output by the reward model; and a parameter adjustment unit, for performing reinforcement learning training on the target second intelligent agent based on the reward score to adjust the parameters of a base large model of the target second intelligent agent.

[0011] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method provided according to the present disclosure.

[0012] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided. The computer instructions are used to cause a computer to execute the method provided according to the present disclosure.

[0013] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the method provided according to the present disclosure is implemented.

[0014] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0016] Figure 1 is a schematic diagram of an exemplary system architecture in which various methods and devices described herein may be implemented according to an embodiment of the present disclosure;

[0017] Figure 2 is a flow chart of a method for generating conversation data based on an agent according to an embodiment of the present disclosure;

[0018] Figure 3 is a schematic diagram of a target first agent and a target second agent performing a dialogue interaction according to an embodiment of the present disclosure;

[0019] Figure 4 is a flow chart of a test interview according to an embodiment of the present disclosure;

[0020] Figure 5 is a flow chart of an agent evaluation method according to an embodiment of the present disclosure;

[0021] Figure 6 is a schematic diagram of a basic entry test assessment according to one embodiment of the present disclosure;

[0022] Figure 7 is a schematic diagram of a conversation experience evaluation according to an embodiment of the present disclosure;

[0023] Figure 8is a schematic diagram of transformation effect evaluation according to an embodiment of the present disclosure;

[0024] Fig. 9 is a flow chart of a method for training an intelligent agent according to an embodiment of the present disclosure;

[0025] Fig.10 is a flow chart for calculating reward scores according to one embodiment of the present disclosure;

[0026] Fig.11 is an architectural diagram of an online environment and a conversation simulation environment according to an embodiment of the present disclosure;

[0027] Fig.12 is a structural block diagram of an intelligent agent according to an embodiment of the present disclosure;

[0028] Fig.13 is a block diagram of an agent-based conversation data generating device according to an embodiment of the present disclosure;

[0029] Fig.14 is a block diagram of an agent evaluation device according to an embodiment of the present disclosure;

[0030] Fig.15 is a block diagram of a training device for an intelligent agent according to an embodiment of the present disclosure;

[0031] Fig.15 is a block diagram of a training device for an intelligent agent according to an embodiment of the present disclosure;

[0032] Fig.16 A schematic block diagram of an example electronic device that can be used to implement embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0033] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0034] In the technical solutions disclosed in this disclosure, the collection, storage, use, processing, transmission, provision, disclosure and application of user personal information involved are in compliance with the relevant laws and regulations, necessary confidentiality measures are taken, and they do not violate public order and good morals. In the technical solutions disclosed in this disclosure, the user's authorization or consent is obtained before obtaining or collecting user personal information.

[0035] Current agents are available to users in the form of chatbots, which are designed to simulate human conversations and can have text or voice conversations with real users. These chatbots are designed to understand users' questions, provide answers, perform tasks, or provide assistance.

[0036] The inventors have found that the current period for accumulating dialogue data of intelligent agents in actual scenarios is long, and there is a lack of sufficient and real dialogue samples, making it impossible to discover potential problems of the intelligent agents in a timely manner, and there is also a lack of sufficient dialogue samples to train the intelligent agents for further iterative optimization.

[0037] Embodiments of the present disclosure provide an agent-based conversation data generation method, an evaluation method, a training method, an agent, an apparatus, an electronic device, a storage medium, and a program product. The agent-based conversation data generation method includes: in response to a received conversation data generation request, obtaining a plurality of first agents configured with respective object portraits, and a plurality of second agents configured with respective entity portraits. A target first agent among the plurality of first agents is selected to conduct a conversation with a target second agent among the plurality of second agents to obtain conversation data, wherein the target first agent outputs a first conversation content according to the configured target object portrait, the first conversation content has the conversation behavior characteristics and language style characteristics of the target object, and the target second agent outputs a second conversation content according to the configured entity portrait.

[0038] According to the embodiments of the present disclosure, by configuring an object portrait for each of the multiple first intelligent agents, and configuring an entity portrait for each of the multiple second intelligent agents, the target first intelligent agent imitates the conversation behavior and language style of the target object and conducts a conversation with the target second intelligent agent, thereby imitating the conversation process of the actual scene, and efficiently generating a sufficient amount of simulated conversation data.

[0039] Figure 1 is a schematic diagram of an exemplary system architecture in which various methods and devices described herein may be implemented according to an embodiment of the present disclosure.

[0040] It should be noted that Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0041] like Figure 1 As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0042] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Terminal devices 101, 102, 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptops, desktop computers, etc.

[0043] Server 105 can be a server that provides various services, such as a background management server that provides support for websites browsed by users using terminal devices 101, 102, and 103 (for example only). The background management server can analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device. Server 105 can also be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system. Server 105 can also be a server of a distributed system, or a server combined with a blockchain.

[0044] It should be noted that the agent-based dialogue data generation method, evaluation method and training method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the agent, agent-based dialogue data generation device, evaluation device and training device provided in the embodiment of the present disclosure can generally be set in the server 105. The agent-based dialogue data generation method, evaluation method and training method provided in the embodiment of the present disclosure can also be executed by a server or a server group that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the agent, agent-based dialogue data generation device, evaluation device and training device provided in the embodiment of the present disclosure can also be set in a server or a server group that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105.

[0045] Alternatively, when the terminal devices 101, 102, 103 also have computing capabilities and computing resources that meet the requirements, the agent-based dialogue data generation method, evaluation method, and training method provided in the embodiments of the present disclosure can generally be executed by the terminal devices 101, 102, 103. Accordingly, the agent, agent-based dialogue data generation device, evaluation device, and training device provided in the embodiments of the present disclosure can generally be set in the terminal devices 101, 102, 103.

[0046] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0047] Figure 2 It is a flowchart of a method for generating conversation data based on an agent according to an embodiment of the present disclosure.

[0048] like Figure 2 As shown, the method 200 may include operations S210 to S220.

[0049] In operation S210, in response to the received dialog data generation request, a plurality of first agents configured with respective object portraits and a plurality of second agents configured with respective entity portraits are acquired.

[0050] In operation S220, a target first agent among multiple first agents is selected to conduct a dialogue with a target second agent among multiple second agents to obtain dialogue data, wherein the target first agent outputs a first dialogue content according to the configured target object portrait, and the first dialogue content has the dialogue behavior characteristics and language style characteristics of the target object, and the target second agent outputs a second dialogue content according to the configured entity portrait.

[0051] The dialog data generation request may be generated based on a manual interaction operation on the front-end page of the dialog data generation application. The interaction operation may include any type of operation, such as a gesture interaction operation such as a click operation, but is not limited thereto.

[0052] Object portraits can include abstracted labeled object models based on the object's identity attributes, social attributes, living habits, preferred behaviors, and other information. For example, when the object is a user, its object portrait can be obtained based on age, gender, occupation, identity, tone, personality, conversation style, historical information, and preferences. For example, the source of user portrait data includes the user's historical conversation records, purchase behavior, browsing history, interest tags, feedback information, etc. Through steps such as data cleaning, feature extraction, and modeling, the portrait features of each user are constructed. These features can include users' common questions, preferred answer methods, purchase tendencies, and commonly used communication languages.

[0053] Entities may include organizations or individuals engaged in social activities. Entity portraits may include abstracted labeled entity models based on entity attributes, social activities, business scope, user group characteristics, and other information. For example, when the entity is an organization or individual that provides services, its entity portrait can be obtained based on information such as industry category, geographic location, enterprise size, business scope, business model, market positioning, revenue scale, and profitability.

[0054] For example, after each conversation, the target first agent can update the characteristics of the target object portrait based on the new interaction data to more realistically imitate the real user. The target second agent can also adjust the relevant characteristics of the object portrait in the target entity portrait based on the new interaction data. This dynamic update mechanism enables the target first agent and the target second agent to continuously learn the new needs and preferences of the object, thereby providing simulated conversations close to actual scenarios in future conversations.

[0055] Intelligent agents include systems or entities that can autonomously perceive the environment, make decisions, and perform actions to complete specific tasks. Large models can provide decision support for intelligent agents and provide them with reasoning analysis and task planning capabilities. Intelligent agents use the analysis results of large models to execute or optimize their decision-making processes. Intelligent agents can integrate multiple large models to handle different types of tasks.

[0056] A large model may refer to a deep learning model with large-scale model parameters. A large model usually contains hundreds of millions, tens of billions, hundreds of billions, trillions, or even more than ten trillion model parameters. A large model may include a large-scale language model (LLM), GPT (Generative Pre-trained Transformer), a large visual model, a multimodal large model, and the like. The large model involved in the embodiments of the present disclosure may be a general large model, or may be an expert large model obtained after fine-tuning (Fine Tune) based on demand, and the embodiments of the present disclosure are not limited to this.

[0057] The first agent can imitate the behavior, language style, knowledge scope, needs, conversation style and personality of the object according to the configured object portrait. The first agent can be constructed based on a large amount of historical conversation data and user behavior patterns. The target first agent can be any one of multiple agents, such as a predetermined or randomly selected agent. During the conversation, the target first agent will call the user portrait features in real time, combine the current conversation context, and generate personalized conversation content, thereby simulating the behavior, preferences and conversation strategies of real users, and can have a natural and realistic conversation with the second agent. For example, for users who prefer concise answers, the first agent will give priority to providing concise replies; for users who like detailed explanations, the first agent will provide more detailed answers. The second agent can imitate the entity's speech according to the configured entity portrait, and output the second conversation content within the scope of the answer and business.

[0058] For example, the dialogue between objects and entities in the actual scene is a pre-step to conclude a transaction. The simulated dialogue between the target first agent and the target second agent can be centered around products or services that may generate a commercial transaction.

[0059] For example, the target first agent outputs questions about the services provided by the target second agent by imitating the dialogue behavior and language style of the target object. The question may include the dialogue behavior characteristics and language style characteristics of the target object. Among them, the dialogue behavior characteristics include the types of behaviors expressed by the target object through language. These behaviors can be direct expressions, such as statements, commands, inquiries, etc., or they can be behaviors implied by language, such as greetings, silence, explanations, etc. Language style characteristics include various characteristics of the target object in language expression, such as vocabulary selection, syntactic structure, voice rhythm, etc. The target second agent can output the second dialogue content including business introduction, question answering and marketing words with the goal of promoting transactions based on the configured entity portrait. The second dialogue content may include business characteristics, question answering characteristics and marketing characteristics.

[0060] For example, the first target agent is used to simulate the user, and can simulate the user's speech and imitate the user's conversation and chat. For example, it can respond to the dialogue content output by the second target agent to promote business transactions, simulate the user's "silence", "leave contact information", "end the conversation", and simulate the real user's language style to conduct a conversation. The second agent can be the same as the merchant agent provided in the actual scene.

[0061] According to the embodiments of the present disclosure, by configuring an object portrait for each of the multiple first intelligent agents, and configuring an entity portrait for each of the multiple second intelligent agents, the target first intelligent agent imitates the conversation behavior and language style of the target object and conducts a conversation with the target second intelligent agent, thereby imitating the conversation process of the actual scene, and efficiently generating a sufficient amount of simulated conversation data.

[0062] Figure 3 It is a schematic diagram of a dialog interaction between a target first agent 301 and a target second agent 302 according to an embodiment of the present disclosure.

[0063] In some embodiments, in operation S220, selecting a target first agent 301 among multiple first agents to have a dialogue with a target second agent 302 among multiple second agents to obtain dialogue data includes: calling a behavior macro model 3011 and a dialogue strategy macro model 3022 of the target first agent 301, and having a dialogue with the target second agent 302 to obtain dialogue data. The behavior macro model 3011 is used to output dialogue behavior features according to the target object portrait, and the dialogue strategy macro model 3022 is used to output a first dialogue content with language style features according to the target object portrait.

[0064] The behavior macro model 3011 can be obtained by fine-tuning the general macro model based on the need to imitate the behavior of the object. For example, the behavior training set is obtained by using the dialogue data between the object based on different object portraits and the second agent in the actual scene. The dialogue samples in the behavior training set include the behavior labels of each object, which are used to fine-tune the general macro model to obtain the behavior macro model 3011.

[0065] The dialogue strategy model 3022 can be obtained by fine-tuning the general model based on the need to imitate the language style of the object. For example, the strategy training set is obtained by using the dialogue data between the object based on different object portraits and the second agent in the actual scene. The dialogue samples in the strategy training set include the language style labels of each object, which are used to fine-tune the general model to obtain the dialogue strategy model 3022. The dialogue strategy model 3022 can automatically formulate and adjust the dialogue strategy by imitating the language style of the object to output the first dialogue content.

[0066] In some embodiments, calling the behavior big model 3011 and the dialogue strategy big model 3022 of the target first agent 301 to conduct a dialogue with the target second agent 302 includes: inputting the second dialogue content into the behavior big model 3011. The behavior big model 3011 is made to output a dialogue behavior feature according to the second dialogue content and the target object portrait. The target first agent 301 is made to imitate the behavior of the target object to send the first dialogue content to the target second agent 302, wherein the behavior of the target object matches the action feature of the dialogue behavior feature.

[0067] For example, behavior prompt words can also be input into the behavior macro model 3011. The behavior prompt words are used to guide the behavior macro model 3011 to output the dialogue behavior features according to the second dialogue content and the target object portrait. The action features in the dialogue behavior features include action information indicating that the target first agent 301 can be used to imitate the behavior of the object, such as a silence action, an end-of-dialogue action, etc. The target first agent 301 imitates the behavior of the target object according to the actions in the action features.

[0068] In some embodiments, after the behavior big model 3011 outputs the dialogue behavior features according to the second dialogue content and the target object portrait, when the dialogue behavior features also include information features, the information features and the second dialogue content are input into the dialogue strategy big model 3022 to obtain the first dialogue content with language style features, wherein the first dialogue content includes information of the target object corresponding to the information features.

[0069] In some embodiments, after the behavior macromodel 3011 outputs the dialogue behavior features based on the second dialogue content and the target object portrait, when the dialogue behavior features do not include information features, the second dialogue content is input into the dialogue strategy macromodel 3022 to obtain the first dialogue content with language style features.

[0070] The information feature is used to indicate whether the dialogue strategy model 3022 provides information about the object. The information feature may include whether information is provided, the type of information, the accuracy of the information, the level of detail of the information, etc. For example, the second dialogue content includes "leave a contact method". The information feature may include whether contact information is provided, the type of contact information (such as mobile phone number, social account, etc.), how many contact methods are provided, etc.

[0071] For example, the dialogue strategy prompt words can also be input into the dialogue strategy macro model 3022. The dialogue strategy prompt words are used to guide the dialogue strategy macro model 3022 to imitate the corresponding language style according to the second dialogue content and the target object portrait, and output the first dialogue content. The dialogue strategy prompt words can also be automatically adjusted according to whether the dialogue behavior feature is an information feature. For example, when the information feature is included, the dialogue strategy prompt words include content that guides the dialogue strategy macro model 3022 to generate information about the target object. When the information feature is not included, the dialogue strategy prompt words include content that guides the dialogue strategy macro model 3022 to refuse to generate information about the target object.

[0072] Exemplarily, depending on whether the dialogue behavior feature includes information features, the content input into the dialogue strategy macro model 3022 is affected, and the amount of information in the first dialogue content returned to the target second agent 302 is also affected. Therefore, based on the target object portrait, the behavior macro model 3011 and the dialogue strategy macro model 3022 cooperate with each other to enable a simulated dialogue between the target first agent 301 and the target second agent 302 that is closer to the actual scene.

[0073] In some embodiments, the target second agent 302 may include multiple domain-specific models. Each domain-specific model focuses on dialogue processing in a specific domain, and may have faster response time and more accurate professional domain capabilities. These specialized models can be implemented by cutting and optimizing the general large model, for example, different specialized models can be switched to respond according to the user's context, thereby providing more efficient multi-round dialogue management capabilities, improving the accuracy and response speed of the dialogue, and achieving better adaptation to complex dialogue situations.

[0074] For example, the target second agent 302 may also include a task planning model, wherein the multiple domain expertise models include a marketing domain expertise model, a knowledge answer domain expertise model, and a service recommendation expert expertise model. After receiving a message from a user in each round, the task planning model in the target second agent 302 is called to infer the tasks and domain expertise models to be performed in this round. Then, one or more domain expertise models inferred by the task planning model are called to generate a reply for this round of dialogue.

[0075] It is understood that the remaining first agents in the plurality of first agents may be the same as the target first agent, or the remaining first agents may be different from the target first agent. The remaining second agents in the plurality of second agents may be the same as the target second agent, or the remaining second agents may be different from the target second agent.

[0076] Figure 4 is a flow chart of a test session according to one embodiment of the present disclosure.

[0077] In some embodiments, reference Figure 4 , operation S220 of selecting a target first agent 401 among multiple first agents to have a dialogue with a target second agent 402 among multiple second agents also includes: in operation S410, generating a test question set based on each functional indicator of the target second agent 402, wherein each functional indicator is obtained according to the target entity portrait. In operation S420, selecting the target first agent 401 to initiate a question based on each test question set. In operation S430, obtaining a question answer set in which the target second agent 402 replies to each test question set. Among them, the target first agent 401 includes a behavior large model 4011 and a dialogue strategy large model 4022.

[0078] Functional indicators are used to characterize the understanding ability of the target second agent 402 of the corresponding functions of the target entity portrait based on the function points extracted from the target entity portrait. For example, the target second agent 402 is an intelligent customer service provided by a swimming training merchant, and the target entity portrait is obtained based on the swimming pool address, contact information, course fees, swimming training results and other information configured by the swimming training merchant. The target second agent 402 may have functional indicators such as course introduction accuracy indicator, address accuracy indicator, contact information accuracy indicator, and swimming effect expression accuracy indicator.

[0079] The test question set includes one or more question contents, each of which includes questions for corresponding functional indicators, such as automated quality inspection tests for target entity portraits, competitor rejection, etc. By initiating questions according to the test question set by the target first agent 401, automated testing of the target second agent 402 can be achieved during the simulation dialogue process. The question answer set output by the target second agent 402 includes one or more answer contents, each of which includes a reply to the question content, thereby showing the performance of the corresponding functional indicator.

[0080] In some embodiments, reference Figure 4 , use the test output model 403 to perform at least one of the following operations: process the target entity portrait to extract at least one functional indicator. Process each functional indicator to generate a test question set for each functional indicator.

[0081] The large model 403 for producing the test paper can be a general large model. Alternatively, the large model 403 for producing the test paper can be a large model obtained by fine-tuning the general large model based on at least one of the functional indicator extraction requirements and the test question generation requirements. For example, at least one functional indicator of each entity sample is extracted manually based on multiple entity portrait samples to obtain a training sample set, and then fine-tuning training can be performed based on the training sample set. It is also possible to further construct a test question set and a standard answer set based on each functional indicator of each entity sample manually so as to perform fine-tuning training as a training sample set.

[0082] For example, the large model 403 for producing the test paper can process the target entity portrait and the indicator prompt words to extract at least one functional indicator, and then manually construct a test question set based on each functional indicator to be tested, wherein the indicator prompt words include the requirements for extracting the functional indicator. Alternatively, at least one functional indicator is first extracted manually based on the target entity portrait, and then each functional indicator and the test prompt words are input into the large model 403 for producing the test paper to obtain a test question set. The test prompt words include the test requirements for generating test questions for the corresponding indicators. Alternatively, the target entity portrait, the indicator prompt words and the test prompt words are input into the large model 403 for producing the test paper to obtain at least one functional indicator output by the large model 403 for producing the test paper, and a test question set for each functional indicator.

[0083] An embodiment of automatically evaluating a target second agent in a simulated dialogue scenario is further described below.

[0084] Figure 5 is a flowchart of an agent evaluation method according to an embodiment of the present disclosure.

[0085] like Figure 5 As shown, the method 500 may include operations S510 to S520.

[0086] In operation S510, in response to an evaluation request for a target second agent, dialog data between a target first agent and a target second agent is obtained, where the dialog data is obtained according to one or more embodiments of the above-mentioned dialog data generation method.

[0087] In operation S520, the dialogue data is input into at least one evaluation model to obtain an evaluation result of the at least one evaluation model on the target second agent.

[0088] Each evaluation big model may include a general big model, and may also include a big model obtained by fine-tuning the general big model according to the evaluation requirements of specific indicators. Each evaluation big model may output the evaluation results of the target second intelligent agent in one or more dimensions based on the input dialogue data. Exemplarily, the evaluation results of the target second intelligent agent may be obtained based on the outputs of all evaluation big models.

[0089] Currently, the evaluation of the target second agent is performed manually based on the dialogue data in the actual scene. However, the dialogue data in the actual scene is scarce, and the manual evaluation depends on the experience and time of the evaluator, so there is a lack of efficient and accurate evaluation methods.

[0090] According to the embodiments of the present disclosure, an automated evaluation method is provided. The use effect of the intelligent agent on the physical material is detected by means of simulated dialogue evaluation, and the problems of the intelligent agent are discovered in time. It can be used as a pre-verification of the strategy effect before the strategy experiment, without waiting for the slow accumulation of actual dialogue data, and the effect of the strategy optimization can be verified in time, thereby improving the efficiency of intelligent agent evaluation and strategy iteration.

[0091] Figure 6 is a schematic diagram of a basic entry test assessment according to one embodiment of the present disclosure.

[0092] In some embodiments, reference Figure 6 , the target first intelligent agent 601 interacts with the target second intelligent agent 602, the dialogue data includes a set of questions and answers for each functional indicator by the target second intelligent agent 602, at least one evaluation model includes a marking model 604, and in operation S520, the dialogue data is input into at least one evaluation model to obtain the evaluation result of the target second intelligent agent 602 by at least one evaluation model, including: inputting the question and answer set of each functional indicator, the test question set and its standard answer set into the marking model 604, and obtaining the evaluation result of the marking model 604 for each functional indicator.

[0093] The scoring model 604 can be a general model. Or the scoring model 604 can be a general model that is fine-tuned based on the functional indicator evaluation requirements. For example, in the fine-tuning stage, a test question set and a standard answer set are constructed based on each functional indicator of each entity sample, and a question answer set of the target second agent 602 is obtained. Fine-tuning training is performed based on the question answer set, test question set, and standard answer set of this stage.

[0094] For example, in addition to inputting the question answer set, test question set and standard answer set of each functional indicator into the marking model 604, the functional evaluation prompt words corresponding to each functional indicator can also be input. The functional evaluation prompt words include the evaluation requirements for the corresponding functional indicator.

[0095] According to an embodiment of the present disclosure, referring to Figure 6 , so that the test-setting model 603 and the scoring model 604 cooperate with each other, and by evaluating the target second intelligent agent 602's ability to understand the target entity portrait, each functional indicator and its own test question set are abstracted. Then, the target second intelligent agent 602's answers to the questions corresponding to each functional indicator are obtained, so that the scoring model 604 can be used to automatically evaluate the answer effect, and the basic entry test of the target second intelligent agent 602 can be completed efficiently.

[0096] Figure 7 is a schematic diagram of conversation experience evaluation according to an embodiment of the present disclosure.

[0097] In some embodiments, the at least one evaluation big model includes an experience evaluation big model, and in operation S520, the dialogue data is input into the at least one evaluation big model, and obtaining the evaluation result of the at least one evaluation big model on the target second agent includes:

[0098] The conversation data and the evaluation prompts of each experience indicator are input into the experience evaluation model to obtain the evaluation results of the corresponding experience indicators, wherein the evaluation prompts include the evaluation requirements for the corresponding experience indicators, and the evaluation results represent the experience value provided by the target second intelligent agent to the target object during the conversation.

[0099] The experience evaluation big model may include a general big model, and may also include a big model after fine-tuning the general big model based on the experience index evaluation requirements. For example, in the fine-tuning stage, based on obtaining a set of dialogue samples, each dialogue sample includes an experience index and a corresponding experience value. The dialogue sample set of this stage is used to fine-tune the general big model.

[0100] The experience index is used to characterize the conversation experience that the target second agent gives to the target object during the conversation. The experience value can be used to characterize the quality of the conversation experience, such as the level of satisfaction. For example, multiple experience indicators can be defined through the four dimensions of basic experience, answering ability, understanding ability, and management ability to test the multiple abilities of the target second agent that affect the conversation experience.

[0101] According to the embodiments of the present disclosure, based on predefined experience evaluation criteria, the ability to identify experience problems can be built to achieve automated evaluation of the conversation experience.

[0102] In some embodiments, the at least one evaluation big model includes a plurality of experience evaluation big models, and the plurality of experience evaluation big models perform at least one of the following operations:

[0103] Output the evaluation results of multiple experience indicators. For example, one of the multiple experience evaluation models can be used to output the evaluation results of multiple experience indicators.

[0104] Output multiple evaluation results for multiple experience indicators in a one-to-one correspondence, wherein each experience evaluation model is obtained by fine-tuning training with training samples related to the corresponding experience indicator. For example, each experience evaluation model in the multiple experience evaluation models outputs an evaluation result for each experience indicator in a one-to-one correspondence.

[0105] like Figure 7 As shown, the multiple experience evaluation big models include A experience evaluation big model 701 and B experience evaluation big model 702 and other big models. For example, experience index 1 and its evaluation prompt word 1 can be input into A experience evaluation big model 701, and experience index 2 and its evaluation prompt word 2 can be input into A experience evaluation big model 701 simultaneously or asynchronously, so that A experience evaluation big model 701 outputs the evaluation result of experience index 1 and the evaluation result of experience index 2. For example, experience index 3 and its evaluation prompt word 3 can be input into B experience evaluation big model 702, so that B experience evaluation big model 702 outputs the evaluation result of experience index 3.

[0106] therefore Figure 7 The A experience evaluation model 701 can output the evaluation results of multiple experience indicators. The B experience evaluation model 702 can correspondingly output the evaluation results for experience indicator 3, wherein the B experience evaluation model 702 is obtained by fine-tuning the training samples related to the corresponding experience indicator 3.

[0107] According to the embodiments of the present disclosure, using a certain experience evaluation big model to output the evaluation results of multiple experience indicators can achieve full utilization of the big model and save computing resources. Deploying a dedicated experience evaluation big model corresponding to a certain experience indicator can obtain a more accurate evaluation effect.

[0108] Figure 8 It is a schematic diagram of transformation effect evaluation according to an embodiment of the present disclosure.

[0109] In some embodiments, at least one evaluation big model includes a conversion evaluation big model 801. In operation S520, the conversation data is input into the at least one evaluation big model, and the evaluation result of the at least one evaluation big model on the target second intelligent agent is obtained, which includes: inputting the conversation data into the conversion evaluation big model, and obtaining the evaluation result of the conversion evaluation big model 801 on each conversion indicator, wherein the evaluation result of the conversion indicator represents the conversion effect of the target second intelligent agent on the target object during the conversation process.

[0110] For example, the conversion evaluation big model 801 may include a general big model, and may also include a big model after fine-tuning the general big model based on the conversion index evaluation requirements. For example, in the fine-tuning stage, based on obtaining a dialogue sample set, each dialogue sample includes a conversion index and a corresponding conversion effect. The dialogue sample set of this stage is used to perform fine-tuning training on the general big model.

[0111] For another example, the conversion evaluation model 801 may include a word embedding layer, a deep neural network layer, and a classification layer that are identical or similar to the general large model. The conversation data is input into the conversion evaluation model 801, and the feature vector of the conversation data is extracted using the word embedding layer. The feature vector is then processed using the deep neural network layer and the output is used as the input of the classification layer. The classification layer may include a classification function (such as a softmax function), and the classification layer outputs the conversion value of each conversion indicator based on the input. The high or low conversion value can represent the quality of the conversion effect.

[0112] For example, in addition to inputting the conversation data into the conversion evaluation model 801, the conversion evaluation prompt words of each conversion indicator can also be used as input. Each conversion evaluation prompt word includes the evaluation requirements of the corresponding conversion indicator.

[0113] For example, the target object is the user, and the target entity object is the merchant. The conversion effect of each conversion indicator is used to characterize whether the user can be successfully guided to the expected business transaction target of the indicator. For example, conversion indicators may include purchase conversion indicators, registration conversion indicators, appointment conversion indicators, continued chat conversion indicators, and contact conversion indicators. The business transaction targets of each conversion indicator include purchase, registration, appointment, continued chat, and leaving contact information.

[0114] Reference Figure 8Taking the chat continuation conversion indicator and the contact retention conversion indicator as examples, the conversation data, the prompt words of the chat continuation conversion indicator, and the prompt words of the contact retention conversion indicator can be input into the conversion evaluation model 801 to obtain the evaluation effect of each conversion indicator, such as the conversion effect described in natural language, or the conversion value output by the classification layer.

[0115] According to the embodiments of the present disclosure, it is possible to realize automated evaluation of conversion effects and improve conversion evaluation efficiency.

[0116] In some embodiments, multiple conversion evaluation big models 801 may be included, and the multiple conversion evaluation big models 801 perform at least one of the following operations:

[0117] Output the evaluation results of multiple conversion indicators. For example, a conversion evaluation model 801 among multiple conversion evaluation models 801 can be used to output the evaluation results of multiple conversion indicators.

[0118] Multiple evaluation results for multiple conversion indicators are outputted one by one, wherein each conversion evaluation model 801 is obtained by fine-tuning training with training samples related to the corresponding conversion indicator. For example, each conversion evaluation model 801 in the multiple conversion evaluation models 801 outputs the evaluation result of each conversion indicator one by one.

[0119] The following further describes an embodiment of using the dialogue data obtained by the above-mentioned dialogue data generation method to perform intelligent agent training.

[0120] Fig. 9 It is a flowchart of a method for training an intelligent agent according to an embodiment of the present disclosure.

[0121] like Fig. 9 As shown, the method 900 may include operations S910 to S930.

[0122] In operation S910, a training sample set is obtained based on the dialogue data between the target first agent and the target second agent, where the dialogue data is obtained according to one or more embodiments of the above-mentioned dialogue data generation method.

[0123] In operation S920, the dialogue samples in the training sample set are input into the reward model to obtain a reward score output by the reward model.

[0124] In operation S930, reinforcement learning training is performed on the target second agent based on the reward score to adjust the base large model parameters of the target second agent.

[0125] The reward model (RM) can score the performance of the target second agent based on the dialogue sample, and then generate a reward score. For example, the reward model can score the functional index, experience index, and conversion index of the target second agent to obtain a reward score for one or more indicators.

[0126] The basic idea of ​​reinforcement learning training is that the agent gains intelligence through continuous interaction with the environment. The reinforcement learning strategy is based on the environment (State), subject (Actor), behavior (Action) and reward (Reward). The environment is the current state, the subject is the object that interacts with the environment and performs actions, the behavior is the action performed by the subject, and the reward is the feedback given to the specific behavior of the subject.

[0127] Corresponding to the simulation dialogue scenario, the subject (Actor) is the target second agent in the current stage, the environment (State) can be the first dialogue content output by the target first agent, the behavior (Action) is the second dialogue content corresponding to the first dialogue content, and the reward (Reward) is the reward score output by the reward model. The reward score is used as a reward signal to control the entire training process, so that the dialogue strategy of the target second agent is optimized.

[0128] For example, the target second agent can integrate multiple base models, such as multiple domain-specific models. During the reinforcement learning training process, optimizing the dialogue strategy of the target second agent is performed by adjusting the parameters of one or more base models.

[0129] Based on one or more embodiments of the above method for generating conversation data, the behaviors, preferences and conversation strategies of real users are simulated, and through continuous simulated interactions, diverse conversation samples are generated. These samples cover different scenarios, questions and user types, greatly enriching the training data.

[0130] According to the embodiments of the present disclosure, not only the shortage of real data is compensated, but also the adaptability and generalization ability of the target second intelligent agent are significantly enhanced, so that the target second intelligent agent after training can still maintain high-quality dialogue output in the scenario of lacking sufficient real dialogue data.

[0131] Fig.10 is a flow chart of calculating a reward score according to one embodiment of the present disclosure.

[0132] like Fig.10 As shown, one embodiment of operation S920 may include operations S1021 to S1022. The reward model includes a plurality of evaluation models.

[0133] In operation S1021, the dialogue samples are input into a plurality of large evaluation models, and the evaluation results of each of a plurality of evaluation indicators are output, wherein the plurality of evaluation indicators are used to evaluate the dialogue performance between the target second intelligent agent and the target first intelligent agent.

[0134] Operation S1022, calculating a reward score based on the multiple evaluation results and their respective weights. For example, each evaluation result is converted into a corresponding evaluation score, which is then multiplied by the respective weight, and finally all multiplication results are summed to obtain a reward score.

[0135] The multiple evaluation models may include the scoring model, experience evaluation model, and conversion evaluation model involved in some embodiments of the present disclosure. The multiple evaluation indicators may include the functional indicators, experience indicators, and conversion indicators involved in some embodiments of the present disclosure. The functional indicators, experience indicators, and conversion indicators can be used to evaluate the dialogue performance of the target second agent and the target first agent.

[0136] Take the simulated dialogue scenario between users and merchants as an example. The importance of various evaluation indicators may vary in different dialogue scenarios and business needs. In operation S1022, the reward model can introduce an adjustable weight coefficient to dynamically adjust the impact of each evaluation result on the final score, so as to achieve a balance of goals in different scenarios. For example, in a customer service scenario, the evaluation results of experience indicators may be given a higher weight, while in a marketing scenario, the evaluation results of conversion indicators may account for a larger proportion. This flexible weight setting method enables the reward model to be adjusted according to specific dialogue scenarios and business needs. This allows the trained target second agent to adapt to different dialogue scenarios and business needs.

[0137] Fig.11 It is an architectural diagram of an online environment and a conversation simulation environment according to an embodiment of the present disclosure.

[0138] Reference Fig.11 , in the actual scenario, an intelligent online delivery platform 1110 is provided. Each merchant can provide merchant information through the configuration page of the B-side to build an entity object portrait, and then publish an online merchant intelligent agent to communicate with real users. Each real user conducts multiple rounds of dialogues with the merchant intelligent agent through the online environment of the online BCP (Business Continuity Plan) dialogue system 1111. Then, a real dialogue log is formed based on the real dialogue data. After obtaining the real dialogue log, clues can be extracted and sent back to the advertiser. In addition, the evaluator can log in to the dialogue quality manual evaluation platform 1112 to view the real interaction data or customer feedback data obtained based on the real dialogue log, and conduct a manual evaluation of the dialogue experience. The merchant intelligent agent is then optimized based on the evaluation results.

[0139] In actual scenarios, however, the accumulation of real dialogue data between real users who want to initiate a conversation and the merchant agent is slow, resulting in a long iterative optimization cycle for the merchant agent. In addition, the manual evaluation method is limited by the experience and time of the evaluator, and the evaluation effect is not ideal. Therefore, this embodiment constructs an agent dialogue simulation system 1120 through user simulation, dialogue environment simulation, and simulated dialogue automated evaluation.

[0140] Continue to refer to Fig.11 , each user is taken as an object, and the corresponding user agent 1121 (i.e., the first agent) is constructed. The agent dialogue simulation path 1122 provides a dialogue simulation environment. The dialogue simulation environment refers to the dialogue interaction environment between the merchant agent (i.e., the second agent) and the user agent, which supports the generation of multiple rounds of simulated dialogues. The dialogue environment simulation is isolated from the online environment, and the merchant agents running in the two environments can be the same. For example, multiple scenarios and dialogue processes can be set to simulate different user needs and interaction methods, such as by simulating scenarios such as user inquiries, requests for help, or transactions.

[0141] Through continuous simulation interactions, a variety of dialogue samples are generated and simulated dialogue logs are obtained. These samples cover different scenarios, questions and user types, greatly enriching the dialogue sample data.

[0142] Then, an evaluation agent 1123 is constructed to provide automated evaluation services. The simulation evaluation test provided by the automated evaluation service is used to improve the experience evaluation efficiency and strategy iteration efficiency of the merchant agent. For example, the simulation evaluation test can be divided into two parts: the basic entry test and the effect simulation test. The basic entry test is used to test the basic entry threshold of the agent, including automated quality inspection tests for customer portraits, competitor rejection, etc. The effect simulation test is a further test of the agent, including the evaluation of the automated evaluation of the conversation experience and the conversion effect.

[0143] For example, the evaluation agent may integrate the evaluation big models involved in some embodiments of the present disclosure, such as one or more of the scoring big model, the experience evaluation big model, and the conversion evaluation big model.

[0144] For example, one or more of the scoring big model, the experience evaluation big model and the conversion evaluation big model can be used as reward models to conduct reinforcement learning training on one or more base big models of the merchant intelligent body, so as to optimize and improve the experience level and conversion ability of the merchant intelligent body.

[0145] According to the embodiments of the present disclosure, not only can the problem of data scarcity be solved, but also the capabilities of the big model can be deeply aligned with the skills of manual customer service, providing merchants with a stable, professional and personalized dialogue service solution, effectively improving the performance of the intelligent dialogue system in actual business scenarios, enabling merchants to better provide services and improve user experience and satisfaction.

[0146] In some embodiments, the diversified dialogue samples obtained by simulation can be summarized and refined to supplement the distribution features in the merchant agent and applied to the advertisement retrieval system to improve the distribution efficiency of the retrieval system. For example, the advertisement retrieval system can obtain an advertisement retrieval request. The retrieval information in the advertisement retrieval request is matched with the merchant's distribution features. After the match is successful, the advertisement retrieval system distributes the merchant's advertisement.

[0147] Fig.12 is a structural block diagram of an intelligent agent according to an embodiment of the present disclosure. The intelligent agent of this embodiment can be obtained based on the training method of the intelligent agent as above.

[0148] In the embodiments of the present disclosure, inspired by the von Neumann structure in modern computer theory, such as Fig.12 As shown, the AI ​​agent 1200 may include five core modules: an input module 1210 , a control module 1220 , a storage module 1230 , a calculation module 1240 and an output module 1250 .

[0149] The input module 1210 is responsible for receiving or sensing information such as queries, requests, instructions, signals or data from the outside world (such as users or external environment), and converting them into a format that can be understood and processed by the AI ​​agent 1200. The input module 1210 is the primary link for the AI ​​agent 1200 to interact with the outside world. It enables the AI ​​agent 1200 to efficiently and accurately obtain the necessary "sensory" information from the outside world and respond to this information.

[0150] In an example, the input module 1210 may input the first conversation content described above.

[0151] In the example, the control module 1220 is the core support for the AI ​​agent 1200 to handle complex tasks. The control module 1220 can execute the large model-based search method described above.

[0152] In the example, the control module 1220 will continuously interact with the storage module 1230, the computing module 1240 and / or the output module 1250 during operation. However, it should be noted that in the embodiment of the present disclosure, the control module 1220 acts as a single initiator to initiate communication with the storage module 1230, the computing module 1240 and / or the output module 1250, and there is no communication coupling between the storage module 1230, the computing module 1240 and the output module 1250.

[0153] In the example, the performance of the control module 1220 may be closely related to the large model on which the AI ​​agent 1200 is based. In order to give full play to the capabilities of the large language model, the internal structure of the control module 1220 may be designed to be highly configurable and extensible to cope with various types of tasks and requirements in real scenarios.

[0154] The storage module 1230 may be responsible for memorizing information such as historical conversations, event flows, etc. The above prompt information and data resources may be included in the storage module 1230 .

[0155] In the example, after obtaining the first dialogue content, the AI ​​agent 1200 can retrieve the relevant prompt words and data resources from the storage module 1230 and feed them back to the control module 1220. Then, the control module 1220 can use the feedback prompt words and data resources to obtain the second dialogue content and pass the second dialogue content to the output module 1250.

[0156] The operation module 1240 can be regarded as a predefined tool library. The tools obtained by encapsulating the large base model as described above can be included in the operation module 1240.

[0157] In the example, when the AI ​​agent 1200 needs to respond to the first dialogue content, the relevant tool encapsulating the corresponding base large model can be called from the operation module 1240 and fed back to the control module 1220. Then, the control module 1220 can use the called tool to obtain the second dialogue content and pass it to the output module 1250. It can be understood that although the large language model has excellent language understanding and generation capabilities, it is like a human being, without the help of any tools, the tasks that can be solved are very limited. When the AI ​​agent 1200 is given the ability to call tools, more complex tasks can be achieved.

[0158] The AI ​​agent 1200 according to the embodiment of the present disclosure can simply and effectively improve the level of intelligence, and improve flexibility and versatility.

[0159] Based on the above-mentioned agent-based dialogue data generation method, agent evaluation method and agent training method, an agent-based dialogue data generation device, an agent evaluation device and an agent training device are also provided.

[0160] Fig.13 It is a block diagram of an agent-based conversation data generating device according to an embodiment of the present disclosure.

[0161] like Fig.13 As shown, the conversation data generating device 1300 may include a first acquisition unit 1310 and a first conversation unit 1320 .

[0162] The first acquisition unit 1310 is used to respond to the received dialogue data generation request and acquire multiple first agents configured with respective object portraits and multiple second agents configured with respective entity portraits. The first dialogue unit 1320 is used to select a target first agent from the multiple first agents to dialogue with a target second agent from the multiple second agents to obtain dialogue data, wherein the target first agent outputs a first dialogue content according to the configured target object portrait, the first dialogue content has the dialogue behavior characteristics and language style characteristics of the target object, and the target second agent outputs a second dialogue content according to the configured entity portrait.

[0163] In some embodiments, the first dialogue unit 1320 may further include a calling unit. The calling unit is used to call the behavior big model and the dialogue strategy big model of the target first agent, and conduct a dialogue with the target second agent to obtain dialogue data. The behavior big model is used to output dialogue behavior features according to the target object portrait, and the dialogue strategy big model is used to output the first dialogue content with language style features according to the target object portrait.

[0164] In some embodiments, the calling unit may also be used to input the second dialogue content into the behavior big model. The behavior big model is made to output the dialogue behavior feature according to the second dialogue content and the target object portrait. The target first agent is made to imitate the behavior of the target object to send the first dialogue content to the target second agent, wherein the behavior of the target object matches the action feature of the dialogue behavior feature.

[0165] In some embodiments, after the behavior macromodel outputs dialogue behavior features based on the second dialogue content and the target object portrait, when the dialogue behavior features also include information features, the information features and the second dialogue content are input into the dialogue strategy macromodel to obtain a first dialogue content with language style features, wherein the first dialogue content includes information of the target object corresponding to the information features.

[0166] In some embodiments, after the behavior macromodel outputs dialogue behavior features based on the second dialogue content and the target object portrait, when the dialogue behavior features do not include information features, the second dialogue content is input into the dialogue strategy macromodel to obtain the first dialogue content with language style features.

[0167] In some embodiments, the dialogue data generating device 1300 may further include: a test question generating unit, configured to generate a test question set based on each functional indicator of the target second agent, wherein each functional indicator is obtained based on the target entity portrait; a questioning unit, configured to select the target first agent to initiate a question based on each test question set; and an answering unit, configured to obtain a question answering set of the target second agent in replying to each test question set.

[0168] In some embodiments, the test-taking unit is further used to use the test-taking large model to perform at least one of the following operations: processing the target entity portrait to extract at least one functional indicator. Processing each functional indicator to generate a test question set for each functional indicator.

[0169] Fig.14 is a block diagram of an agent evaluation device according to an embodiment of the present disclosure.

[0170] like Fig.14 As shown, the agent evaluation device 1400 may include a second acquisition unit 1410 and an agent evaluation unit 1420 .

[0171] The second acquisition unit 1410 is used to respond to the evaluation request for the target second agent and obtain the dialogue data between the target first agent and the target second agent, and the dialogue data is obtained according to the dialogue data generating device 1300. The agent evaluation unit 1420 is used to input the dialogue data into at least one evaluation large model to obtain the evaluation result of the at least one evaluation large model on the target second agent.

[0172] In some embodiments, the dialogue data includes a set of questions and answers for each functional indicator by the target second agent, at least one evaluation model includes a grading model, and the agent evaluation unit 1420 is also used to input the question and answer set of each functional indicator, the test question set and its standard answer set into the grading model to obtain the evaluation result of the grading model for each functional indicator.

[0173] In some embodiments, at least one evaluation big model includes an experience evaluation big model, and the agent evaluation unit 1420 is also used to input the dialogue data and the evaluation prompt words of each experience indicator into the experience evaluation big model to obtain the evaluation results of the corresponding experience indicators, wherein the evaluation prompt words include the evaluation requirements for the corresponding experience indicators, and the evaluation results represent the experience value provided by the target second agent to the target object during the dialogue process.

[0174] In some embodiments, the at least one evaluation big model includes a plurality of experience evaluation big models, and the plurality of experience evaluation big models perform at least one of the following operations:

[0175] Output the evaluation results of multiple experience indicators.

[0176] Multiple evaluation results for multiple experience indicators are outputted in a one-to-one correspondence, wherein each large experience evaluation model is obtained by fine-tuning training with training samples related to the corresponding experience indicator.

[0177] In some embodiments, at least one evaluation model includes a conversion evaluation model, and the agent evaluation unit 1420 is also used to input the conversation data into the conversion evaluation model to obtain the evaluation results of the conversion evaluation model for each conversion indicator, wherein the evaluation results of the conversion indicators represent the conversion effect of the target second agent on the target object during the conversation process.

[0178] Fig.15 is a block diagram of a training device for an intelligent agent according to an embodiment of the present disclosure.

[0179] like Fig.15 As shown, the training device 1500 of the agent may include a training sample unit 1510, a reward score unit 1520 and a parameter adjustment unit 1530.

[0180] The training sample unit 1510 is used to obtain a training sample set based on the dialogue data between the target first agent and the target second agent, and the dialogue data is obtained according to the dialogue data generating device 1300. The reward score unit 1520 is used to input the dialogue samples in the training sample set into the reward model to obtain the reward score output by the reward model. The parameter adjustment unit 1530 is used to perform reinforcement learning training on the target second agent based on the reward score to adjust the base large model parameters of the target second agent.

[0181] In some embodiments, the reward model includes multiple evaluation models, and the reward scoring unit 1520 is further used to: input the dialogue sample into the multiple evaluation models, output the evaluation result of each evaluation indicator in the multiple evaluation indicators, wherein the multiple evaluation indicators are used to evaluate the dialogue performance of the target second agent and the target first agent. The reward score is calculated based on the multiple evaluation results and their respective weights.

[0182] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0183] Fig.16A schematic block diagram of an example electronic device 1600 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0184] like Fig.16 As shown, the device 1600 includes a computing unit 1601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1602 or a computer program loaded from a storage unit 1608 to a random access memory (RAM) 1603. In the RAM 1603, various programs and data required for the operation of the device 1600 can also be stored. The computing unit 1601, the ROM 1602, and the RAM 1603 are connected to each other via a bus 1604. An input / output (I / O) interface 1605 is also connected to the bus 1604.

[0185] A number of components in the device 1600 are connected to the I / O interface 1605, including: an input unit 1606, such as a keyboard, a mouse, etc.; an output unit 1607, such as various types of displays, speakers, etc.; a storage unit 1608, such as a disk, an optical disk, etc.; and a communication unit 1609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1609 allows the device 1600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0186] The computing unit 1601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSP), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1601 performs the various methods and processes described above. For example, in some embodiments, the method involved in the present disclosure may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 1608. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1600 via the ROM 1602 and / or the communication unit 1609. When the computer program is loaded into the RAM 1603 and executed by the computing unit 1601, one or more steps of the method involved in the present disclosure described above may be performed. Alternatively, in other embodiments, the computing unit 1601 may be configured to execute the method involved in the present disclosure in any other appropriate manner (for example, by means of firmware).

[0187] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard parts (ASSPs), system on chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0188] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0189] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories, read-only memories, erasable programmable read-only memories (EPROM) or flash memories, optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0190] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) display or a liquid crystal display (LCD)) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0191] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a Local Area Network (LAN), a Wide Area Network (WAN), and the Internet.

[0192] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0193] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0194] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for generating dialogue data based on an agent, comprising: In response to the received dialog data generation request, obtaining a plurality of first agents configured with respective object portraits, and a plurality of second agents configured with respective entity portraits; Selecting a target first agent from the plurality of first agents to conduct a dialogue with a target second agent from the plurality of second agents to obtain dialogue data, Among them, the target first intelligent agent outputs the first dialogue content according to the configured target object portrait, the first dialogue content has the dialogue behavior characteristics and language style characteristics of the target object, and the target second intelligent agent outputs the second dialogue content according to the configured entity portrait.

2. The method according to claim 1, wherein: The step of selecting a target first agent from among the plurality of first agents to conduct a dialogue with a target second agent from among the plurality of second agents to obtain dialogue data comprises: Calling the behavior macromodel and the dialogue strategy macromodel of the target first intelligent agent, and conducting a dialogue with the target second intelligent agent to obtain the dialogue data; Among them, the behavior big model is used to output the dialogue behavior characteristics according to the target object portrait, and the dialogue strategy big model is used to output the first dialogue content with the language style characteristics according to the target object portrait.

3. The method according to claim 2, wherein: The calling of the behavior macromodel and the dialogue strategy macromodel of the target first intelligent agent to conduct a dialogue with the target second intelligent agent comprises: inputting the second dialogue content into the behavior macro model; causing the behavior macro model to output the dialogue behavior feature according to the second dialogue content and the target object portrait; The target first agent is caused to imitate the behavior of the target object to send the first dialogue content to the target second agent, wherein the behavior of the target object matches the action feature of the dialogue behavior feature.

4. The method according to claim 3, further comprising: After causing the behavior macro model to output the dialogue behavior features according to the second dialogue content and the target object portrait, In the case where the dialogue behavior feature also includes an information feature, the information feature and the second dialogue content are input into the dialogue strategy macro model to obtain the first dialogue content having the language style feature, wherein the first dialogue content includes information of the target object corresponding to the information feature.

5. The method according to claim 3, further comprising: After causing the behavior macro model to output the dialogue behavior features according to the second dialogue content and the target object portrait, In the case where the dialogue behavior feature does not include an information feature, the second dialogue content is input into the dialogue strategy macro model to obtain the first dialogue content having the language style feature.

6. The method according to claim 1, wherein: The selecting a target first agent among the plurality of first agents to have a dialogue with a target second agent among the plurality of second agents comprises: Generate a test question set based on each functional indicator of the target second intelligent agent, wherein each functional indicator is obtained according to the target entity portrait; Selecting the target first agent to initiate a question based on each of the test question sets; Obtain a set of question answers in which the target second agent replies to each of the test question sets.

7. The method according to claim 6, wherein: Use the large model to perform at least one of the following operations: Processing the target entity portrait to extract at least one of the functional indicators; Each of the functional indicators is processed to generate a test question set for each of the functional indicators.

8. An agent evaluation method, comprising: In response to an evaluation request for a target second agent, acquiring dialogue data between a target first agent and the target second agent, wherein the dialogue data is obtained according to the method according to any one of claims 1 to 7; The dialogue data is input into at least one evaluation model to obtain the evaluation result of the at least one evaluation model on the target second intelligent agent.

9. The method according to claim 8, wherein: The dialogue data includes a set of answers to questions of each functional indicator given by the target second agent, and the at least one evaluation model includes a scoring model. The step of inputting the dialogue data into at least one evaluation model to obtain an evaluation result of the at least one evaluation model on the target second agent comprises: The question and answer set, test question set and standard answer set of each functional indicator are input into the marking model to obtain the evaluation result of each functional indicator by the marking model.

10. The method according to claim 8, wherein: The at least one evaluation big model includes an experience evaluation big model, and the inputting the dialogue data into the at least one evaluation big model to obtain the evaluation result of the at least one evaluation big model on the target second intelligent agent includes: The conversation data and the evaluation prompt words of each experience indicator are input into the experience evaluation big model to obtain the evaluation results of the corresponding experience indicators, wherein the evaluation prompt words include the evaluation requirements for the corresponding experience indicators, and the evaluation results represent the experience value provided by the target second intelligent agent to the target object during the conversation.

11. The method according to claim 10, wherein: The at least one evaluation big model includes a plurality of the experience evaluation big models, The plurality of experience evaluation models perform at least one of the following operations: Outputting evaluation results of multiple experience indicators; Multiple evaluation results for multiple experience indicators are outputted in a one-to-one correspondence, wherein each large experience evaluation model is obtained by fine-tuning training with training samples related to the corresponding experience indicator.

12. The method according to claim 8, wherein: The at least one evaluation big model includes a conversion evaluation big model, and the inputting the dialogue data into the at least one evaluation big model to obtain the evaluation result of the at least one evaluation big model on the target second agent includes: The dialogue data is input into the conversion evaluation model to obtain the evaluation result of each conversion indicator of the conversion evaluation model, wherein the evaluation result of the conversion indicator represents the conversion effect of the target second intelligent agent on the target object during the dialogue process.

13. A method for training an intelligent agent, comprising: Acquire a training sample set based on the dialogue data between the target first agent and the target second agent, wherein the dialogue data is obtained according to the method described in any one of claims 1 to 7; Inputting the dialogue samples in the training sample set into the reward model to obtain the reward score output by the reward model; Reinforcement learning training is performed on the target second intelligent agent based on the reward score to adjust the base large model parameters of the target second intelligent agent.

14. The method according to claim 13, wherein: The reward model includes multiple evaluation models. The step of inputting the dialogue samples in the training sample set into the reward model to obtain the reward score output by the reward model comprises: Inputting the dialogue sample into the multiple evaluation models, and outputting the evaluation result of each evaluation indicator in multiple evaluation indicators, wherein the multiple evaluation indicators are used to evaluate the dialogue performance between the target second intelligent agent and the target first intelligent agent; The reward score is calculated based on the multiple evaluation results and their respective weights.

15. An intelligent agent configured to be trained according to the method according to claim 13 or 14.

16. A device for generating dialogue data based on an intelligent agent, comprising: A first acquisition unit, configured to acquire, in response to the received dialog data generation request, a plurality of first agents configured with respective object portraits, and a plurality of second agents configured with respective entity portraits; The first dialogue unit is used to select a target first agent from the plurality of first agents to dialogue with a target second agent from the plurality of second agents to obtain dialogue data. Among them, the target first intelligent agent outputs the first dialogue content according to the configured target object portrait, the first dialogue content has the dialogue behavior characteristics and language style characteristics of the target object, and the target second intelligent agent outputs the second dialogue content according to the configured entity portrait.

17. An intelligent agent evaluation device, comprising: a second acquisition unit, configured to acquire, in response to an evaluation request for a target second agent, dialogue data between the target first agent and the target second agent, wherein the dialogue data is obtained by the apparatus according to claim 16; The agent evaluation unit is used to input the dialogue data into at least one evaluation model to obtain the evaluation result of the at least one evaluation model on the target second agent.

18. A training device for an intelligent agent, comprising: A training sample unit, configured to obtain a training sample set based on the dialogue data between the target first agent and the target second agent, wherein the dialogue data is obtained by the apparatus according to claim 16; A reward scoring unit, used to input the dialogue samples in the training sample set into the reward model to obtain the reward score output by the reward model; A parameter adjustment unit is used to perform reinforcement learning training on the target second intelligent agent based on the reward score to adjust the base large model parameters of the target second intelligent agent.

19. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in any one of claims 1 to 7, or the method described in any one of claims 8 to 12, or the method described in claims 13 to 14.

20. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method of any one of claims 1 to 7, or the method of any one of claims 8 to 12, or the method of claims 13 to 14.

21. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 7, or the method according to any one of claims 8 to 12, or the method according to claims 13 to 14 is implemented.

Images