Information processing systems, information processing methods, and programs

Abstract

This technology provides information for controlling dialogue. [Solution] The information processing system comprises at least one memory and at least one processor, the at least one processor acquiring dialogue data indicating the content of the dialogue and inputting the dialogue data into a first machine learning model to generate input information to be input into a second machine learning model in order to cause the dialogue agent to perform the dialogue.

Description

Technical Field

[0001] This disclosure relates to an information processing system, an information processing method, and a program.

Background Art

[0002] Techniques for controlling a device that interacts with a human are known. For example, there is a technique that generates a response sentence when a user inputs text and displays the response to the user in text.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] This disclosure provides a technique for generating information for controlling an interaction.

Means for Solving the Problems

[0005] An information processing system according to a first aspect of this disclosure includes at least one memory and at least one processor. The at least one processor acquires dialogue data indicating dialogue content, and generates input information to be input to a second machine learning model in order to cause a dialogue agent to execute a dialogue by inputting the dialogue data into a first machine learning model.

[0006] An information processing system according to a second aspect of this disclosure comprises at least one memory and at least one processor, wherein the at least one processor acquires first input information for causing a first dialogue agent to perform a dialogue, acquires second input information for causing a second dialogue agent to perform a dialogue, inputs the first input information and the second input information into a first machine learning model, and inputs the second input information into a second machine learning model to perform a dialogue between the first dialogue agent and the second dialogue agent, wherein the first input information includes at least one of scenario information or phase information as information necessary for dialogue control, and the second input information contains less information necessary for dialogue control than the first input information. [Brief explanation of the drawing]

[0007] [Figure 1] This is a block diagram showing an example of the overall structure of a dialogue system. [Figure 2] This is a block diagram showing an example of the functional configuration of a control device. [Figure 3] This is a diagram showing an example of a chat screen. [Figure 4] This figure shows an example of a settings screen. [Figure 5] This figure shows an example of a dialogue example generation screen. [Figure 6] This figure shows an example of a prompt for generating dialogue examples. [Figure 7] This figure shows the first example of the dialogue example display screen. [Figure 8] This figure shows a second example of the dialogue example display screen. [Figure 9] This figure shows the first example of the profile generation screen. [Figure 10] This figure shows an example of a profile generation prompt. [Figure 11] This figure shows a second example of the profile generation screen. [Figure 12] This figure shows an example of a prompt display screen. [Figure 13] This figure shows an example of a debug screen. [Figure 14] This is a diagram showing the first example of the evaluation screen. [Figure 15] This figure shows a second example of the evaluation screen. [Figure 16] This is a processing flow illustrating an example of how prompts are created. [Figure 17] This is a processing flow illustrating an example of an agent verification method. [Figure 18] A block diagram showing an example of a computer hardware configuration. [Modes for carrying out the invention]

[0008] Hereinafter, embodiments of this disclosure will be described with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant descriptions will be omitted.

[0009] This disclosure relates to the technical fields of Human-Computer Interaction (HCI) and Artificial Intelligence (AI). One embodiment of this disclosure relates to a technology for streamlining the development and debugging of role-playing agents based on machine learning models such as Large Language Models (LLMs). In particular, one embodiment of this disclosure provides a framework that leverages the natural language generation capabilities of Large Language Models to reduce the effort required to create prompts for input to machine learning models and to debug them.

[0010] Note that a role-playing agent is a software agent that embodies a specific personality. A role-playing agent interacts with humans through various media such as text, voice, or video. A role-playing agent based on a machine learning model obtains messages from the interlocutor and generates messages to respond to the messages from the interlocutor based on the machine learning model. A role-playing agent is an example of a dialogue agent (or a first dialogue agent).

[0011] In recent years, large language models have made remarkable progress. Large language models have high natural language generation capabilities and are widely used in various fields. Large language models are not only providing information but also evolving into autonomous task execution agents. In particular, the development of role-playing agents that embody specific personalities is progressing rapidly. Role-playing agents are being considered for use in various applications such as education, recommendation, or counseling.

[0012] However, it is very difficult to create a role-playing agent that accurately reflects the specific requirements of users. In particular, the design of prompts for constructing a role-playing agent requires a great deal of time and expertise. Also, in order to verify whether the role-playing agent operates as intended, it is necessary to interact with the role-playing agent repeatedly, and this process is also a major burden. In addition, since large language models tend to generate general or ambiguous responses, improving the response quality of role-playing agents is also an issue.

[0013] One embodiment of the present disclosure proposes a framework that utilizes the capabilities of large language models to semi-automate the development and debugging of role-playing agents to address these issues. This embodiment mainly performs the following two tasks: (1) semi-automatically generates desirable dialogue examples and extracts prompts for setting roles for the role-playing agent; (2) verifies the role-playing agent using a debugging agent that serves as the dialogue partner of the role-playing agent.

[0014] Note that the debugging agent is a software agent used for debugging the role-playing agent. The debugging agent interacts with the role-playing agent on behalf of the user. The debugging agent is an example of an interaction agent (or the second interaction agent).

[0015] According to one embodiment of the present disclosure, the quality of the role-playing agent can be improved. For example, one embodiment of the present disclosure can reduce the tendency of large language models to generate general or ambiguous responses. Also, one embodiment of the present disclosure has the advantage that the user edits dialogue examples instead of editing prompts. Furthermore, one embodiment of the present disclosure can automate the debugging process, allowing the user to evaluate the quality of the prompt without entering messages. According to one embodiment of the present disclosure, since the manual labor of the user is significantly reduced, the effect of reducing the subjective workload of the user is obtained.

[0016] <Overall Configuration of the Dialogue System> One embodiment of this disclosure is an example of an information processing system for performing dialogue. Hereinafter, the information processing system according to this embodiment will be referred to as the "dialogue system." The dialogue system may perform dialogue with a dialogue partner by controlling a dialogue agent, or it may perform dialogue with another dialogue agent. Hereinafter, the dialogue partner of a dialogue agent will be referred to as the "dialogue partner." The dialogue partner may be any entity that can be the dialogue partner of a dialogue agent. The dialogue partner may be a user of the dialogue system, or it may not be a user of the dialogue system. The dialogue partner may be a human being, or it may be a non-human entity (for example, a CG character, a robot, a non-human organism, etc.).

[0017] This embodiment describes an example in which a dialogue system performs a text-based dialogue. A text-based dialogue may include, for example, text chat, email, messaging services, social networking services, etc. The dialogue performed by the dialogue system is not limited to text-based dialogue. The dialogue system may also perform, for example, voice-based dialogue, video-based dialogue, or dialogue via a robot.

[0018] Voice-based dialogue may, for example, involve the dialogue being conducted by capturing the dialoguer's speech using a microphone on a terminal device operated by the dialoguer, and outputting the dialogue agent's speech through the terminal device's speaker. The dialogue agent may synthesize speech that possesses the voice quality and speaking style characteristics of a specific person. This specific person may be a real person or a virtual character. In voice-based dialogue, the dialoguer interacts with a dialogue agent that mimics a specific person solely through voice via the terminal device.

[0019] Video-based interaction may involve, for example, displaying a computer graphics (CG) character on a display device and controlling the CG character to conduct the interaction with the participant. The CG character may be a virtual human, also known as an avatar. The CG character is not limited to humans and may be a real-world creature, a fictional creature, a robot, an anthropomorphic object, or an animated geometric pattern. In video-based interaction, the participant interacts with a virtual CG character displayed on a display device.

[0020] Robot-mediated dialogue may involve, for example, operating a robot capable of voice input and output to conduct a conversation with a person. The robot could be, for example, a humanoid modeled after a human. However, the robot is not limited to humans; it could be a real-world organism, a fictional organism, or an anthropomorphic object. In robot-mediated dialogue, the person interacting would be speaking with a robot that has a physical form.

[0021] The overall configuration of the dialogue system according to this embodiment will be described with reference to Figure 1. Figure 1 is a block diagram showing an example of the overall configuration of the dialogue system.

[0022] As shown in Figure 1, the dialogue system 1000 includes a control device 10, a generation device 20, and a terminal device 30. The control device 10, the generation device 20, and the terminal device 30 may be connected to each other via a communication network to enable data communication, or they may be integrated as a single piece of hardware. The communication network may be, for example, a LAN (Local Area Network), a WAN (Wide Area Network), a VPN (Virtual Private Network), or the Internet.

[0023] The control device 10 is an example of an information processing device that controls a dialogue agent AG, which is the dialogue partner of the interlocutor S. The control device 10 may be a computer such as a personal computer, workstation, or server. The control device 10 may construct the dialogue agent AG by inputting a role setting prompt to the machine learning model M possessed by the generation device 20. The control device 10 may generate a role setting prompt for constructing the dialogue agent AG. The control device 10 may verify the role setting prompt for constructing the dialogue agent AG.

[0024] A role-setting prompt is an example of a system prompt for setting a role for a dialogue agent AG in a dialogue. A system prompt is an example of a prompt for giving a permanent instruction to a machine learning model M. A prompt is an example of input information to be input to the machine learning model M in order to give instructions to the machine learning model M. A prompt may be text data, electronic data representing text, audio, still images, or video, or multimodal electronic data containing at least two of the following: text, audio, still images, or video.

[0025] The control device 10 may accept operations from user U. The control device 10 may generate or verify a role setting prompt in response to operations from user U. The control device 10 may display the screen operated by user U on the display device. The control device 10 may receive electronic data indicating operations from a terminal device (not shown) operated by user U. The control device 10 may transmit electronic data to a terminal device (not shown) operated by user U to display the screen operated by user U.

[0026] The control device 10 may acquire messages input by the interlocutor S (hereinafter referred to as "user messages") from the terminal device 30. The control device 10 may output messages to be presented to the interlocutor S (hereinafter referred to as "system messages") to the terminal device 30. The control device 10 may generate system messages based on user messages. The control device 10 may control the machine learning model M of the generation device 20 to generate system messages.

[0027] The generation device 20 is an example of an information processing device that performs a predetermined task based on a machine learning model. The generation device 20 may be a computer such as a personal computer, workstation, or server. The generation device 20 may perform a predetermined task based on a machine learning model M. The machine learning model M is an example of a machine learning model that has been trained to perform a predetermined task. The machine learning model M may be a machine learning model that can generate various types of data such as text, audio, images, and videos. The machine learning model M may be a neural network, a transformer, a large-scale language model, a generative model, or a foundational model, for example.

[0028] The machine learning model M may support multimodal operation. That is, the machine learning model M may take at least two inputs, such as text data, audio data, image data, or video data. The machine learning model M may also output at least two outputs, such as text data, audio data, image data, or video data.

[0029] The machine learning model M may be implemented using a single machine learning model. The machine learning model M may be implemented by the collaboration of multiple machine learning models. The machine learning model M may consist of multiple machine learning models depending on the task being performed.

[0030] The generation device 20 may perform a task to generate a role setting prompt in response to a request from the control device 10. The generation device 20 may perform a task to generate one or more pieces of information used to generate a role setting prompt in response to a request from the control device 10. The generation device 20 may perform a task to generate one or more pieces of information used to verify a role setting prompt in response to a request from the control device 10. The generation device 20 may perform a task to generate a system message in response to a request from the control device 10.

[0031] Terminal device 30 is an example of an information processing device operated by the interlocutor S. Terminal device 30 may be an information processing terminal such as a personal computer, smartphone, or tablet. Terminal device 30 may display a screen on its display device for performing dialogue with the dialogue agent AG. Terminal device 30 may receive user messages from the interlocutor S via the screen displayed on its display device. Terminal device 30 may present system messages to the interlocutor S via the screen displayed on its display device. Terminal device 30 may perform dialogue between the interlocutor S and the dialogue agent AG by repeatedly receiving user messages and presenting system messages.

[0032] Presenting information to a user of the dialogue system 1000 may include the processor performing at least a portion of the processing necessary to display the information on a display device. The display device may be provided in the same device as the processor, or in a different device from the processor. There may be multiple display devices.

[0033] Note that the overall configuration of the dialogue system 1000 shown in Figure 1 is just one example, and various system configurations are possible depending on the application and purpose. The dialogue system 1000 may consist of one or more devices. Each device included in the dialogue system 1000 may be part of a system composed of multiple devices. Each function included in the dialogue system 1000 may be implemented by any device that makes up the system. Each component included in the dialogue system 1000 may be included in any device that makes up the system.

[0034] The dialogue system 1000 may include multiple units of one or more of the control devices 10, generation devices 20, or terminal devices 30. The control devices 10, generation devices 20, or terminal devices 30 may be one or more information processing devices, or they may be information processing systems that include multiple information processing devices. The control devices 10, generation devices 20, or terminal devices 30 may be implemented by one or more computers, or they may be implemented as a cloud computing service. The dialogue system 1000 may be implemented by a standalone computer that integrates the control devices 10, generation devices 20, and terminal devices 30. The terminal device 30 may be implemented by other devices included in the dialogue system 1000. The terminal device 30 may be implemented by an information processing device or information processing system outside of the dialogue system 1000. Note that "external" means not included in the dialogue system 1000. The classification of devices such as the control devices 10, generation devices 20, and terminal devices 30 shown in Figure 1 is just one example.

[0035] As an example, the dialogue system 1000 may consist of one or more server devices and one or more client devices. The one or more server devices may have one or more of the functions of the control device 10 and the generation device 20. The one or more client devices may have one or more of the functions of the terminal device 30. The server device may be implemented as a system including multiple information processing devices. The server device may be implemented as a cloud computing service.

[0036] As another example, the dialogue system 1000 may consist of a single information processing device. This single information processing device may include the functions of a control device 10, a generation device 20, and a terminal device 30.

[0037] <Functional Configuration of Control Device> The functional configuration of the control device 10 will be explained with reference to Figure 2. Figure 2 is a block diagram showing an example of the functional configuration of the control device.

[0038] As shown in Figure 2, the control device 10 comprises an agent control unit 100, a prompt creation support unit 110, and an agent verification support unit 120. The control device 10 functions as the agent control unit 100, the prompt creation support unit 110, and the agent verification support unit 120 when one or more pre-installed control programs are executed.

[0039] The agent control unit 100 controls the conversational agent AG. The agent control unit 100 may perform a conversation with the conversational agent S by controlling the conversational agent AG. The agent control unit 100 may perform a text-based conversation (for example, text chat) with the conversational agent S by controlling the conversational agent AG. The agent control unit 100 may perform a voice-based conversation with the conversational agent S by controlling the conversational agent AG. The agent control unit 100 may perform a video-based conversation with the conversational agent S by controlling the conversational agent AG.

[0040] The agent control unit 100 may set a role setting prompt in the machine learning model M. Based on the machine learning model M in which the role setting prompt has been set, the agent control unit 100 may execute a dialogue between the interlocutor S and the dialogue agent AG. Since the role setting prompt defines the roles of the interlocutor S and the dialogue agent AG, the agent control unit 100 can control the dialogue agent AG while maintaining its role during the dialogue.

[0041] The agent control unit 100 may repeatedly receive user messages and present system messages until the dialogue between the interlocutor S and the dialogue agent AG is completed. The agent control unit 100 may determine whether the dialogue between the interlocutor S and the dialogue agent AG has ended based on the role setting prompt. For example, the agent control unit 100 may determine that the dialogue between the interlocutor S and the dialogue agent AG has ended if the scenario indicated in the role setting prompt has ended. For example, the agent control unit 100 may determine that the dialogue between the interlocutor S and the dialogue agent AG has ended if it is unable to obtain a user message after a predetermined time has elapsed.

[0042] The prompt creation support unit 110 assists user U in creating role setting prompts. The prompt creation support unit 110 may generate dialogue data showing example dialogues based on input from user U. The prompt creation support unit 110 may modify the generated dialogue data. The prompt creation support unit 110 may generate a role setting prompt based on the dialogue data confirmed by user U. The prompt creation support unit 110 may modify the generated role setting prompt.

[0043] The prompt creation support unit 110 stores the role setting prompt in the storage device of the control device 10. Multiple role setting prompts may be stored in the storage device of the control device 10. The multiple role setting prompts may differ in at least one of the following: the role of the dialogue agent AG, the role of the dialogue participant S, the dialogue scenario, or the dialogue phase.

[0044] The agent verification support unit 120 assists the user U in verifying the conversational agent AG. The agent verification support unit 120 may construct the conversational agent AG based on the role setting prompt. The agent verification support unit 120 may construct a debug agent that will be the conversational partner of the conversational agent AG. The agent verification support unit 120 may execute the conversation between the conversational agent AG and the debug agent. The agent verification support unit 120 may generate conversational data that shows the content of the conversation between the conversational agent AG and the debug agent.

[0045] The agent verification support unit 120 may evaluate the content of the dialogue between the conversational agent AG and the debug agent. The agent verification support unit 120 may generate evaluation items for evaluating the content of the dialogue between the conversational agent AG and the debug agent. The agent verification support unit 120 may evaluate the content of the dialogue between the conversational agent AG and the debug agent based on the generated evaluation items. For example, the agent verification support unit 120 may automatically evaluate whether the content of the dialogue between the conversational agent AG and the debug agent conforms to the scenario and behavior specified in the role setting prompt.

[0046] (Detailed configuration of the agent control unit) As shown in Figure 2, the agent control unit 100 includes an agent construction unit 101, a message acquisition unit 102, a message generation unit 103, and a message output unit 104.

[0047] The agent construction unit 101 constructs a conversational agent AG. The agent construction unit 101 may construct the conversational agent AG based on a role setting prompt read from the storage device of the control device 10. The agent construction unit 101 may construct the conversational agent AG based on a role setting prompt generated by the prompt creation support unit 110. The agent construction unit 101 may construct the conversational agent AG by inputting the role setting prompt of the conversational agent AG into the machine learning model M.

[0048] The message acquisition unit 102 acquires user messages. The message acquisition unit 102 may receive user messages from the terminal device 30. The message acquisition unit 102 may acquire user messages entered into the terminal device 30 by the interlocutor S. The terminal device 30 may receive user messages entered by the interlocutor S via the screen displayed on the display device of the terminal device 30. The terminal device 30 may receive user messages by performing speech recognition on the voice of the interlocutor S picked up by the microphone of the terminal device 30. The terminal device 30 may display the received user messages on the screen displayed on the display device.

[0049] The message generation unit 103 generates system messages. The message generation unit 103 may also generate system messages in response to user messages. The message generation unit 103 may also generate system messages based on a conversational agent AG. The message generation unit 103 may also generate system messages based on a machine learning model M in which the role setting prompt for the conversational agent AG has been configured.

[0050] The message generation unit 103 may generate a system message in response to a user message based on the user message and the machine learning model M. The message generation unit 103 may send a prompt to the generation device 20 instructing it to generate a system message, and receive a system message generated by the generation device 20 inputting the prompt to the machine learning model M from the generation device 20. The message generation unit 103 may generate a system message in response to a user message by inputting a prompt to the machine learning model M instructing it to generate a system message. Hereinafter, the prompt instructing the generation of a system message will be referred to as the "message generation prompt".

[0051] The message generation prompt may include a user message and dialogue history information. Furthermore, the message generation prompt may also include a role setting prompt. The dialogue history information may be information in which the utterances of the interlocutor S (user messages) and the utterances of the dialogue agent AG (system messages) are linked in chronological order. The dialogue history information may include utterances from the start of the dialogue up to the immediately preceding utterance. The dialogue history information may include a predetermined number of utterances, going backward in time from the immediately preceding utterance. Each utterance may be associated with information indicating the speaker (interlocutor S or dialogue agent AG). The information indicating the speaker may be added before or after the utterance. The information indicating the speaker may be added in a way that makes it identifiable by a predetermined symbol. The boundaries of each utterance may be indicated by a predetermined symbol. The dialogue history information may include information indicating the speaker for which utterances are to be generated. The information indicating the speaker for which utterances are to be generated does not need to be associated with utterances.

[0052] Furthermore, generating other information based on certain information and a machine learning model may include performing one or more of the following processes: • The process of inputting certain information into a machine learning model to generate other information. • The process of generating information based on existing information, inputting that information into a machine learning model, and then generating other information from that model. • Using information extracted from certain data, a machine learning model is used to perform a generation process that generates other information.

[0053] The message output unit 104 outputs a system message. The message output unit 104 may output a system message in order to present the system message to the interlocutor S. The dialogue control unit 170 may send the system message to the terminal device 30. The terminal device 30 may present the system message to the interlocutor S via a screen displayed on a display device. The terminal device 30 may display the system message to be presented to the interlocutor S on a screen displayed on a display device. The display device that displays the screen may be a display device provided by the terminal device 30, or a display device connected to the terminal device 30.

[0054] (Detailed configuration of the prompt creation support unit) As shown in Figure 2, the prompt creation support unit 110 includes a dialogue data acquisition unit 111, a dialogue data modification unit 112, a prompt generation unit 113, and a prompt modification unit 114.

[0055] The dialogue data acquisition unit 111 acquires dialogue data. Dialogue data is an example of electronic data showing the content of a conversation between multiple participants. Dialogue data may also be data showing the content of a conversation between a first participant and a second participant. Dialogue data may be used to generate a role setting prompt. Dialogue data may also be data showing an example of a conversation to be executed by a dialogue agent AG. Dialogue data may also be data in which the utterances of each of the multiple participants are arranged in utterance order.

[0056] Dialogue data may have identification information of each utterance of the interlocutors associated with the interlocutors. The interlocutor identification information may also be information indicating the interlocutor's role. The information indicating the interlocutor's role may be information that explicitly indicates the interlocutor's role, or information that indirectly indicates the interlocutor's role. The information indicating the interlocutor's role may also be information that indicates the relationship between the interlocutors. The information indicating the relationship between the interlocutors may include information indicating the role of the first speaker and information indicating the role of the second speaker.

[0057] The relationship between the interlocutors may include, for example, at least one or a combination of at least two of the following: a relationship in which the first interlocutor provides services or goods to the second interlocutor; a relationship in which the first interlocutor instructs the second interlocutor; a relationship in which the first interlocutor pays remuneration to the second interlocutor; or a relationship in which the first interlocutor provides information to the second interlocutor. Specifically, the relationship between the interlocutors may include, for example, at least one of the following: a relationship between an examinee and an interviewer; a relationship between a retail store clerk and a customer; a relationship between a superior and a subordinate; a relationship between a teacher and a student; a relationship between a speaker and an audience; a relationship between a doctor and a patient; or a relationship between a client and a representative. The client may include, for example, a professional such as a lawyer or consultant.

[0058] The dialogue data acquisition unit 111 may generate dialogue data based on the machine learning model M. The dialogue data acquisition unit 111 may send a prompt to the generation device 20 instructing the generation of dialogue data, and receive the generated dialogue data from the generation device 20 when the generation device 20 inputs the prompt to the machine learning model M. If the control device 10 has a machine learning model M, the dialogue data acquisition unit 111 may acquire dialogue data generated by the machine learning model M by inputting a prompt to the machine learning model M instructing the generation of dialogue data. Hereinafter, the prompt input to the machine learning model M to generate dialogue data will be referred to as the "dialogue generation prompt".

[0059] The dialogue generation prompt may include information indicating dialogue conditions. These dialogue conditions may be conditions that must be met for a dialogue between multiple dialoguers. Examples of information indicating dialogue conditions may include at least one of the following: information indicating the role of each dialoguer, information indicating the location (setting) where the dialogue takes place, information indicating the scene (phase) of the dialogue, information indicating the situation of the dialogue, information indicating the purpose of the dialogue, the number of utterances during the dialogue, or the number of phases during the dialogue. Information indicating the scene of the dialogue may include at least one of the scene where the dialogue begins or the scene where the dialogue ends.

[0060] The dialogue data acquisition unit 111 may accept input of information indicating dialogue conditions from user U. The dialogue data acquisition unit 111 may accept input of information indicating dialogue conditions via a screen displayed on a display device. The dialogue data acquisition unit 111 may read information indicating dialogue conditions that has been pre-stored in a storage device.

[0061] The dialogue data acquisition unit 111 may acquire parameters for generating dialogue data. The parameters for generating dialogue data may include information indicating the machine learning model M used to generate the dialogue data. The parameters for generating dialogue data may also include a temperature parameter. The temperature parameter is a parameter that indicates the randomness of the information generated by the machine learning model M. For example, if the value of the temperature parameter is large, the randomness of the output of the machine learning model M may be high. On the other hand, if the value of the temperature parameter is small, the randomness of the output of the machine learning model M may be low.

[0062] The dialogue data acquisition unit 111 may accept dialogue data input from user U. User U may create dialogue data manually. User U may create dialogue data by transcribing the dialogue audio of multiple participants. User U may create dialogue data by performing speech recognition on the dialogue audio of multiple participants. User U may create dialogue data by editing the transcription or speech recognition results of the dialogue audio.

[0063] The dialogue data modification unit 112 modifies the dialogue data. Modification of the dialogue data may include editing the dialogue data or refining the dialogue data. Modification of the dialogue data may also include adding, changing, or deleting one or more utterances from one or more dialoguers.

[0064] The dialogue data modification unit 112 may accept modifications to the dialogue data by the user U. The dialogue data modification unit 112 may accept modifications to the dialogue data via a screen displayed on a display device. The dialogue data modification unit 112 may display the generated dialogue data on the screen and modify the dialogue data based on the modification instructions entered by the user U on the screen. The dialogue data modification unit 112 may acquire the modified dialogue data entered by the user U on the screen.

[0065] The dialogue data modification unit 112 may modify the dialogue data based on the machine learning model M. The dialogue data modification unit 112 may send a prompt to the generation device 20 instructing modification of the dialogue data, and receive the modified dialogue data generated by the generation device 20 when the generation device 20 inputs the prompt to the machine learning model M. If the control device 10 has a machine learning model M, the dialogue data acquisition unit 111 may acquire the modified dialogue data generated by the machine learning model M by inputting a prompt to the machine learning model M instructing modification of the dialogue data. The prompt instructing modification of the dialogue data may include, for example, the dialogue data before modification and the modification instructions input by the user U.

[0066] The prompt generation unit 113 generates a role setting prompt. The prompt generation unit 113 may generate a role setting prompt for constructing a dialogue agent AG. The prompt generation unit 113 may generate a role setting prompt based on dialogue data acquired by the dialogue data acquisition unit 111. The prompt generation unit 113 may generate a role setting prompt based on dialogue data modified by the dialogue data modification unit 112.

[0067] The role setting prompt may include background information, profile information, scenario information, and phase information. Background information may include basic instructions and additional conditions. Basic instructions may include information specifying the role of the conversational agent AG (e.g., a natural language sentence such as "You are [role]"), response length limits, and information indicating that the response should be conversational. Additional conditions may include information indicating conditions for leading to the desired behavior of the conversational agent AG. Basic instructions and additional conditions may be presented, for example, as a bulleted list.

[0068] Profile information is an example of information that shows the profile of the conversational agent AG (for example, attributes, basic information, personal information, etc.). Profile information may also include personality traits or situational factors, as an example. By including profile information in the role setting prompt, the control device 10 can fine-tune the behavior of the conversational agent AG and change the persona of the conversational agent AG to suit the scenario and the conversational partner S.

[0069] Profile information may include at least one of the following: attributes, basic information, personal information, personality, characteristics, or situational factors. For example, profile information may include age, gender, occupation, personality, hobbies, interests, annual income, family, lifestyle, reason for visiting the conversation location (motivation), beliefs, expectations, intended actions, or attitude towards the conversation partner. Profile information may also include any profile elements that may influence the scenario. For example, in a conversation scenario at a travel agency, profile elements that may influence the scenario may include family structure, place of residence, means of transportation, etc.

[0070] Profile information may be structured text data in a specified format, for example. This specified format may include, for example, JSON (JavaScript Object Notation), Markdown, HTML (Hyper Text Markup Language), XML (Extensible Markup Language), CSV (Comma Separated Values), Mermaid format, etc.

[0071] Scenario information is an example of information that indicates the scenario of the dialogue that the dialogue agent AG will execute (for example, it can be called the setting, context, assumptions, storyline, etc.). Scenario information may, for example, be text data that expresses the overall dialogue scenario in natural language. By including scenario information in the role setting prompt, the control device 10 can enable the dialogue agent AG to behave consistently in a contextual manner throughout the entire dialogue.

[0072] Scenario information may include at least one of the following: setting, context, assumptions, or plot. For example, in a scenario involving a dialogue at a travel agency, the scenario information may include the following: A 38-year-old man visits a travel agency. He is planning a family trip during the winter holidays and is looking for an itinerary that includes hot springs and skiing. • The staff suggested the Niseko area of ​​Hokkaido and explained the details of accommodation and meals. • The customer has shown interest in a specific set menu and is considering a 3-night, 4-day plan. Ultimately, the customer is satisfied with the plan, provides their name and contact information to proceed with the reservation, and leaves the store.

[0073] Phase information is an example of information that indicates the phases (for example, flow, development, scene, stage, etc.) of the dialogue performed by the dialogue agent AG. Phase information may also be text data that expresses the content of each phase in the dialogue and the conditions for transitioning to the next phase in natural language. By including phase information in the role setting prompt, the control device 10 can guide the dialogue agent AG to proceed with a complex dialogue and ensure that important steps are not omitted.

[0074] Phase information may include at least one of the following: phase transition information, flow, development, scene, or stage. For example, phase information may include the phase division of the dialogue, a summary of the content of each phase, and the conditions for transitioning to the next phase. For example, in a dialogue scenario at a travel agency, the phase information may be divided into an entry phase, a request confirmation phase, and a plan proposal phase, and may include the specific content of each phase and the transition conditions (for example, transitioning to the next phase when the customer shows interest in a particular region).

[0075] The prompt generation unit 113 may generate profile information. The prompt generation unit 113 may generate profile information based on the dialogue data acquired by the dialogue data acquisition unit 111. The prompt generation unit 113 may generate profile information based on the dialogue data modified by the dialogue data modification unit 112. The prompt generation unit 113 may generate profile information for the dialogue agent AG based on the dialogue content shown in the dialogue data, specifically the utterances of the dialoguer that may be designated as the role of the dialogue agent AG.

[0076] The prompt generation unit 113 may generate profile information based on the machine learning model M. The prompt generation unit 113 may send a prompt instructing the generation of profile information to the generation device 20, and receive the generated profile information from the generation device 20 when the generation device 20 inputs the prompt to the machine learning model M. If the control device 10 has a machine learning model M, the prompt generation unit 113 may obtain the profile information generated by the machine learning model M by inputting a prompt instructing the generation of profile information to the machine learning model M. Hereinafter, the prompt instructing the generation of profile information will be referred to as the "profile generation prompt".

[0077] The profile generation prompt may, for example, include the dialogue content shown in the dialogue data. The profile generation prompt may, for example, include information instructing the system to estimate the dialoguer's profile from the dialogue content shown in the dialogue data.

[0078] The prompt generation unit 113 may generate multiple profile candidates. The prompt generation unit 113 may accept an operation by user U to select profile information from the multiple profile candidates. The prompt generation unit 113 may accept the selection of profile information via a screen displayed on a display device. The prompt generation unit 113 may display the generated multiple profile candidates on the screen and select the profile candidate selected by user U on the screen as the profile information to be included in the role setting prompt.

[0079] The prompt generation unit 113 may display a predetermined number (one or more) of profile candidates on the screen and accept a selection from user U whether or not to adopt a profile candidate. If user U chooses not to adopt a profile candidate, the prompt generation unit 113 may display a predetermined number of other profile candidates on the screen. If user U chooses to adopt a profile candidate, the prompt generation unit 113 may select that profile candidate as the profile information to be included in the role setting prompt.

[0080] The prompt generation unit 113 may generate scenario information. The prompt generation unit 113 may generate scenario information based on the dialogue data acquired by the dialogue data acquisition unit 111. The prompt generation unit 113 may generate scenario information based on the dialogue data modified by the dialogue data modification unit 112. The prompt generation unit 113 may generate scenario information by summarizing the entire dialogue content shown in the dialogue data.

[0081] The prompt generation unit 113 may generate scenario information based on the machine learning model M. The prompt generation unit 113 may send a prompt instructing the generation of scenario information to the generation device 20, and receive the generated scenario information from the generation device 20 when the generation device 20 inputs the prompt to the machine learning model M. If the control device 10 has a machine learning model M, the prompt generation unit 113 may obtain the scenario information generated by the machine learning model M by inputting a prompt instructing the generation of scenario information to the machine learning model M. Hereinafter, the prompt instructing the generation of scenario information will be referred to as the "scenario generation prompt".

[0082] The scenario generation prompt may, for example, include the dialogue content shown in the dialogue data. The scenario generation prompt may, for example, include information instructing the user to summarize the entire dialogue content shown in the dialogue data.

[0083] The prompt generation unit 113 may generate phase information. The prompt generation unit 113 may generate phase information based on the dialogue data acquired by the dialogue data acquisition unit 111. The prompt generation unit 113 may generate phase information based on the dialogue data modified by the dialogue data modification unit 112. The prompt generation unit 113 may generate phase information by dividing the dialogue content shown in the dialogue data into multiple phases and summarizing the dialogue content of each phase.

[0084] The prompt generation unit 113 may generate phase information based on the machine learning model M. The prompt generation unit 113 may send a prompt instructing the generation of phase information to the generation device 20, and receive the generated phase information from the generation device 20 when the generation device 20 inputs the prompt to the machine learning model M. If the control device 10 has a machine learning model M, the prompt generation unit 113 may obtain the phase information generated by the machine learning model M by inputting a prompt instructing the generation of phase information to the machine learning model M. Hereinafter, the prompt instructing the generation of phase information will be referred to as the "phase generation prompt".

[0085] A phase generation prompt may, for example, include the dialogue content shown in the dialogue data. A phase generation prompt may, for example, include information instructing the user to divide the dialogue content shown in the dialogue data into one or more phases and to summarize the content of each phase and the conditions for transitioning to the next phase.

[0086] The prompt generation unit 113 may generate a role setting prompt that includes one of the following: profile information, scenario information, or phase information. The prompt generation unit 113 may generate a role setting prompt that includes at least two of the following: profile information, scenario information, or phase information. The prompt generation unit 113 may generate a role setting prompt that includes all of the following: profile information, scenario information, and phase information.

[0087] Furthermore, the inclusion of information in a prompt may include the information being included directly in the prompt, other information generated based on that information being included in the prompt, other information generated by a machine learning model based on that information being included in the prompt, the information being directly embedded in the prompt, or the information being embedded in a prompt for generating the prompt. For example, the inclusion of profile information in a role setting prompt may include the profile information being included directly in the role setting prompt, other information generated based on the profile information being included in the role setting prompt, other information generated by a machine learning model based on the profile information being included in the role setting prompt, the profile information being directly embedded in the role setting prompt, or the profile information being embedded in a prompt for generating the role setting prompt. For example, the inclusion of scenario information in a role setting prompt may include the scenario information being included directly in the role setting prompt, other information generated based on the scenario information being included in the role setting prompt, other information generated by a machine learning model based on the scenario information being included in the role setting prompt, the scenario information being directly embedded in the role setting prompt, or the scenario information being embedded in a prompt for generating the role setting prompt. For example, the inclusion of phase information in a role setting prompt may include: the phase information being directly included in the role setting prompt; other information generated based on the phase information being included in the role setting prompt; other information generated by a machine learning model based on the phase information being included in the role setting prompt; the phase information being directly embedded in the role setting prompt; or the phase information being embedded in a prompt for generating the role setting prompt.

[0088] The prompt generation unit 113 may generate a role setting prompt for constructing a debug agent. The prompt generation unit 113 may generate a role setting prompt for a debug agent based on the dialogue data acquired by the dialogue data acquisition unit 111. The prompt generation unit 113 may generate a role setting prompt for a debug agent based on the dialogue data modified by the dialogue data modification unit 112. The prompt generation unit 113 may generate profile information for a debug agent based on the dialogue content shown in the dialogue data, specifically the utterances of the interlocutor S that can be designated as the role of the interlocutor S. That is, the role setting prompt for a debug agent may be generated based on the same dialogue data as the role setting prompt for the dialogue agent AG.

[0089] The prompt generation unit 113 may generate a debug agent role setting prompt such that it requires less information for dialogue control than the dialogue agent AG role setting prompt. The information required for dialogue control may include, for example, at least one of scenario information or phase information. The debug agent role setting prompt may not include at least a portion of the scenario information included in the dialogue agent AG role setting prompt. The debug agent role setting prompt may not include at least a portion of the phase information included in the dialogue agent AG role setting prompt. For example, the prompt generation unit 113 may generate a debug agent role setting prompt by inputting a prompt to the machine learning model M that instructs it to more concisely summarize the dialogue content shown in the dialogue data. The dialogue agent AG role setting prompt is an example of first input information. The debug agent role setting prompt is an example of second input information.

[0090] The prompt generation unit 113 may accept input from user U for the debug agent role setting prompt. User U may manually create the debug agent role setting prompt. User U may create the debug agent role setting prompt by editing the interactive agent AG's role setting prompt. User U may create the debug agent role setting prompt by deleting some of the information from the interactive agent AG's role setting prompt.

[0091] The prompt modification unit 114 modifies the role setting prompt. Modifying the role setting prompt may include editing the role setting prompt or refining the role setting prompt. Modifying the role setting prompt may also include adding, changing, or deleting information contained in the role setting prompt.

[0092] The prompt correction unit 114 may accept corrections to the role setting prompt by user U. The prompt correction unit 114 may accept corrections to the role setting prompt via a screen displayed on a display device. The prompt correction unit 114 may display the generated role setting prompt on the screen and correct the role setting prompt based on the correction instructions entered by user U on the screen. The prompt correction unit 114 may acquire the corrected role setting prompt entered by user U on the screen.

[0093] The prompt modification unit 114 may modify one of the following from the role setting prompts: profile information, scenario information, or phase information. The prompt modification unit 114 may modify at least two of the following from the role setting prompts: profile information, scenario information, or phase information. The prompt modification unit 114 may modify all of the following from the role setting prompts: profile information, scenario information, and phase information.

[0094] (Detailed structure of the agent verification support unit) As shown in Figure 2, the agent verification support unit 120 includes a debugger construction unit 121, an interaction execution unit 122, an evaluation item generation unit 123, and an evaluation execution unit 124.

[0095] The debugger construction unit 121 constructs a debug agent. The debugger construction unit 121 may construct the debug agent based on the role setting prompt generated by the prompt creation support unit 110. The debugger construction unit 121 may also construct the debug agent by inputting the role setting prompt of the debug agent into the machine learning model M.

[0096] The debugger construction unit 121 may construct a dialogue agent AG. The debugger construction unit 121 may construct a dialogue agent AG based on a role setting prompt generated by the prompt creation support unit 110. The debugger construction unit 121 may construct a dialogue agent AG by inputting the dialogue agent AG's role setting prompt into the machine learning model M. The debugger construction unit 121 may obtain the dialogue agent AG constructed by the agent control unit 100.

[0097] The debugger construction unit 121 may construct a debug agent in the control device 10, or it may construct a debug agent in an information processing device other than the control device 10. The debugger construction unit 121 may construct a dialogue agent AG in the control device 10, or it may construct a dialogue agent AG in an information processing device other than the control device 10. The information processing device in which the debugger construction unit 121 constructs the debug agent and the information processing device in which the debugger construction unit 121 or the agent construction unit 101 constructs the dialogue agent AG may be the same information processing device or different information processing devices.

[0098] The dialogue execution unit 122 executes the dialogue between the dialogue agent AG and the debug agent. The dialogue execution unit 122 may generate messages from the dialogue agent AG. The dialogue execution unit 122 may generate messages from the debug agent. The dialogue execution unit 122 may execute the dialogue between the dialogue agent AG and the debug agent by repeatedly generating messages from the dialogue agent AG and messages from the debug agent.

[0099] The dialogue execution unit 122 may generate a message from the dialogue agent AG based on the dialogue agent AG. The dialogue execution unit 122 may generate a message from the dialogue agent AG based on the machine learning model M in which the dialogue agent AG's role setting prompt is set. The dialogue execution unit 122 may generate a message from the dialogue agent AG by inputting a prompt to the machine learning model M instructing it to generate a message from the dialogue agent AG. The prompt instructing the machine learning model M to generate a message from the dialogue agent AG may include a message from the debug agent. The prompt instructing the machine learning model M to generate a message from the dialogue agent AG may include the content of the dialogue between the dialogue agent AG and the debug agent.

[0100] The dialogue execution unit 122 may generate a message from the debug agent based on the debug agent. The dialogue execution unit 122 may generate a message from the debug agent based on the machine learning model M in which the debug agent's role setting prompt has been set. The dialogue execution unit 122 may generate a message from the debug agent by inputting a prompt to the machine learning model M instructing it to generate a message from the debug agent. The prompt instructing the generation of a message from the debug agent may include a message from the dialogue agent AG. The prompt instructing the generation of a message from the debug agent may include the content of the dialogue between the dialogue agent AG and the debug agent.

[0101] The dialogue execution unit 122 may accept message input from user U. The dialogue execution unit 122 may accept messages input by user U as messages from dialogue agent AG. The dialogue execution unit 122 may accept messages input by user U as messages from debug agent. The dialogue execution unit 122 may accept message input from user U via a screen displayed on a display device.

[0102] The dialogue execution unit 122 may re-execute the dialogue between the dialogue agent AG and the debug agent. The dialogue execution unit 122 may re-execute the dialogue from a specific message during the dialogue as the starting point. The dialogue execution unit 122 may accept a message from user U to be used as the starting point for re-execution. The dialogue execution unit 122 may replace the message specified as the starting point for re-execution with a message entered by user U and re-execute the dialogue.

[0103] The evaluation item generation unit 123 generates evaluation items for evaluating the content of the dialogue. The evaluation item generation unit 123 may generate evaluation items for evaluating the content of the dialogue between the dialogue agent AG and the debug agent. The evaluation items may include items for determining whether or not predetermined evaluation criteria are met. The predetermined evaluation criteria may include, as an example, at least one of the following: dialogue consistency, dialogue coherence, role matching, degree of following / adhering to / fitting of role setting prompts, factual accuracy, dialogue fluency, or dialogue style. Furthermore, as an example, it may include at least one of the following / adhering to / fitting of profile information, scenario information, or phase information used to create the role setting prompts. The evaluation items may also be information that lists the main points in the dialogue scenario. For example, the evaluation items may include information that indicates information that should be included in the utterances of the dialogue agent AG. For example, the evaluation items may include information that indicates the conditions that the flow of the dialogue between the debug agent and the dialogue agent AG should satisfy.

[0104] The evaluation item generation unit 123 may generate evaluation items based on dialogue data. The evaluation item generation unit 123 may generate evaluation items based on dialogue data acquired by the dialogue data acquisition unit 111. The evaluation item generation unit 123 may generate evaluation items based on dialogue data modified by the dialogue data modification unit 112. The evaluation item generation unit 123 may generate evaluation items based on the role setting prompt of the dialogue agent AG. The evaluation item generation unit 123 may generate evaluation items based on the role setting prompt of the debug agent. The evaluation item generation unit 123 may generate evaluation items based on at least one of the profile information, scenario information, or phase information included in each role setting prompt.

[0105] The evaluation item generation unit 123 may generate evaluation items based on the machine learning model M. The evaluation item generation unit 123 may send a prompt to the generation device 20 to instruct the generation of evaluation items, and receive the generated evaluation items from the generation device 20 when the generation device 20 inputs the prompt to the machine learning model M. The evaluation item generation unit 123 may obtain evaluation items generated by the machine learning model M by inputting a prompt to the machine learning model M to instruct the generation of evaluation items. Hereinafter, the prompt instructing the generation of evaluation items will be referred to as the "evaluation item generation prompt".

[0106] The evaluation item generation prompt may, for example, include the dialogue content shown in the dialogue data. The evaluation item generation prompt may, for example, include information instructing the extraction of points that can evaluate the consistency, coherence, or role matching of the dialogue content shown in the dialogue data. The evaluation item generation prompt may, for example, include information instructing the extraction of points that can evaluate the adherence to the role setting prompt and the profile information, scenario information, or phase information used to create it.

[0107] The evaluation execution unit 124 performs an evaluation of the dialogue content. The evaluation execution unit 124 may evaluate the dialogue content between the dialogue agent AG and the debug agent. The evaluation execution unit 124 may evaluate the dialogue content between the dialogue agent AG and the debug agent based on the evaluation items generated by the evaluation item generation unit 123.

[0108] The evaluation execution unit 124 may evaluate the content of the dialogue between the dialogue agent AG and the debug agent based on the machine learning model M. The evaluation execution unit 124 may send a prompt to the generation device 20 to instruct it to evaluate the dialogue content, and receive the evaluation result generated by the generation device 20 when the generation device 20 inputs the prompt to the machine learning model M. The evaluation execution unit 124 may obtain the evaluation result generated by the machine learning model M by inputting a prompt to the machine learning model M to instruct it to evaluate the dialogue content. Hereinafter, the prompt instructing the evaluation of the dialogue content will be referred to as the "evaluation result generation prompt".

[0109] The evaluation result generation prompt may, for example, include dialogue data showing the content of the dialogue between the dialogue agent AG and the debug agent. The evaluation result generation prompt may, for example, include instruction information that instructs the system to determine whether the dialogue content satisfies each evaluation item. The evaluation result generation prompt may, for example, include instruction information that instructs the system to output the reasons why the dialogue content satisfies or fails to satisfy the evaluation items.

[0110] The evaluation results of the dialogue content may include information indicating whether or not the evaluation criteria are met. If the evaluation results of the dialogue content determine that the evaluation criteria are met, the reasons for that determination may be included. If the evaluation results of the dialogue content determine that the evaluation criteria are not met, the reasons for that determination may be included. The evaluation results of the dialogue content may include information indicating whether or not each of the multiple evaluation criteria is met. The evaluation results of the dialogue content may include, for each of the multiple evaluation criteria, the reasons why that evaluation criterion is met or not met.

[0111] The evaluation of dialogue content based on machine learning models (especially large-scale language models) has the following characteristics, for example: First, the evaluation results from machine learning models tend to closely match those from humans. On the other hand, evaluation results from machine learning models have limitations in evaluating factual accuracy and emotional content. In particular, the evaluation results from machine learning models do not always match the user's expectations regarding subjective matters and expertise in role-playing dialogues.

[0112] The functional configuration of the control device 10 shown in Figure 2 is just one example, and it goes without saying that there are various functional configurations depending on the application and purpose. The division of the storage unit or processing unit of the control device 10, such as the agent control unit 100, agent construction unit 101, message acquisition unit 102, message generation unit 103, message output unit 104, prompt creation support unit 110, dialogue data acquisition unit 111, dialogue data modification unit 112, prompt generation unit 113, prompt modification unit 114, agent verification support unit 120, debugger construction unit 121, dialogue execution unit 122, evaluation item generation unit 123, and evaluation execution unit 124 shown in Figure 2 is just one example.

[0113] <User Interface> The user interface provided by the dialogue system 1000 will be described with reference to Figures 3 to 15. The user interface of the dialogue system 1000 may, for example, be displayed on a display device provided by the control device 10 or the terminal device 30. For example, a user interface for creating or evaluating role setting prompts may be displayed on the display device of the control device 10. For example, a user interface for interacting with a dialogue agent AG may be displayed on the display device of the terminal device 30.

[0114] The user interface of the dialogue system 1000 may be provided as standard web content. The user interface of the dialogue system 1000 may also be displayed in a web browser pre-installed on the terminal device 30. For example, the user interface of the dialogue system 1000 may be implemented as a web page written in HTML (Hyper Text Markup Language). The web page may include applications such as JavaScript (registered trademark).

[0115] Figure 3 shows an example of a chat screen. The chat screen is an example of a screen for conducting a dialogue (text chat) between a conversational agent AG and a dialogueal agent S. The chat screen provides a user-friendly interface for real-time interaction with the dialogueal agent AG. As shown in Figure 3, the chat screen 600 has a message display area 601, a message input field 602, a send button 603, and an initialization button 604.

[0116] The message display area 601 is an example of an area that displays the content of the dialogue between the interlocutor S and the dialogue agent AG. The message display area 601 displays user messages m (m1~m3) from the interlocutor S and system messages r (r1~r2) from the dialogue agent AG, arranged chronologically in a predetermined direction (up and down in Figure 3). The user messages m from the interlocutor S display a re-execution button b (b1~b3). When the interlocutor S presses a re-execution button b, the dialogue can be restarted from that user message m.

[0117] The message input field 602 is an example of an input field that accepts user messages from the interlocutor S. After the interlocutor S enters a user message in the message input field 602, they press the send button 603, and the user message entered in the message input field 602 is accepted. The accepted user message is added to the end of the message display area 601 (the very bottom in Figure 3).

[0118] The initialization button 604 is used to initialize (reset) the dialogue between the interlocutor S and the dialogue agent AG. When the interlocutor S presses the initialization button 604, the dialogue content displayed in the message display area 601 is erased. After initializing the dialogue by pressing the initialization button 604, if the interlocutor S enters a user message, a new dialogue will start from the entered user message.

[0119] In the example shown in Figure 3, the dialogue begins with user message m1 entered by the interlocutor S. System message r1 is the response of the dialogue agent AG to user message m1. User message m2 is the response of the interlocutor S to system message r1. System message r2 is the response of the dialogue agent AG to user message m2. User message m3 is the response of the interlocutor S to system message r2. System message r3 is the response of the dialogue agent AG to user message m3.

[0120] Figure 4 shows an example of a settings screen. The settings screen is an example of a screen for setting the essential elements of the conversational agent AG. As shown in Figure 4, the settings screen 610 has a model selection field 611, a temperature input field 612, a user role input field 613, an agent role input field 614, a background information input field 615, a profile input field 616, a scenario input field 617, a phase input field 618, and a prompt display area 619.

[0121] The Model Selection field 611 is an example of a selection field that accepts the selection of a machine learning model M to be used to construct the conversational agent AG. The Model Selection field 611 displays the options for machine learning models M that can be used to construct the conversational agent AG. The Temperature Input field 612 is an example of an input field that accepts the input of a temperature parameter. The User Role Input field 613 is an example of an input field that accepts the input of the role of the conversational participant S. The Agent Role Input field 614 is an example of an input field that accepts the input of the role of the conversational agent AG.

[0122] The background information input field 615 is an example of an input field that accepts input of background information, including basic instructions. The background information input field 615 may also accept input of background information, including basic instructions and additional information. The profile input field 616 is an example of an input field that accepts input of profile information. The scenario input field 617 is an example of an input field that accepts input of scenario information. The phase input field 618 is an example of an input field that accepts input of phase information.

[0123] The prompt display area 619 is an example of an area that displays the role setting prompt for the conversational agent AG. The role setting prompt for the conversational agent AG is generated based on the information entered in the settings screen 610. User U or conversational participant S can check the role setting prompt for the conversational agent AG by referring to the prompt display area 619.

[0124] The settings screen 610 may only have the scenario input field 617 filled in. The dialogue system 1000 can operate with only the scenario input field 617 filled in. The settings screen 610 is also suitable for dialogue participant S or user U who want to use the dialogue system 1000 with minimal customization.

[0125] Figure 5 shows a first example of the dialogue example generation screen. The dialogue example generation screen is an example of a screen used to perform operations to generate a desired dialogue example. As shown in Figure 5, the dialogue example generation screen 620A has a model selection field 621, a temperature input field 622, a user role input field 623, an agent role input field 624, a dialogue location input field 625, a start point input field 626, an end point input field 627, a condition input field 628, a start creation button 629, and a profile generation button 630.

[0126] The model selection field 621, temperature input field 622, user role input field 623, and agent role input field 624 are the same as the model selection field 611, temperature input field 612, user role input field 613, and agent role input field 614 on the settings screen 610.

[0127] The dialogue location input field 625 is an example of an input field that accepts input for the location (stage) where the dialogue between the dialogue participant S and the dialogue agent AG takes place. The start point input field 626 is an example of an input field that accepts input for the scene where the dialogue between the dialogue participant S and the dialogue agent AG begins. The end point input field 627 is an example of an input field that accepts input for the scene where the dialogue between the dialogue participant S and the dialogue agent AG ends. The condition input field 628 is an example of an input field that accepts input for conditions related to the dialogue example. The condition input field 628 is an input field where you describe what kind of dialogue you want the dialogue agent AG to have. As illustrated in Figure 5, "The customer is taking paid leave in winter and is planning a trip within Japan. They will ask the store clerk several times about the available options to gather information," the dialogue example will be created according to the conditions described here. You can write the conditions of the dialogue, a description of the situation, the motivations of the conversation on the user's and agent's sides, what you want to ask the user, the flow of the dialogue, or the roles of the user and agent.

[0128] The "Start Creation" button 629 is an example of a button used to begin creating a dialogue example. When user U presses the "Start Creation" button 629, the generation of the dialogue example begins based on the information entered on the dialogue example generation screen 620A. The "Generate Profile" button 630 is an example of a button used to proceed to generating profile information to be included in the role setting prompt without generating a dialogue example.

[0129] Figure 6 shows an example of a dialogue generation prompt. As shown in Figure 6, the dialogue generation prompt 700 includes placeholders to embed information entered into the dialogue example generation screen 620A. {user} is a placeholder to embed the role of the dialogue participant S. {assistant} is a placeholder to embed the role of the dialogue agent AG. {story_start} is a placeholder to embed the starting point of the dialogue. {story_end} is a placeholder to embed the ending point of the dialogue. {utter_min} and {utter_max} are placeholders to embed the lower and upper limits of the number of utterances, respectively. {phase_min} and {phase_max} are placeholders to embed the lower and upper limits of the number of phases, respectively. {dialog_generate_condition} is a placeholder to embed the dialogue generation conditions.

[0130] Figure 6 shows an example where a placeholder is represented by a string enclosed in curly braces. A placeholder only needs to contain information that can be identified as corresponding to an item. Placeholders may be written in a prescribed format. Placeholders may also be information that identifies an item and is accompanied by a prescribed symbol or graphic. The information that identifies an item may be the item name. The information that identifies an item may also include descriptive information about the item. The prescribed symbol may, for example, be a bracket symbol. That is, a placeholder may be a string enclosed in a prescribed bracket symbol. The bracket symbol may be any symbol, including, for example, square brackets, round brackets, quotation marks, black brackets, curly braces, etc. Placeholders may also contain initial values ​​for the information corresponding to the item.

[0131] Figure 7 shows a second example of the dialogue example generation screen. Figure 7 also shows an example of the dialogue example generation screen for modifying the generated dialogue example. As shown in Figure 7, the dialogue example generation screen 620B has a dialogue example display area 631, a condition specification button 632, a modification input button 633, a modification execution button 634, and a profile generation button 635.

[0132] The dialogue example display area 631 is an example of an area that displays a dialogue example generated based on the information entered in the dialogue example generation screen 620A. The dialogue example display area 631 may display only a part of the dialogue example. The dialogue example display area 631 may display the dialogue example for each phase. The dialogue example display area 631 may display all the dialogue examples separated by phase.

[0133] The condition specification button 632 is a button to return to the dialogue example generation screen 620A. When user U presses the condition specification button 632, the generated dialogue example is discarded and the user transitions to the dialogue example generation screen 620A. User U can select the condition specification button 632 if they want to restart the generated dialogue example from condition input.

[0134] The edit input button 633 is a button for entering comments (correction instructions) that indicate corrections to the generated example dialogue. When user U presses the edit input button 633, the example dialogue generation screen 620B transitions to a state where comments can be entered.

[0135] The edit execution button 634 is used to modify the example dialogue based on the entered comment. After user U enters a comment, pressing the edit execution button 634 will modify the example dialogue displayed in the example dialogue display area 631 based on the comment.

[0136] The profile generation button 635 is a button that proceeds to the generation of profile information to be included in the role setting prompt. When user U presses the profile generation button 635, the generated example dialogue is confirmed as the desired example dialogue, and the user is redirected to a screen for generating profile information.

[0137] Figure 8 shows a third example of the dialogue example generation screen. Figure 8 shows an example of the dialogue example generation screen in a state where comments instructing corrections to the dialogue example can be entered. As shown in Figure 8, the dialogue example generation screen 620C displays a comment input area 642 when a predetermined operation is performed on the utterance content 641 to be corrected in the dialogue example display area 631. The predetermined operation may include selecting the utterance content 641, right-clicking with the utterance content 641 selected, or selecting a predetermined menu from the context menu displayed when right-clicking with the utterance content 641 selected.

[0138] The comment input area 642 includes a comment input field 643, an OK button 644, and a cancel button 645. When user U enters a comment in the comment input field 643 and then presses the OK button 644, the comment is added to the utterance content 641 to be modified. On the other hand, when user U presses the cancel button 645, the comment entered in the comment input field 643 is discarded, and the comment input area 642 closes. The dialogue example display area 631 also returns to an unselected state.

[0139] Figure 9 shows a first example of a profile generation screen. The profile generation screen is an example of a screen used to perform operations for generating profile information. As shown in Figure 9, the profile generation screen 650A has a model selection field 651, a temperature input field 652, a user role input field 653, an agent role input field 654, a candidate number input field 655, a dialogue location input field 656, a condition input field 657, an additional condition input field 658, a start creation button 659, a dialogue example usage selection field 660, and a dialogue example generation button 661.

[0140] The model selection field 651, temperature input field 652, user role input field 653, and agent role input field 654 are the same as the model selection field 611, temperature input field 612, user role input field 613, and agent role input field 614 on the settings screen 610. The condition input field 657 is the same as the condition input field 628 on the dialogue example generation screen 620A.

[0141] The candidate number input field 655 is an example of an input field that accepts input for the number of profile candidates to generate. The additional conditions input field 658 is an example of an input field that accepts input for additional conditions to be included in the background information.

[0142] The "Start Creation" button 659 is an example of a button used to begin creating profile candidates. When user U presses the "Start Creation" button 659, the generation of profile candidates begins based on the information entered on the profile generation screen 650. The "Use Dialogue Example" selection field 660 accepts the user's choice of whether or not to use a dialogue example when generating profile candidates. If the "Use Dialogue Example" selection field 660 is selected, the profile candidates will be generated using the dialogue example generated on the dialogue example generation screen 620A, or the dialogue example modified on the dialogue example generation screen 620B. On the other hand, if the "Use Dialogue Example" selection field 660 is not selected, the profile candidates will be generated without using a dialogue example.

[0143] The dialogue example generation button 661 is an example of a button used to return to the dialogue example generation screen 620A. When user U presses the dialogue example generation button 661, the system transitions to the dialogue example generation screen 620A without generating profile candidates. User U can select the dialogue example generation button 661 if they wish to regenerate the dialogue example and generate profile candidates.

[0144] Figure 10 shows an example of a profile generation prompt. As shown in Figure 10, the profile generation prompt 710 includes placeholders to embed the information entered in the profile generation screen 650A. {user} is a placeholder to embed the role of the interlocutor S. {assistant} is a placeholder to embed the role of the conversational agent AG. {story_setting} is a placeholder to embed the basic instructions or the setting of the conversation. {num_candidates} is a placeholder to embed the number of profile candidates to generate. {persona_generate_condition} is a placeholder to embed the conditions for generating profile candidates. {desirable_dialogue_example} is a placeholder to embed a desirable example conversation.

[0145] Figure 11 shows a second example of the profile generation screen. Figure 11 shows an example of a profile generation screen for selecting a generated profile candidate. As shown in Figure 11, the profile generation screen 650B has a background information display field 671, one or more profile display fields 672 (672-1, 672-2), one or more profile selection fields 673 (673-1, 673-2), a dialogue example generation button 674, a prompt generation button 675, and a generation condition setting button 676.

[0146] The background information display area 671 is an example of a display area that shows the background information used to generate the profile candidates. The background information display area 671 may also display the information entered in the condition input area 657 and the additional condition input area 658 of the profile generation screen 650A.

[0147] The profile display area 672 is an example of a display area that shows profile candidates generated based on the information entered in the profile generation screen 650A. A profile selection area 673 is displayed near each profile display area 672. The profile selection area 673 accepts the selection of a profile candidate. When user U selects one of the profile selection areas 673, the selected profile candidate is determined to be the profile information to be included in the role setting prompt. The selection state of multiple profile selection areas 673 is controlled exclusively. User U can select one profile candidate from multiple profile candidates using the profile selection area 673.

[0148] The dialogue example generation button 674 is an example of a button used to return to the dialogue example generation screen 620A. When user U presses the dialogue example generation button 674, the user transitions to the dialogue example generation screen 620A, which has the profile candidate selected in the profile selection field 673 set. On the dialogue example generation screen 620A, the dialogue example is generated based on that profile candidate.

[0149] The prompt generation button 675 is an example of a button used to generate a role setting prompt. When user U presses the prompt generation button 675, scenario information and phase information are generated based on the profile candidate selected in the profile selection field 673.

[0150] The generation condition setting button 676 is an example of a button used to return to the profile generation screen 650A. When user U presses the generation condition setting button 676, the generated profile candidates are discarded and the user transitions to the profile generation screen 650A. User U can select the generation condition setting button 676 if they want to change the generation conditions and regenerate the profile candidates.

[0151] Figure 12 shows an example of a prompt display screen. The prompt display screen is an example of a screen for confirming the role setting prompt. As shown in Figure 12, the prompt display screen 680 has a background information display area 681, a profile display area 682, a scenario display area 683, a phase display area 684, an apply button 685, an edit dialogue example button 686, and an initial screen button 687.

[0152] Background information display area 681 is an example of a display area that displays the background information included in the role setting prompt in an editable format. Profile display area 682 is an example of a display area that displays the profile information included in the role setting prompt in an editable format. Scenario display area 683 is an example of a display area that displays the scenario information included in the role setting prompt in an editable format. Phase display area 684 is an example of a display area that displays the phase information included in the role setting prompt in an editable format.

[0153] The Reflect button 685 is an example of a button used to confirm a role setting prompt. When user U presses the Reflect button 685, the role setting prompt, including the information displayed on the prompt display screen 680, is confirmed. The prompt generation unit 113 of the control device 10 stores the confirmed role setting prompt in its storage device.

[0154] The dialogue example editing button 686 is an example of a button used to return to the dialogue example generation screen 620B. When user U presses the dialogue example editing button 686, the generated role setting prompt is discarded, and the user transitions to the dialogue example generation screen 620B. User U can select the dialogue example editing button 686 if they want to edit the generated dialogue example and start the profile generation process again.

[0155] The initial screen button 687 is an example of a button used to return to the dialogue example generation screen 620A. When user U presses the initial screen button 687, the generated role setting prompt and generated dialogue example are discarded, and the user transitions to the dialogue example generation screen 620A. User U can select the initial screen button 687 if they want to restart the generated dialogue example from the condition input stage.

[0156] Figure 13 shows an example of a debug screen. The debug screen is an example of a screen used to debug the interactive agent AG. As shown in Figure 13, the debug screen 800 has a model selection field 801, a temperature input field 802, a debugger role input field 803, an agent role input field 804, a prompt input field 805, a message display area 806, a message generation button 807, a message input field 808, a send button 809, and an initialization button 810.

[0157] Model selection field 801 is an example of a selection field that accepts the selection of a machine learning model M to be used to build the conversational agent AG and the debug agent. Temperature input field 802 is an example of an input field that accepts the input of temperature parameters. Debugger role input field 803 is an example of an input field that accepts the input of the role of the debug agent. Agent role input field 804 is an example of an input field that accepts the input of the role of the conversational agent AG.

[0158] The prompt input field 805 is an example of an input field that accepts input for the debug agent role setting prompt. The prompt input field 805 may automatically display the debug agent role setting prompt generated by the prompt generation unit 113.

[0159] The message display area 806 is an example of an area that displays the content of the dialogue between the debug agent and the dialogue agent AG. The message display area 806 is the same as the message display area 601 of the chat screen 600, except that it displays a message from the debug agent instead of a user message m from the dialoguer S.

[0160] The message generation button 807 is used to automatically generate messages for the debug agent. When user U presses the message generation button 807, a message for the debug agent is automatically generated based on the machine learning model M configured with the debug agent role setting prompt. The generated message is added to the end of the message display area 806 (the bottom in Figure 13) as a debug agent message.

[0161] The message input field 808 is an example of an input field that accepts messages from user U. After user U enters a message in the message input field 808 and presses the send button 809, the message entered in the message input field 808 is accepted as a message by the debug agent. The accepted message is added to the end of the message display area 806 (the very bottom in Figure 13) as a message by the debug agent.

[0162] The initialization button 810 is an example of a button used to initialize (reset) the dialogue between the debug agent and the dialogue agent AG. When user U presses the initialization button 810, the dialogue content displayed in the message display area 806 is cleared. After initializing the dialogue by pressing the initialization button 810, if user U inputs a message to the debug agent, a new dialogue will start from the input message. After initializing the dialogue, user U may also press the message generation button 807. When user U presses the message generation button 807, a new dialogue will start from an automatically generated message from the debug agent.

[0163] User U can use the debug screen 800 to evaluate the responses of the conversational agent AG and modify the responses based on the flow of the conversation. This allows the conversational system 1000 to efficiently debug the conversational agent. The debug screen 800 allows adjustment of parameters such as the machine learning model M, temperature parameters, and the role of the conversational agent AG, so the debugging process can be flexibly applied to various scenarios.

[0164] Figure 14 shows a first example of an evaluation screen. The evaluation screen is an example of a screen used to perform operations for evaluating the content of the dialogue between the debug agent and the dialogue agent AG. As shown in Figure 14, the evaluation screen 820A has an evaluation item display area 821, an item add button 822, an item delete button 823, an execute button 824, and a result delete button 825.

[0165] The evaluation item display area 821 is an example of an area that displays evaluation items for evaluating the content of the dialogue. The evaluation item display area 821 displays one or more evaluation items 826. One or more evaluation items 826 are extracted from the dialogue example used to generate the role setting prompt. In Figure 14, for visibility purposes, only representative evaluation items 826 are labeled.

[0166] The item addition button 822 is an example of a button used to add an evaluation item 826 to the evaluation item display area 821. When user U presses the item addition button 822, an input field for the evaluation item is added to the evaluation item display area 821. When user U enters information into the input field, a new evaluation item 826 is added.

[0167] The item delete button 823 is an example of a button used to delete an evaluation item from the evaluation item display area 821. When user U presses the item delete button 823, each of the evaluation items 826 in the evaluation item display area 821 becomes selectable. When user U selects an evaluation item 826 to delete, the selected evaluation item 826 is deleted from the evaluation item display area 821. The item delete button 823 may also delete items without selecting any input fields that are not yet filled in.

[0168] The execute button 824 is an example of a button used to perform an evaluation of the dialogue content. When user U presses the execute button 824, it is determined whether the dialogue content between the debug agent and the dialogue agent AG satisfies each of the evaluation items 826 in the evaluation item display area 821. After the evaluation of the dialogue content is complete, the evaluation item display area 821 displays the result of the evaluation for each of the evaluation items 826. The result may include whether the dialogue content satisfies the evaluation item (e.g., True) or whether the dialogue content does not satisfy the evaluation item (e.g., False). The result may also include the reason why the dialogue content satisfies the evaluation item (the basis for the determination that it satisfies) or the reason why the dialogue content does not satisfy the evaluation item (the basis for the determination that it does not satisfy). The result may also include a number representing the degree to which the evaluation item is satisfied (e.g., a real number from 0 to 1, where a number closer to 1 means it satisfies, and a number closer to 0 means it does not satisfy).

[0169] The result delete button 825 is an example of a button used to delete the evaluation results of the dialogue content. The evaluation results of the dialogue content include the judgment results for each of the evaluation items 826 included in the evaluation item display area 821. When user U presses the result delete button 825, the judgment results for each of the evaluation items 826 displayed in the evaluation item display area 821 are deleted.

[0170] Figure 15 shows a second example of the evaluation screen. Figure 15 shows an example of an evaluation screen in which the evaluation results of the dialogue content are displayed. As shown in Figure 15, the evaluation screen 820B includes an evaluation item display area 821 that displays one or more evaluation items 826. The evaluation item display area 821 displays a selectable judgment result 831 corresponding to each evaluation item 826. When a judgment result 831 is selected, the basis 832 for the selected judgment result 831 is displayed. The evaluation item display area 821 may also automatically select a judgment result 831 corresponding to an evaluation item 826 that is determined not to be met.

[0171] <Prompt creation support method flow> The prompt creation support method executed by the dialogue system 1000 will be explained with reference to Figure 16. Figure 16 is a processing flow showing an example of the prompt creation support method. The prompt creation support method is an example of an information processing method that assists in creating role setting prompts for constructing a dialogue agent AG.

[0172] In step S101, user U displays the dialogue example generation screen 620A (see Figure 5). The dialogue example generation screen 620A may be displayed on the display device of the control device 10, or on the display device of a terminal device connected to the control device 10 via a communication network. User U inputs dialogue conditions into the dialogue example generation screen 620A. The dialogue data acquisition unit 111 of the control device 10 receives the dialogue conditions entered into the dialogue example generation screen 620A.

[0173] In step S102, the dialogue data acquisition unit 111 of the control device 10 generates a dialogue generation prompt based on the dialogue conditions received in step S101. The dialogue data acquisition unit 111 transmits the generated dialogue generation prompt to the generation device 20. The generation device 20 receives the dialogue generation prompt from the control device 10. The generation device 20 inputs the dialogue generation prompt to the machine learning model M.

[0174] The machine learning model M performs a task to generate dialogue data based on the input dialogue generation prompt. The machine learning model M outputs the dialogue data obtained as a result of executing the task. The dialogue data acquisition unit 111 acquires the dialogue data output from the machine learning model M. The dialogue data acquisition unit 111 displays the dialogue data generated by the machine learning model M on the dialogue example generation screen 620B (see Figure 7).

[0175] In step S103, the dialogue data acquisition unit 111 of the control device 10 determines whether or not to modify the dialogue data. For example, if the modification execution button 634 is pressed on the dialogue example generation screen 620B, the dialogue data acquisition unit 111 may determine that the dialogue data should be modified. For example, if the profile generation button 635 is pressed on the dialogue example generation screen 620B without the modification execution button 634 being pressed, the dialogue data acquisition unit 111 may determine that the dialogue data should not be modified.

[0176] If it is determined that the dialogue data should be modified (YES), the dialogue data acquisition unit 111 sends the dialogue data to the dialogue data modification unit 112 and proceeds to step S104. On the other hand, if it is determined that the dialogue data should not be modified (NO), the dialogue data acquisition unit 111 sends the dialogue data to the prompt generation unit 113 and proceeds to step S105.

[0177] In step S104, the dialogue data modification unit 112 of the control device 10 receives dialogue data from the dialogue data acquisition unit 111. The dialogue data modification unit 112 modifies the dialogue data. The dialogue data modification unit 112 may also modify the dialogue data based on comments entered in the dialogue example generation screen 620C. The dialogue data modification unit 112 displays the modified dialogue data on the dialogue example generation screen 620B. The dialogue data modification unit 112 returns to step S103 and determines again whether or not to modify the dialogue data.

[0178] The control device 10 may choose whether or not to accept modifications to the dialogue data. The control device 10 may use the dialogue data generated in step S102, or dialogue data generated by any other method, without modification. The control device 10 may also accept input of dialogue data that has been modified by an external information processing device or information processing system. If the control device 10 does not accept modifications to the dialogue data, steps S103 and S104 may be omitted.

[0179] In step S105, the prompt generation unit 113 of the control device 10 receives dialogue data from the dialogue data acquisition unit 111. Based on the dialogue data, the prompt generation unit 113 generates a profile generation prompt. The prompt generation unit 113 transmits the generated profile generation prompt to the generation device 20. The generation device 20 receives the profile generation prompt from the control device 10. The generation device 20 inputs the profile generation prompt to the machine learning model M.

[0180] The machine learning model M performs a task to generate profile candidates based on the input profile generation prompt. The machine learning model M outputs the profile candidates obtained as a result of executing the task. The prompt generation unit 113 acquires the profile candidates output from the machine learning model M. The prompt generation unit 113 displays the profile candidates generated by the machine learning model M on the profile generation screen 650B (see Figure 11).

[0181] In step S106, the prompt generation unit 113 of the control device 10 accepts the selection of a profile candidate. For example, when any of the profile selection fields 673 are selected on the profile generation screen 650B, the prompt generation unit 113 determines that the selected profile candidate will be included as profile information in the role setting prompt. The prompt generation unit 113 displays the determined profile information on the prompt display screen 680 (see Figure 12).

[0182] In step S107, the prompt generation unit 113 of the control device 10 generates a scenario generation prompt based on the dialogue data received in step S105. The prompt generation unit 113 transmits the generated scenario generation prompt to the generation device 20. The generation device 20 receives the scenario generation prompt from the control device 10. The generation device 20 inputs the scenario generation prompt to the machine learning model M.

[0183] The machine learning model M performs a task to generate scenario information based on the input scenario generation prompt. The machine learning model M outputs the scenario information obtained as a result of executing the task. The prompt generation unit 113 acquires the scenario information output from the machine learning model M. The prompt generation unit 113 displays the acquired scenario information on the prompt display screen 680 (see Figure 12).

[0184] In step S108, the prompt generation unit 113 of the control device 10 generates a phase generation prompt based on the dialogue data received in step S105. The prompt generation unit 113 transmits the generated phase generation prompt to the generation device 20. The generation device 20 receives the phase generation prompt from the control device 10. The generation device 20 inputs the phase generation prompt to the machine learning model M.

[0185] The machine learning model M performs a task to generate phase information according to the input phase generation prompt. The machine learning model M outputs the phase information obtained as a result of executing the task. The prompt generation unit 113 acquires the phase information output from the machine learning model M. The prompt generation unit 113 displays the acquired phase information on the prompt display screen 680 (see Figure 12).

[0186] In step S109, the prompt generation unit 113 of the control device 10 determines whether or not to modify the role setting prompt. For example, if at least one of the background information display field 681, profile display field 682, scenario display field 683, or phase display field 684 is edited on the prompt display screen 680, the prompt generation unit 113 may determine to modify the role setting prompt. For example, if none of the background information display field 681, profile display field 682, scenario display field 683, and phase display field 684 are edited on the prompt display screen 680, and the reflect button 685 is pressed, the prompt generation unit 113 may determine not to modify the role setting prompt.

[0187] If it is determined that the role setting prompt should be modified (YES), the prompt generation unit 113 sends the role setting prompt to the prompt modification unit 114 and proceeds to step S110. On the other hand, if it is determined that the role setting prompt should not be modified (NO), the prompt generation unit 113 proceeds to step S111.

[0188] In step S110, the prompt correction unit 114 of the control device 10 receives a role setting prompt from the prompt generation unit 113. The prompt correction unit 114 corrects the role setting prompt. For example, the prompt correction unit 114 may correct the role setting prompt based on the edits made on the prompt display screen 680. The prompt correction unit 114 displays the corrected role setting prompt on the prompt display screen 680. The prompt correction unit 114 returns to step S109 and determines again whether or not to correct the role setting prompt.

[0189] The control device 10 may choose whether or not to accept modifications to the role setting prompt. The control device 10 may use the role setting prompt generated in step S108 without making any modifications. The control device 10 may also accept input of a role setting prompt that has been modified by an external information processing device or information processing system. If the control device 10 does not accept modifications to the role setting prompt, steps S109 and S110 may be omitted.

[0190] In step S111, the prompt generation unit 113 of the control device 10 confirms the role setting prompt. For example, if the reflect button 685 is pressed on the prompt display screen 680, the prompt generation unit 113 may confirm the role setting prompt displayed on the prompt display screen 680. The prompt generation unit 113 stores the confirmed role setting prompt in the storage device of the control device 10.

[0191] <Flowchart of Agent Verification Support Method> The agent verification support method executed by the dialogue system 1000 will be explained with reference to Figure 17. Figure 17 is a processing flow showing an example of the agent verification support method. The agent verification support method is an example of an information processing method that supports the verification of the dialogue agent AG. The dialogue agent AG may be constructed based on a role setting prompt created by the prompt creation support method, or it may be constructed based on a role setting prompt created independently of the prompt creation support method.

[0192] In step S201, user U displays the debug screen 800 (see Figure 13). The debug screen 800 may be displayed on the display device of the control device 10, or on the display device of a terminal device connected to the control device 10 via a communication network. The debugger construction unit 121 of the control device 10 reads the role setting prompt from the storage device of the control device 10. The debugger construction unit 121 displays the information contained in the role setting prompt on the debug screen 800.

[0193] In step S202, the debugger construction unit 121 of the control device 10 constructs the conversational agent AG based on the role setting prompt read in step S201. The debugger construction unit 121 sends the role setting prompt for the conversational agent AG to the generation device 20. The generation device 20 receives the role setting prompt for the conversational agent AG from the control device 10. The generation device 20 inputs the role setting prompt for the conversational agent AG to the machine learning model M. The machine learning model M constructs the conversational agent AG based on the input role setting prompt for the conversational agent AG.

[0194] In step S203, the debugger construction unit 121 of the control device 10 constructs a debug agent based on the role setting prompt read in step S201. The debugger construction unit 121 sends the debug agent's role setting prompt to the generation device 20. The generation device 20 receives the debug agent's role setting prompt from the control device 10. The generation device 20 inputs the debug agent's role setting prompt to the machine learning model M. The machine learning model M constructs a debug agent based on the input debug agent's role setting prompt.

[0195] In step S204, user U performs an operation to start debugging on the debug screen 800. For example, user U may press the message generation button 807 on the debug screen 800. For example, user U may enter a message in the message input field 808 on the debug screen 800 and then press the send button 809.

[0196] The dialogue execution unit 122 of the control device 10 performs dialogue between the dialogue agent AG and the debug agent in response to an operation to start debugging. For example, the dialogue execution unit 122 generates a message for the debug agent based on the debug agent and displays it in the message display area 806 of the debug screen 800. The dialogue execution unit 122 also generates a message for the dialogue agent AG in response to the debug agent's message based on the dialogue agent AG and displays it in the message display area 806 of the debug screen 800.

[0197] The dialogue execution unit 122 executes the dialogue between the dialogue agent AG and the debug agent by repeatedly generating messages from the dialogue agent AG and messages from the debug agent. When the dialogue execution unit 122 determines that the dialogue between the dialogue agent AG and the debug agent has ended, it sends dialogue data indicating the content of the dialogue between the dialogue agent AG and the debug agent to the evaluation execution unit 124.

[0198] In step S205, the evaluation item generation unit 123 of the control device 10 acquires the dialogue data used to generate the role setting prompt acquired in step S201. The dialogue data used to generate the role setting prompt may be stored in the storage device of the control device 10, for example, in association with the role setting prompt. Based on the dialogue data, the evaluation item generation unit 123 generates an evaluation item generation prompt. The evaluation item generation unit 123 transmits the generated evaluation item generation prompt to the generation device 20. The generation device 20 receives the evaluation item generation prompt from the control device 10. The generation device 20 inputs the evaluation item generation prompt to the machine learning model M.

[0199] The machine learning model M executes a task to generate evaluation items based on the input evaluation item generation prompt. The machine learning model M outputs the evaluation items obtained as a result of executing the task. The evaluation item generation unit 123 acquires the evaluation items output from the machine learning model M. The evaluation item generation unit 123 displays the evaluation items generated by the machine learning model M on the evaluation screen 820A (see Figure 14).

[0200] In step S206, the evaluation item generation unit 123 of the control device 10 determines whether or not to modify the evaluation items. For example, if an evaluation item 826 is added, edited, or deleted on the evaluation screen 820A, the evaluation item generation unit 123 may determine that the evaluation items should be modified. For example, if the execute button 824 is pressed on the evaluation screen 820A without any addition, editing, or deletion of the evaluation items 826, the evaluation item generation unit 123 may determine that the evaluation items should not be modified.

[0201] If it is determined that the evaluation items should be modified (YES), the evaluation item generation unit 123 proceeds to step S207. On the other hand, if it is determined that the evaluation items should not be modified (NO), the evaluation item generation unit 123 sends the evaluation items to the evaluation execution unit 124 and proceeds to step S208.

[0202] In step S207, the evaluation item generation unit 123 of the control device 10 modifies the evaluation items. For example, the evaluation item generation unit 123 may modify the evaluation items based on the edits made on the evaluation screen 820A. The evaluation item generation unit 123 displays the modified evaluation items on the evaluation screen 820A. The evaluation item generation unit 123 returns to step S206 and determines again whether or not to modify the evaluation items.

[0203] In step S208, the evaluation execution unit 124 of the control device 10 receives evaluation items from the evaluation item generation unit 123. The evaluation execution unit 124 also receives dialogue data from the dialogue execution unit 122 that shows the content of the dialogue between the dialogue agent AG and the debug agent. Based on the evaluation items and dialogue data, the evaluation execution unit 124 generates an evaluation result generation prompt. The evaluation execution unit 124 transmits the generated evaluation result generation prompt to the generation device 20. The generation device 20 receives the evaluation result generation prompt from the control device 10. The generation device 20 inputs the evaluation result generation prompt to the machine learning model M.

[0204] The machine learning model M performs a task to evaluate the dialogue content based on the input evaluation result generation prompt. The machine learning model M outputs the evaluation result obtained as a result of executing the task. The evaluation execution unit 124 acquires the evaluation result output from the machine learning model M. The evaluation execution unit 124 displays the evaluation result generated by the machine learning model M on the evaluation screen 820B (see Figure 15).

[0205] <Effects of the dialogue system> A user survey was conducted to confirm the effectiveness of the dialogue system 1000. The user survey revealed the following significant effects:

[0206] Firstly, the dialogue system 1000 can significantly reduce workload. Subjective workload when using the user interface of the dialogue system 1000 was measured using NASA-TLX (National Aeronautics and Space Administration Task Load Index) and compared with the workload when using the user interface of the baseline system. The user interface of the dialogue system 1000 showed a statistically significant reduction in subjective workload compared to the baseline user interface (p<0.05). In particular, significant improvements were observed in five dimensions: "mental demands," "physical demands," "time demands," "effort," and "dissatisfaction." Specifically, "mental demands" were 76.25 for the baseline and 39.38 for the dialogue system 1000. "Physical demands" were 50.63 for the baseline and 25.00 for the dialogue system 1000. "Time demands" were 78.13 for the baseline and 38.75 for the dialogue system 1000. For "effort," the baseline score was 78.13, compared to 42.50 for Dialogue System 1000. For "dissatisfaction," the baseline score was 69.38, compared to 39.38 for Dialogue System 1000.

[0207] Secondly, the dialogue system 1000 can reduce cognitive load. In the dialogue system 1000, the role-playing agent and the debugging agent receive role-setting prompts through the generation of dialogue examples. This clarifies the dialogue between the role-playing agent and the debugging agent, allowing the user to focus on the overall flow of the dialogue and the control of the agents, thereby reducing cognitive load.

[0208] Thirdly, the dialogue system 1000 improves control over the dialogue agent. For example, some users who are good at creating prompts can effectively control the machine learning model even in the baseline system. However, many users preferred the user interface of the dialogue system 1000 over the baseline system. Objective evaluations showed that the user interface of the dialogue system 1000 was useful in facilitating specific conversations, including personal profiles. This suggests that scenarios or phases generated through dialogue examples are more effective in controlling user instructions than simple, manually created explanations.

[0209] Fourth, the dialogue system 1000 improves efficiency and reduces cognitive load. Many users highlighted the difficulty of continuous interaction with role-playing agents and were attracted to the ability to quickly generate debugging-related utterances through the dialogue system 1000's user interface. This reduced the burden on users, allowing them to focus on controlling the overall flow of the conversation rather than manually adjusting specific item settings or role setting prompts.

[0210] Fifth, the dialogue system 1000 provides expert-level quality. The dialogue system 1000 demonstrated that it can generate role-playing agents equivalent to those created with role-setting prompts created by experts.

[0211] Sixth, the Dialogue System 1000 achieves high user satisfaction. In a user survey, 6 out of 8 participants felt that the Dialogue System 1000's user interface helped them achieve task goals more effectively (average score 4.0). Also, 7 out of 8 participants felt that it improved the efficiency of creating role-setting prompts (average score 4.375). Furthermore, 7 out of 8 participants reported high satisfaction with the Dialogue System 1000's user interface (average score 4.5). And, 6 out of 8 participants reported that they were able to better imagine specific interactions when using the Dialogue System 1000's user interface (average score 4.125).

[0212] [Other embodiments] In each of the embodiments described above, a configuration was described in which the control device 10 performs all generation tasks using the machine learning model M possessed by the generation device 20. Here, there may be multiple machine learning models M that perform the generation tasks. Each of the multiple machine learning models may perform one or more generation tasks.

[0213] In each of the embodiments described above, the machine learning model used to generate the role setting prompt is the first example of the first machine learning model. The machine learning model used to generate the dialogue data is the second example of the first machine learning model. The machine learning model used to construct the dialogue agent AG is the third example of the first machine learning model. The machine learning model used for dialogue with the interlocutor S is the first example of the second machine learning model. The machine learning model used to construct the debug agent is the second example of the second machine learning model. The machine learning model used to generate the role setting prompt for the dialogue agent AG is an example of the third machine learning model. The machine learning model used to generate the role setting prompt for the debug agent is an example of the fourth machine learning model. The machine learning model used to generate the evaluation results of the dialogue content is an example of the fifth machine learning model. The machine learning model used to generate evaluation items for evaluating the dialogue content is an example of the sixth machine learning model.

[0214] The machine learning model used to generate role setting prompts (first example of the first machine learning model) may be the same as at least one of the machine learning models used to generate dialogue data, the machine learning model used to build dialogue agent AG, the machine learning model used for dialogue with dialoguer S, the machine learning model used to build debug agent, the machine learning model used to generate role setting prompts for dialogue agent AG, the machine learning model used to generate evaluation results of dialogue content, and the machine learning model used to generate evaluation items for evaluating dialogue content, or it may be a machine learning model that is different from any of these.

[0215] The machine learning model used to generate dialogue data (a second example of the first machine learning model) may be the same as at least one of the machine learning models used to generate role setting prompts, the machine learning model used to build dialogue agent AG, the machine learning model used for dialogue with dialoguer S, the machine learning model used to build debug agent, the machine learning model used to generate role setting prompts for dialogue agent AG, the machine learning model used to generate role setting prompts for debug agent, the machine learning model used to generate evaluation results of dialogue content, and the machine learning model used to generate evaluation items for evaluating dialogue content, or it may be a machine learning model that is different from any of these.

[0216] The machine learning model used to construct the conversational agent AG (the third example of the first machine learning model) may be the same as at least one of the machine learning models used to generate role setting prompts, machine learning models used to generate dialogue data, machine learning models used for dialogue with the interlocutor S, machine learning models used to construct the debug agent, machine learning models used to generate role setting prompts for the conversational agent AG, machine learning models used to generate role setting prompts for the debug agent, machine learning models used to generate evaluation results of the dialogue content, and machine learning models used to generate evaluation items for evaluating the dialogue content, or it may be a machine learning model that is different from any of these.

[0217] The machine learning model used for dialogue with the interlocutor S (first example of the second machine learning model) may be the same as at least one of the machine learning models used for generating role setting prompts, generating dialogue data, building dialogue agent AG, building debug agent, generating role setting prompts for dialogue agent AG, generating role setting prompts for debug agent, generating evaluation results for dialogue content, and generating evaluation items for evaluating dialogue content, or it may be a machine learning model that is different from any of these.

[0218] The machine learning model used to construct the debug agent (second example of the second machine learning model) may be the same as at least one of the machine learning models used to generate role setting prompts, machine learning models used to generate dialogue data, machine learning models used to construct the dialogue agent AG, machine learning models used for dialogue with the interlocutor S, machine learning models used to generate role setting prompts for the dialogue agent AG, machine learning models used to generate role setting prompts for the debug agent, machine learning models used to generate evaluation results of the dialogue content, and machine learning models used to generate evaluation items for evaluating the dialogue content, or it may be a machine learning model that is different from any of these.

[0219] The machine learning model used to generate the role setting prompt for the conversational agent AG (the third machine learning model) may be the same as at least one of the machine learning models used to generate the role setting prompt, the machine learning model used to generate conversational data, the machine learning model used to build the conversational agent AG, the machine learning model used for conversation with the conversational partner S, the machine learning model used to build the debug agent, the machine learning model used to generate the role setting prompt for the debug agent, the machine learning model used to generate the evaluation results of the conversation content, and the machine learning model used to generate evaluation items for evaluating the conversation content, or it may be a machine learning model that is different from any of these.

[0220] The machine learning model used to generate the role setting prompt for the debug agent (the fourth machine learning model) may be the same as at least one of the machine learning models used to generate the role setting prompt, the machine learning model used to generate dialogue data, the machine learning model used to build the dialogue agent AG, the machine learning model used for dialogue with the interlocutor S, the machine learning model used to build the debug agent, the machine learning model used to generate the role setting prompt for the dialogue agent AG, the machine learning model used to generate the evaluation results of the dialogue content, and the machine learning model used to generate evaluation items for evaluating the dialogue content, or it may be a machine learning model that is different from any of these.

[0221] The machine learning model used to generate the evaluation results of the dialogue content (the fifth machine learning model) may be the same as at least one of the machine learning models used to generate role setting prompts, the machine learning model used to generate dialogue data, the machine learning model used to construct dialogue agent AG, the machine learning model used for dialogue with dialoguer S, the machine learning model used to construct the debug agent, the machine learning model used to generate role setting prompts for dialogue agent AG, the machine learning model used to generate role setting prompts for the debug agent, and the machine learning model used to generate evaluation items for evaluating the dialogue content, or it may be a machine learning model that is different from any of these.

[0222] The machine learning model used to generate evaluation items for evaluating the content of the dialogue (the sixth machine learning model) may be the same as at least one of the machine learning models used to generate role setting prompts, the machine learning model used to generate dialogue data, the machine learning model used to construct the dialogue agent AG, the machine learning model used for dialogue with the dialogue partner S, the machine learning model used to construct the debug agent, the machine learning model used to generate the role setting prompts for the dialogue agent AG, the machine learning model used to generate the role setting prompts for the debug agent, or the machine learning model used to generate the evaluation results of the content of the dialogue, or it may be a machine learning model that is different from any of them.

[0223] Furthermore, the machine learning model used to generate the role setting prompt and the machine learning model used for dialogue with the interlocutor S may be the same machine learning model. By using the same machine learning model for generating the role setting prompt and controlling the dialogue agent AG, for example, it is expected that discrepancies between the behavior of the dialogue system 1000 during development and its behavior in production can be minimized.

[0224] In each of the embodiments described above, an example was explained in which the dialogue by the dialogue agent AG is performed by the control device 10 and the terminal device 30. The dialogue by the dialogue agent AG may be performed by an information processing device or information processing system different from the dialogue system 1000. The dialogue by the dialogue agent AG may be performed by at least a part of the dialogue system 1000. The dialogue agent AG may be built into an information processing device that performs dialogue using a machine learning model M. As an example, the dialogue agent AG may be built into at least a part of the dialogue system 1000, at least a part of another information processing system different from the dialogue system 1000, or another information processing device different from the dialogue system 1000.

[0225] <Summary> As is clear from the above description, the dialogue system 1000 according to one embodiment of the present disclosure acquires dialogue data indicating the content of the dialogue and inputs the dialogue data into a machine learning model M to generate a role setting prompt for the dialogue agent AG to execute the dialogue.

[0226] The dialogue system 1000 may accept input of dialogue conditions from the user and generate dialogue data by inputting the dialogue conditions into a machine learning model M.

[0227] The dialogue system 1000 may present dialogue data generated by the machine learning model M to the user U and may accept modifications to the dialogue data by the user U.

[0228] The dialogue system 1000 may accept input of dialogue data from user U.

[0229] The dialogue system 1000 may generate a role setting prompt that includes profile information of the dialogue agent AG.

[0230] The dialogue system 1000 may input dialogue data into a machine learning model M to generate multiple candidate profile information, and accept the user U's selection of profile information.

[0231] Profile information may include at least one of the following: attributes, basic information, personal information, personality, characteristics, or situational factors.

[0232] The dialogue system 1000 may generate a role setting prompt that includes dialogue scenario information to be executed by the dialogue agent AG.

[0233] The dialogue system 1000 may present scenario information generated by the machine learning model M to the user U and may accept modifications to the scenario information from the user U.

[0234] Scenario information may include at least one of the following: setting, context, premise, or plot outline.

[0235] The dialogue system 1000 may generate a role setting prompt that includes dialogue phase information to be executed by the dialogue agent AG.

[0236] The dialogue system 1000 may present the phase information generated by the machine learning model M to the user U and accept modifications to the phase information from the user U.

[0237] Phase information may include at least one of the following: phase transition information, flow, development, scene, or stage.

[0238] The dialogue system 1000 may perform a dialogue between the dialoguer S and the dialogue agent AG by inputting a role setting prompt into the machine learning model M.

[0239] The machine learning model used to generate the role setting prompt and the machine learning model used to perform the dialogue with the interlocutor S may be the same machine learning model.

[0240] The role setting prompt can also simply be a prompt.

[0241] An interaction system 1000 according to one embodiment of the present disclosure obtains a first role setting prompt for a dialogue agent AG to execute a dialogue, obtains a second role setting prompt for a debug agent to execute a dialogue, inputs the first role setting prompt to a machine learning model M, and inputs the second role setting prompt to the machine learning model M to execute a dialogue between the dialogue agent AG and the debug agent. The first role setting prompt includes at least one of scenario information or phase information as information necessary for dialogue control, and the second role setting prompt contains less information necessary for dialogue control than the first role setting prompt.

[0242] The second role setting prompt does not need to include at least some of the scenario information contained in the first role setting prompt.

[0243] The second role setting prompt does not need to include at least some of the phase information contained in the first role setting prompt.

[0244] The machine learning model used to generate the first role-setting prompt and the machine learning model used to generate the second role-setting prompt may be the same machine learning model.

[0245] The dialogue system 1000 may acquire dialogue data indicating the content of the dialogue between the first interlocutor and the second interlocutor, input the dialogue data into a machine learning model M to generate a first role setting prompt for executing the dialogue as the first interlocutor, and input the dialogue data into the machine learning model M to generate a second role setting prompt for executing the dialogue as the second interlocutor.

[0246] The dialogue system 1000 may generate an evaluation result of the dialogue content by inputting dialogue data, which shows the content of the dialogue between the dialogue agent AG and the debug agent, into a machine learning model M.

[0247] The dialogue system 1000 may input dialogue data into a machine learning model M to generate evaluation items for evaluating the content of the dialogue between the dialogue agent AG and the debugging agent.

[0248] The evaluation results may include information indicating whether or not the evaluation criteria are met.

[0249] The evaluation results may include at least one reason for meeting the evaluation criteria, or one reason for failing to meet the evaluation criteria.

[0250] The dialogue system 1000 may present evaluation items generated by the machine learning model M to the user U and may accept modifications to the evaluation items by the user U.

[0251] The evaluation items may include items for determining at least one of the following: dialogue consistency, dialogue coherence, role matching, degree of following role setting prompts, degree of adherence to role setting prompts, degree of suitability to role setting prompts, factual accuracy, dialogue fluency, or dialogue style.

[0252] As a result, according to one embodiment of this disclosure, information for controlling the conversational agent can be generated. In one aspect, according to this embodiment, the amount of information that the user inputs to generate a role setting prompt is reduced, so the effort required by the user to create a role setting prompt can be greatly reduced. In another aspect, according to this embodiment, the role setting prompt can be automatically debugged by having the conversational agents interact with each other, so the effort required by the user to debug the role setting prompt can be greatly reduced.

[0253] [Hardware configuration of information processing equipment] In the embodiments described above, some or all of the devices (control device 10, generation device 20, and terminal device 30) may be composed of hardware, or they may be composed of information processing by software (programs) executed by a CPU (Central Processing Unit), GPU (Graphics Processing Unit), etc. If they are composed of information processing by software, the software that realizes at least some of the functions of each device in the embodiments described above may be stored on a non-temporary storage medium (non-temporary computer-readable medium) such as a CD-ROM (Compact Disc-Read Only Memory) or USB (Universal Serial Bus) memory, and the information processing by the software may be executed by loading it into a computer. Alternatively, the software may be downloaded via a communication network. Furthermore, all or part of the processing by the software may be implemented in a circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array), so that the information processing by the software is executed by hardware.

[0254] The storage medium for the software may be a removable medium such as an optical disc, or a fixed storage medium such as a hard disk or memory. Furthermore, the storage medium may be located inside the computer (main memory, auxiliary storage, etc.) or outside the computer.

[0255] Figure 18 is a block diagram showing an example of the hardware configuration of each device (control device 10, generation device 20, and terminal device 30) in the embodiment described above. Each device may be implemented as a computer 7, for example, comprising a processor 71, main memory 72 (memory), auxiliary memory 73 (memory), network interface 74, and device interface 75, which are connected via a bus 76.

[0256] The computer 7 in Figure 18 has one of each component, but it may have multiple identical components. Also, although Figure 18 shows one computer 7, the software may be installed on multiple computers, and each of these computers may execute the same or different parts of the software's processing. In this case, it may be a distributed computing configuration in which each computer communicates via a network interface 74 or the like to execute processing. In other words, each device in the above-described embodiment (control device 10, generation device 20, and terminal device 30) may be configured as a system that realizes its function by having one or more computers execute instructions stored in one or more storage devices. Alternatively, it may be configured so that information transmitted from the terminal is processed by one or more computers located on the cloud, and the processing results are transmitted to the terminal.

[0257] The various calculations performed by each device (control device 10, generation device 20, and terminal device 30) in the embodiments described above may be executed in parallel using one or more processors, or using multiple computers via a network. Alternatively, the various calculations may be distributed to multiple processing cores within a processor and executed in parallel. Furthermore, some or all of the processing and means of this disclosure may be implemented by at least one of a processor and a storage device located on a cloud that can communicate with computer 7 via a network. Thus, each device in the embodiments described above may be in the form of parallel computing using one or more computers.

[0258] The processor 71 may be an electronic circuit (processing circuit, processing circuitry, CPU, GPU, FPGA, ASIC, etc.) that performs either control or calculations of a computer. The processor 71 may also be a general-purpose processor, a dedicated processing circuit designed to perform specific calculations, or a semiconductor device that includes both a general-purpose processor and a dedicated processing circuit. Furthermore, the processor 71 may include optical circuits or quantum computing-based calculation functions.

[0259] The processor 71 may perform calculations based on data and software input from various devices within the computer 7, and may output calculation results and control signals to these devices. The processor 71 may also control the various components of the computer 7 by executing the computer 7's OS (Operating System) or applications.

[0260] Each of the devices in the above-described embodiment (control device 10, generation device 20, and terminal device 30) may be implemented by one or more processors 71. Here, the processor 71 may refer to one or more electronic circuits arranged on one chip, or one or more electronic circuits arranged on two or more chips or two or more devices. When multiple electronic circuits are used, each electronic circuit may communicate by wire or wireless.

[0261] The main memory 72 may store instructions executed by the processor 71 and various data, and the information stored in the main memory 72 may be read by the processor 71. The auxiliary memory 73 is a memory device other than the main memory 72. These memory devices refer to any electronic component capable of storing electronic information, and may be semiconductor memory. The semiconductor memory may be either volatile memory or non-volatile memory. In the above-described embodiment, the memory devices for storing various data in each device (control device 10, generation device 20, and terminal device 30) may be implemented by the main memory 72 or the auxiliary memory 73, or by the built-in memory of the processor 71. For example, each storage unit in the above-described embodiment may be implemented by the main memory 72 or the auxiliary memory 73.

[0262] In the embodiments described above, if each device (control device 10, generation device 20, and terminal device 30) consists of at least one storage device (memory) and at least one processor connected to (coupled with) this at least one storage device, then at least one processor may be connected to one storage device. Also, at least one storage device may be connected to one processor. Furthermore, the configuration may include at least one processor among a plurality of processors being connected to at least one storage device among a plurality of storage devices. This configuration may also be realized by storage devices and processors included in a plurality of computers. Moreover, the configuration may include a storage device integrated with a processor (for example, a cache memory including an L1 cache and an L2 cache).

[0263] The network interface 74 is an interface for connecting to the communication network 8 wirelessly or via a wired connection. The network interface 74 can be any appropriate interface, such as one conforming to existing communication standards. Information may be exchanged between the computer 7 and an external device 9A connected via the communication network 8 through the network interface 74. The communication network 8 may be a WAN (Wide Area Network), LAN (Local Area Network), PAN (Personal Area Network), or a combination thereof, as long as information is exchanged between the computer 7 and the external device 9A. An example of a WAN is the Internet, an example of a LAN is IEEE 802.11 or Ethernet (registered trademark), and an example of a PAN is Bluetooth (registered trademark) or NFC (Near Field Communication).

[0264] The device interface 75 is an interface such as USB that connects directly to the external device 9B.

[0265] External device 9A is a device connected to computer 7 via a network. External device 9B is a device directly connected to computer 7.

[0266] External device 9A or external device 9B may, for example, be an input device. The input device may be a camera, microphone, motion capture device, various sensors, keyboard, mouse, touch panel, etc., and provides the acquired information to the computer 7. Alternatively, it may be a device equipped with an input unit, memory, and processor, such as a personal computer, tablet terminal, or smartphone.

[0267] Furthermore, external device 9A or external device 9B may, for example, be an output device. The output device may be a display device such as an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) panel, or a speaker that outputs sound, etc. It may also be a device equipped with an output unit, memory, and a processor, such as a personal computer, tablet terminal, or smartphone.

[0268] Furthermore, external devices 9A and 9B may be storage devices (memory). For example, external device 9A may be network storage, and external device 9B may be storage such as an HDD.

[0269] Furthermore, the external device 9A or external device 9B may be a device that has some of the functions of the components of each device (control device 10, generation device 20, and terminal device 30) in the embodiment described above. In other words, the computer 7 may transmit some or all of the processing results to the external device 9A or external device 9B, or may receive some or all of the processing results from the external device 9A or external device 9B.

[0270] In this specification (including the claims), when the expression "at least one of a, b, and c" or "at least one of a, b, or c" (including similar expressions) is used, it includes any of a, b, c, ab, ac, bc, or abc. Furthermore, any element may have multiple instances, such as aa, abb, aabbcc, etc. In addition, it is also possible to add other elements other than the enumerated elements (a, b, and c), such as abcd which has d.

[0271] In this specification (including the claims), when expressions such as "using data as input / based on data / according to / in accordance with data" (including similar expressions) are used, unless otherwise specified, this includes using the data itself or using data that has been processed in some way (e.g., data with added noise, normalized data, features extracted from the data, intermediate representations of the data, etc.). Furthermore, when it is stated that some result is obtained "using data as input / based on data / according to / in accordance with data" (including similar expressions), unless otherwise specified, this includes cases where the result is obtained based solely on the data in question or where the result is influenced by other data, factors, conditions, and / or states other than the data in question. Furthermore, when it is stated that "data is output" (including similar expressions), unless otherwise specified, this includes cases where the data itself is used as output or where data that has been processed in some way (e.g., data with added noise, normalized data, features extracted from the data, intermediate representations of various types of data, etc.) is used as output.

[0272] In this specification (including the claims), the terms “connected” and “coupled” are intended to be non-restrictive terms that include any direct connection / coupling, indirect connection / coupling, electrical connection / coupling, communicative connection / coupling, operational connection / coupling, physical connection / coupling, etc. The terms should be interpreted as appropriate in the context in which they are used, but any form of connection / coupling that is not intentionally or naturally excluded should be interpreted non-restrictively as being included in the terms.

[0273] In this specification (including the claims), when the expression "A is configured to B" is used, the physical structure of element A has a configuration capable of performing operation B, and it may include that the permanent or temporary setting / configuration of element A is set to actually perform operation B. For example, when element A is a general-purpose processor, the processor has a hardware configuration capable of performing operation B, and it may be set to actually perform operation B by a permanent or temporary program (instruction) setting. Also, when element A is a dedicated processor, a dedicated arithmetic circuit, etc., regardless of whether control instructions and data are actually attached, the circuit structure, etc. of the processor may be implemented to actually perform operation B.

[0274] In this specification (including the claims), when terms meaning containment or possession (such as "comprising / including", "having", etc.) are used, they are intended as open-ended terms, including cases where they contain or possess things other than the object indicated by the object of such terms. When the object of these terms meaning containment or possession does not specify a quantity or is an expression suggesting a singular number (an expression with "a" or "an" as the article), such an expression should be interpreted as not being limited to a specific number.

[0275] In this specification (including the claims), even if expressions such as "one or more" and "at least one" are used in one place, and expressions that do not specify a quantity or imply a singular number (expressions with "a" or "an" as the article) are used in other places, the latter expression is not intended to mean "one". Generally, expressions that do not specify a quantity or imply a singular number (expressions with "a" or "an" as the article) should be interpreted as not necessarily being limited to a specific number.

[0276] In this specification, if it is described that a specific effect (advantage / result) is obtained for a specific configuration of a certain embodiment, unless there are other reasons, it should be understood that the same effect can also be obtained for one or more other embodiments having the same configuration. However, the presence or absence of the effect generally depends on various factors, conditions, and / or states, and it should be understood that the effect is not necessarily obtained by the configuration. The effect is only obtained by the configuration described in the embodiment when various factors, conditions, and / or states are satisfied, and the effect is not necessarily obtained in the invention according to the claim that defines the configuration or a similar configuration.

[0277] In this specification (including the claims), when a plurality of hardware performs a predetermined process, each hardware may cooperate to perform the predetermined process, or some hardware may perform all of the predetermined process. Also, some hardware may perform a part of the predetermined process, and another hardware may perform the remainder of the predetermined process. In this specification (including the claims), when expressions such as "one or more hardware perform a first process and the one or more hardware perform a second process" (including similar expressions) are used, the hardware performing the first process and the hardware performing the second process may be the same or different. That is, it is sufficient that the hardware performing the first process and the hardware performing the second process are included in the one or more hardware. Note that the hardware may include an electronic circuit, a device including an electronic circuit, and the like.

[0278] In this specification (including the claims), when multiple memory devices store data, each of the multiple memory devices may store only a portion of the data or the entire data. Furthermore, a configuration in which some of the multiple memory devices store data is also included.

[0279] In this specification (including the claims), terms such as “first,” “second,” etc., are used merely as a way of distinguishing between two or more elements and are not necessarily intended to impose technical meanings such as temporal, spatial, order, or quantity on the subject. Therefore, for example, references to a first element and a second element do not necessarily mean that only two elements can be employed therein, that the first element must precede the second element, or that the first element must exist for the second element to exist.

[0280] While embodiments of this disclosure have been described in detail above, this disclosure is not limited to the individual embodiments described above. Various additions, modifications, substitutions, and partial deletions are possible, provided that they do not depart from the conceptual idea and spirit of the present invention derived from the claims and their equivalents. For example, where numerical values ​​or mathematical formulas are used in the description of the embodiments described above, these are provided for illustrative purposes only and do not limit the scope of this disclosure. Similarly, the sequence of operations shown in the embodiments is also illustrative and does not limit the scope of this disclosure.

[0281] Furthermore, the following forms are possible for disclosure technology.

[0282] (Note 1) At least one memory, Equipped with at least one processor, The aforementioned at least one processor is We obtain dialogue data that shows the content of the conversation, By inputting the aforementioned dialogue data into the first machine learning model, input information is generated to be input to the second machine learning model in order to have the dialogue agent perform the dialogue. Information processing system.

[0283] (Note 2) The aforementioned at least one processor is The system accepts input of dialogue conditions from the user. The dialogue data is generated by inputting the aforementioned dialogue conditions into the first machine learning model. The information processing system described in Appendix 1.

[0284] (Note 3) The aforementioned at least one processor is The dialogue data generated by the first machine learning model is presented to the user. The system accepts modifications to the dialogue data by the user. The information processing system described in Appendix 2.

[0285] (Note 4) The aforementioned at least one processor is The system accepts input of the aforementioned dialogue data from the user. An information processing system as described in any one of the items 1 to 3 of the appendix.

[0286] (Note 5) The aforementioned at least one processor is The input information, including the profile information of the dialogue agent, is generated. An information processing system as described in any one of the items 1 to 4 of the appendix.

[0287] (Note 6) The aforementioned at least one processor is By inputting the aforementioned dialogue data into the first machine learning model, a plurality of candidate profile information is generated. The system accepts user selection of the aforementioned profile information. The information processing system described in Appendix 5.

[0288] (Appendix 7) The profile information includes at least any one of attributes, basic information, personal information, personality, characteristics, or situational factors. The information processing system according to Appendix 5 or 6.

[0289] (Appendix 8) The at least one processor generates the input information including scenario information of the dialogue to be executed by the dialogue agent. The information processing system according to any one of Items 1 to 7 of the appendix.

[0290] (Appendix 9) The at least one processor presents the scenario information generated by the first machine learning model to the user, and accepts the correction of the scenario information by the user. The information processing system according to Appendix 8.

[0291] (Appendix 10) The scenario information includes at least any one of settings, context, premise, or script. The information processing system according to Appendix 8 or 9.

[0292] (Appendix 11) The at least one processor generates the input information including phase information of the dialogue to be executed by the dialogue agent. The information processing system according to any one of Items 1 to 10 of the appendix.

[0293] (Appendix 12) The at least one processor presents the phase information generated by the first machine learning model to the user, and accepts the correction of the phase information by the user. The information processing system according to Appendix 11.

[0294] (Note 13) The aforementioned phase information includes at least one of the following: phase transition information, flow, development, scene, or stage. The information processing system described in Appendix 11 or 12.

[0295] (Note 14) The aforementioned at least one processor is By inputting the aforementioned input information into the second machine learning model, a dialogue between the interlocutor and the dialogue agent is performed. An information processing system as described in any one of the items in Appendix 1 through Appendix 13.

[0296] (Note 15) The first machine learning model and the second machine learning model are the same machine learning model. The information processing system described in Appendix 14.

[0297] (Note 16) The aforementioned input information is a prompt. An information processing system as described in any one of the items in Appendix 1 through Appendix 15.

[0298] (Note 17) At least one memory, Equipped with at least one processor, The aforementioned at least one processor is Obtain first input information to allow the first dialogue agent to perform a dialogue, Obtain second input information to allow the second dialogue agent to perform the dialogue, By inputting the first input information into the first machine learning model and the second input information into the second machine learning model, a dialogue between the first dialogue agent and the second dialogue agent is performed. The first input information includes, as information necessary for dialogue control, at least one of either scenario information or phase information. The second input information contains less information necessary for dialogue control than the first input information. Information processing system.

[0299] (Note 18) The second input information does not include at least a portion of the scenario information included in the first input information. The information processing system described in Appendix 17.

[0300] (Note 19) The second input information does not include at least a portion of the phase information included in the first input information. The information processing system described in Appendix 17 or 18.

[0301] (Note 20) The first machine learning model and the second machine learning model are the same machine learning model. An information processing system as described in any one of the items 17 to 19 of the appendices.

[0302] (Note 21) The aforementioned at least one processor is Dialogue data showing the content of the conversation between the first and second dialogue participants is obtained. By inputting the aforementioned dialogue data into a third machine learning model, the first input information for performing the dialogue as the first dialoguer is generated. By inputting the aforementioned dialogue data into a fourth machine learning model, the second input information for performing the dialogue as the second dialoguer is generated. An information processing system as described in any one of the items 17 to 20 of the appendix.

[0303] (Note 22) The aforementioned at least one processor is By inputting second dialogue data, which represents the content of the dialogue between the first dialogue agent and the second dialogue agent, into a fifth machine learning model, an evaluation result of the dialogue content is generated. The information processing system described in Appendix 21.

[0304] (Note 23) The aforementioned at least one processor is By inputting the aforementioned dialogue data into a sixth machine learning model, evaluation items are generated for evaluating the content of the dialogue between the first dialogue agent and the second dialogue agent. The information processing system described in Appendix 22.

[0305] (Note 24) The evaluation results include information indicating whether or not the evaluation items are met. The information processing system described in Appendix 23.

[0306] (Note 25) The evaluation result includes at least one reason for satisfying the evaluation item, or one reason for not satisfying the evaluation item. The information processing system described in Appendix 23 or 24.

[0307] (Note 26) The aforementioned at least one processor is The evaluation items generated by the sixth machine learning model are presented to the user. The system accepts modifications to the evaluation items by the user. An information processing system as described in any one of the items 23 to 25 of the appendix.

[0308] (Note 27) The aforementioned evaluation items include items for determining at least one of the following: dialogue consistency, dialogue coherence, role matching, degree of following role setting prompts, degree of adherence to role setting prompts, degree of suitability to role setting prompts, factual accuracy, dialogue fluency, or dialogue style. An information processing system as described in any one of the items 23 to 26 of the appendix.

[0309] (Note 28) Using the information processing system described in any one of the items in Appendix 1 to Appendix 16, To enable the dialogue agent to perform a dialogue, input information is generated to be input to the second machine learning model. Information processing methods.

[0310] (Note 29) Using the information processing system described in any one of the items in Appendix 17 to Appendix 27, Perform the dialogue between the preceding first dialogue agent and the preceding second dialogue agent. Information processing methods.

[0311] (Note 30) At least one processor is made to execute the information processing method described in Appendix 28. program.

[0312] (Note 31) At least one processor is made to execute the information processing method described in Appendix 29. program. [Explanation of Symbols]

[0313] 10: Control device 20:Generation device 30: Terminal device 100: Agent Control Unit 101: Agent Development Department 102: Message acquisition unit 103: Message generation unit 104: Message output section 110: Prompt Creation Support Department 111: Dialogue Data Acquisition Unit 112: Dialogue Data Correction Unit 113: Prompt generation unit 114: Prompt Correction Section 120: Agent Verification Support Department 121: Debugger Construction Section 122: Dialogue Execution Unit 123: Evaluation Item Generation Unit 124: Evaluation Execution Unit 1000: Dialogue System AG: Conversational Agent M: Machine learning model

Claims

1. At least one memory, It comprises at least one processor, The aforementioned at least one processor is We obtain dialogue data that shows the content of the conversation, By inputting the aforementioned dialogue data into the first machine learning model, input information is generated to be input to the second machine learning model in order to have the dialogue agent perform the dialogue. Information processing system.

2. The aforementioned at least one processor is The system accepts input of dialogue conditions from the user. The dialogue data is generated by inputting the aforementioned dialogue conditions into the first machine learning model. The information processing system according to claim 1.

3. The aforementioned at least one processor is The dialogue data generated by the first machine learning model is presented to the user. The system accepts modifications to the dialogue data by the user. The information processing system according to claim 2.

4. The aforementioned at least one processor is The system accepts input of the aforementioned dialogue data from the user. The information processing system according to claim 1.

5. The aforementioned at least one processor is The input information, including the profile information of the dialogue agent, is generated. The information processing system according to claim 1.

6. The aforementioned at least one processor is By inputting the aforementioned dialogue data into the first machine learning model, a plurality of candidate profile information is generated. The system accepts user selection of the aforementioned profile information. The information processing system according to claim 5.

7. The aforementioned profile information includes at least one of the following: attributes, basic information, personal information, personality, characteristics, or situational factors. The information processing system according to claim 5.

8. The aforementioned at least one processor is The input information includes dialogue scenario information to be executed by the dialogue agent, The information processing system according to claim 1.

9. The aforementioned at least one processor is The scenario information generated by the first machine learning model is presented to the user. The system accepts the user's modification of the scenario information. The information processing system according to claim 8.

10. The aforementioned scenario information includes at least one of the following: setting, context, premise, or plot. The information processing system according to claim 8.

11. The aforementioned at least one processor is The input information, which includes the phase information of the dialogue to be performed by the dialogue agent, generates the input information. The information processing system according to claim 1.

12. The aforementioned at least one processor is The phase information generated by the first machine learning model is presented to the user. The system accepts the user's modification of the phase information. The information processing system according to claim 11.

13. The aforementioned phase information includes at least one of the following: phase transition information, flow, development, scene, or stage. The information processing system according to claim 11.

14. The aforementioned at least one processor is By inputting the aforementioned input information into the second machine learning model, a dialogue between the interlocutor and the dialogue agent is performed. The information processing system according to any one of claims 1 to 13.

15. The first machine learning model and the second machine learning model are the same machine learning model. The information processing system according to claim 14.

16. The aforementioned input information is a prompt. The information processing system according to any one of claims 1 to 13.

17. At least one memory, It comprises at least one processor, The aforementioned at least one processor is Obtain first input information to allow the first dialogue agent to perform a dialogue, Obtain second input information to allow the second dialogue agent to perform the dialogue, By inputting the first input information into the first machine learning model and the second input information into the second machine learning model, a dialogue between the first dialogue agent and the second dialogue agent is performed. The first input information includes, as information necessary for dialogue control, at least one of either scenario information or phase information. The second input information contains less information necessary for dialogue control than the first input information. Information processing system.

18. The second input information does not include at least a portion of the scenario information included in the first input information. The information processing system according to claim 17.

19. The second input information does not include at least a portion of the phase information included in the first input information. The information processing system according to claim 17.

20. The first machine learning model and the second machine learning model are the same machine learning model. The information processing system according to claim 17.

21. The aforementioned at least one processor is Dialogue data showing the content of the conversation between the first and second interlocutors is obtained. By inputting the aforementioned dialogue data into a third machine learning model, the first input information for executing the dialogue as the first dialoguer is generated. By inputting the aforementioned dialogue data into a fourth machine learning model, the second input information for performing the dialogue as the second dialoguer is generated. The information processing system according to claim 17.

22. The aforementioned at least one processor is By inputting second dialogue data, which represents the content of the dialogue between the first dialogue agent and the second dialogue agent, into a fifth machine learning model, an evaluation result of the dialogue content is generated. The information processing system according to claim 21.

23. The aforementioned at least one processor is By inputting the aforementioned dialogue data into a sixth machine learning model, evaluation items are generated for evaluating the content of the dialogue between the first dialogue agent and the second dialogue agent. The information processing system according to claim 22.

24. The evaluation results include information indicating whether or not the evaluation items are met. The information processing system according to claim 23.

25. The evaluation result includes at least one reason for satisfying the evaluation item, or one reason for not satisfying the evaluation item. The information processing system according to claim 23.

26. The aforementioned at least one processor is The evaluation items generated by the sixth machine learning model are presented to the user. The system accepts modifications to the evaluation items by the user. The information processing system according to claim 23.

27. The aforementioned evaluation items include items for determining at least one of the following: dialogue consistency, dialogue coherence, role matching, degree of following role setting prompts, degree of adherence to role setting prompts, degree of suitability to role setting prompts, factual accuracy, dialogue fluency, or dialogue style. The information processing system according to claim 23.

28. Using the information processing system described in any one of claims 1 to 13, An information processing method for generating input information to be input to the second machine learning model in order to cause the dialogue agent to perform a dialogue.

29. Using the information processing system described in any one of claims 17 to 27, The first dialogue agent and the second dialogue agent perform a dialogue. Information processing methods.

30. The information processing method described in claim 28 is performed on at least one processor. program.

31. To cause at least one processor to execute the information processing method described in claim 29, program.

Images