Information processing device, information processing method, information processing system, and program

Abstract

To enable appropriate responses to messages sent by the user while multiple characters are conversing. [Solution] The system includes a conversation execution unit that executes a program to generate character speech data and causes multiple characters to perform character-to-character conversations, an output unit that outputs character speech data indicating the content of the character-to-character conversations, and a data acquisition unit that acquires user speech data indicating the user's speech on the information processing device after the output unit has output the character speech data. The conversation execution unit then causes multiple characters and the user to perform character-to-user conversations in a manner based on the user speech data acquired by the data acquisition unit, based on the status of the character-to-character conversations.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, an information processing system, and a program.

Background Art

[0002] Conventionally, there has been known an information processing system in which a computer executes a program so that a character selected by a user can issue a message addressed to the user based on the context of the message sent by the user (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a conventional information processing system, it has been assumed that a user and a character have a one-on-one conversation. When the user issues a message while the user and the character are having a one-on-one conversation, since the message is considered to be a message addressed to the character, the information processing system creates a message of the character based on the context of the message issued by the user.

[0005] However, when a plurality of characters can have a conversation, there has been a problem that if a plurality of characters always respond to a message issued by the user during the conversation of the plurality of characters, the conversation becomes unnatural.

[0006] Therefore, the present invention has been made in view of these points, and an object thereof is to be able to appropriately respond to a message issued by a user while a plurality of characters are having a conversation. [Means for solving the problem]

[0007] An information processing device according to a first aspect of the present invention includes: a conversation execution unit that executes a program to generate character speech data and causes a conversation between multiple characters to take place; an output unit that outputs the character speech data indicating the content of the conversation between characters; and a data acquisition unit that, after the output unit has output the character speech data, acquires user speech data indicating the speech of a user using a service provided by the information processing device. The conversation execution unit, based on the status of the conversation between characters, causes a conversation between the multiple characters and the user in a manner based on the user speech data acquired by the data acquisition unit.

[0008] The conversation execution unit may, based on the content of the user utterance data acquired by the data acquisition unit, identify which of the multiple characters the user utterance data is addressed to, and generate character utterance data to be spoken by the identified character.

[0009] The information processing device may further include a storage unit that stores the personalities of each of the multiple characters, and the conversation execution unit may generate character speech data to be spoken by a selected character based on the personalities of each of the multiple characters.

[0010] The character speech data and the user speech data are audio data, and the conversation execution unit may stop the character conversation if the data acquisition unit begins to acquire the user speech data while the characters are having a conversation.

[0011] The conversation execution unit may not execute the character-user conversation while it is executing the character-user conversation, and may execute the character-user conversation if the data acquisition unit acquires the user utterance data while the character-user conversation is not being executed.

[0012] The conversation execution unit may, while executing the character-to-character conversation, determine, based on the content of the character-to-character conversation, whether or not it is a period during which it is permissible to execute the character-to-user conversation. If the data acquisition unit acquires the user utterance data during a period during which it is determined that it is permissible to execute the character-to-user conversation, it may execute the character-to-user conversation.

[0013] If the conversation execution unit is executing the conversation between characters and the data acquisition unit acquires the user utterance data, the conversation execution unit may generate character utterance data for at least one of the multiple characters that includes a predetermined word stored in the storage unit beforehand.

[0014] The conversation execution unit may control the content of the character utterance data or the timing of the character utterance data uttered by the multiple characters in the character-user conversation after the data acquisition unit has acquired the user utterance data, based on at least one of the words contained in the user utterance data, the context of the user utterance data, the intonation of the user utterance data, the emotion of the user estimated from the user utterance data, the speaking speed of the user utterance data, and the time from when the previous user utterance data was uttered until when the latest user utterance data was uttered.

[0015] If the conversation execution unit determines that a response to the user utterance data acquired by the data acquisition unit is unnecessary, it may continue the character-user conversation without executing the character-user conversation.

[0016] The character speech data and the user speech data are audio data, and the conversation execution unit may choose not to execute the character-user conversation if the volume of the user's voice indicated by the user speech data is below a threshold.

[0017] The information processing device further includes a storage unit that stores relational data indicating the relationship between at least one of the plurality of characters and the user, and the conversation execution unit may, based on the relational data, control at least one of the following: the content of the character utterance data spoken by the plurality of characters in the character-user conversation after the data acquisition unit has acquired the user utterance data, whether or not any of the plurality of characters respond, and the timing of when the plurality of characters utter the character utterance data.

[0018] The data acquisition unit may acquire user utterance data based on the utterances of multiple users, identify the user corresponding to the acquired user utterance data, and the conversation execution unit may generate character utterance data for the user utterance data based on the result of the data acquisition unit identifying the user.

[0019] The conversation execution unit may pause the character conversation for a predetermined period of time if the content of the character conversation or the content of the user speech data acquired by the data acquisition unit satisfies predetermined conditions.

[0020] The conversation execution unit may suspend the speech of each of the multiple characters for a predetermined period of time associated with each of the characters.

[0021] The conversation execution unit may, when the conversation between characters or the user speech data acquired by the data acquisition unit satisfies predetermined conditions, cause one of the multiple characters to speak alone for a predetermined period of time or longer.

[0022] The conversation execution unit may, if the user utterance data acquired by the data acquisition unit during the conversation between the character users satisfies predetermined conditions, have at least one of the multiple characters utter an utterance during the user's utterance.

[0023] The data acquisition unit acquires the user speech data from a plurality of the users, identifies the user who transmitted the acquired user speech data, and stores the user speech data in the storage unit in association with the identified user. The conversation execution unit may generate character speech data for the user who transmitted the user speech data acquired immediately before by the data acquisition unit by referring to a plurality of past user speech data stored in the storage unit in association with the user who transmitted the user speech data acquired immediately before by the data acquisition unit.

[0024] The data acquisition unit creates summary data of the past user speech data stored in the storage unit, and the conversation execution unit may generate character speech data for the user speech data acquired immediately before by the data acquisition unit by referring to the summary data.

[0025] When the user speech data is input, the conversation execution unit inputs the user speech data acquired by the data acquisition unit into a machine learning model that outputs data indicating the emotion at the time of receiving the user speech data, identifies the emotion at the time of receiving the user speech data, and may generate character speech data for the user speech data based on the identified emotion.

[0026] The conversation execution unit may identify the emotion at the time of receiving the user speech data based on the relationship between the content of the topic during the conversation between the characters and the content indicated by the user speech data acquired by the data acquisition unit, and generate character speech data for the user speech data based on the identified emotion.

[0027] The information processing apparatus further includes an image data generation unit that generates face image data of the plurality of characters, and the conversation execution unit stores, in the storage unit, a tag indicating a specified emotion in association with the user speech data acquired by the data acquisition unit. The image data generation unit may generate the face image data corresponding to the emotion indicated by the tag by referring to the tag stored in the storage unit.

[0028] The information processing method according to the second aspect of the present invention includes steps in which a computer executes a character-to-character conversation among a plurality of characters by generating character speech data, outputs the character speech data indicating the content of the character-to-character conversation, acquires user speech data indicating a user's speech after outputting the character speech data, and executes a character-to-user conversation between the plurality of characters and the user in a manner based on the acquired user speech data according to the situation of the character-to-character conversation.

[0029] The program according to the third aspect of the present invention is a program for causing a computer to execute steps in which a character-to-character conversation is executed among a plurality of characters by generating character speech data, the character speech data indicating the content of the character-to-character conversation is output, user speech data indicating a user's speech is acquired after outputting the character speech data, and a character-to-user conversation between the plurality of characters and the user is executed in a manner based on the acquired user speech data according to the situation of the character-to-character conversation.

[0030] A fourth aspect of the present invention provides an information processing system comprising an information processing device and an information terminal used by a user who uses a service provided by the information processing device, wherein the information processing device includes a conversation execution unit that executes a program to generate character speech data and causes a conversation between multiple characters to take place, an output unit that outputs the character speech data indicating the content of the conversation between characters, and a data acquisition unit that, after the output unit has output the character speech data, acquires user speech data indicating the user's speech from the information terminal, wherein the conversation execution unit causes a conversation between the multiple characters and the user in a manner based on the user speech data acquired by the data acquisition unit, based on the status of the conversation between characters, and the information terminal transmits the input user speech data to the information processing device and outputs the character speech data received from the information processing device. [Effects of the Invention]

[0031] According to the present invention, the system will be able to respond appropriately to messages sent by the user while multiple characters are conversing. [Brief explanation of the drawing]

[0032] [Figure 1] This is a diagram illustrating the overview of the information processing system S. [Figure 2] This diagram shows the content of the statements made by both characters during a conversation between them. [Figure 3] This is a diagram showing the configuration of the information processing device 1. [Figure 4] This figure shows an example of character data stored in the memory unit 12. [Figure 5] This figure shows an example of relational data stored in the memory unit 12. [Figure 6] This is a flowchart illustrating an example of the processing flow in the information processing device 1. [Modes for carrying out the invention]

[0033] [Overview of Information Processing System S] Figure 1 is a diagram illustrating the overview of the information processing system S. The information processing system S is a system for user U to send and receive messages with a character that can converse using artificial intelligence (AI). User U is a user of the services provided by the information processing device 1. The character and user U may converse by sending and receiving messages containing text data, or by sending and receiving messages containing voice data. The character and user U may converse while sending and receiving messages containing image data or video data.

[0034] The number of characters and users U that can converse is arbitrary, and in Figure 1, users U-1 through Un (where n is an integer) are shown. In the information processing system S, one character may converse with one user U, multiple characters may converse with one user U, one character may converse with multiple users U, or multiple characters may converse with multiple users U.

[0035] In the information processing system S, a character is associated with facial image data, such as an animated image of an animal's face. For example, multiple characters may have different facial images, but multiple characters may have the same facial image. Multiple characters may each have different attributes. Attributes are determined, for example, by the character's type, age, voice quality, and personality. Multiple characters may have the same attributes.

[0036] As shown in Figure 1, the information processing system S comprises an information processing device 1 and one or more information terminals 2. The information processing device 1 and the one or more information terminals 2 transmit and receive data via a network.

[0037] Information processing device 1 has, for example, a generative AI based on an LLM (Large-Scale Language Model), and generates character speech data that a character will utter by executing a program. Based on the content of the user speech data received from the information terminal 2 and at least one of the character's attributes, information processing device 1 determines the content of the character speech data and the timing at which the character transmits the character speech data.

[0038] Information terminal 2 is a device used by user U, and is, for example, a smartphone, tablet, or computer. Information terminal 2 transmits user speech data corresponding to the voice or text input from user U to information processing device 1. Information terminal 2 also outputs voice corresponding to character speech data transmitted from information processing device 1 through its speaker, or displays text corresponding to character speech data on its display. Information terminal 2 converts the input voice into digital data and transmits it to information processing device 1, and converts the digital data received from information processing device 1 into voice before outputting it.

[0039] Figure 2 shows the content of statements made by both parties during a conversation between multiple characters and user U (hereinafter referred to as "character-user conversation"). In Figure 2, the content of the character statements is displayed on the display of information terminal 2, but the character statements may also be output as audio.

[0040] Figure 2(a) shows a state of character-to-character conversation where multiple characters are talking. Although Figure 2(a) shows a state where two characters are talking, three or more characters may be talking in a character-to-character conversation.

[0041] Figure 2(b) shows a situation where a character-user conversation begins when user U joins the conversation while a conversation between characters is taking place. In Figure 2(b), user U hears the first character say, "Ohtani was amazing today!" and the second character say, "That was an incredible home run," and then says, "He also successfully stole a base."

[0042] Thus, in the information processing system S, user U can join a conversation between multiple characters and converse with the characters. This function can be applied to various applications for providing information to users, but it is particularly suitable for story-based games because users can converse while viewing images of the characters. The configuration and operation of the information processing device 1 will be described in detail below.

[0043] [Configuration of Information Processing Device 1] Figure 3 shows the configuration of the information processing device 1. The information processing device 1 includes a communication unit 11, a storage unit 12, and a control unit 13. The control unit 13 includes a data acquisition unit 131, a conversation execution unit 132, an image data generation unit 133, and an output unit 134.

[0044] The communication unit 11 has a communication interface for sending and receiving various types of data with the information terminal 2. The communication unit 11 transmits character speech data input from the output unit 134 to the information terminal 2 and inputs user speech data received from the information terminal 2 to the data acquisition unit 131.

[0045] The memory unit 12 has storage media such as ROM (Read Only Memory), RAM (Random Access Memory), and SSD (Solid State Drive). The memory unit 12 stores the program executed by the control unit 13. The memory unit 12 also stores various data used to realize character-to-character conversations and character-to-user conversations. For example, the memory unit 12 temporarily stores user utterance data associated with user identification information. The memory unit 12 may also store character data and related data used by the conversation execution unit 132 to generate character utterance data.

[0046] Figure 4 shows an example of character data stored in the memory unit 12. The character data includes information about each of the multiple characters that can appear in character-to-character conversations and character-to-user conversations. In the character data shown in Figure 4, the character ID, the character's official name, the character's abbreviation, and the character's personality are associated.

[0047] Figure 5 shows an example of relational data stored in the memory unit 12. Relational data indicates the compatibility between a character and user U. In the example shown in Figure 5, A indicates good compatibility, B indicates moderate compatibility, and C indicates poor compatibility. For example, user U1 has good compatibility with characters CH1 and CH4, but poor compatibility with character CH2.

[0048] The control unit 13, for example, has a CPU (Central Processing Unit) and functions as a data acquisition unit 131, a conversation execution unit 132, an image data generation unit 133, and an output unit 134 by executing a program stored in the memory unit 12.

[0049] The data acquisition unit 131 acquires various types of data transmitted from the information terminal 2. For example, after the output unit 134 outputs character speech data, the data acquisition unit 131 acquires user speech data indicating the speech of user U. The data acquisition unit 131 has a function to convert between speech and text, and may have a speech recognition function that converts digital speech data received from the information terminal 2 into text data. The data acquisition unit 131 acquires user speech data, for example, and notifies the conversation execution unit 132 of the acquired user speech data. The data acquisition unit 131 may also notify the conversation execution unit 132 by storing the acquired user speech data in the storage unit 12.

[0050] The conversation execution unit 132 executes character conversations between multiple characters by running a program to generate character utterance data. This program includes, for example, a large-scale language model. The conversation execution unit 132 generates character utterance data by inputting, for example, user utterance data obtained from the data acquisition unit 131 and a prompt to instruct the LLM to generate character utterance data corresponding to the user utterance data, and then acquiring the character utterance data output from the LLM. The conversation execution unit 132 may use a different LLM for each character to acquire character utterance data.

[0051] The image data generation unit 133 generates face image data for multiple characters. The image data generation unit 133 may generate multiple face image data and store the face image data selected by the user U in the storage unit 12, associating it with information for identifying the character. The face image data is displayed on the information terminal 2's display at the time the character speaks, for example, as shown in Figure 2.

[0052] The output unit 134 outputs character speech data that indicates the content of conversations between characters. The output unit 134 has a speech synthesis function that generates digital audio data based on the character speech data generated by the conversation execution unit 132, and outputs the character speech data in audio form. Specifically, the output unit 134 transmits the character speech data generated by the conversation execution unit 132 to the information terminal 2 via the communication unit 11.

[0053] [Operation of the conversation execution unit 132] The conversation execution unit 132 performs various processes based on character data and relationship data, etc., in order to enable user U to converse with multiple characters as if they were talking to a human. The conversation execution unit 132 determines which character will speak, when the character will speak, and the volume, emotion, speed, etc. of the character's speech, so that the conversation between the character and user U takes place in a manner predetermined by user U, for example. If the length of time during which character-user conversation is possible is predetermined, the conversation execution unit 132 generates character speech data so that the conversation ends within the predetermined time.

[0054] The conversation execution unit 132 determines the content of character utterance data spoken by multiple characters in a character-user conversation after the data acquisition unit 131 has acquired user utterance data. For example, the conversation execution unit 132 determines the content of character utterance data based on the content of user utterance data. Based on data stored in the memory unit 12 that shows the relationship between words included in the user utterance data and the emotions of the characters, the conversation execution unit 132 controls the priority of character utterances, volume, emotion, speed, pauses between utterances, and whether or not to interrupt user U's utterances.

[0055] The conversation execution unit 132 executes character-user conversations between multiple characters and the user in a manner based on the user utterance data acquired by the data acquisition unit 131, based on the status of the character-to-character conversation. The manner based on the user utterance data is, for example, as follows:

[0056] Based on what User U says, the system identifies the character User U is speaking to, and that character then speaks. Based on what User U says and the character's personality, a character is selected to respond to User U, and that character speaks.

[0057] If the timing of acquiring user speech data does not coincide with the timing of generating character speech data, the character will speak in a manner corresponding to user U's speech. If the timing of user speech data acquisition coincides with the timing of character speech data generation, the character will stop speaking.

[0058] If the content of User U's statement, as indicated by the user statement data, is related to the topic of conversation between characters, the character will respond to User U's statement. If the content of User U's statement, as indicated by the user statement data, is not related to the topic of the character conversation, the character will respond in a manner that does not address User U's statement (for example, by ignoring or interrupting User U's statement).

[0059] If the content of a user's statement, as indicated by the user's statement data, is offensive to a certain character, that character's speaking priority will be increased, and they will speak at a faster speed than usual. If the user's emotional state, as indicated by the user's speech data, is determined to be aggressive, increase the speed or volume of the character's response. If the intonation of the user's speech data indicates that user U will continue speaking, all characters will stop speaking.

[0060] The following describes in detail the process by which the conversation execution unit 132 generates character speech data in response to the data acquisition unit 131 acquiring user speech data.

[0061] (Based on the user's comments, the character that should respond is identified.) The conversation execution unit 132 identifies which of the multiple characters the user utterance data is addressed to based on the content of the user utterance data, and generates character utterance data spoken by the identified character. For example, the conversation execution unit 132 identifies the destination character to whom user U spoke based on the relationship between the words contained in each of the multiple character utterance data before the data acquisition unit 131 acquired the user utterance data and the words contained in the user utterance data. In other words, the conversation execution unit 132 identifies the destination character based on the context of the multiple character utterances in the preceding predetermined period and the content of the user's utterance.

[0062] Specifically, the conversation execution unit 132 identifies a destination character that made a statement containing words related to the words in the user statement data at the time closest to the time user U made the statement, within a predetermined period prior to the time user U made the statement, by referring to relational data that shows a group of multiple related words. The conversation execution unit 132 generates character statement data, including a response to the user statement data, by inputting the user statement data into the LLM corresponding to the identified destination character.

[0063] The conversation execution unit 132 may identify the target character based on the character's name included in the user's utterance data. The character's name may be the character's official name as shown in Figure 4, or it may be an abbreviation or nickname. For example, suppose that while the character with ID CH1 and the character with ID CH2, as shown in Figure 4, are having a conversation, the user says, "Teacher, what do you think about Ohtani?"

[0064] In this case, the conversation execution unit 132 determines whether any of the multiple words included in the user utterance data are words that are also included in the full names or abbreviations of characters CH1 and CH2 in the character data shown in Figure 4. The conversation execution unit 132 determines that the destination character is character CH1 by determining that "Sensei" included in the user utterance data is an abbreviation of character CH1. If the conversation execution unit 132 determines that the user utterance data does not contain the names of the characters, it may identify the destination character based on the context of the multiple characters' utterances and the content of the user's utterances.

[0065] (Select a character to respond based on their personality) As the character data in Figure 4 shows, the frequency of speech and the speed of response to speeches from other characters or user U differ among characters. Therefore, the conversation execution unit 132 may generate character speech data for selected characters based on the personalities of each of the multiple characters. For example, the conversation execution unit 132 generates character speech data so that, in response to a speech from user U, characters with a tendency to speak frequently are given priority in responding. Specifically, in the example shown in Figure 4, the conversation execution unit 132 generates character speech data so that character CH2, which has the personality of being "talkative," speaks preferentially over character CH1, which has the personality of being "somewhat taciturn."

[0066] However, doing so may result in character CH1 having no opportunity to respond to user U. Therefore, the conversation execution unit 132 may, on the condition that it could not identify the target character based on the content of user U's statement, select a character to respond to user U's statement based on the character's personality.

[0067] The character data may include, along with the character's personality, or instead of the character's personality, a numerical value indicating how easy it is for the character to speak (hereinafter referred to as the "speaking frequency value"). In this case, the conversation execution unit 132 selects a character to respond to the user U at a frequency corresponding to the speaking frequency value, for example.

[0068] If character CH1 has a speaking frequency value of 1.0 and character CH2 has a speaking frequency value of 4.0, and user U speaks while character CH1 and character CH2 are conversing, the conversation execution unit 132 will have character CH1 respond to user U with a 20% probability and character CH2 respond with an 80% probability. By operating in this way, the conversation execution unit 132 ensures that the characters speak at a frequency that matches their personalities, making it less likely for user U to feel any incongruity regarding the relationship between the characters' personalities and the circumstances of their speech.

[0069] (Conditions for the character to respond) If user U speaks while multiple characters are having a conversation with each other, the conversation will be interrupted. Therefore, the conversation execution unit 132, for example, does not execute a character-user conversation while a character-user conversation is being executed, and only executes a character-user conversation when the data acquisition unit 131 acquires user speech data while a character-user conversation is not being executed. This prevents the conversation of multiple characters from being interrupted, while still allowing user U to participate in the conversation.

[0070] Even while a conversation between characters is taking place, depending on the topic of the conversation, it may not cause any problems if user U interrupts midway through. Therefore, while the conversation is being executed between characters, the conversation execution unit 132 determines, based on the content of the conversation, whether or not it is appropriate to execute a conversation between the character and the user. If the data acquisition unit 131 acquires user utterance data during a period in which it is deemed appropriate to execute a conversation between the character and the user, the conversation between the character and the user may be executed.

[0071] For example, when multiple characters converse according to pre-created scenario data, the conversation execution unit 132 refers to a flag included in the scenario data that indicates whether or not it is a period in which interruptions by user U are permitted, and determines whether or not to execute the character-user conversation. The scenario data is stored in the storage unit 12 and associates the elapsed time since the start of the character-user conversation, the content of what the multiple characters say, and information (e.g., a flag) indicating whether or not it is a period in which interruptions by user U are permitted.

[0072] Based on this scenario data, the conversation execution unit 132 determines whether or not to allow a transition from a conversation between characters to a conversation between character users. This prevents the transition to a conversation involving user U when multiple characters are discussing a topic in which user U is undesirable to participate, while making it easier for user U to join a conversation when multiple characters are discussing a topic in which user U is acceptable to participate.

[0073] The conversation execution unit 132 may send information to the information terminal 2 indicating that user U may join the conversation during a period in which user U interruptions are permitted. When the information terminal 2 receives this information, it displays that user U can join the conversation. This makes it easier for user U to speak at an appropriate time while a conversation between characters is taking place.

[0074] If the data acquisition unit 131 acquires user utterance data while the conversation execution unit 132 is executing a conversation between characters, it may generate character utterance data for at least one of the multiple characters that includes a predetermined word previously stored in the storage unit 12. The predetermined word is, for example, a word that indicates that user U cannot participate in the conversation, such as "Wait a minute" or "Not now." The predetermined word may also be a word that encourages user U to continue the conversation, such as "Okay, and?"

[0075] The conversation execution unit 132 may determine a predetermined word based on what multiple characters are saying by referring to data stored in the memory unit 12 that associates topics with predetermined words. For example, if the topic of the conversation between characters is important to user U, the conversation execution unit 132 selects a predetermined word to strongly prohibit participation. If the topic of the conversation between characters is one in which user U may participate, the conversation execution unit 132 selects a predetermined word to encourage participation. By having the conversation execution unit 132 have the characters utter predetermined words in this way, the processing load is reduced because it is not necessary to analyze the content of user U's statements before generating character statement data.

[0076] The conversation execution unit 132 may determine that it can transition from a character-to-character conversation to a character-to-user conversation if the content of the character's statement includes content that invites user U to join the conversation. For the conversation execution unit 132 to operate in this way, the scenario data associates the content of the character's statement with information indicating whether or not user U is allowed to interrupt after the statement. For example, the conversation execution unit 132 allows user U to join the conversation after the character makes a statement that encourages user U to speak, such as "What do you all think?" or "I'd like to hear your opinions." By operating in this way, the conversation execution unit 132 can transition to a character-to-user conversation without user U feeling any discomfort.

[0077] (Controlling the timing of speech) In order to enable user U to engage in character-user conversations with the same feeling as when talking to a person face-to-face, the conversation execution unit 132 may control the content of the character utterance data or the timing of when multiple characters utter character utterance data in character-user conversations after the data acquisition unit 131 has acquired the user utterance data.

[0078] The conversation execution unit 132 controls the content of the character speech data or the timing of issuing the character speech data based on, for example, at least one of the following: words included in the user speech data, the context of the user speech data, the intonation of the user speech data, the user's emotions estimated from the user speech data, the speed at which the user speech data is spoken, and the time elapsed between the issuance of the previous user speech data and the issuance of the latest user speech data.

[0079] The conversation execution unit 132 determines the timing for issuing character speech data by referring to time data that associates each of the following: words, context, intonation, user U's emotions, speaking speed, and the time until user U speaks, with the time from when user U finishes speaking until when the character speaks.

[0080] Specifically, if the conversation execution unit 132 contains words that indicate user U is dissatisfied (e.g., "This is troublesome," "What are we going to do?") or words that indicate user U is in a hurry (e.g., "Hurry up," "It's too slow"), it will release the character speech data earlier than the standard timing. The conversation execution unit 132 will also release the character speech data earlier than the standard timing if it determines that user U is dissatisfied or in a hurry based on the context identified from multiple past user speech data and character speech data, or based on the intonation of user U's speech.

[0081] The conversation execution unit 132 may generate character speech data requesting that user U slow down if their speech speed is too fast. The conversation execution unit 132 may also generate character speech data requesting that user U change their intonation if their intonation is inappropriate.

[0082] The conversation execution unit 132 may issue character statement data earlier than the standard timing if it determines, based on user utterance data, that user U is feeling dissatisfied. The conversation execution unit 132 may issue character statement data earlier than the standard timing if user U's utterance speed is above a threshold. The conversation execution unit 132 may issue character statement data earlier than the standard timing if the time between the issuance of the previous user utterance data and the issuance of the latest user utterance data is below a threshold.

[0083] In this way, when the conversation execution unit 132 determines, based on the user's utterance data, that user U is dissatisfied or in a hurry, it will utter the character's utterance data earlier than usual, making it less likely for user U to become dissatisfied with the character-user conversation.

[0084] If the conversation execution unit 132 determines that a response is not necessary to the user statement data acquired by the data acquisition unit 131, it may continue the character-to-character conversation without executing the character-to-user conversation. For example, if the conversation execution unit 132 determines that user U's statement is addressed to another character, it will ignore user U's statement. If the content of the user statement data is unrelated to the topic of the character-to-character conversation, or if the user statement data contains a predetermined prohibited word, the conversation execution unit 132 may ignore user U's statement and continue the character-to-character conversation.

[0085] The conversation execution unit 132 may decide whether or not to ignore user U's statements based on its relationship with user U. For example, the conversation execution unit 132 may increase the probability of ignoring user U's statements when the relationship between user U and the character is poor compared to when the relationship between user U and the character is good.

[0086] The conversation execution unit 132 may prevent the character-user conversation from being executed if the character speech data and user speech data are audio data, and the volume of the voice of user U indicated by the user speech data is below a threshold. The threshold is determined, for example, based on the probability that the audio input to the information terminal 2 can be correctly recognized.

[0087] The conversation execution unit 132 may generate character speech data indicating that the user U needs to raise their voice, such as "I can't hear you," if the volume of the user U's voice is below a threshold. By operating in this way, the conversation execution unit 132 makes it easier for the user U to raise their voice, thus preventing the character from making inappropriate remarks due to misrecognition of the user U's voice. It also allows the user U to feel as if they are actually talking to another person.

[0088] By the way, if user U speaks while multiple characters are conversing via voice, user U's voice will overlap with the voices of the multiple characters, making it impossible to correctly identify the content of user U's voice. Therefore, the conversation execution unit 132 may stop the character conversation if the data acquisition unit 131 begins acquiring user speech data while the characters are conversing.

[0089] However, depending on the topic of the conversation between characters, there may be cases where the conversation should not be stopped. In such cases, the conversation execution unit 132 may either continue the conversation between characters as is, or increase the volume of the characters' voices and allow them to continue speaking.

[0090] In order for the conversation execution unit 132 to appropriately determine whether or not to stop the conversation between characters, the scenario data used by the conversation execution unit 132 to generate character statement data may include a stop determination flag that indicates whether or not it is permissible to stop the conversation, in relation to the topic or the content of what the character says. In this case, the conversation execution unit 132 will refer to the stop determination flag to determine whether or not to stop the conversation between characters when user U speaks during the conversation between characters.

[0091] In human conversations, there are times when silence follows a statement. Therefore, the conversation execution unit 132 may pause the character conversation for a predetermined period of time if the content of the character conversation or the content of the user statement data acquired by the data acquisition unit 131 satisfies predetermined conditions. These predetermined conditions include when the user statement data contains words that have been registered in advance, when the user statement data is longer than a predetermined length, when the user's tone is aggressive, or when the topic has come to a close.

[0092] The conversation execution unit 132 may pause the character-to-character conversation if the part of speech at the end of the user's utterance data is not a sentence-ending particle or auxiliary verb, or if the intonation of the user's utterance indicates that the utterance will continue, as this may indicate that the next word may be uttered. By pausing the character's utterance and creating silence, the conversation execution unit 132 makes it easier for the user to formulate the next word. This operation by the conversation execution unit 132 makes it easier for user U to feel as if they are having a conversation with a person during character-to-user conversations.

[0093] If the predetermined time for pausing character conversations is constant regardless of the character, it becomes uninteresting for user U. Therefore, the conversation execution unit 132 may pause a character's speech for a predetermined time associated with each of the multiple characters. For example, the conversation execution unit 132 refers to the character data shown in Figure 4 and determines the predetermined time based on the character's personality. Specifically, character CH1, who has a taciturn personality, will have a longer pause time for speaking than character CH2, who talks a lot. By operating in this way, user U can feel the individuality of the characters, which increases their desire to converse with them.

[0094] The conversation execution unit 132 may have one of the multiple characters speak alone for a predetermined period of time or longer if the conversation between characters or the user speech data acquired by the data acquisition unit 131 satisfies predetermined conditions. For example, in the normal state, the conversation execution unit 132 switches the character speaking within the maximum speaking time that each character can speak continuously. On the other hand, if the conversation execution unit 132 determines, based on the context, that a character should continue speaking, it will have that character speak alone for a predetermined period of time or longer.

[0095] The conversation execution unit 132 may cause one of the characters to speak beyond the maximum speaking time if the topic is in an area of ​​expertise for one of the characters. The conversation execution unit 132 may also cause one of the characters to speak beyond the maximum speaking time if the user's spoken data is a question directed at that character.

[0096] The conversation execution unit 132 may have at least one of the multiple characters speak during a conversation between character users if the user utterance data acquired by the data acquisition unit 131 during the conversation satisfies predetermined conditions. For example, if the conversation execution unit 132 detects that there is a difference of opinion between user U and a character, it may have the character speak during user U's utterance.

[0097] The conversation execution unit 132 may have a character speak while user U is speaking if user U continues speaking beyond a predetermined maximum speaking time stored in the memory unit 12, or if the user's speech data contains inappropriate words. The conversation execution unit 132 may also have a second character start speaking while a first character is speaking, thereby stopping the first character from speaking. By having the conversation execution unit 132 interrupt others in this way, user U can experience a certain degree of tension, similar to when talking to a person in the real world, thus increasing the enjoyment user U has when conversing with the character.

[0098] (Control based on the relationship between the character and user U) The conversation execution unit 132 may control the character-user conversation based on the relationship between the character and user U indicated by the relationship data stored in the memory unit 12. Based on the relationship between the character and user U, the conversation execution unit 132 controls at least one of the following after the data acquisition unit 131 acquires the user utterance data: the content of the character utterance data spoken by multiple characters in the character-user conversation, whether or not any of the multiple characters respond, and the timing of when the multiple characters utter the character utterance data.

[0099] For example, if the relationship between User U and any of the multiple characters engaged in a character-to-character conversation is good (i.e., the relationship shown in Figure 5 is rank A), the conversation execution unit 132 will generate character statement data with positive content in response to User U's statement as soon as User U speaks. On the other hand, if the relationship between User U and both of the multiple characters engaged in a character-to-character conversation is not good, the conversation execution unit 132 will not generate character statement data even if the data acquisition unit 131 acquires User statement data.

[0100] Even when the conversation execution unit 132 generates character dialogue data, it may generate character dialogue data that is negative in response to user U's statements, or it may generate character dialogue data a long time after user U has made a statement. Because the conversation execution unit 132 operates in this way, the responses to user U's statements will differ depending on the character, which increases the enjoyment user U gets from conversing with various characters.

[0101] The conversation execution unit 132 may determine how often the character speaks while user U is speaking, based on the relationship between user U and the character. For example, if the relationship between user U and the character is poor, the conversation execution unit 132 will speak more frequently while user U is speaking than if the relationship is good. By operating in this way, user U can converse with the character with a feeling even more similar to talking to a person in the real world.

[0102] (Handling when multiple users U join the conversation) If multiple users U are listening to a conversation between multiple characters, there is a possibility that multiple users U may try to join the conversation. In this case, if the characters make the same statement regardless of which user U made the statement, the users U may feel that something is off.

[0103] Therefore, the data acquisition unit 131 may acquire user utterance data based on the utterances of multiple users U, identify the user U corresponding to the acquired user utterance data, and the conversation execution unit 132 may generate character utterance data for the user utterance data based on the result of the data acquisition unit 131's identification of the user U. For example, the data acquisition unit 131 identifies the user U who made the utterance based on the identification information of the information terminal 2 that transmitted the user utterance data. The conversation execution unit 132 generates character utterance data with content based on the relationship between the identified user U and the character, for example by referring to the relationship data shown in Figure 5.

[0104] The method by which the data acquisition unit 131 identifies the speaking user U is arbitrary, and the user U may be identified based on at least one of the user U's tone of voice, writing style, and voice quality that has been learned in advance. The data acquisition unit 131 may also identify the user U by acquiring face image data of the user U from the information terminal 2 and comparing the acquired face image data with the face image data of the user U that has been stored in advance. By operating in this manner, the data acquisition unit 131 and the conversation execution unit 132 enable natural conversations between multiple user Us and multiple characters, making it easier for multiple user Us to enjoy conversations with multiple characters.

[0105] (Character statements based on User U's statement history) If the content of the character's statements does not take into account user U's past statements, there is a risk that user U may end up repeating the same explanation or that the content may be undesirable to user U. Therefore, the conversation execution unit 132 may generate character statement data based on user U's past statement history.

[0106] To achieve this, the data acquisition unit 131 acquires user utterance data from multiple users U, identifies the user U that sent the acquired user utterance data, and stores the user utterance data in the storage unit 12 in association with the identified user U. The conversation execution unit 132 generates character utterance data for the user U that sent the user utterance data acquired by the data acquisition unit 131 by referring to multiple past user utterance data stored in the storage unit 12 in association with the user U that sent the user utterance data acquired by the data acquisition unit 131 immediately before.

[0107] The conversation execution unit 132 inputs, for example, multiple past user utterance data, along with the most recently acquired user utterance data, into the LLM for generating character utterance data. As an example, the conversation execution unit 132 inputs a prompt to the LLM that has a flag to distinguish between the most recently acquired user utterance data and past user utterance data. This allows the conversation execution unit 132 to generate character utterance data that takes past user utterances into consideration.

[0108] However, if a large amount of past user utterance data is stored in the storage unit 12, the storage unit 12 will require a large capacity. Also, the processing load for the conversation execution unit 132 to generate character utterance data based on the history of user utterance data will increase. Therefore, the data acquisition unit 131 may create summary data of past user utterance data stored in the storage unit 12, and the storage unit 12 may store the summary data in association with user U. In this case, the conversation execution unit 132 generates character utterance data for the user utterance data acquired by the data acquisition unit 131 immediately after by referring to the summary data. This allows the conversation execution unit 132 to efficiently generate character utterance data that takes into account user U's past utterances while suppressing the capacity of the storage unit 12.

[0109] The data acquisition unit 131 may create summary data of predetermined items based on past user utterance data. These predetermined items could be, for example, things that user U likes, dislikes, work, places they have traveled to, etc. This allows the conversation execution unit 132 to generate character utterance data by referencing the summary data of predetermined items related to the topic of the character-user conversation, thereby enabling a character-user conversation suitable for user U.

[0110] The larger the amount of past user utterance data stored in the memory unit 12, the richer the content of the summary data becomes, and the easier it is for the conversation execution unit 132 to generate character utterance data suitable for user U. Therefore, the memory unit 12 may store past user utterance data for different periods for each user U.

[0111] For example, the period for which the storage unit 12 stores user utterance data of user U who pays a higher fee for the service of character-user conversations may be longer than the period for which the storage unit 12 stores user utterance data of user U who pays a lower fee. With this configuration, user U can improve the quality of character-user conversations by paying a higher fee, so the information processing device 1 can provide character-user conversation services of a quality that meets the needs of user U.

[0112] (Estimation of user U's emotions) If a character makes a statement without considering user U's feelings, user U may become offended and unable to enjoy the conversation between the character and user. Therefore, the conversation execution unit 132 may identify the user's feelings upon receiving user statement data by inputting the user statement data acquired by the data acquisition unit 131 into a machine learning model that outputs data indicating the feelings at the time of receiving the user statement data. This machine learning model may be, for example, a model trained using training data in which a large amount of statement data is associated with the feelings corresponding to that statement data. To improve the accuracy of emotion identification, the user statement data input into the machine learning model may include information indicating the tone of voice of user U.

[0113] The conversation execution unit 132 generates character dialogue data for the user dialogue data based on the identified emotion. For example, the conversation execution unit 132 generates character dialogue data by inputting a prompt to the LLM that includes user dialogue data and data indicating the identified emotion. Specifically, if user U says in an angry tone, "Manager A's decisions are outrageous," the conversation execution unit 132 inputs a prompt to the LLM that includes information such as "Dialogue: Manager A's decisions are outrageous, Emotion: Angry." The conversation execution unit 132 then generates character dialogue data that alleviates user U's anger, such as, "That's right. They should have made a pitching change earlier."

[0114] The conversation execution unit 132 may identify the emotion felt when the user received the user's statement data, based on the relationship between the content of the topic during the character-to-character conversation and the content of the user's statement data acquired by the data acquisition unit 131, and generate character statement data in response to the user's statement data based on the identified emotion. For example, the conversation execution unit 132 identifies the emotion of user U corresponding to the content of the topic during the character-to-character conversation and the content of the user's statement data by referring to emotion estimation data that associates the content of the topic, the content of user U's statement, and user U's emotion. By operating in this manner, the conversation execution unit 132 can hear statements from the character that are appropriate to its own emotion, thereby increasing user U's satisfaction with the character-to-user conversation.

[0115] The image data generation unit 133 may determine a character's face image based on the emotions of user U identified by the conversation execution unit 132. To do this, the conversation execution unit 132 stores a tag indicating the identified emotion in the storage unit 12, associated with the user utterance data acquired by the data acquisition unit 131. The image data generation unit 133 generates face image data corresponding to the emotion indicated by the tag by referring to the tag stored in the storage unit 12. The image data generation unit 133 generates face image data having the characteristics corresponding to the emotion indicated by the tag by referring to image generation data that associates the emotion with the characteristics of the face image to be generated.

[0116] The image data generation unit 133, for example, if the identified user U's emotion is "angry," generates face image data with a calm expression to help user U calm down. If the identified user U's emotion is "depressed," the image data generation unit 133 generates face image data with a cheerful expression to help user U become more energetic. The conversation execution unit 132 may generate character speech data with a tone or content that corresponds to the characteristics of the face image data generated by the image data generation unit 133. By having the conversation execution unit 132 and the image data generation unit 133 operate in this way, user U can see a character's face image with an expression that matches their emotion and hear character speech with content that matches their emotion, thus increasing user U's satisfaction with the character-user conversation.

[0117] [Processing flow in Information Processing Device 1] Figure 6 is a flowchart showing an example of the processing flow in the information processing device 1. The flowchart in Figure 6 shows the processing flow when a user makes a statement while a conversation between characters is taking place (S11), and the data acquisition unit 131 acquires the user's statement data (S12).

[0118] The conversation execution unit 132 determines whether the conditions for ignoring the user statement data are met, based on the relationship between the topic of the character-to-character conversation and the content of the user statement data, or the timing at which the data acquisition unit 131 acquired the user statement data (S13). If the conversation execution unit 132 determines that the conditions for ignoring the user statement data are met (YES in S13), it returns to S11 and continues the character-to-character conversation without proceeding to the character-to-user conversation.

[0119] If the conversation execution unit 132 determines that the conditions for ignoring user utterance data are not met (NO in S13), it stops the character-to-character conversation and transitions to a character-to-user conversation (S14). Subsequently, the conversation execution unit 132 determines whether the destination of the user utterance data is clear or not (S15).

[0120] If the conversation execution unit 132 determines that the destination of the user's utterance data is clear (YES in S15), it generates character utterance data for the character corresponding to the destination (S16). If the conversation execution unit 132 determines that the destination of the user's utterance data is unclear (NO in S15), it generates character utterance data for a character selected based on the character's personality indicated by the character data (S17). The output unit 134 transmits the generated character utterance data to the information terminal 2 (S18).

[0121] The conversation execution unit 132 determines whether the data acquisition unit 131 has acquired user utterance data within a predetermined time period after the output unit 134 transmits character utterance data (S19). If the conversation execution unit 132 determines that user utterance data has been acquired (YES in S19), it repeats the processes from S13 to S18. If the predetermined time period has elapsed without the data acquisition unit 131 acquiring user utterance data (NO in S19), the conversation execution unit 132 determines whether the data acquisition unit 131 has received an instruction to end the conversation from the information terminal 2 (S20).

[0122] If the data acquisition unit 131 has not received an instruction to end the conversation (NO in S20), the conversation execution unit 132 repeats the processes from S11 to S19. If the data acquisition unit 131 receives an instruction to end the conversation from the information terminal 2 (YES in S20), the conversation execution unit 132 terminates the generation of the spoken data.

[0123] [Effects of Information Processing Device 1] As explained above, the conversation execution unit 132 executes character-user conversations between multiple characters and the user in a manner based on the user utterance data acquired by the data acquisition unit 131, based on the status of the conversations between characters. As a result, the conversation execution unit 132 can appropriately respond to messages made by user U while multiple characters are conversing, and generate character utterance data that makes user U feel as if they are conversing with a real person. Consequently, user U's satisfaction with the conversations with multiple characters increases.

[0124] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of its gist. For example, all or part of the apparatus can be configured by functionally or physically distributing and integrating in any unit. Furthermore, new embodiments resulting from any combination of multiple embodiments are also included in the embodiments of the present invention. The effects of the new embodiments resulting from the combinations are combined with the effects of the original embodiments. [Explanation of symbols]

[0125] 1. Information Processing Device 2 Information terminals 11 Communications Department 12 Storage section 13 Control Unit 131 Data Acquisition Unit 132 Conversation Execution Unit 133 Image Data Generation Unit 134 Output section

Claims

1. An information processing device, A conversation execution unit that executes a program to generate character dialogue data and enables character-to-character conversations between multiple characters, An output unit that outputs the character dialogue data showing the content of the conversation between the characters, After the output unit outputs the character speech data, the data acquisition unit acquires user speech data indicating speech made by a user using the service provided by the information processing device. It has, The conversation execution unit, based on the status of the character-to-character conversation, executes a character-to-user conversation between the multiple characters and the user in a manner based on the user utterance data acquired by the data acquisition unit. Information processing device.

2. The conversation execution unit identifies which of the multiple characters the user utterance data is addressed to based on the content of the user utterance data acquired by the data acquisition unit, and generates character utterance data to be spoken by the identified character. The information processing apparatus according to claim 1.

3. It further has a memory unit that stores the personalities of each of the aforementioned multiple characters, The conversation execution unit generates character speech data to be spoken by the selected character based on the personality of each of the multiple characters. The information processing apparatus according to claim 1.

4. The aforementioned character dialogue data and user dialogue data are audio data. The conversation execution unit stops the character-to-character conversation if the data acquisition unit begins acquiring the user utterance data while the character-to-character conversation is taking place. The information processing apparatus according to any one of claims 1 to 3.

5. The conversation execution unit does not execute the character-user conversation while it is executing the character-user conversation, and when the data acquisition unit acquires the user utterance data while the character-user conversation is not being executed, it executes the character-user conversation. The information processing apparatus according to any one of claims 1 to 3.

6. The conversation execution unit, while executing the character-to-character conversation, determines, based on the content of the character-to-character conversation, whether or not it is permissible to execute the character-to-user conversation, and if it is determined that it is permissible to execute the character-to-user conversation, the data acquisition unit will execute the character-to-user conversation if it acquires the user utterance data during that period. The information processing apparatus according to any one of claims 1 to 3.

7. When the conversation execution unit is executing the conversation between characters and the data acquisition unit acquires the user utterance data, it generates character utterance data for at least one of the multiple characters that includes a predetermined word stored in the storage unit beforehand. The information processing apparatus according to any one of claims 1 to 3.

8. The conversation execution unit controls the content of the character utterance data or the timing of character utterance data spoken by the multiple characters in the character-user conversation after the data acquisition unit has acquired the user utterance data, based on at least one of the words contained in the user utterance data, the context of the user utterance data, the intonation of the user utterance data, the emotion of the user estimated from the user utterance data, the speaking speed of the user utterance data, and the time from when the previous user utterance data was spoken until when the latest user utterance data was spoken. The information processing apparatus according to any one of claims 1 to 3.

9. If the conversation execution unit determines that a response to the user utterance data acquired by the data acquisition unit is unnecessary, it will not execute the character-user conversation and will continue the character-user conversation. The information processing apparatus according to claim 8.

10. The aforementioned character dialogue data and user dialogue data are audio data. The conversation execution unit will not execute the character-user conversation if the volume of the user's voice, as indicated by the user's utterance data, is below a threshold. The information processing apparatus according to claim 9.

11. The system further includes a storage unit that stores relationship data indicating the relationship between at least one of the aforementioned multiple characters and the user. The conversation execution unit controls, based on the relational data, at least one of the following after the data acquisition unit acquires the user utterance data: the content of the character utterance data spoken by the multiple characters in the character-user conversation, whether or not any of the multiple characters respond, and the timing at which the multiple characters utter the character utterance data. The information processing apparatus according to any one of claims 1 to 3.

12. The data acquisition unit acquires user statement data based on statements made by multiple users, identifies the user corresponding to the acquired user statement data, The conversation execution unit generates character speech data for the user speech data based on the results of the data acquisition unit identifying the user. The information processing apparatus according to any one of claims 1 to 3.

13. The conversation execution unit pauses the character conversation for a predetermined period of time if the content of the character conversation or the content of the user speech data acquired by the data acquisition unit satisfies predetermined conditions. The information processing apparatus according to any one of claims 1 to 3.

14. The conversation execution unit pauses the speech of each of the plurality of characters for the predetermined time period associated with each of the characters. The information processing apparatus according to claim 13.

15. The conversation execution unit, when the conversation between characters or the user speech data acquired by the data acquisition unit satisfies predetermined conditions, causes one of the multiple characters to speak alone for a predetermined period of time or longer. The information processing apparatus according to any one of claims 1 to 3.

16. The conversation execution unit, when the user utterance data acquired by the data acquisition unit during the conversation between the character users satisfies predetermined conditions, causes at least one of the plurality of characters to utter during the user's utterance. The information processing apparatus according to any one of claims 1 to 3.

17. The data acquisition unit acquires user utterance data from multiple users, identifies the user who sent the acquired user utterance data, and stores the user utterance data in the storage unit in association with the identified user. The conversation execution unit generates character statement data for the user who made the user statement data acquired by the data acquisition unit immediately before, by referring to a plurality of past user statement data stored in the storage unit in association with the user who made the user statement data acquired by the data acquisition unit immediately before. The information processing apparatus according to any one of claims 1 to 3.

18. The data acquisition unit creates summary data of past user statements stored in the storage unit. The conversation execution unit generates character utterance data corresponding to the user utterance data acquired immediately before by the data acquisition unit by referring to the summary data. The information processing apparatus according to claim 17.

19. When the conversation execution unit receives the user utterance data, it inputs the user utterance data acquired by the data acquisition unit into a machine learning model that outputs data indicating the emotion at the time of receiving the user utterance data, thereby identifying the emotion at the time of receiving the user utterance data, and generating character utterance data for the user utterance data based on the identified emotion. The information processing apparatus according to claim 1.

20. The conversation execution unit identifies the emotion felt when the user statement data was received, based on the relationship between the content of the topic during the conversation between the characters and the content of the user statement data acquired by the data acquisition unit, and generates character statement data in response to the user statement data based on the identified emotion. The information processing apparatus according to claim 1.

21. The system further includes an image data generation unit that generates face image data of the aforementioned multiple characters, The conversation execution unit stores a tag indicating a specific emotion in the memory unit, in association with the user utterance data acquired by the data acquisition unit. The image data generation unit generates the face image data corresponding to the emotion indicated by the tag by referring to the tag stored in the storage unit. The information processing apparatus according to claim 19 or 20.

22. Computers The steps include generating character dialogue data to enable character-to-character conversations between multiple characters, and The steps include: outputting the character dialogue data that shows the content of the conversation between the characters; After outputting the aforementioned character dialogue data, the step is to obtain user dialogue data that represents the user's dialogue. Based on the status of the character-to-character conversation, the step of performing a character-to-user conversation between the multiple characters and the user in a manner based on the acquired user utterance data, An information processing method having

23. On the computer, The steps include generating character dialogue data to enable character-to-character conversations between multiple characters, and The steps include: outputting the character dialogue data that shows the content of the conversation between the characters; After outputting the aforementioned character dialogue data, the step is to obtain user dialogue data that represents the user's dialogue. Based on the status of the character-to-character conversation, the step of performing a character-to-user conversation between the multiple characters and the user in a manner based on the acquired user utterance data, A program to execute.

24. The system comprises an information processing device and an information terminal used by a user who utilizes the services provided by the information processing device, The aforementioned information processing device is A conversation execution unit that executes a program to generate character dialogue data and enables character-to-character conversations between multiple characters, An output unit that outputs the character dialogue data showing the content of the conversation between the characters, After the output unit outputs the character speech data, a data acquisition unit acquires user speech data indicating the user's speech from the information terminal. It has, The conversation execution unit, based on the status of the character-to-character conversation, executes a character-to-user conversation between the multiple characters and the user in a manner based on the user utterance data acquired by the data acquisition unit. The information terminal transmits the input user speech data to the information processing device and outputs the character speech data received from the information processing device. Information processing system.

Images