Response device, response method, and response program

Abstract

To facilitate timely responses to user conversations. [Solution] If the processing based on the first large-scale language model that generates a response in response to the conversation input by the user is not yet complete, the response device 100 inputs a prompt, expressed in natural language text, to the second large-scale language model, which is a command to estimate the user's emotions based on the input conversation, and estimates the user's emotions. Based on the estimated user emotions, the response device 100 generates a response to the user who input the conversation based on a predetermined model.

Description

Technical Field

[0001] The present invention relates to a response device, a response method, and a response program.

Background Art

[0002] There is known a technique for generating a response to a user's conversation input using generative AI (Artificial Intelligence) such as a large language model. For example, there is known a conventional technique for generating the response content of a dialogue agent based on the user's language information and the user's non-verbal information using a large language model (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the prior art, it may be difficult to make a timely response to a user's conversation. For example, the prior art uses natural language processing by a large language model to generate the response content. However, the large language model used by the prior art may have a waiting time such as a processing wait, and there is a problem in generating a response to a user's conversation in real time.

Means for Solving the Problems

[0005] Therefore, in order to solve the above-mentioned problems and achieve the objective, the response device of the present invention is characterized by having an estimation unit that, when processing based on a first large-scale language model that generates a response corresponding to a conversation input by a user is incomplete, inputs a prompt expressed in natural language text that instructs the input conversation to estimate the user's emotions, to a second large-scale language model to estimate the user's emotions, and a generation unit that generates a response to the user who input the conversation based on a predetermined model, based on the user's emotions estimated by the estimation unit. [Effects of the Invention]

[0006] The present invention has the effect of facilitating timely responses to user conversations. [Brief explanation of the drawing]

[0007] [Figure 1] Figure 1 is a diagram illustrating the overall processing of the response device according to the embodiment. [Figure 2] Figure 2 shows the configuration of the response device according to this embodiment. [Figure 3] Figure 3 is a table diagram showing an example of response conditions according to the embodiment. [Figure 4] Figure 4 is a table diagram showing an example of a response history according to the embodiment. [Figure 5] Figure 5 shows an example of the response processing according to this embodiment. [Figure 6] Figure 6 shows an example of the output of the response according to the present invention. [Figure 7] Figure 7 is a flowchart showing the processing performed by the response device according to this embodiment. [Figure 8] Figure 8 shows an example of the effects according to the embodiment. [Figure 9] Figure 9 shows an example of a computer that implements the response device according to this embodiment. [Modes for carrying out the invention]

[0008] Hereinafter, embodiments for carrying out the present invention (hereinafter referred to as "embodiments") will be described with reference to the drawings. However, each embodiment is not limited to those described below.

[0009] <Overview> (background) Technologies that use large-scale language models to automatically execute dialogues in response to user conversations are known. For example, a reference technology is known that generates the content of a dialogue agent's response based on the user's linguistic and non-linguistic information.

[0010] However, the referenced technology may experience processing delays, posing a challenge to generating real-time responses to user conversations. Therefore, timely responses to user conversations may be difficult with the referenced technology.

[0011] (Processing by response device 100) Therefore, in this embodiment, while waiting for the completion of the response generation process for the user's conversation based on the first large-scale language model, the response device 100 performs a primary response such as nodding or expressing emotion according to the user's emotions estimated using the second large-scale language model.

[0012] Note that the "first large language model" in this embodiment is a generation model that generates an "answer" corresponding to a conversation including the content of questions, problem-solving, requests, etc. input by the user, and may hereinafter be referred to as "LLM (Large Language Models) for answer generation". Also, the "second large language model" in this embodiment is a generation model that estimates the emotion of the user using the conversation content input by the user, and may hereinafter be referred to as "LLM for emotion estimation". Note that the "LLM for answer generation" and the "LLM for emotion estimation" may be realized by different LLMs respectively, or may be realized by inputting different prompts to the same LLM. Also, the "conversation" is a speech in natural language input by the user, and is the content of the user's speech including the user's questions, inquiries, and information generation commands.

[0013] Here, an overall picture of the processing by the response device 100 will be described. FIG. 1 is a diagram for explaining an overall picture of the processing of the response device 100 according to the embodiment. The response device 100 shown in FIG. 1 is an example of a computer that provides a technology for realizing the information processing described below.

[0014] First, the response device 100 receives a conversation input by the user (FIG. 1 (1-1)). Then, the response device 100 inputs the received user conversation to an LLM for answer generation used to generate an answer to the conversation (FIG. 1 (1-2)).

[0015] Next, the response device 100 determines whether or not the generation process of the answer based on the LLM for answer generation is completed (FIG. 1 (2)). Here, when the generation process based on the LLM for answer generation is completed, the response device 100 outputs the generated answer to the user (not shown in FIG. 1).

[0016] When the process based on the LLM for answer generation that generates an answer in response to the conversation input by the other party or the user is incomplete, the response device 100 inputs a prompt in which a command for estimating the user's emotion based on the input conversation is expressed in text in natural language to the LLM for emotion estimation, and estimates the user's emotion ( (3) in FIG. 1).

[0017] Based on the estimated emotion of the user ( (4-1) in FIG. 1), the response device 100 generates a response (such as "echo", "emotion expression", "repetition", "filler", "fixed-form response", etc.) to the user who input the conversation based on a predetermined model ( (4-2) in FIG. 1). Then, the response device 100 outputs the generated responses such as "echo", "emotion expression", "repetition", "filler", "fixed-form response", etc. to the user ( (5) in FIG. 1).

[0018] In this way, until an answer to the input of a conversation including the user's question or inquiry is generated, the response device 100 according to the present embodiment can generate a primary response according to the conversation and output it to the user. As a result, the response device 100 has the effect of facilitating a timely response to the user's conversation.

[0019] <Description of the response device 100> Hereinafter, the configuration of the response device 100 according to the present embodiment will be described. FIG. 2 is a diagram showing the configuration of the response device 100 according to the embodiment. As shown in FIG. 2, the response device 100 includes a communication unit​​​​​​​​​The communication unit 110 performs data communication related to the input of information about conversations entered by the user via a terminal device, etc. The communication unit 110 also performs data communication related to the output of generated response information and answer information. The communication unit 110 is implemented using a NIC (Network Interface Card), etc., and controls communication via telecommunication lines such as a LAN (Local Area Network) or the Internet. Furthermore, the communication unit 110 can be connected to the network via wired or wireless connection as needed, and can send and receive information bidirectionally with terminal devices operated by the user.

[0022] (Storage unit 120) The storage unit 120 stores data and programs used for various processes performed by the control unit 130, as well as various data acquired through the operation of the control unit 130. The storage unit 120 is implemented using semiconductor memory elements such as RAM (Random Access Memory) and flash memory, or storage devices such as hard disks and optical discs. As shown in Figure 2, the storage unit 120 also includes a response condition DB 121, a response history DB 122, and a model DB 123.

[0023] (Response condition DB121) The response condition DB121 is a database that stores the selection criteria (response conditions) for the response generation process in response to conversations entered by the user. Specifically, the response condition DB121 stores the type of response method and the conditions for selecting the response method.

[0024] Here, an example of a response condition stored in the response condition DB121 will be explained using Figure 3. Figure 3 is a table diagram showing an example of a response condition according to the embodiment. As shown in Figure 3, the response condition DB121 stores information related to each item, such as "No," which is information that identifies the individual data included in the response condition, "response method," and "condition," in a table format or the like.

[0025] "Response method" refers to information that identifies the response method selected in response to the user's conversation, and includes information such as "acknowledgment," "expression of emotion," "repetition," "addition of filler," and "standard response." "Conditions" refers to the conditions for selecting a response method, and includes information that defines which response method will be selected based on the user's conversation, emotions, response situation, etc.

[0026] For example, as shown in Figure 3, the response condition DB121 stores the response method "acknowledgment" and the condition "user speaks" as response conditions, identified by No. "1". The response condition identified by No. "1" above means "when the user is speaking, generate and output acknowledgments such as 'yes' or 'I understand' in response to the utterance."

[0027] Furthermore, the response condition DB121 stores the response method "emotional expression" and the condition "user's emotion changes," identified by No. "2," as response conditions. The response condition identified by No. "2" means that "when the user's emotion estimated by the estimation unit 133 changes, the avatar's emotional expression is changed to match the user's emotion."

[0028] Furthermore, the response condition DB121 stores the response method "repetition" and the condition "user utterance completed" as response conditions, identified by No. "3". The response condition identified by No. "3" means that "when the user's utterance is completed or has reached a conclusion, the content of the utterance will be summarized into a short message (for example, within 100 characters) and repeated to the user."

[0029] Furthermore, the response condition DB121 stores the response method "add filler" and the condition "a predetermined amount of time is required to respond" as response conditions, identified by No. "4". The response condition identified by No. "4" above means that "if a predetermined amount of time (for example, 1 second or more) is required to respond to a user's utterance, a filler such as 'Um' or 'Please wait a moment' will be output."

[0030] Furthermore, the response condition DB121 stores the response method "Standard response" and the condition "User's utterance or emotion satisfies a predetermined condition" as response conditions, identified by No. "5". The response condition identified by No. "5" above means "outputting standard questions, responses, etc., according to the user's utterance or emotion."

[0031] (Response history DB122) The response history DB 122 is a database that stores information (response history) regarding the response history of the response device 100 to the user. Specifically, the response history DB 122 stores information such as information that identifies the user (user identification information) and information regarding the response history for that user (response history).

[0032] Here, an example of a response history stored in the response history DB122 will be explained using Figure 4. Figure 4 is a table diagram showing an example of a response history according to the embodiment. As shown in Figure 4, the response history DB122 stores information related to each item, such as "No," which is information that identifies individual data included in the response history, "user identification information," and "response history," in a table format or the like.

[0033] For example, as shown in Figure 4, the response history DB122 stores the user identification information "A," identified by No. "1," in association with the response history "B." The letters "A" and "B" listed for each item in the table diagram shown in Figure 4 are legends for the information contained in each item.

[0034] "User identification information" is information that identifies the user who is having a conversation with the response device 100, and includes information that is expressed by a predetermined combination of strings, numbers, symbols, etc. Note that information that could identify the individual user may be deleted or replaced in the user identification information using known technology. "Response history" includes information such as the history of conversations between the response device 100 and the user, and what kind of responses the response device 100 gave to the user.

[0035] (Model DB123) Model DB123 is a database that stores predetermined models used to generate responses. For example, Model DB123 can store speech analysis AI (Artificial Intelligence) trained to generate interjections in response to conversations with users, large-scale language models (LLM for emotion estimation) that estimate user emotions and generate emotion information, large-scale language models for generating summaries of user conversations, motion AI that changes the emotional expression of an avatar based on the avatar's emotion information, and so on.

[0036] The speech analysis AI, large-scale language models, motion AI, etc., mentioned above may be models based on publicly known technologies. For example, the large-scale language model can be either "ChatGPT®," a large-scale language model with general-purpose knowledge, or "tsuzumi®," a predetermined large-scale language model that undergoes adapter tuning (see, for example, references 1 and 2).

[0037] (Reference 1):ChatGPT(OpenAI),<URL:https: / / openai.com / chatgpt> ,<Searched on August 25, 2020> (Reference 2): NTT version of large-scale language model "tsuzumi",<URL:https: / / www.rd.ntt / research / LLM_tsuzumi.html> ,<Searched on August 25, 2020>

[0038] (Control unit 130) Now, let's return to Figure 2 and continue the explanation. The control unit 130 has an internal memory for temporarily storing programs and processing data that define various processing procedures of the response device 100, and is realized by electronic circuits such as a CPU (Central Processing Unit) and an MPU (Micro Processing Unit), and integrated circuits such as an ASIC (Application Specific Integrated Circuit) and an FPGA (Field Programmable Gate Array). As shown in Figure 2, the control unit 130 has a reception unit 131, a determination unit 132, an estimation unit 133, a generation unit 134, a storage unit 135, and an output unit 136.

[0039] (Reception desk 131) The reception unit 131 receives information (conversation, utterances) related to natural language conversations uttered by the user via a terminal device operated by the user. The reception unit 131 can also receive information related to conversations, such as the user's voice information, voice information and video information, and text information obtained by converting voice to text.

[0040] (Judgment unit 132) The determination unit 132 performs predetermined determination processing in the processing performed by the response device 100. Specifically, the determination unit 132 determines whether or not the response generation process based on the response generation LLM has been completed. The determination unit 132 also determines whether or not there is insufficient information to perform the generation process using the response generation LLM. An example of the determination processing performed by the determination unit 132 will be explained in the section on "Examples of Processing" described later.

[0041] (Estimation part 133) The estimation unit 133 inputs a prompt to the emotion estimation LLM containing a command to estimate the user's emotion based on the conversation input by the user, and estimates the user's emotion. For example, the estimation unit 133 estimates the user's emotion "confusion" from conversation content such as, "Actually, I'm having trouble with XX and I don't know what to do." The estimation unit 133 can perform the estimation of the user's emotion based on a large-scale language model using, for example, the well-known technology "tsuzumi(registered trademark)" (see Reference 2 above).

[0042] (Generation unit 134) The generation unit 134 generates a response based on a predetermined model that corresponds to the user's emotions estimated by the estimation unit 133 and the conversation input by the user. An example of response generation by the generation unit 134 will be explained in the "Example of Processing" section below.

[0043] (Storage section 135) The storage unit 135 associates the response generated by the generation unit 134 with information identifying the user to whom the response is output (user identification information) and stores it in the response history DB 122 as a response history.

[0044] (Output section 136) The output unit 136 outputs the response generated by the generation unit 134 to the terminal device of the user engaging in the conversation. An example of the response output by the output unit 136 will be explained in the section on "Example of Processing" below.

[0045] (An example of processing) From here, an example of processing by the response device 100 will be explained using Figure 5. Figure 5 is a diagram showing an example of response processing according to this embodiment.

[0046] First, the response device 100 (reception unit) receives the user's conversation based on chat, video call, voice call, etc. (Figure 5 (1-1)) (Figure 5 (1-2)). Here, the response device 100 inputs a prompt to the response generation LLM that includes a command to generate a response about the received user conversation (Figure 5 (2)).

[0047] The response device 100 (determination unit) makes a determination regarding the progress of the response generation process based on the response generation LLM (Figure 5 (3-1)). For example, if the response device 100 is in a status such as waiting for prompt input to the response generation LLM or waiting for the response generation process in the response generation LLM to be completed, the response device 100 (determination unit) determines that "the process based on the response generation LLM that generates a response corresponding to the conversation input by the user is incomplete" (Figure 5 (3-2)).

[0048] If the above determination process determines that "the process is incomplete" (Figure 5 (3-2)), the response device 100 (estimation unit) inputs a prompt to the emotion estimation LLM to estimate the user's emotions using the conversation input by the user, and estimates the user's emotions based on that conversation (Figure 5 (4-1) and (4-2)).

[0049] On the other hand, if the response device 100 determines that "processing is complete" in the above determination process, it outputs the generated response to the user (not shown in Figure 5).

[0050] Next, the response device 100 (generation unit) performs processing to generate responses that correspond to the user's conversation, such as "generating interjections," "generating avatar emotions," "generating repetitions," "generating fillers," and "selecting standard responses," based on the estimated user's emotions and the input conversation (Figure 5(5)). Then, the response device 100 (output unit) outputs the generated responses to the user (Figure 5(6)).

[0051] Specifically, the response device 100 (generation unit) inputs the estimated user's emotions and the conversation to a predetermined speech analysis AI model that has been trained to generate interjections corresponding to the conversation, and generates interjections that correspond to the conversation (Figure 5 (5-1)). Then, the response device 100 (output unit) outputs the generated interjections according to the user's conversation (Figure 5 (6-1)).

[0052] Furthermore, the response device 100 (generation unit) inputs a prompt to a predetermined model, a large-scale language model, which includes a command to generate avatar emotion information corresponding to the user's emotions, based on the estimated user's emotions and the conversation, and generates avatar emotion information corresponding to the conversation (Figure 5 (5-2)). Then, the response device 100 (output unit) changes and displays the avatar's emotion expression to the user based on the generated avatar emotion information (Figure 5 (6-2)).

[0053] Furthermore, the response device 100 (generation unit) inputs a prompt, which includes a command to generate a summary of the conversation based on the estimated user's emotions and the conversation, to a predetermined model, a large-scale language model, and generates a summary of the conversation (Figure 5 (5-3)). The response device 100 (output unit) outputs the generated summary of the conversation as a repetition to the user (Figure 5 (6-3)).

[0054] Furthermore, the response device 100 (generation unit) generates fillers as responses that correspond to the estimated user's emotions and conversation (Figure 5 (5-4)). For example, the response device 100 (generation unit) generates fillers as responses that correspond to the estimated user's emotions and conversation, based on a pre-set correspondence between user emotions and conversation and fillers. Then, the response device 100 (output unit) outputs the generated fillers according to the user's conversation (Figure 5 (6-4)).

[0055] The response device 100 (generation unit) selects a target standard phrase based on the estimated user's emotions and the pre-set standard phrases corresponding to the conversation, and generates a response (Figure 5 (5-5)). Then, the response device 100 (output unit) outputs the response using the generated standard phrase according to the user's conversation (Figure 5 (6-5)).

[0056] On the other hand, if the response device 100 (determination unit) determines that there is insufficient information to generate a response based on the response generation LLM (Figure 5 (7-1)), the response device 100 (output unit) displays text or an avatar to the user informing them that there is insufficient information (Figure 5 (7-2)).

[0057] Here, an example of the response output processing by the response device 100 described above will be explained using Figure 6. Figure 6 is a diagram showing an example of the response output according to the embodiment. Figure 6 shows the utterance by the user (Figure 6(1)) and the utterance by the response device 100 (Figure 6(2)). Although Figure 6 shows an example of output in chat format, other formats such as telephone, video call, or email may also be used.

[0058] First, the response device 100 receives an utterance from the user, such as "I would like to ask about XX" (Figure 6 (1-1)). Then, the response device 100 generates and outputs an acknowledgment such as "Yes" in response to the utterance in Figure 6 (1-1) (Figure 6 (2-1)).

[0059] Next, the response device 100 receives the user's utterance, "Actually, I'm having trouble with XX and I don't know what to do" (Figure 6 (1-2)). At this point, the response device 100 changes the avatar's emotion to "empathy" or the like, in accordance with the user's emotion "confusion" estimated from the utterance in Figure 6 (1-2) (Figure 6 (2-2)). Then, the response device 100 generates and outputs an acknowledgment such as "I see, that's terrible" (Figure 6 (2-3)).

[0060] Next, before generating a response corresponding to the conversation input by the user, the response device 100 repeats the input content to the user. At this time, for example, if it takes a predetermined amount of time to generate the repetition, the response device 100 generates and outputs a filler such as "Um..." (Figure 6 (2-4)). Then, when the generation of the repetition is complete, the response device 100 outputs a repetition such as "Is it correct that the problem you are having is that you don't know what to do in the case of XX?" (Figure 6 (2-5)).

[0061] Next, the response device 100 receives the user's input "Yes, that's correct." in response to the response output in Figure 6 (2-5) (Figure 6 (1-3)). Then, the response device 100 changes the avatar's emotion to "smiling" or the like in order to alleviate the estimated user emotion of "confusion" (Figure 6 (2-6)). Then, the response device 100 generates and outputs an acknowledgment such as "Understood. I will look into XX, so please wait patiently" (Figure 6 (2-7)).

[0062] Furthermore, if generating a response takes time, the response device 100 can generate and output small talk such as, "By the way, it's been really hot lately..." (Figure 6 (2-8)). At this time, the response device 100 can also change the avatar's emotion to "confused" or similar depending on the content of the small talk (Figure 6 (2-9)).

[0063] Then, once the response device 100 has finished generating the response, it changes the avatar's emotion to "smiling" or the like (Figure 6 (2-10)) and outputs a response such as "Sorry to keep you waiting. I have looked into XX. XX is..." (Figure 6 (2-11)).

[0064] Furthermore, the timing of "acknowledgments," "fillers," etc., in the conversation between the response device 100 and the user is not limited to the timing shown in Figure 6, but will be output by the response device 100 at an appropriate timing in response to the user's conversation.

[0065] (Procedure for processing by the response device 100) Next, the processing procedure implemented by the response device 100 according to this embodiment will be explained with reference to Figure 7. Figure 7 is a flowchart showing the processing performed by the response device 100 according to this embodiment.

[0066] The reception unit 131 receives a conversation from the user (S101). Next, the response device 100 executes the response generation process based on the response generation LLM (S102).

[0067] If generating a response takes a predetermined amount of time (Yes in S103), the estimation unit 133 estimates the user's emotion based on the emotion estimation LLM (S104). Next, the generation unit 134 generates a response based on the input conversation and the estimated emotion (S105). Then, the output unit 136 outputs the generated response (S106). Finally, the response device 100 terminates processing.

[0068] On the other hand, if the generation of the answer does not take the predetermined time (No. in S103), the output unit 136 outputs the answer generated based on the answer generation LLM (S107). Then, the response device 100 terminates processing.

[0069] (effect) Next, we will explain the effects of the response device 100 according to this embodiment. Conventionally, when using a large-scale language model to automatically execute a dialogue in response to a user's conversation, there may be waiting times such as processing delays, making it difficult to respond to the user's conversation in a timely manner.

[0070] Therefore, in the response device 100 according to this embodiment, if the processing based on the response generation LLM that generates a response in response to the conversation input by the user is not yet complete, the estimation unit 133 inputs a prompt expressed in natural language text, which is a command to estimate the user's emotions based on the input conversation, to the emotion estimation LLM, and estimates the user's emotions. Based on the user's emotions estimated by the estimation unit 133, the generation unit 134 of the response device 100 generates a response to the user who input the conversation based on a predetermined model.

[0071] Here, the effects realized by the response device 100 according to this embodiment will be explained using Figure 8. Figure 8 is a diagram showing an example of the effects according to this embodiment. Figure 8 is a diagram showing a comparison of the response between the conventional technology and the response device 100 according to this embodiment.

[0072] For example, in the conventional technology, when a user inputs "I'm having trouble with XX and I don't know what to do," no response is given, which can cause the user to feel anxious, wondering, "There's no response, but is my message getting through...?" (see "During Speech" in the "Conventional Technology" section). In comparison to the above, the response device 100 can alleviate the user's anxiety by providing affirmative responses such as "Yes," "I see," and "That's terrible," or by using predefined phrases in response to the user's input (see "During Speech" in the "Response Device 100 According to This Embodiment" section).

[0073] Furthermore, in conventional technology, no response is made when generating an answer to user input, which can cause users to feel anxious or dissatisfied, such as "the response is slow..." (see "Waiting" section in "Conventional Technology"). In comparison to the above, the response device 100 can alleviate user anxiety by repeating the user's input, such as "You asked about XX, so please wait a moment while I check." (see "Waiting" section in "Response Device 100 According to This Embodiment").

[0074] Furthermore, in conventional technology, even if a response to the user's input is generated without problems, if the user feels anxious or dissatisfied during the processing, they may feel dissatisfied, saying things like, "I was kept waiting for so long..." (see "During Response" in the "Conventional Technology" section). In comparison to the above, the response device 100 provides an appropriate response to the user's input, which has the effect of making it easier for the user to feel satisfied with the response, saying things like, "They answered me properly!" (see "During Response" in the "Response Device 100 According to This Embodiment" section).

[0075] Therefore, the response device 100 according to this embodiment has the effect of increasing user satisfaction by facilitating timely responses to user conversations. Furthermore, the response device 100 according to this embodiment achieves predetermined effects by executing the processes described below.

[0076] The generation unit 134 inputs the user's emotions estimated by the estimation unit 133 and the conversation to a predetermined model (speech analysis AI) that has been trained to generate interjections corresponding to the conversation, and generates interjections corresponding to the conversation as a response.

[0077] Through the processing described above, the response device 100 can output appropriate responses in accordance with the user's conversation and the user's emotions based on that conversation. As a result, the response device 100 reduces user discomfort by informing the user that the response device 100 is processing appropriately during the waiting time after the user issues a command to generate a response.

[0078] The generation unit 134 inputs a prompt to a predetermined model (large-scale language model) that includes a command to generate avatar emotion information corresponding to the user's emotions, based on the user's emotions estimated by the estimation unit 133 and the conversation, and generates avatar emotion information corresponding to the conversation as a response. The output unit 136 changes and displays the avatar's emotion expression to the user based on the avatar emotion information generated by the generation unit 134.

[0079] Through the process described above, the response device 100 can appropriately change the emotional expression of its avatar in accordance with the user's conversation and the user's emotions based on that conversation. As a result, the response device 100 can provide an appropriate response in line with the user's emotions during the waiting time after the user issues a command to generate a response.

[0080] The generation unit 134 inputs a prompt to a predetermined model (large-scale language model) that includes a command to generate a summary of the conversation, based on the user's emotions estimated by the estimation unit 133 and the conversation, and generates a summary of the conversation.

[0081] Through the processing described above, the response device 100 can appropriately repeat the user's emotions and a summary of the user's conversation. As a result, the response device 100 reduces user discomfort by informing the user that the response device 100 is processing the information appropriately during the waiting time after the user issues a command to generate a response.

[0082] The generation unit 134 generates a filler response corresponding to the user's emotions estimated by the estimation unit 133 and the conversation. Through the above-described process, the response device 100 outputs an appropriate filler when outputting a response according to the user's conversation and the user's emotions based on the conversation, thereby indicating that the response device 100 is processing appropriately. As a result, the response device 100 has the effect of reducing the user's discomfort.

[0083] The generation unit 134 generates a response based on the user's emotions estimated by the estimation unit 133 and pre-set standard phrases corresponding to the conversation. Through the above process, the response device 100 can output a response using standard phrases according to the user's conversation and the user's emotions based on that conversation. As a result, the response device 100 can quickly inform the user that it is processing appropriately using pre-registered standard phrases. Consequently, the response device 100 has the effect of reducing user discomfort.

[0084] The determination unit 132 determines whether there is insufficient information for the LLM (Letter Generation Machine) to perform the generation process for the answer. If the output unit 136 determines that there is insufficient information, it displays text or an avatar to the user informing them that there is insufficient information.

[0085] Through the processing described above, the response device 100 can appropriately inform the user that information is missing when generating an answer based on the answer generation LLM. As a result, the response device 100 can improve the accuracy of the answer by having the user input the missing information in advance before generating the answer, and can prevent the generation of an answer that the user did not expect. Furthermore, the response device 100 can reduce the discomfort the user feels by changing the emotional expression of the avatar to indicate that information is missing.

[0086] <Variation> The following describes modifications implemented by the response device 100 according to this embodiment.

[0087] (Data, etc.) The large-scale language model, interjections, emotional expressions, repetition, fillers, fixed phrase responses, names of the functional parts of the response device 100, steps, processes, names of steps or processes, etc., used in the description of the above embodiment are merely examples and can be changed at will.

[0088] For example, while it was explained that the response condition DB121 stores information related to each item, such as "No," which identifies individual data included in the response condition, "response method," and "condition," in a table format, the items and their contents are not limited. Similarly, while it was explained that the response history DB122 stores information related to each item, such as "No," which identifies individual data included in the response history, "user identification information," and "response history," in a table format, the items and their contents are not limited.

[0089] (An example of checking the accuracy of a summary) In this embodiment, the response device 100 is described as generating and outputting a summary of the user's conversation. At this time, the response device 100 can determine the accuracy of the generated summary.

[0090] Specifically, the response device 100 (output unit) outputs a summary of the conversation generated by the generation unit to the user. Next, the response device 100 (reception unit) receives information from the user regarding whether the summary is correct, such as "the user's answer such as Yes / No regarding whether the summary is correct" or "the user's summary of the conversation content."

[0091] The response device 100 (determination unit) determines whether the summary of the conversation is accurate based on the user's input. For example, regarding the user's response, such as Yes / No, to whether the summary is correct, the response device 100 (determination unit) determines that the summary is accurate if the user's input is "Yes," and that the summary is inaccurate if the user's input is "No." The response device 100 (determination unit) also compares the summary generated by the response device 100 with the summary input by the user, and determines that the summary generated by the response device 100 is accurate if the similarity of the summaries, calculated based on known techniques such as calculating the similarity of sentences, exceeds a predetermined threshold.

[0092] Through the process described above, when the response device 100 generates a response based on the response generation LLM, the user can check the summary in advance, preventing the device from generating a response that the user did not anticipate.

[0093] (Regarding the use of response history) In this embodiment, the response device 100 is described as storing the response history with the user in the response history DB 122. Therefore, the response device 100 can use this response history to respond to the user.

[0094] Specifically, when the response device 100 receives a user's conversation, it searches the user's past response history based on the user identification information and the content of the conversation. If a response history exists for the user, the response device 100 can generate a response to the user using the retrieved response history. For example, the response device 100 can generate a response tailored to the user using information such as past responses to the user, emotions, fillers, summaries, and selected standard phrases.

[0095] (Regarding the natural language used in responses) In this embodiment, the response device 100 can generate a response based on natural language that is appropriate for the user. For example, the response device 100 can generate a response using natural language based on a predetermined "dialect" detected from the user's conversation.

[0096] Furthermore, the response device 100 can estimate the user's attribute information, such as gender, age, place of residence, family structure, and occupation, from the user's conversation, and generate a response using natural language and phrasing appropriate to the estimated attribute information. For example, if the user is elderly, the response device 100 can generate a response that uses language and topics suitable for elderly people.

[0097] (Regarding the use of the model) In this embodiment, the model (large-scale language model) used by the response device 100 is described as being stored in the model DB 123 of the storage unit 120, but this is not limited to this. For example, the response device 100 can access an external information processing device (server, etc.) and use a predetermined model.

[0098] (Flowcharts, etc.) In flowcharts, each step may be rearranged as long as it does not create inconsistencies, and some steps may be omitted. Furthermore, conjunctions such as "next," "continue," "in addition," "at this time," and "on this occasion" in flowchart descriptions do not limit the order or timing of the processes in the flowchart.

[0099] <Hardware Configuration> Each component of the illustrated device is a functional concept and does not necessarily have to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions. Furthermore, each processing function performed by each device can be implemented, all or any part of it, by a CPU and the program that is analyzed and executed by that CPU, or by hardware using wired logic.

[0100] Furthermore, among the processes described in this embodiment, all or part of those described as being performed automatically can be performed manually using known methods. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the drawings can be arbitrarily changed unless otherwise specified.

[0101] <Program> In one embodiment, the various devices constituting the response device 100 can be implemented by installing a response program as packaged software or online software on a desired computer. For example, by having the above-mentioned response program executed on an information processing device, the various devices constituting the response device 100 can be made to function. The information processing device referred to here includes desktop or notebook personal computers. In addition, the information processing device also includes mobile communication terminals such as smartphones and mobile phones, and slate terminals such as PDAs (Personal Digital Assistants).

[0102] Figure 9 shows an example of a computer that implements the response device 100 according to the embodiment. The computer 1000 has, for example, memory 1010 and CPU 1020. The computer 1000 also has a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0103] Memory 1010 includes ROM (Read Only Memory) 1011 and RAM 1012. ROM 1011 stores, for example, a boot program such as BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1090. The disk drive interface 1040 is connected to the disk drive 1100. For example, a removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to, for example, a mouse 1110 and a keyboard 1120. The video adapter 1060 is connected to, for example, a display 1130.

[0104] The hard disk drive 1090 stores, for example, an OS (Operating System) 1091, an application program 1092, a program module 1093, and program data 1094. That is, the program that defines each process of the various devices constituting the response device 100 is implemented as a program module 1093 in which code executable by a computer is written. The program module 1093 is stored, for example, in the hard disk drive 1090. For example, a program module 1093 for performing processes similar to the functional configuration of the various devices constituting the response device 100 is stored in the hard disk drive 1090. Note that the hard disk drive 1090 may be replaced by an SSD (Solid State Drive).

[0105] Furthermore, the configuration data used in the processing of the embodiment described above is stored as program data 1094 in, for example, memory 1010 or hard disk drive 1090. The CPU 1020 then reads the program module 1093 and program data 1094 stored in memory 1010 or hard disk drive 1090 into RAM 1012 as needed and executes the processing of the embodiment described above.

[0106] Furthermore, the program module 1093 and program data 1094 are not limited to being stored in the hard disk drive 1090; for example, they may be stored in a removable storage medium and read by the CPU 1020 via a disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (LAN, WAN (Wide Area Network), etc.). The program module 1093 and program data 1094 may then be read from the other computer by the CPU 1020 via a network interface 1070.

[0107] <Other> Although this embodiment has been described above, this embodiment is not limited by the description and drawings that constitute part of the disclosure. That is, all other embodiments, examples, and operational techniques made by those skilled in the art based on this embodiment are included in the scope of this embodiment. [Explanation of Symbols]

[0108] 100 Response device 110 Communications Department 120 Storage section 121 Response Condition DB 122 Response History DB 123 Model DB 130 Control Unit 131 Reception Department 132 Judgment section 133 Estimation Department 134 Generation part 135 Storage Unit 136 Output section

Claims

1. If the processing based on the first large-scale language model that generates responses in response to the conversation entered by the user is not yet complete, An estimation unit that estimates the user's emotions by inputting a prompt, expressed in natural language text, as a command to estimate the user's emotions based on the input conversation, into a second large-scale language model, Based on the user's emotions estimated by the estimation unit, a generation unit generates a response to the user who input the conversation based on a predetermined model. A response device characterized by having the following features.

2. The generating unit is The user's emotions estimated by the estimation unit and the conversation are input to the predetermined model, which has been trained to generate responses corresponding to the conversation. The aforementioned interjections corresponding to the aforementioned conversation are generated as the aforementioned responses. The response device according to feature 1.

3. The generating unit is Based on the user's emotions estimated by the estimation unit and the conversation, a prompt including a command to generate emotional information for an avatar corresponding to the user's emotions is input to the predetermined model, thereby generating the emotional information for the avatar corresponding to the conversation as the response. The system further includes an output unit that changes and displays the emotional expression of the avatar displayed to the user based on the emotional information of the avatar generated by the generation unit. The response device according to feature 1.

4. The generating unit is Based on the user's emotions estimated by the estimation unit and the conversation, a prompt including a command to generate a summary of the conversation is input to the predetermined model to generate the summary of the conversation. The response device according to feature 1.

5. An output unit that outputs the summary of the conversation generated by the generation unit to the user, The system further includes a determination unit that determines whether the summary of the conversation is accurate based on the input from the user. The response device according to feature 4.

6. The generating unit is The estimation unit generates fillers corresponding to the user's emotions estimated by the estimation unit and the conversation as the response. The response device according to feature 1.

7. The generating unit is Based on the user's emotions estimated by the estimation unit and pre-set standard phrases corresponding to the conversation, the response is generated. The response device according to feature 1.

8. A determination unit that determines whether or not there is insufficient information for performing generation processing using the first large-scale language model, If it is determined that the aforementioned information is insufficient, the system further includes an output unit that displays text or an avatar informing the user that the information is insufficient. The response device according to any one of claims 1 to 7.

9. A response method to be executed by a computer-based response device, If the processing based on the first large-scale language model that generates responses in response to the conversation entered by the user is not yet complete, The response device inputs a prompt, expressed in natural language text, into a second large-scale language model, which is a command to estimate the user's emotions based on the input conversation, and performs an estimation step of estimating the user's emotions. The response device includes a generation step in which, based on the user's emotions estimated in the estimation step, a response to the user who input the conversation is generated based on a predetermined model, A response method characterized by including the above steps and having a computer perform each of the above steps.

10. If the processing based on the first large-scale language model that generates responses in response to the conversation entered by the user is not yet complete, An estimation step in which a prompt, expressed in natural language text, is input to a second large-scale language model to estimate the user's emotions based on the input conversation, Based on the user's emotions estimated in the estimation step, a generation step is performed to generate a response to the user who input the conversation based on a predetermined model. A response program that instructs a computer to execute a command.

Images