Information processing apparatus, response method, and response program

Abstract

To improve responsiveness in a conversation using a language model.SOLUTION: When a receiving unit receives a second speech in a period starting with reception of a first speech and ending with the completion of presentation of a first answer to the first speech in an information processing apparatus, a generation control unit starts generating a second answer based on the second speech, and a presentation unit presents the generated second answer to a user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, a response method, and a response program. [Background technology]

[0002] There is known a technology that allows an information processing device to exchange natural language responses with a user by using a language model generated by machine learning of natural language. For example, a text generation device described in Patent Document 1 generates an answer to a question input via a user terminal using a large-scale language model, and outputs the generated answer to the user terminal. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 7313757 Summary of the Invention [Problem to be solved by the invention]

[0004] The text generation device described in Patent Document 1 has room for improvement in terms of responsiveness in a dialogue with a user. For example, in a dialogue using the text generation device, a user inputs a question and then realizes that there is an error in the content. In this case, the user must wait for the answer to the previous question to be presented, and then input a new question to obtain the answer. In this case, a considerable waiting time occurs before the user receives the desired answer.

[0005] Such room for improvement is not limited to the above-described sentence generation device, but is common to all cases where a dialogue is conducted using a language model. An exemplary object of the present disclosure is to provide a technology that enables improvement of responsiveness in a dialogue using a language model. [Means for solving the problem]

[0006] An information processing device according to an exemplary aspect of the present disclosure is an information processing device that enables a user to respond in natural language using a machine-learned language model, and includes: a reception means for receiving utterances from the user; a generation control means for causing the language model to generate a response to the received utterance; and a presentation means for presenting the generated response to the user; when the reception means receives a second utterance during the period from when a first utterance is received to when presentation of the first response to the first utterance is completed, the generation control means starts generating a second response based on the second utterance, and the presentation means presents the generated second response to the user.

[0007] A response method according to an exemplary aspect of the present disclosure is a response method that enables a user to respond in natural language using a machine-learned language model, in which at least one processor executes a reception process that receives an utterance from the user, a generation control process that causes the language model to generate a response to the received utterance, and a presentation process that presents the generated response to the user, and when the at least one processor receives a second utterance during the period from when a first utterance is received to when presentation of the first response to the first utterance is completed, the at least one processor starts generating a second response based on the second utterance and presents the generated second response to the user.

[0008] A response program according to an exemplary aspect of the present disclosure is a response program that enables a user to respond in natural language using a machine-learned language model, and causes a computer to function as a reception means for receiving an utterance from the user, a generation control means for causing the language model to generate a response to the received utterance, and a presentation means for presenting the generated response to the user, wherein when the reception means receives a second utterance during the period from when a first utterance is received to when the presentation of the first response to the first utterance is completed, the generation control means begins generating a second response based on the second utterance, and the presentation means presents the generated second response to the user. [Effects of the Invention]

[0009] According to an exemplary aspect of the present disclosure, an exemplary effect is achieved in that it is possible to improve responsiveness in a dialogue using a language model. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram illustrating a configuration of an information processing device according to the present disclosure. [Figure 2] FIG. 1 is a flow diagram illustrating the flow of a response method according to the present disclosure. [Figure 3] FIG. 10 is a block diagram showing a configuration of another information processing device according to the present disclosure. [Figure 4] FIG. 10 is a diagram illustrating an example of a text dialogue. [Figure 5] FIG. 10 is a diagram illustrating an example of a dialogue by voice. [Figure 6] 4 is a flowchart showing an example of processing executed by the information processing device shown in FIG. 3. [Figure 7] FIG. 1 is a block diagram illustrating a configuration of a computer that functions as an information processing device according to the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0011] The following are examples of embodiments of the present invention. However, the present invention is not limited to the exemplary embodiments shown below, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technologies (part or all of the products or methods) employed in the exemplary embodiments shown below may also be included in the scope of the present invention. Furthermore, embodiments obtained by appropriately omitting some of the technologies employed in the exemplary embodiments shown below may also be included in the scope of the present invention. Furthermore, the effects mentioned in the exemplary embodiments shown below are examples of effects expected in the exemplary embodiments, and do not define the scope of the present invention. In other words, embodiments that do not exhibit the effects mentioned in the exemplary embodiments shown below may also be included in the scope of the present invention.

[0012] First Exemplary Embodiment A first exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. This exemplary embodiment is a basic form for each of the exemplary embodiments described below. Note that the scope of application of each technique employed in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technique employed in this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical obstacles arise. Furthermore, each technique shown in the drawings referenced to explain this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical obstacles arise.

[0013] (Configuration of information processing device 1) The configuration of an information processing device 1 according to this exemplary embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of the information processing device 1. The information processing device 1 is a device that uses a machine-learned language model to enable responses to a user in natural language, that is, to accept input in natural language and return a response in natural language. As shown in Fig. 1, the information processing device 1 includes a receiving unit 101, a generation control unit 102, and a presentation unit 103.

[0014] The reception unit 101 receives an utterance from a user. Here, "utterance" includes not only utterances uttered as voice but also utterances input as text. That is, the reception unit 101 may receive input of voice data as an utterance, or may receive input of text data as an utterance. When receiving input of voice data, the reception unit 101 may convert the voice data into text data. Note that the conversion into text data may be performed by a processing block different from the reception unit 101 provided in the information processing device 1, or may be performed by a device other than the information processing device 1. Furthermore, the content of the utterance is arbitrary, and may be, for example, an utterance for conversation, an utterance as a question, or an utterance as a command.

[0015] The generation control unit 102 causes a language model to generate a response to the utterance received by the receiving unit 101. The language model may be a model generated by machine learning so as to be capable of generating a response to a user's utterance. For example, a general-purpose language model that has been machine-learned to learn the arrangement of components (such as words) of a sentence written in a natural language or the arrangement of sentences in a piece of writing may be used as the language model. The language model may be one provided in the information processing device 1 or another device. In the latter case, the generation control unit 102 transmits voice data or text data indicating the content of the utterance received by the receiving unit 101 to the other device, and acquires the response generated by the language model from the other device.

[0016] The presentation unit 103 presents the answer generated under the control of the generation control unit 102 to the user. The presentation may be performed in any manner. For example, the presentation unit 103 may present the answer by having an audio output device output the generated answer as a sound, or may present the answer by having a display device output the answer as a display. The audio output device and the display device used to present the answer may be included in the information processing device 1, or may be included in another device.

[0017] Here, it is assumed that the receiving unit 101 receives a second utterance during the period from when the receiving unit 101 receives a first utterance until when the presentation of a first answer to the first utterance is completed. In this case, the generation control unit 102 starts generating a second answer based on the second utterance. Then, the presentation unit 103 presents the generated second answer to the user.

[0018] Note that the "first utterance" here refers to the utterance immediately preceding the "second utterance," and does not necessarily mean the first utterance in a dialogue. For example, an utterance uttered in the middle of a dialogue may be the "first utterance," in which case the next utterance would be the "second utterance."

[0019] Furthermore, during the period from when the first utterance is received until the presentation of the first answer to the first utterance is completed, the presentation unit 103 may present the first answer sequentially, starting from the generated portion. The same applies to the second answer, and the presentation unit 103 may present the second answer sequentially, starting from the generated portion. Furthermore, when starting the generation of the second utterance, the generation control unit 102 may suspend or continue the generation of the first answer.

[0020] Furthermore, the second answer may be generated based on the second utterance. For example, the generation control unit 102 may input the second utterance directly into a language model to generate the second answer, or may input both the first utterance and the second utterance into a language model to generate the second answer. Furthermore, for example, the generation control unit 102 may input a processed version of the second utterance into a language model to generate the second answer.

[0021] As described above, the information processing device 1 is a device that enables interaction with a user in natural language using a machine-learned language model, and includes a reception unit 101 that receives a user utterance, a generation control unit 102 that causes the language model to generate a response to the received utterance, and a presentation unit 103 that presents the generated response to the user. If the reception unit 101 receives a second utterance during the period from when it receives a first utterance to when it completes presentation of the first response to the first utterance, the generation control unit 102 starts generating a second response based on the second utterance, and the presentation unit 103 presents the generated second response to the user.

[0022] According to the above configuration, the user can make a second utterance and receive a second answer based on the second utterance without having to wait for a response to the first utterance to be presented or performing an operation to interrupt the response to the first utterance. In this way, the information processing device 1 has the effect of improving responsiveness in a dialogue using a language model.

[0023] (Response Program) The functions of the information processing device 1 described above can also be realized by a program. A response program according to this exemplary embodiment is a response program that enables a user to respond in natural language using a machine-learned language model. The response program causes a computer to function as a reception unit that receives a user's utterance, a generation control unit that causes the language model to generate a response to the received utterance, and a presentation unit that presents the generated response to the user. When the reception unit receives a second utterance during a period from the time the reception unit receives a first utterance to the time the presentation of the first response to the first utterance is completed, the generation control unit starts generating a second response based on the second utterance, and the presentation unit presents the generated second response to the user. This response program enables improved responsiveness in dialogues using language models.

[0024] (Response method flow) The flow of the response method according to this exemplary embodiment will be described with reference to Fig. 2. Fig. 2 is a flow diagram showing the flow of the response method. Note that the execution entity of each step in this response method may be a processor provided in the information processing device 1, or a processor provided in another device, or each step may be executed by a processor provided in a different device.

[0025] In S1 (reception process), at least one processor receives a first utterance from a user. The first utterance may be input as voice data or text data.

[0026] In S2 (generation control process), at least one processor causes the language model to start generating a first answer to the first utterance accepted in S1. Note that the at least one processor may present the first answer generated by the language model sequentially starting from the generated part, or may present the first answer when generation is completed.

[0027] In S3 (reception process), at least one processor receives a second utterance from the user. Note that Fig. 2 shows an example in which the second utterance is received within a period from the reception of the first utterance to the completion of presentation of a first response to the first utterance.

[0028] In S4 (generation control process), at least one processor starts generating a second answer based on the second utterance.

[0029] In S5 (presentation process), at least one processor presents the second answer generated under the control of S4 to the user. This ends the process of FIG. 2. The process of S5 may be started immediately after the start of the process of S4. In other words, the at least one processor may present the second answer generated by the language model sequentially starting from the generated part, or may present it when generation is completed.

[0030] As described above, the response method according to this exemplary embodiment is a response method that enables a user to respond in natural language using a machine-learned language model, in which at least one processor executes a reception process for receiving a user utterance, a generation control process for causing the language model to generate a response to the received utterance, and a presentation process for presenting the generated response to the user, and if the at least one processor receives a second utterance during a period from when a first utterance is received to when the presentation of the first response to the first utterance is completed, the at least one processor starts generating a second response based on the second utterance and presents the generated second response to the user. Therefore, the response method according to this embodiment makes it possible to improve responsiveness in dialogues using a language model.

[0031] Second Exemplary Embodiment A second exemplary embodiment, which is one example of an embodiment of the present invention, will be described in detail with reference to the drawings. Note that the scope of application of each technique employed in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technique employed in this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical hindrance occurs. Furthermore, each technique shown in each drawing referenced to explain this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical hindrance occurs.

[0032] (Configuration of information processing device 1A) The configuration of an information processing device 1A according to this exemplary embodiment will be described with reference to FIG. 3. FIG. 3 is a block diagram showing the configuration of the information processing device 1A. The information processing device 1A is a device that enables a user to respond in natural language using a machine-learned language model. Note that the information processing device 1A may be a device whose main function is to respond to a user, or may be a general-purpose device that also has other functions. Furthermore, the information processing device 1A may be a stationary device or a portable device.

[0033] As shown in FIG. 3, the information processing device 1A includes a control unit 10A that controls the various units of the information processing device 1A and a storage unit 11A that stores various data used by the information processing device 1A. The information processing device 1A also includes a communication unit 12A that enables the information processing device 1A to communicate with other devices, an input unit 13A that accepts input to the information processing device 1A, and an output unit 14A that enables the information processing device 1A to output data. The control unit 10A includes a reception unit 101A, a generation control unit 102A, a presentation unit 103A, a determination method determination unit 104A, an interrupt determination unit 105A, a feeling estimation unit 106A, an evaluation unit 107A, and an optimization unit 108A. The storage unit 11A also stores a language model 111A and a method determination model 112A. The determination method determination unit 104A, the evaluation unit 107A, the optimization unit 108A, and the method determination model 112A will be described in detail below.

[0034] The receiving unit 101A receives a user utterance, similar to the receiving unit 101 described in exemplary embodiment 1. Similarly to the generation control unit 102 described in exemplary embodiment 1, the generation control unit 102A causes the language model 111A to generate a response to the utterance received by the receiving unit 101A. Similarly to the language model described in exemplary embodiment 1, the language model 111A may be a model generated by machine learning so as to be able to generate a response to a user utterance. Similarly to the presentation unit 103 described in exemplary embodiment 1, the presentation unit 103A presents the response generated under the control of the generation control unit 102A to the user.

[0035] In the information processing device 1A, similarly to the information processing device 1, when the receiving unit 101A receives a second utterance during the period from the time when the first utterance is received until the time when the presentation of the first answer to the first utterance is completed, the generation control unit 102A starts generating a second answer based on the second utterance, and the presentation unit 103A presents the generated second answer to the user. This provides an effect that it is possible to improve responsiveness in a dialogue using the language model 111A, similarly to the information processing device 1.

[0036] When a second utterance is received during the period from when a first utterance is received until the presentation of a first answer to the first utterance is completed, the interrupt determination unit 105A determines whether to start generating a second answer. Hereinafter, this determination is referred to as interrupt determination. When the interrupt determination unit 105A determines to start generating a second answer, the generation control unit 102A causes the language model 111A to start generating a second answer, and the presentation unit 103A presents the generated second answer to the user. On the other hand, when the interrupt determination unit 105A determines not to start generating a second answer, the generation control unit 102A does not cause the language model 111A to start generating a second answer. In this case, the generation of the first answer continues, and the presentation unit 103A presents the generated first answer to the user.

[0037] The feeling deduction unit 106A deduces the user's feeling during at least a part of the period from the end of the first utterance to the end of the second utterance. The feeling deduction unit 106A may deduces the user's feeling or may deduces whether the user's feeling has changed. The estimation result by the feeling deduction unit 106A is used for interrupt determination by the interrupt determination unit 105A.

[0038] (About interrupt judgment) As described above, when a second utterance is received during the period from when a first utterance is received until the presentation of a first answer to the first utterance is completed, the interrupt determination unit 105A determines whether or not to start generating a second answer.

[0039] The interrupt determination unit 105A may determine whether to start generating a second response based on, for example, at least one of the content of the second utterance, the audio of the second utterance, an image of the user taken during at least a portion of the period from the end of the first utterance to the end of the second utterance, and biometric information of the user measured during at least a portion of that period. This configuration achieves, in addition to the effects of the information processing device 1, an effect of enabling a more natural conversation with the user. Below, a specific example of interrupt determination using the above-mentioned various information will be described.

[0040] (Decision based on speech content) For example, when the interrupt determination unit 105A determines whether to start generating a second answer based on the content of the second utterance, if the second utterance is merely a backchannel or an utterance without any particular meaning, the generation of the first answer can be continued and the first answer can be presented, thereby realizing a more natural dialogue with the user.

[0041] In the interrupt determination based on the content of the second utterance, the interrupt determination unit 105A may determine whether or not to start generating a second answer, for example, based on whether or not a predetermined word or phrase is included in the second utterance.

[0042] The predetermined word or phrase may be a word or phrase (e.g., "yes," "uh," "right," "well," etc.) that appears relatively frequently in utterances that should not be accepted as interruptions, such as simple backchannels or meaningless vocalizations. In this case, interrupt determination unit 105A will not start generating a second answer if the second utterance contains any of these.

[0043] The predetermined word or phrase may be a word or phrase that appears relatively frequently in utterances that are preferably accepted as an interrupt (for example, "wait a moment," "in other words," "correctly," "by the way," etc.). In this case, interrupt determination unit 105A starts generating a second answer when the second utterance contains any of these words or phrases.

[0044] (Decision based on voice) When having a conversation with a user through voice, it is also effective to determine whether to start generating a second answer based on the voice of the second utterance, because the user's emotions and intentions are reflected in the voice.

[0045] The interrupt determination unit 105A may also make a determination taking into consideration the voice of the first utterance. Note that determination based on the voice of the second utterance includes determination made directly using the voice data of the second utterance, determination made using features extracted from the voice data of the second utterance, and determination made using information obtained by analyzing the voice data of the second utterance. These matters also apply to image-based determination, which will be described later.

[0046] When determining whether to start generating a second answer based on the audio of the second utterance, interrupt determination unit 105A may acquire at least one of pitch, volume, frequency, and speed as a feature from the audio data of the second utterance. By defining in advance the relationship between the value of such a feature and whether to start generating a second answer, interrupt determination unit 105A can determine whether to start generating a second answer based on the value of the feature generated from the acquired audio data.

[0047] Furthermore, the interrupt determination unit 105A may acquire time-series data of the feature amount and determine whether to start generating a second answer based on the pattern of the time-series change. For example, the interrupt determination unit 105A may determine to start generating a second answer when the speed of the second utterance is faster than the speed of the first utterance. In such a case, it is considered that the user noticed an error or omission in the first utterance and quickly made the second utterance, and it is preferable to accept such a second utterance as an interrupt.

[0048] (Image-based judgment) If an image of the user taken during at least a part of the period from the end of the first utterance to the end of the second utterance can be acquired, it is also effective to determine whether to start generating a second answer based on the image. This is because the user's emotions and intentions are also reflected in the image of the user. The image may be a moving image or a still image.

[0049] When determining whether to start generating a second answer based on the image described above, the interrupt determination unit 105A may acquire features indicating at least one of the user's gaze, the detection results of landmarks on the user's face, and facial expression in the acquired image. Acquiring such features can utilize technologies such as FACS (Facial Action Coding System), OpenFace, and Dlib. By predefining the relationship between the values ​​of the features acquired from the image of the user and whether to start generating a second answer, the interrupt determination unit 105A can determine whether to start generating a second answer based on the values ​​of the features generated from the acquired image.

[0050] Furthermore, the interrupt determination unit 105A may acquire time-series data of the feature amount and determine whether to start generating a second answer based on the pattern of the time-series change. For example, the interrupt determination unit 105A may determine to start generating a second answer when the acquired feature amount value indicates that the user's facial expression has changed from a neutral expression to a negative expression (for example, an expression of anger or disappointment).

[0051] The interrupt determination unit 105A may also determine whether to start generating a second answer based on whether a predetermined action of the user is detected from a moving image or a time-series still image of the user. The predetermined action may be determined in advance. For example, the interrupt determination unit 105A may determine to start generating a second answer when a gesture requesting stopping generation of the first answer is detected.

[0052] (Determination based on biological information) If it is possible to acquire biometric information of the user for at least a part of the period from the end of the first utterance to the end of the second utterance, it is also effective to determine whether to start generating a second answer based on the biometric information. This is because the user's biometric information also reflects the user's emotions and intentions. The biometric information used here may be any information that reflects the user's emotions, such as heart rate, pulse rate, and blood pressure.

[0053] When determining whether to start generating a second answer based on the biometric information, the interrupt determination unit 105A may acquire the biometric information and use the acquired biometric information as a feature amount as is, or may generate a feature amount from the acquired biometric information. By defining in advance the relationship between the acquired biometric information or the value of the feature amount generated from the acquired biometric information and whether to start generating a second answer, the interrupt determination unit 105A can determine whether to start generating a second answer based on the acquired biometric information.

[0054] Furthermore, the interrupt determination unit 105A may acquire time-series data of the biometric information and determine whether to start generating a second answer based on a pattern of time-series changes in the biometric information or features generated from the biometric information. For example, the interrupt determination unit 105A may determine to start generating a second answer when the user's heart rate increases during the period from the end of the first utterance to the end of the second utterance.

[0055] (Determination based on a combination of multiple types of features) The interrupt determination unit 105A may determine whether to start generating a second answer by combining multiple types of feature amounts. This can improve the accuracy of the determination compared to when the determination is made based on only one type of feature amount. For example, the interrupt determination unit 105A may determine to start generating a second answer when the feature amounts acquired from the image of the user indicate that the user's facial expression has changed negatively and that the user has averted their gaze. Furthermore, for example, the interrupt determination unit 105A may determine to start generating a second answer when the user's heart rate has increased and the user's speaking rate has increased.

[0056] (Decision based on a trained model) The interrupt determination unit 105A may determine whether to start generating a second answer using a trained model that has been trained to output a determination result on whether to start generating an answer in response to input of data indicating the content of an utterance. In this case, the interrupt determination unit 105A determines whether to start generating a second answer from an output obtained by inputting a second utterance to the trained model.

[0057] For example, the language model 111A used to generate an answer can be used as the trained model. When the language model 111A is used, the interrupt determination unit 105A may generate a prompt including text data of the second utterance and asking whether the utterance requests an answer, and input the prompt to the language model 111A. In this case, if the output of the language model 111A indicates that an answer is requested, the interrupt determination unit 105A starts generating the second answer. On the other hand, if the output of the language model 111A indicates that an answer is not requested, the interrupt determination unit 105A does not start generating the second answer. Furthermore, the interrupt determination unit 105A may include text data of the first utterance in the prompt. This realizes a more accurate interrupt determination that takes into account the flow of utterances from the first utterance to the second utterance.

[0058] The trained model used for interruption determination may be one trained for interruption determination. Such a trained model can be generated by machine learning using training data in which various data serving as criteria for determining whether to accept the second utterance as an interrupt are associated with correct answer data indicating whether to accept the second utterance as an interrupt.

[0059] The data used as a basis for determining whether to accept the second utterance as an interrupt may be, for example, text data or voice data of the second utterance, or a feature amount generated from these data. Furthermore, the data used as a basis for determination may be an image of the user taken during at least a part of the period from the end of the first utterance to the end of the second utterance, or biometric information measured during that period, or a feature amount generated from these data.

[0060] (Decision based on emotion estimation results) The information processing device 1A includes a feeling deduction unit 106A that deduces the user's feeling during at least a part of a period from the end of the first utterance to the end of the second utterance. Therefore, the interrupt determination unit 105A may determine whether to start generating a second answer based on the estimation result of the feeling deduction unit 106A. This configuration not only provides the effects of the information processing device 1, but also provides the effect of realizing a natural dialogue that takes the user's feeling into consideration.

[0061] As described above, the feeling deduction unit 106A may deduce the user's feeling or may deduce whether the user's feeling has changed. The feeling deduction unit 106A may apply any estimation method. For example, the feeling deduction unit 106A may use a sentiment polarity dictionary indicating the positivity and negativity of various words and phrases to determine the positivity and negativity of each word or phrase included in the second utterance. The feeling deduction unit 106A may then combine these determination results to determine the positivity or negativity of the second utterance. In this case, the interrupt determination unit 105A may determine to start generating a second answer when the positivity of the second utterance is equal to or less than a threshold or the negativity of the second utterance is equal to or greater than a threshold.

[0062] Furthermore, the feeling estimation unit 106A may estimate the user's feeling at the time of the second utterance using the language model 111A. In this case, the feeling estimation unit 106A may generate a prompt including the second utterance and instructing the language model 111A to estimate the speaker's feeling, and input the prompt to the language model 111A. As a result, an estimation result of the speaker's feeling is output from the language model 111A. Note that the language model used for feeling estimation may be a language model other than the language model 111A (for example, a language model fine-tuned for emotion estimation).

[0063] The emotion estimation unit 106A may estimate the emotion based on at least any of the content of the second utterance, the audio of the second utterance, an image of the user captured during at least a part of the period from the end of the first utterance to the end of the second utterance, and biometric information measured during that period. The emotion estimation can be performed by comparing the various data or feature amounts acquired from the data with a predetermined threshold, or by using a trained model that has learned the relationship between the feature amounts and the emotion.

[0064] In addition, when the interrupt judgment unit 105A judges whether or not to start generating a second answer based on the estimation result as described above, it can be said that the judgment whether or not to start generating a second answer is based on at least any of the content of the second utterance, the audio of the second utterance, an image of the user taken during at least a part of the period from the end of the first utterance to the end of the second utterance, and the user's biometric information measured during at least a part of that period.

[0065] (Regarding the use of multiple interrupt determination methods) As described above, various interrupt determination methods (hereinafter simply referred to as interrupt determination methods) can be applied. The interrupt determination methods include those with different characteristics, such as those that require a long time to obtain a determination result but have high determination accuracy, and those that obtain a determination result quickly but have slightly lower determination accuracy. Therefore, the interrupt determination unit 105A may apply an interrupt determination method suited to the user from among a plurality of interrupt determination methods for determining whether or not to start generating a second answer. This configuration provides the effect of being able to determine an appropriate interrupt determination method suited to the user, in addition to the effect provided by the information processing device 1.

[0066] The interrupt determination method to be applied may be determined by the determination method determination unit 104A. For example, the determination method determination unit 104A may allow the user to select in advance which interrupt determination method to apply, in which case the interrupt determination method selected in advance by the user is applied. Furthermore, for example, the determination method determination unit 104A may allow the user to input in advance his / her preferences regarding dialogue. In this case, the determination unit 104A determines to apply an interrupt determination method according to the user's preferences. For example, for a user who places importance on response speed in dialogue, the determination method determination unit 104A may determine to apply an interrupt determination method that takes a relatively short time to obtain a determination result.

[0067] Furthermore, for example, the determination method determination unit 104A may have the user try out a dialogue using various interrupt determination methods. In this case, the determination method determination unit 104A may have the user select which interrupt determination method to apply after the trial. Furthermore, the determination method determination unit 104A may have the evaluation unit 107A (described later) evaluate each interrupt determination method based on the trial results of the interrupt determination methods. In this case, the determination method determination unit 104A may decide to apply the interrupt determination method with the best evaluation result.

[0068] (Details of the Judgment Method Determination Unit 104A / Evaluation Unit 107A / Optimization Unit 108A) The following describes in detail the determination method determination unit 104A, the evaluation unit 107A, and the optimization unit 108A. The following also describes in detail the method determination model 112A.

[0069] The determination method determination unit 104A determines the interrupt determination method to be applied by the interrupt determination unit 105A. Various methods can be applied as the method for determining the interrupt determination method. For example, as described above, the determination method determination unit 104A can also determine the interrupt determination method to be applied based on the user's selection.

[0070] The determination method determination unit 104A can also determine the interrupt determination method using a method determination model 112A. The method determination model 112A is a model for determining the interrupt determination method to be applied. For example, the method determination model 112A may be a trained model generated by machine learning the correspondence between various information that is preferably considered when determining the optimal interrupt determination method and the optimal interrupt determination method corresponding to that information. For example, the method determination model 112A may be a model generated by reinforcement learning.

[0071] Furthermore, for example, the method determination model 112A may be an optimization model that determines an optimal interrupt determination method according to given conditions. When the method determination model 112A is an optimization model, the optimization unit 108A repeatedly updates the method determination model 112A, as described below, thereby optimizing the method determination model 112A.

[0072] The language model 111A can also be used as the method determination model 112A. In this case, the determination method determination unit 104A generates a prompt including a list of candidate interrupt determination methods and a description of the case, instructing the user to select an interrupt determination method suitable for the case from the candidates, and inputs the generated prompt to the language model 111A. This allows the language model 111A to output an interrupt determination method suitable for the case. Note that the list of interrupt determination methods preferably includes a description of the characteristics of each interrupt determination method and the type of user for which it is suitable. Furthermore, the description of the case preferably includes user attribute information (e.g., age, gender, occupation, personality, etc.), the second utterance to be the target of interrupt determination, the first utterance preceding it, etc.

[0073] The evaluation unit 107A evaluates the interrupt determination method applied by the interrupt determination unit 105A. The evaluation method is not particularly limited. For example, the evaluation unit 107A may receive feedback from the user regarding a dialogue when the interrupt determination method to be evaluated is applied or an answer presented in the dialogue, and use the feedback as an evaluation result of the interrupt determination method.

[0074] The feedback received by the evaluation unit 107A may be feedback explicitly given by the user, such as verbal feedback or feedback via the communication unit 12A or the input unit 13A. In this case, the evaluation unit 107A may directly use the input feedback as the evaluation result of the interrupt determination method.

[0075] Furthermore, the evaluation unit 107A may receive, as feedback, at least one of text, speech, images, and biometric information indicating the content of the user's utterances during and after the dialogue. In this case, the evaluation unit 107A may generate an evaluation result by analyzing at least one of text, speech, images, and biometric information indicating the content of the user's utterances. The evaluation criteria may be determined in advance. For example, if the user's facial expression or voice detected during the dialogue (especially when a second answer to a second utterance is presented) is calm, the evaluation unit 107A may determine the evaluation result of the applied interruption determination method as good. On the other hand, if the user's facial expression or voice detected is in a bad mood or indicates dissatisfaction, the evaluation unit 107A may determine the evaluation result of the applied interruption determination method as bad.

[0076] The evaluation result of the evaluation unit 107A can be used to change the interrupt determination method to be applied. That is, the determination method determination unit 104A may determine the interrupt determination method to be applied from the next time onwards, based on the evaluation result of the evaluation unit 107A for the interrupt determination method applied by the interrupt determination unit 105A. This configuration not only provides the effects of the information processing device 1, but also has the effect of automatically applying an interrupt determination method that will make the answer presented to the user more preferable.

[0077] Furthermore, the evaluation result of the evaluation unit 107A can be used to optimize the method determination model 112A. The optimization unit 108A optimizes the method determination model 112A based on the evaluation result by the evaluation unit 107A. More specifically, the optimization unit 108A may optimize the method determination model 112A by performing, for each determined interrupt determination method, a process of updating the method determination model 112A based on the evaluation result for the interrupt determination method determined using the method determination model 112A. This configuration not only provides the effects of the information processing device 1, but also provides the effect of optimizing the method determination model 112A while repeatedly determining an interrupt determination method using the method determination model 112A. For example, the optimization unit 108A can optimize the method determination model 112A by applying a technique called online optimization.

[0078] (Example of a text dialogue) FIG. 4 is a diagram showing an example of a text dialogue between the information processing device 1A and its user. The text dialogue is conducted via a UI (User Interface) screen displayed on an arbitrary display device. FIG. 4 shows a UI screen example 4 during the period from when the input of a first utterance is accepted until the input of a second utterance is accepted, and a UI screen example 4' after the input of the second utterance is accepted. These UI screen examples include an input field 41 and an answer display field 43, and also display progress information 42 indicating the progress of answer generation.

[0079] The input field 41 is a display object for receiving text input by the user, that is, the user's utterance. The input field 41 is also a display object for displaying the text input by the user. In other words, the receiving unit 101A receives the text input in the input field 41 as the user's utterance. Then, the presentation unit 103A displays the input text in the input field 41.

[0080] For example, the input field 41 of UI screen example 4 displays the text "I've been feeling unwell since this morning, so please tell me what to do." This is the text of the first utterance input by the user. The generation control unit 102A inputs this text into the language model 111A to generate an answer to this text. The generated answer is displayed in the answer display field 43 by the presentation unit 103A. In UI screen example 4, the presentation unit 103A sequentially displays the answers generated under the control of the generation control unit 102A, starting from the generated part.

[0081] Furthermore, in UI screen example 4, presentation unit 103A displays a progress bar as progress information 42 that indicates the progress of answer generation. The progress bar in UI screen example 4 indicates the progress of the first answer generation, with the progress rate at the start of the first answer generation being 0% and the progress rate at the completion of generation being 100%.

[0082] Furthermore, in UI screen example 4, an arrow indicating a threshold of the progress rate is displayed in association with the progress bar. The threshold of the progress rate is a threshold used to determine whether to accept an interrupt. More specifically, the generation control unit 102A causes the language model 111A to generate a second answer on the condition that the progress rate of the generation of the first answer is less than a predetermined threshold when the second utterance is accepted.

[0083] Meanwhile, the input field 41 in the example UI screen 4' displays the text "I have a bad headache, but I don't seem to have a fever." This is the text of the second utterance entered by the user when the progress rate of the generation of the first answer is below the threshold. In the example of FIG. 4, the example UI screen 4 displays the generated first answers sequentially, starting from the generated portion. Therefore, if the displayed first answer is not what the user intended, the user can enter the text of the second utterance described above before the generation and presentation of the first answer is completed.

[0084] If the text of the second utterance is input during a period in which the progress rate of the generation of the first answer is less than a threshold (for example, when the progress rate is 30%, as in UI screen example 4), the generation control unit 102A starts generating a second answer based on the text of the second utterance, and the presentation unit 103 presents the generated second answer to the user.

[0085] In the answer display field 43 of the UI screen example 4', a new answer, i.e., a second answer, generated in consideration of the user's symptoms added by the second utterance is displayed. In addition, in the UI screen example 4', the progress bar display is also updated to indicate the progress rate of the second answer to the second utterance. In this way, the information processing device 1A can promptly present the answer desired by the user. Furthermore, as shown in the example of FIG. 4, the information processing device 1A can also be used in the fields of medicine and healthcare.

[0086] As described above, by using the information processing device 1A, even if the user makes utterances (inputs text) in a sporadic manner, an answer that takes into consideration the content of each utterance can be presented. Note that when the information processing device 1A receives a third utterance while the progress rate of the second answer is below the threshold, it starts generating and presenting an answer to the third utterance. In this way, the information processing device 1A can also handle multiple interrupts.

[0087] Furthermore, as described above, the presenting unit 103A may present to the user progress information indicating the progress of generation of the first answer during the period from when the first utterance is accepted until presentation of the first answer to the first utterance is completed. This configuration provides, in addition to the effects of the information processing device 1, an effect that the user can determine whether to make a second utterance in consideration of the progress of generation of the first answer. Note that the progress information need only be information indicating the progress of generation of the first answer, and is not limited to a progress bar. For example, the presenting unit 103A may present, as the progress information, a numerical value or the like indicating the progress of generation of the first answer.

[0088] Furthermore, as described above, the generation control unit 102A may cause the language model 111A to generate a second answer on the condition that the progress rate of generation of the first answer is less than a predetermined threshold when the second utterance is received. In this case, the presentation unit 103A may present the threshold and progress information indicating the progress rate of generation of the first answer to the user. This configuration provides, in addition to the effects of the information processing device 1, an effect that the user can determine whether to make a second utterance and the timing of making the second utterance, taking into consideration the progress of generation of the first answer and the end of the period during which interruption is permitted.

[0089] (Example of voice dialogue) FIG. 5 is a diagram showing an example of a voice dialogue between the information processing device 1A and its user. The voice dialogue is carried out via a voice input device that acquires the voice uttered by the user and a voice output device that outputs a voice response to the user. In the example of FIG. 5, the information processing device 1A and the user U converse via a voice input device and a voice output device provided in a telephone. In this way, the information processing device 1A can also be used for automatic responses to inquiries over the phone. Note that in FIG. 5, the content of the utterance output by the information processing device 1A is shown in a speech bubble associated with the information processing device 1A, but in reality, the voice of the utterance output by the information processing device 1A is output from the voice output device of the receiver.

[0090] In scene 5 shown in FIG. 5, as shown in speech bubble 51, user U asks a question about "product X." Receiving unit 101A of information processing device 1A receives voice data of this question as a first utterance and converts the voice data into text data. Then, generation control unit 102A inputs the text data of the first utterance into language model 111A to generate a first answer. The generated first answer is output by voice from a voice output device included in the telephone by presentation unit 103A. In FIG. 5, the output first answer is shown in speech bubble 52. At the time of scene 5, the generation and presentation of the first answer have not been completed, and only the beginning of the first answer has been presented.

[0091] 5 is a scene in which user U makes a second utterance and an answer to the second utterance is presented before the generation and presentation of the first answer is completed. Specifically, as shown in speech bubble 53, user U makes the second utterance to the effect that the previous question was not about "product X" but about "product Y."

[0092] The second utterance is received during the period from when the receiving unit 101A receives the first utterance until the presentation of the first answer to the first utterance is completed. Therefore, the generation control unit 102A starts generating a second answer based on the second utterance, and the presentation unit 103 presents the generated second answer to the user U. As shown in speech bubble 54, the second answer is an answer to the question in which "Product X" in the first question shown in speech bubble 51 is corrected to "Product Y" based on the second utterance. Such a second answer can be generated by inputting both the first and second utterances into language model 111A.

[0093] In this way, by using the information processing device 1A, when the user U realizes an error in a first utterance, he or she can make a second utterance to correct it, and be presented with a second answer that reflects the correction.

[0094] (Processing flow) The flow of processing executed by the information processing device 1A will be described with reference to Fig. 6. Fig. 6 is a flow diagram showing an example of processing executed by the information processing device 1A. The flow diagram of Fig. 6 includes each step of a response method according to this exemplary embodiment.

[0095] In S11 (reception process), the reception unit 101A receives a first utterance from a user. As described above, the utterance may be made by inputting voice data or text data. Furthermore, when receiving the input of the voice data of the first utterance, the reception unit 101A may convert the voice data into text data.

[0096] In S12 (generation control process), the generation control unit 102A causes the language model 111A to start generating a first answer to the first utterance accepted in S11. Also in S12, the presentation unit 103A starts presenting progress information indicating the progress of the generation of the first answer. Furthermore, after starting to present the progress information, the presentation unit 103A updates the content of the presented progress information according to the progress of the generation of the first answer.

[0097] Furthermore, the presenting unit 103A may present the first answers generated under the control of the generation control unit 102A, starting from the generated part (presenting process). Note that the presenting unit 103A may omit presenting the progress information when presenting the answers by voice. These matters also apply to the process of S18 described later.

[0098] In S13, the reception unit 101A determines whether or not a second utterance by the user has been received. If the determination in S13 is YES, the reception unit 101A performs a reception process for the second utterance, and the process proceeds to S15. On the other hand, if the determination in S13 is NO, the process proceeds to S14.

[0099] Regarding the process of S13, the receiving unit 101A may wait for the second utterance during the period from when the first utterance is received until the presentation of the first answer to the first utterance is completed. Furthermore, the receiving unit 101A may wait for the second utterance during the period in which the progress rate of the generation of the first answer is less than a predetermined threshold. Whether the progress rate of the generation of the first answer is less than the predetermined threshold may be determined in S17.

[0100] In S14, the generation control unit 102A completes the generation of the first answer, and the presentation unit 103A then presents the generated first answer to the user (presentation process). The presentation unit 103A also ends the presentation of progress information about the first answer. This completes the process in FIG. 6. Note that when the answer is presented by voice, both the generation and presentation of the first answer are completed in S14. The same applies to S19, which will be described later.

[0101] In S15, the determination method determination unit 104A determines the interrupt determination method. More specifically, the determination method determination unit 104A inputs predetermined input data (e.g., text data of the second utterance) to the method determination model 112A, and determines the interrupt determination method based on the output data of the method determination model 112A.

[0102] In S16, the feeling deduction unit 106A deduces the user's feeling. More specifically, the feeling deduction unit 106A deduces the feeling taken into account in the interruption determination method determined in S15. For example, assume that the interruption determination method determined in S15 uses the positivity and negativity of the user's feeling at the time of the first utterance and the time of the second utterance. In this case, the feeling deduction unit 106A deduces the positivity and negativity of the user's feeling at the time of the first utterance and the time of the second utterance, respectively. Note that if the emotion estimation result is not used in the interruption determination method determined in S15, the processing of S16 is omitted.

[0103] In S17, the interrupt determination unit 105A applies the interrupt determination method determined in S15 and determines whether or not to cause the language model 111A to generate a second answer based on the emotion estimation result in S16. If the determination in S17 is YES, the process proceeds to S18. On the other hand, if the determination in S17 is NO, the process proceeds to S14.

[0104] In S18 (generation control process), the generation control unit 102A causes the language model 111A to start generating a second answer to the second utterance. At this time, the generation control unit 102A may suspend the generation of the first answer. Also, in S18, the presentation unit 103A starts presenting progress information indicating the progress of the generation of the second answer.

[0105] In S19, the generation control unit 102A completes the generation of the second answer, and the presentation unit 103A presents the second answer that has been completely generated to the user (presentation process). The presentation unit 103A also ends the presentation of progress information about the second answer. Note that if the generation of the first answer has continued, the presentation unit 103A may also present the first answer.

[0106] In S20, the evaluation unit 107A evaluates the interruption determination method determined in S15. In S21, the optimization unit 108A updates the method determination model 112A based on the evaluation result of S20. This ends the processing in FIG. 6. Note that after S14, S19, or S21 is completed, the processing may return to S11 to accept further utterance input. Furthermore, the processing of S20 and S21 may be performed after the dialogue is completed.

[0107] [Software implementation example] Some or all of the functions of the information processing devices 1 and 1A may be realized by hardware such as an integrated circuit (IC chip), or may be realized by software.

[0108] In the latter case, the information processing devices 1 and 1A are realized by, for example, a computer that executes instructions of a program, which is software that realizes each function. An example of such a computer (hereinafter referred to as computer C) is shown in Fig. 7. Fig. 7 is a block diagram showing the hardware configuration of computer C that functions as information processing devices 1 and 1A.

[0109] The computer C includes at least one processor C1 and at least one memory C2. The memory C2 stores a program (response program) P for causing the computer C to operate as the information processing device 1 or 1A. In the computer C, the processor C1 reads and executes the program P from the memory C2, thereby realizing each function of the information processing device 1 or 1A.

[0110] The processor C1 may be, for example, a central processing unit (CPU), a graphic processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a tensor processing unit (TPU), a quantum processor, a microcontroller, or a combination thereof. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.

[0111] The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may also include a communication interface for transmitting and receiving data to and from other devices. The computer C may also include an input / output interface for connecting input / output devices such as a keyboard, mouse, display, and printer.

[0112] Furthermore, the program P can be recorded on a non-transitory tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can acquire the program P via such a recording medium M. The program P can also be transmitted via a transmission medium. Such a transmission medium can be, for example, a communication network or broadcast waves. The computer C can also acquire the program P via such a transmission medium.

[0113] Furthermore, each of the above functions of the information processing devices 1 and 1A may be realized by a single processor provided in a single computer, by multiple processors provided in a single computer working together, or by multiple processors provided in each of multiple computers working together. Furthermore, a program for causing the information processing device 1 or 1A to realize each of the above functions may be stored in a single memory provided in a single computer, or may be distributed and stored in multiple memories provided in a single computer, or may be distributed and stored in multiple memories provided in each of multiple computers.

[0114] [Additional Notes] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0115] (Appendix A) (Appendix A1) An information processing device that enables a user to respond in natural language using a machine-learned language model, comprising: a reception means for receiving utterances from the user; a generation control means for causing the language model to generate a response to the received utterance; and a presentation means for presenting the generated response to the user, wherein when the reception means receives a second utterance during a period from when a first utterance is received until when presentation of the first response to the first utterance is completed, the generation control means starts generating a second response based on the second utterance, and the presentation means presents the generated second response to the user.

[0116] (Appendix A2) The information processing device described in Appendix A1, wherein the presentation means presents progress information indicating the progress of generating the first answer to the user during the period from when the first utterance is accepted to when presentation of the first answer to the first utterance is completed.

[0117] (Appendix A3) The information processing device described in Appendix A2, wherein the generation control means causes the language model to generate the second answer on condition that a progress rate of generation of the first answer is less than a predetermined threshold when the second utterance is accepted, and the presentation means presents the progress information indicating the progress rate of generation of the first answer and the threshold to the user.

[0118] (Appendix A4) An information processing device described in any of Appendices A1 to A3, comprising an interrupt determination means that determines whether to start generating the second answer based on at least any of the content of the second utterance, the audio of the second utterance, an image of the user taken during at least a portion of the period from the end of the first utterance to the end of the second utterance, and biometric information of the user measured during at least a portion of that period.

[0119] (Appendix A5) The information processing device according to appendix A4, wherein the interrupt determination means performs the determination by applying an interrupt determination method suited to the user from among a plurality of interrupt determination methods for determining whether or not to start generating the second answer.

[0120] (Appendix A6) The information processing device according to appendix A4 or A5, further comprising a determination method determination means for determining an interrupt determination method to be applied from the next time onwards, based on an evaluation result of the interrupt determination method applied by the interrupt determination means.

[0121] (Appendix A7) The information processing device according to appendix A4 or A5, wherein the interrupt judgment method applied by the interrupt judgment means is determined using a method judgment model for determining the interrupt judgment method, and the information processing device is provided with an optimization means for optimizing the method judgment model by performing the following process for each of the determined interrupt judgment methods: updating the method judgment model based on an evaluation result for the interrupt judgment method determined using the method judgment model.

[0122] (Appendix A8) The information processing device according to any one of Appendix A1 to A3, further comprising: a feeling estimation means for estimating a feeling of the user during at least a part of a period from the end of the first utterance to the end of the second utterance; and an interrupt determination means for determining whether to start generating the second answer based on an estimation result of the feeling estimation means.

[0123] [Appendix B] (Appendix B1) A response method that enables a user to respond in natural language using a machine-learned language model, wherein at least one processor executes a reception process that receives an utterance from the user, a generation control process that causes the language model to generate a response to the received utterance, and a presentation process that presents the generated response to the user, and when the at least one processor receives a second utterance during a period from when a first utterance is received to when presentation of the first response to the first utterance is completed, the at least one processor starts generating a second response based on the second utterance and presents the generated second response to the user.

[0124] (Appendix B2) The response method described in Appendix B1, wherein the at least one processor presents progress information to the user indicating the progress of generating the first answer during the period from when the first utterance is accepted to when the presentation of the first answer to the first utterance is completed.

[0125] (Appendix B3) The response method described in Appendix B2, wherein, in the generation control process, the at least one processor causes the language model to generate the second answer on condition that a progress rate of generation of the first answer is less than a predetermined threshold when the second utterance is accepted, and the at least one processor presents the progress information indicating the progress rate of generation of the first answer and the threshold to the user.

[0126] (Appendix B4) A response method described in any of Appendices B1 to B3, including an interrupt determination process in which the at least one processor determines whether to start generating the second answer based on at least any of the content of the second utterance, the audio of the second utterance, an image of the user taken during at least a portion of the period from the end of the first utterance to the end of the second utterance, and biometric information of the user measured during at least a portion of that period.

[0127] (Appendix B5) The response method described in Appendix B4, wherein in the interrupt determination process, the at least one processor applies an interrupt determination method appropriate to the user from among a plurality of interrupt determination methods for determining whether or not to start generating the second answer, to make the determination.

[0128] (Appendix B6) The response method according to appendix B4 or B5, further comprising a determination method determination process in which the at least one processor determines an interrupt determination method to be applied from the next time onwards based on an evaluation result of the applied interrupt determination method.

[0129] (Appendix B7) The response method described in Appendix B4 or B5, wherein the interrupt judgment method applied in the interrupt judgment process is determined using a method judgment model for determining an interrupt judgment method, and the at least one processor optimizes the method judgment model by performing the process of updating the method judgment model based on an evaluation result for the interrupt judgment method determined using the method judgment model for each of the determined interrupt judgment methods.

[0130] (Appendix B8) The response method according to any one of Supplementary Notes B1 to B3, further comprising: an emotion estimation process in which the at least one processor estimates an emotion of the user during at least a part of a period from the end of the first utterance to the end of the second utterance; and an interrupt determination process in which the at least one processor determines whether to start generating the second answer based on an estimation result of the emotion estimation process.

[0131] [Appendix C] (Appendix C1) A response program that enables a user to respond in natural language using a machine-learned language model, the response program causing a computer to function as a reception means for receiving utterances from the user, a generation control means for causing the language model to generate a response to the received utterance, and a presentation means for presenting the generated response to the user, wherein when the reception means receives a second utterance during a period from when a first utterance is received to when presentation of the first response to the first utterance is completed, the generation control means starts generating a second response based on the second utterance, and the presentation means presents the generated second response to the user.

[0132] (Appendix C2) The response program described in Appendix C1, wherein the presentation means presents progress information indicating the progress of generating the first answer to the user during the period from when the first utterance is accepted to when the presentation of the first answer to the first utterance is completed.

[0133] (Appendix C3) The response program according to Appendix C2, wherein the generation control means causes the language model to generate the second answer on condition that a progress rate of generation of the first answer is less than a predetermined threshold when the second utterance is accepted, and the presentation means presents the progress information indicating the progress rate of generation of the first answer and the threshold to the user.

[0134] (Appendix C4) A response program described in any of Appendices C1 to C3, which causes the computer to function as an interrupt determination means that determines whether to start generating the second answer based on at least any of the content of the second utterance, the audio of the second utterance, an image of the user taken during at least a portion of the period from the end of the first utterance to the end of the second utterance, and biometric information of the user measured during at least a portion of that period.

[0135] (Appendix C5) The response program according to appendix C4, wherein the interrupt determination means performs the determination by applying an interrupt determination method corresponding to the user from among a plurality of interrupt determination methods for determining whether or not to start generating the second response.

[0136] (Appendix C6) The response program according to appendix C4 or C5, which causes the computer to function as a determination method determination means that determines an interrupt determination method to be applied from the next time onwards, based on an evaluation result of the interrupt determination method applied by the interrupt determination means.

[0137] (Appendix C7) The response program described in Appendix C4 or C5 causes the computer to function as an optimization means that optimizes the method determination model by executing the following process for each determined interrupt judgment method: the interrupt judgment method applied by the interrupt judgment means is determined using a method determination model for determining the interrupt judgment method, and the method determination model is updated based on an evaluation result for the interrupt judgment method determined using the method determination model.

[0138] (Appendix C8) The response program according to any one of appendices C1 to C3, causing the computer to function as: emotion estimation means for estimating an emotion of the user during at least a part of a period from the end of the first utterance to the end of the second utterance; and interrupt determination means for determining whether to start generating the second answer based on an estimation result of the emotion estimation means.

[0139] [Appendix D] (Appendix D1) An information processing device that enables natural language interaction with a user using a machine-learned language model, the information processing device comprising at least one processor, wherein the at least one processor executes a reception process for receiving utterances from the user, a generation control process for causing the language model to generate an answer to the received utterance, and a presentation process for presenting the generated answer to the user, and when the at least one processor receives a second utterance during a period from when a first utterance is received to when presentation of the first answer to the first utterance is completed, the information processing device starts generating a second answer based on the second utterance and presents the generated second answer to the user.

[0140] The information processing device may further include a memory, and the memory may store a program for causing the at least one processor to execute each of the processes.

[0141] (Appendix D2) The information processing device described in Appendix D1, wherein the at least one processor presents progress information to the user indicating the progress of generating the first answer during the period from when the first utterance is accepted to when the presentation of the first answer to the first utterance is completed.

[0142] (Appendix D3) The information processing device described in Appendix D2, wherein, in the generation control process, the at least one processor causes the language model to generate the second answer, on condition that a progress rate of generation of the first answer is less than a predetermined threshold when the second utterance is accepted, and presents the progress information indicating the progress rate of generation of the first answer and the threshold to the user during generation of the first answer.

[0143] (Appendix D4) An information processing device described in any of Appendices D1 to D3, wherein the at least one processor executes an interrupt determination process to determine whether to start generating the second answer based on at least any of the content of the second utterance, the audio of the second utterance, an image of the user taken during at least a portion of the period from the end of the first utterance to the end of the second utterance, and biometric information of the user measured during at least a portion of that period.

[0144] (Appendix D5) The information processing device described in Appendix D4, wherein in the interrupt determination process, the at least one processor applies an interrupt determination method corresponding to the user from among a plurality of interrupt determination methods for determining whether or not to start generating the second answer to make the determination.

[0145] (Appendix D6) The information processing device according to appendix D4 or D5, wherein the at least one processor executes a determination method determination process to determine an interrupt determination method to be applied from the next time onwards, based on an evaluation result of the interrupt determination method applied in the interrupt determination process.

[0146] (Appendix D7) The information processing device described in Appendix D4 or D5, wherein the interrupt determination method applied in the interrupt determination process is determined using a method determination model for determining an interrupt determination method, and the at least one processor performs an optimization process to optimize the method determination model by performing, for each of the determined interrupt determination methods, a process of updating the method determination model based on an evaluation result for the interrupt determination method determined using the method determination model.

[0147] (Appendix D8) The information processing device according to any one of Appendices D1 to D3, wherein the at least one processor executes an emotion estimation process that estimates an emotion of the user during at least a part of a period from the end of the first utterance to the end of the second utterance, and an interrupt determination process that determines whether to start generating the second answer based on an estimation result of the emotion estimation process.

[0148] [Appendix E] A non-transient recording medium having recorded thereon a response program, the program causing a computer to function as an information processing device that enables responses to a user in natural language using a machine-learned language model, the program causing the computer to execute a reception process for receiving utterances from the user, a generation control process for causing the language model to generate a response to the received utterance, and a presentation process for presenting the generated response to the user, wherein, in the reception process, if the computer receives a second utterance during the period from when a first utterance is received until the presentation of the first response to the first utterance is completed, the computer starts generating a second response based on the second utterance and presents the generated second response to the user. [Explanation of symbols]

[0149] 1. Information processing equipment 101 Reception section (reception means) 102 Generation control unit (generation control means) 103 Presentation unit (presentation means) 1A Information processing equipment 101A Reception unit (reception means) 102A Generation control unit (generation control means) 103A Presentation unit (presentation means) 104A Judgment method determining unit (judgment method determining means) 105A Interrupt judgment unit (interrupt judgment means) 106A Emotion estimation unit (emotion estimation means) 108A Optimization unit (optimization means) 111A Language Model

Claims

1. An information processing device that enables a user to respond in natural language using a machine-learned language model, A receiving means for receiving an utterance from the user; a generation control means for causing the language model to generate a response to the received utterance; a presentation means for presenting the generated answer to the user; An information processing device wherein, when the receiving means receives a second utterance during the period from when the first utterance is received to when the presentation of a first answer to the first utterance is completed, the generation control means starts generating a second answer based on the second utterance, and the presentation means presents the generated second answer to the user.

2. 2. The information processing device according to claim 1, wherein the presentation means presents to the user progress information indicating the progress of generating the first answer during a period from when the first utterance is accepted to when presentation of the first answer to the first utterance is completed.

3. the generation control means causes the language model to generate the second answer on condition that a progress rate of generation of the first answer is less than a predetermined threshold when the second utterance is accepted; The information processing apparatus according to claim 2 , wherein the presenting means presents the progress information indicating a progress rate of the generation of the first answer and the threshold value to the user.

4. An information processing device as described in any one of claims 1 to 3, comprising an interrupt determination means for determining whether or not to start generating the second answer based on at least one of the content of the second utterance, the audio of the second utterance, an image of the user taken during at least a portion of the period from the end of the first utterance to the end of the second utterance, and biometric information of the user measured during at least a portion of that period.

5. The information processing apparatus according to claim 4 , wherein the interrupt determination means performs the determination by applying an interrupt determination method suited to the user from among a plurality of interrupt determination methods for determining whether or not to start generating the second answer.

6. 5. The information processing apparatus according to claim 4, further comprising: a determination method determining means for determining an interrupt determination method to be applied from the next time onward, based on an evaluation result of the interrupt determination method applied by said interrupt determination means.

7. the interruption determination method applied by the interruption determination means is determined using a method determination model for determining an interruption determination method; 5. The information processing apparatus according to claim 4, further comprising: an optimization unit that optimizes the method determination model by executing, for each of the determined interrupt determination methods, a process of updating the method determination model based on an evaluation result for the interrupt determination method determined using the method determination model.

8. a feeling estimation means for estimating a feeling of the user during at least a part of a period from the end of the first utterance to the end of the second utterance; The information processing device according to claim 1 , further comprising: an interrupt determination means configured to determine whether to start generating the second answer based on an estimation result of the emotion estimation means.

9. A response method that enables a user to respond in natural language using a machine-learned language model, At least one processor a reception process for receiving an utterance from the user; a generation control process for causing the language model to generate a response to the received utterance; a presentation process for presenting the generated answer to the user; A response method in which, when the at least one processor receives a second utterance during the period from when the first utterance is received to when the presentation of the first answer to the first utterance is completed, the processor starts generating a second answer based on the second utterance and presents the generated second answer to the user.

10. A response program that enables a user to respond in natural language using a machine-learned language model, Computer, a receiving means for receiving an utterance from the user; generation control means for causing the language model to generate a response to the received utterance; and a presentation means for presenting the generated answer to the user; A response program in which, when the receiving means receives a second utterance during the period from when the first utterance is received to when the presentation of a first answer to the first utterance is completed, the generation control means starts generating a second answer based on the second utterance, and the presentation means presents the generated second answer to the user.

Images