Dialogue device, dialogue method, and dialogue program

Abstract

Based on the user's past utterances, it can provide accurate responses to the user's utterances. [Solution] The dialogue device includes an acquisition unit that, when it detects a user's utterance, uses a first trained language model to acquire a summary related to the current utterance from utterance data that contains at least summaries created in relation to the user's past utterances; a creation unit that, when the acquisition unit is able to acquire a related summary, creates a prompt to input to a second trained language model, including the summary related to the current utterance; and when the acquisition unit is unable to acquire a related summary, creates a prompt based on the current utterance.

Description

Technical Field

[0001] The present disclosure relates to an interaction device, an interaction method, and an interaction program.

Background Art

[0002] There is known a voice input device that outputs the dialogue history before interruption when voice input resumes and provides it to the speaker.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By including the past conversation history in the prompt input to the trained language model, the prior art realizes a conversation that takes into account the content of past conversations. However, if all past conversations are included in the prompt, it will cause limitations in the amount of the prompt that can be input to the trained language model, an increase in response time, an increase in usage fees, etc.

[0005] The present disclosure has been made in view of the above, and an object thereof is to provide an interaction device, an interaction method, and an interaction program capable of interacting while grasping the user's intention.

Means for Solving the Problems

[0006] To solve the above-mentioned problems and achieve the objectives, the dialogue device according to this disclosure is characterized by comprising: an acquisition unit that, when it detects a user's utterance, acquires a summary related to the current utterance from utterance data in which at least summaries created in relation to the user's past utterances are recorded, using a first trained language model; a creation unit that, when the acquisition unit has been able to acquire the related summary, creates a prompt to input the current utterance, including the related summary, into a second trained language model; and when the acquisition unit has been unable to acquire the related summary, creates the prompt based on the current utterance.

[0007] Furthermore, the dialogue method relating to this disclosure is a dialogue method performed by a dialogue device, and is characterized by including: an acquisition step in which, when a user's utterance is detected, a summary related to the current utterance is obtained from utterance data in which at least summaries created in relation to the user's past utterances are recorded, using a first trained language model; a creation step in which, if the related summary can be obtained by the acquisition step, a prompt is created to input the current utterance, including the related summary, into a second trained language model; and if the related summary cannot be obtained by the acquisition step, a prompt is created based on the current utterance.

[0008] Furthermore, the dialogue program relating to this disclosure is characterized in that, when it detects a user's utterance, it causes a computer to execute an acquisition procedure to acquire a summary related to the current utterance using a first trained language model from utterance data which contains at least summaries created in relation to the user's past utterances; a creation procedure to create a prompt to input the current utterance, including the related summary, into a second trained language model if the related summary can be acquired by the acquisition procedure; and a creation procedure to create the prompt based on the current utterance if the related summary cannot be acquired by the acquisition procedure. [Brief explanation of the drawing]

[0009] [Figure 1]Figure 1 shows an example of how the dialogue device is used. [Figure 2] Figure 2 is a block diagram showing an example configuration of the dialogue device according to Embodiment 1. [Figure 3] Figure 3 shows an example of speech data generated based on the utterance according to Embodiment 1. [Figure 4] Figure 4 shows an example of the processing content for obtaining a new utterance and related summary from speech data according to Embodiment 1. [Figure 5] Figure 5 shows an example of the processing content for obtaining a new utterance and related summary from utterance data according to Embodiment 1. [Figure 6] Figure 6 shows an example of the process for creating a prompt, including a new utterance and a related summary, according to Embodiment 1. [Figure 7] Figure 7 shows an example of the processing content for responding to a new utterance according to Embodiment 1. [Figure 8] Figure 8 is a flowchart showing an example of the dialogue processing flow of the dialogue device according to Embodiment 1. [Figure 9] Figure 9 shows an example of the processing content for obtaining a summary of relevance levels above a threshold from speech data according to Embodiment 3. [Figure 10] Figure 10 shows an example of the processing content for obtaining the most recent summary from the speech data among the related summaries according to Embodiment 4. [Figure 11] Figure 11 shows an example of the processing content for obtaining summaries from speech data that share a common speech environment among related summaries according to Embodiment 5. [Figure 12] Figure 12 shows an example of the application of the dialogue device. [Figure 13] Figure 13 is a hardware configuration diagram showing an example of a computer that implements the functions of the dialogue devices of Embodiment 1, Embodiment 2, Embodiment 3, Embodiment 4, Embodiment 5, and Embodiment 6. [Modes for carrying out the invention]

[0010] Embodiments of the dialogue device, dialogue method, and dialogue program relating to this disclosure will be described in detail below with reference to the drawings. However, this disclosure is not limited to the embodiments described below. Furthermore, each embodiment can be combined as appropriate within a non-consistent scope.

[0011] [Embodiment 1] Figure 1 shows an example of how the dialogue device 1 is used. As shown in Figure 1, the dialogue device 1 is a computer or similar device installed in the vehicle, and is a device that performs dialogue with the vehicle user (hereinafter sometimes simply referred to as "user"), who is the driver or passenger. For example, the vehicle user activates the dialogue device 1 installed in the vehicle and speaks to the dialogue device 1. Upon receiving this speech, the dialogue device 1 responds to the user's speech.

[0012] The dialogue device 1 may be a dedicated navigation device built into or mounted in the vehicle. For example, the dialogue device 1 may be a single device having a navigation function, a display function, and a dialogue function.

[0013] Furthermore, users can substitute their everyday portable devices (e.g., smartphones, tablet devices, notebook PCs, desktop PCs, PDAs, etc.) with the dialogue device 1 by installing a predetermined application on them. For example, when a portable device is used as the dialogue device 1, it may be installed on the dashboard of a vehicle while driving.

[0014] By the way, a conventional dialogue device creates a prompt that includes a new utterance and all past dialogues and inputs it into a trained language model. By inputting it into the trained language model, the device answers based on the user's past utterances. However, since the conventional dialogue device creates a prompt that includes a new utterance and all of the user's past utterances, the amount of the prompt increases. Along with this, problems such as a limitation on the amount of the prompt input into the trained language model, an increase in the response time of the trained language model, or an increase in the usage fee of the trained language model occur in the conventional dialogue device.

[0015] Therefore, in the present disclosure, a dialogue device 1 that can dialogue with a user on the premise of past dialogues is described by using a trained language model to obtain a summary most relevant to the content of the user's utterance, creating a prompt that includes the obtained summary in the user's utterance, and inputting the prompt into the trained language model.

[0016] Specifically, when the dialogue device 1 of the present disclosure detects a user's utterance, it obtains a summary related to the current utterance by using a first trained language model 62 from utterance data 61 in which at least a summary created related to the user's past utterances is recorded. When the dialogue device 1 can obtain a relevant summary, it creates a prompt for inputting into a second trained language model 63 that includes the summary related to the current utterance, and when it cannot obtain a relevant summary, it creates a prompt based on the current utterance. Then, the dialogue device 1 inputs the created prompt into the second trained language model 63 and answers the user's current utterance.

[0017] For example, a user might say to dialogue device 1, "What kind of Christmas cake would you like?" Dialogue device 1 then detects the utterance and uses the first trained language model 62 to obtain a summary related to Christmas cakes from the utterance data 61. Dialogue device 1 then creates a prompt including the obtained summary and the current utterance and inputs it into the second trained language model 63. Dialogue device 1 then responds, "Last year, you chose the chocolate Noel from S Confectionery. How about the following options? Chocolate Noel from S Confectionery, Fresh cream Noel from S Confectionery..."

[0018] Thus, the dialogue device 1 can include a summary related to the user's utterances in the prompt, even without including all of the user's past utterances, and can provide an appropriate response based on the user's past utterances.

[0019] (Configuration of Dialogue Device 1) Next, an example of the functional configuration of the dialogue device 1 according to Embodiment 1 will be described. Figure 2 is a block diagram showing an example of the configuration of the dialogue device 1 according to Embodiment 1.

[0020] As shown in Figure 2, the dialogue device 1 consists of a communication unit 2, an audio input unit 3, an output unit 4, a control unit 5, a storage unit 6, and the like.

[0021] The communication unit 2, under the control of the control unit 5, connects to the server via the network and sends and receives various types of information. For example, if speech data or trained language models contained in the memory unit 6 of the dialogue device 1 are stored in an external device such as a server, the communication unit 2 sends and receives data via the network.

[0022] The voice input unit 3 includes a microphone or the like for capturing sound, and generates voice information based on the captured sound. For example, under the control of the control unit 5, the voice input unit 3 generates voice information when it detects user speech. The voice input unit 3 may also generate voice information for speech environments other than user speech. The voice input unit 3 then outputs the generated voice information to the control unit 5.

[0023] The output unit 4 outputs information under the control of the control unit 5. For example, the output unit 4 includes an audio output device and outputs the information input from the control unit 5 as audio. The output unit 4 also includes a display device and, in addition to audio output, may display characters, symbols, etc., on the in-vehicle display of the dialogue device 1 or on various displays connected to the dialogue device 1.

[0024] The memory unit 6 stores various programs executed by the control unit 5 (dialogue programs according to this embodiment), as well as data necessary for the control unit 5 to perform processing. As shown in Figure 2, the memory unit 6 includes speech data 61, a first trained language model 62, a second trained language model 63, and the like.

[0025] The speech data 61 stores summaries of utterances previously made by the user. Figure 3 shows an example of the speech data 61. As shown in Figure 3, the speech data 61 is information that associates "date and time" with "summary". Here, the date and time records the date and time when the user's utterance was detected. The summary records a summary of what the user said to the dialogue device 1.

[0026] In the example in Figure 3, it is shown that a summary of what the user said to the dialogue device 1 at the date and time "10 / 17 14:13" – "The user is searching for information about local products of Kawagoe..." – was recorded in the speech data 61.

[0027] The first trained language model 62 is a language model trained to search for data and output search results according to input prompts. For example, if the first trained language model 62 is given a prompt created to retrieve a summary related to the current utterance from the utterance data 61, it will search for the relevant summary from the utterance data 61 and output it.

[0028] The second trained language model 63 is a language model trained to create and output responses according to input prompts. For example, if the second trained language model 63 receives a prompt that is designed to respond based on a summary related to the current utterance, it will output the response to the output unit 4 according to the instructions in the prompt.

[0029] Note that the first trained language model 62 and the second trained language model 63 may be the same. For example, the first trained language model 62 and the second trained language model 63 may be, for example, large language models (LLMs). Also, the first trained language model 62 and the second trained language model 63 may be a group of language models that include multiple language models. Note that the configuration and type of the models are not particularly limited, as long as the first trained language model 62 and the second trained language model 63 are machine learning models that have been trained to handle the processing described later.

[0030] The control unit 5 is implemented by a controller such as a CPU (Central Processing Unit) or MPU (Micro Processing Unit) executing various programs stored in the memory unit 6, thereby controlling the operation of the entire interactive device 1. Note that the control unit 5 may be composed of integrated circuits other than a CPU or MPU, such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array).

[0031] As shown in Figure 2, this control unit 5 includes an acquisition unit 51, a creation unit 52, and the like.

[0032] When the voice input unit 3 detects human speech, the acquisition unit 51 uses the first trained language model 62 to acquire a summary related to the user's speech from the speech data 61.

[0033] Here, we will use Figures 4 and 5 to explain how to obtain relevant summaries. Figure 4 shows the creation of prompts to be input to the first trained language model 62, and Figure 5 shows the response from the first trained language model 62.

[0034] Figure 4 shows the process of inputting a new utterance and associated summary related to Embodiment 1 into the first trained language model 62 in order to obtain it from the utterance data 61. For example, as shown in Figure 4, first the acquisition unit 51 receives the user's utterance, "What would be a good Christmas present for my junior high school daughter? Come to think of it, what did I get her for her birthday? I'd like something different." via the voice input unit 3.

[0035] Next, as shown in Figure 4A, the acquisition unit 51 creates a prompt in which it sets the question content in "Q:" to indicate a question and the content of the current utterance in "Current Utterance:" to indicate that it is the current utterance, in order to acquire a summary associated with the detected utterance. In the example in Figure 4A, the question content indicating the instruction to the first trained language model 62 is set to "Please acquire the following summary associated with the current utterance from the utterance data." The content of the current utterance is set to "What would be a good Christmas present for my junior high school daughter? Come to think of it, what did I get her for her birthday? I'd like something different." The acquisition unit 51 then inputs the created prompt into the first trained language model 62.

[0036] Next, as shown in Figure 5B, the first trained language model 62 outputs the summary #1 "9 / 15 10:13 The user said to his first-year junior high school daughter..." which is the search result from the speech data 61, to "A:", which indicates the answer content, and "The following summary related to the current utterance has been obtained." Then, the acquisition unit 51 acquires summary #1 and outputs it to the creation unit 52.

[0037] Returning to Figure 2, if the acquisition unit 51 is able to acquire a summary, the creation unit 52 creates a prompt for input to the second trained language model 63, including the current utterance and the acquired summary. If the acquisition unit 51 is unable to acquire a summary, the creation unit 52 creates a prompt for input to the second trained language model 63 based on the current utterance.

[0038] Here, we will use Figures 6 and 7 to explain the process of creating a prompt that includes a relevant summary and responding to the user. Figure 6 is a diagram showing the process of creating a prompt, and Figure 7 is a diagram showing the process of responding to the user.

[0039] As shown in Figure 6C, the creation unit 52 creates a prompt to input to the second trained language model 63 to respond to the current utterance, including the detected current utterance "[Current Utterance] My daughter is in the first year of junior high school..." and the acquired summary "Summary #1 The user is...", in addition to the question "Q: Based on Summary #1, please answer the following utterance," which provides instructions to the second trained language model 63. Next, the creation unit 52 inputs the created prompt to the second trained language model 63.

[0040] Next, as shown in Figure 7, the second trained language model 63, following the prompt's instructions, creates the answer "A: When thinking of Christmas present ideas..." and outputs it to the output unit 4. The output unit 4 responds to the user's current utterance by outputting "When thinking of Christmas present ideas..." in voice. The output unit 4 may also display "When thinking of Christmas present ideas..." as text on the in-car display, either along with or instead of voice. As a result, the user can recall that they gave their junior high school daughter a metronome for her birthday and think of a present based on that birthday present. They can also choose a Christmas present from the present candidates suggested by the dialogue device 1.

[0041] (Processing flow of Dialogue Device 1) Next, an example of the processing procedure by the dialogue device 1 according to Embodiment 1 will be described using Figure 8. Figure 8 is a flowchart showing an example of the dialogue processing flow of the dialogue device 1 according to Embodiment 1. Note that each step in the flowchart shown in Figure 8 can be executed in a different order, and additional or omitted processes may be included.

[0042] First, the dialogue device 1 detects the user's utterance (S1). Next, the dialogue device 1 uses the first trained language model 62 to obtain a summary of past utterances related to the current utterance from the utterance data 61 (S2). If the dialogue device 1 is able to obtain a summary of past utterances related to the current utterance (S2; YES), it creates a prompt that includes the detected utterance and the obtained summary (S3). If the dialogue device 1 is unable to obtain a summary of past utterances related to the current utterance (S2; NO), it creates a prompt based on the currently detected utterance without including a summary of past utterances (S4).

[0043] Subsequently, the dialogue device 1 inputs the created prompt into the second trained language model 63 (S5). Then, the dialogue device 1 responds to the user based on the response created by the second trained language model 63 (S6), and the process ends.

[0044] (effect) The above-described embodiment 1 provides the following effects. The dialogue device 1 creates a prompt that includes a summary of past utterances related to the detected utterance. This allows the dialogue device 1 to reduce the amount of prompts while maintaining the quality of the prompts.

[0045] Dialogue device 1 can accurately respond to user utterances even when limited by the amount of prompts, by reducing the amount of prompts while maintaining the quality of the prompts. Furthermore, dialogue device 1 can shorten the response time of the trained language model, improving user convenience. In addition, dialogue device 1 can reduce the usage fees for the trained language model.

[0046] [Embodiment 2] Embodiment 1 described an example in which the dialogue device 1 retrieves a summary related to the user's utterance by searching for it in the summaries contained in the utterance data 61. However, if a large amount of summaries are stored in the utterance data 61, the search may take a long time. Therefore, Embodiment 2 describes a process to shorten the search time by including "tags" in the utterance data 61 of the dialogue device 1 and using those tags.

[0047] Note that a detailed explanation of the configuration and processing of the dialogue device 1 according to Embodiment 2, which is the same as that of the dialogue device 1 according to Embodiment 1, will be omitted. Below, the configuration and processing content that differ from Embodiment 1 will be described. In this Embodiment 2, the speech data 61 and acquisition unit 51, which differ from those in Embodiment 1, will be described in detail.

[0048] The speech data 61 stores a summary of the user's past utterances and identification information such as tags. The speech data 61 is information associated with "tags" in addition to the "date and time" and "summary" described in the embodiment. Here, the tags record keywords from the summary of the user's utterances.

[0049] For example, the speech data 61 records a summary of what the user said to the dialogue device 1 at the date and time "10 / 17 14:13": "The user is searching for information about local products of Kawagoe..." and the tag "Kawagoe local products...".

[0050] When the acquisition unit 51 acquires a summary related to the user's utterance, it creates a prompt to acquire relevant tags from the utterance data 61 and inputs it into the first trained language model 62. The acquisition unit 51 then outputs the acquired tags to the creation unit 52.

[0051] Alternatively, the acquisition unit 51 may select from the acquired tags, obtain a summary corresponding to the selected tag, and output it to the creation unit 52.

[0052] The acquisition unit 51 may also create a prompt to acquire the summary and tags simultaneously and input it to the first trained language model 62. In this case, the acquisition unit 51 outputs the acquired summary and tags to the creation unit 52.

[0053] Here, we will explain a specific example of the process from obtaining tags to creating prompts. For example, the acquisition unit 51 receives the user's utterance, "What would be a good Christmas present for my junior high school daughter? Come to think of it, what did I get her for her birthday? I'd like something different." from the voice input unit 3.

[0054] Next, the acquisition unit 51 creates a prompt containing the question "Q: Please acquire the following tags related to the current utterance from the utterance data..." which instructs the first trained language model 62 to acquire tags related to the received utterance, and inputs it into the first trained language model 62. The first trained language model 62 then searches the utterance data 61 contained in the input prompt for the related tag "Birthday present, my daughter in the first year of junior high school..." and outputs it.

[0055] Next, the acquisition unit 51 obtains summaries associated with the output tags from the speech data 61. For example, the acquisition unit 51 obtains summaries associated with both "birthday present" and "my daughter in the first year of junior high school" from the speech data 61. Alternatively, the acquisition unit 51 may create a prompt instructing the first trained language model 62 to obtain summaries related to the tags, and input this prompt into the first trained language model 62.

[0056] As described above, the dialogue device 1 includes information identification information in the speech data 61, thereby facilitating the retrieval of summaries related to the user's utterances and contributing to faster processing. This is particularly useful when a large amount of data is stored in the speech data 61.

[0057] [Embodiment 3] The dialogue device 1 of this disclosure can improve the accuracy of its responses by utilizing relevance when the dialogue device 1 searches for summaries. Specifically, in Embodiment 3, when the dialogue device 1 searches for summaries related to the user's utterance from the utterance data 61, it calculates relevance using the first trained language model 62 and obtains summaries whose relevance is above a predetermined threshold.

[0058] Therefore, Embodiment 3 describes a dialogue device 1 that utilizes relevance when searching for summaries. Furthermore, the dialogue device 1 according to Embodiment 3 will be described in detail, including the first trained language model 62 and acquisition unit 51, which differ from those in Embodiment 1.

[0059] The first trained language model 62 is a language model trained to search for data according to input prompts, and to calculate and output the degree of relevance between the current utterance and the retrieved data. For example, if the first trained language model 62 is given prompts to retrieve a summary related to the current utterance from the utterance data 61 and to calculate the degree of relevance between the retrieved summary and the current utterance, it will search for the relevant summary from the utterance data 61, calculate the degree of relevance of the retrieved summary and output it.

[0060] When the voice input unit 3 detects human speech, the acquisition unit 51 uses the first trained language model 62 to acquire a summary of the user's speech that exceeds a predetermined threshold from the speech data 61.

[0061] Specifically, when the acquisition unit 51 acquires summaries related to the user's utterance, it inputs information to the first trained language model 62 in order to calculate the relevance of the relevant summaries. For example, the acquisition unit 51 creates a prompt that includes the question "Q: Acquire the following summaries related to the current utterance from the utterance data. Calculate the relevance of the acquired summaries..." and inputs it to the first trained language model 62. The first trained language model 62 then acquires the relevant summaries from the utterance data 61 and calculates the relevance.

[0062] Figure 9 shows an example of the process for obtaining summaries including relevance. As shown in Figure 9D, the first trained language model 62 outputs a summary obtained from speech data 61, for example, "#1 9 / 15 10:13 The user is thinking about a birthday present for his junior high school daughter..." along with a calculated relevance of "Relevance 80". The acquisition unit 51 obtains summaries with a predetermined relevance or higher from the output summaries.

[0063] For example, the acquisition unit 51 assumes that the relevance threshold is 60. Then, as shown in Figure 9D, the acquisition unit 51 acquires the summary of "#1" whose relevance is 60 or higher. Alternatively, the acquisition unit 51 may create a prompt instructing the first trained language model 62 to acquire a summary of a certain threshold or higher from the speech data 61, and input this prompt to acquire a summary of a certain threshold or higher.

[0064] As described above, the dialogue device 1 improves the accuracy of the summaries it obtains by calculating the degree of relevance and acquiring summaries with a relevance level above a predetermined threshold. As a result, the dialogue device 1 can provide more accurate responses to the user's utterances.

[0065] [Embodiment 4] Incidentally, the disclosed dialogue device 1 can respond based on the most recent conversation by acquiring the latest summary when it acquires multiple summaries related to the user's utterance. Therefore, Embodiment 4 describes a dialogue device 1 that acquires the latest summary with the most recent utterance date and time when it acquires multiple summaries related to the user's utterance. Furthermore, the acquisition unit 51 of the dialogue device 1 according to Embodiment 4, which has a different configuration and processing content from Embodiment 1, will be described in detail.

[0066] The acquisition unit 51 acquires the latest summary when the first trained language model 62 acquires multiple summaries related to the user's utterance.

[0067] Figure 10 shows an example of the process for obtaining the latest summary. For example, as shown in Figure 10E, the first trained language model 62 obtains multiple related summaries from the utterance data 61. The acquisition unit 51 obtains the summary that is newest to the date and time the utterance was detected from among the obtained summaries as the latest summary. Alternatively, the acquisition unit 51 may create a prompt instructing the first trained language model 62 to obtain the latest related summary from the utterance data 61, and input this prompt to the first trained language model 62 to obtain the latest related summary.

[0068] For example, as shown in Figure 10E, the acquisition unit 51 acquires the summary of the most recent utterance detected at "9 / 15 10:13".

[0069] Furthermore, as shown in Embodiment 3, if the acquired summary includes a relevance score, the acquisition unit 51 may acquire the most recent summary among multiple summaries with a relevance score above a predetermined threshold if such summaries are acquired. The acquisition unit 51 may also create a prompt from the speech data 61 indicating that it wants to acquire the most recent summary that is above a predetermined threshold, and input this prompt to the first trained language model 62 to acquire the most recent relevant summary.

[0070] As described above, the dialogue device 1 can obtain the latest summary related to the user's utterance, enabling it to provide a response based on the most recent utterance. For example, the process described in Embodiment 4 is particularly useful when the user and the dialogue device 1 are having a continuous conversation.

[0071] [Embodiment 5] Embodiment 1 described an example in which a summary related to the user's utterance is retrieved by searching for it in the summaries included in the utterance data 61. However, depending on the speaker and the utterance environment, it is possible that summaries that should not be retrieved may be included, even if they are related to the utterance content. Therefore, Embodiment 5 describes a process in which the utterance data 61 of the dialogue device 1 includes the utterance environment, which indicates the environment at the time of the utterance, and retrieves a summary of the utterance environment common to the current utterance.

[0072] Note that a detailed explanation of the dialogue device 1 according to Embodiment 5, which has the same configuration and processing as the dialogue device 1 according to Embodiment 1, will be omitted. Below, the speech data 61 and acquisition unit 51, which differ from those in Embodiment 1, will be described in detail.

[0073] The speech data 61 stores summaries of past utterances made by the user, as well as information about the utterance environment. In addition to the "date and time" and "summary" described in the embodiment, the speech data 61 also includes information about the "utterance environment." Here, the utterance environment records the in-vehicle environment in which the user is speaking.

[0074] For example, speech data 61 records a summary of what the user said to the dialogue device 1 at the date and time "10 / 17 14:13": "The user is searching for information about local products in Kawagoe..." Furthermore, the speech data 61 is processed by the voice input unit 3 of the dialogue device 1 to determine the user who is speaking based on the type of voice, and information such as the number of users and the user's attributes (father, mother) is recorded. For example, if the father was the only passenger in the car when the user was speaking, the speaking environment would be recorded as "1 person" and the passenger's attribute "father". The user may also set the passengers to the dialogue device 1 when boarding. Alternatively, the speaking user may be estimated using general voice analysis or other methods.

[0075] Figure 11 shows the process for obtaining summaries common to the speech environment. For example, as shown in F in Figure 11, for the current utterance, the speech environment is "1 crew member, attribute: father," and the utterance content is "What would be a good Christmas present for my junior high school daughter? Come to think of it, what did I get her for her birthday? I'd like something different."

[0076] The acquisition unit 51 creates a prompt containing the question "Q: Please obtain the following summary related to the current utterance from the speech data..." which is an instruction to the first trained language model 62, in order to obtain a summary related to the detected utterance, and inputs it to the first trained language model 62. The first trained language model 62 then outputs a summary including the speech environment from the speech data 61, as shown in "A: The following summary related to the current utterance..." in G of Figure 11.

[0077] The acquisition unit 51 obtains the summary "#2" which is common to the speech environment "1 occupant, attribute: father" of the current utterance from the acquisition results shown in G of Figure 11. Alternatively, the acquisition unit 51 may create a prompt instructing the first trained language model 62 to obtain summaries from the speech data 61 that are related to the current utterance and share a common speech environment, and input this prompt into the first trained language model 62.

[0078] As described above, in the dialogue device 1 according to Embodiment 5, by including the speech environment in the speech data 61, it is possible to obtain a summary of speech in the same environment as the current speech in response to the user's speech. This makes it possible to obtain summaries of speech from the same person, and to provide even more accurate responses to the user's speech.

[0079] [Embodiment 6] Now, while embodiments of this disclosure have been described, this disclosure may be implemented in various other forms besides those described above.

[0080] (Examples of application) By the way, although the above embodiment was described using an example where the dialogue device 1 is an in-vehicle device, it is not limited to this. The dialogue device 1 can be used not only in in-vehicle devices but also in various other devices that can interact with a user. Here, we will illustrate some examples of applications for the dialogue device 1.

[0081] Figure 12 shows an example of the application of the dialogue device 1. For example, as shown in Figure 12(1), the dialogue device 1 can be used with a smart speaker. In this case, the user activates a smart speaker that has the same functions as the dialogue device 1 and speaks to the smart speaker. Upon receiving this speech, the smart speaker executes the processes described in the above embodiment and responds to the user's speech.

[0082] Furthermore, as shown in Figure 12(2), for example, the dialogue device 1 can be used as a dialogue application for a smartphone. In this case, the user launches a dialogue application with functions equivalent to those of the dialogue device 1 and speaks to the smartphone. The dialogue application, upon receiving this utterance, executes the processes described in the above embodiment and responds to the user's utterance. Note that the dialogue device 1 is not limited to smartphones; it can also be applied to mobile phones, tablets, and other devices.

[0083] Furthermore, as shown in Figure 12(3), for example, it can be used as the dialogue device 1 in a personal computer. In this case, the user launches dialogue software or the like that has the same functionality as the dialogue device 1 and speaks to the personal computer. The dialogue software or the like that receives this utterance executes the processes described in the above embodiment and responds to the user's utterance. Note that the dialogue device 1 is not limited to personal computers, but can be similarly applied to various computers such as game consoles with internet communication capabilities.

[0084] (modified version) The first trained language model 62 and the second trained language model 63 of the dialogue device 1 can determine the content of an instruction from the context and respond even without inputting the instruction "Q:".

[0085] In Embodiment 3, in addition to using the first trained language model 62, a vector database can be used to calculate the relevance of the summary. In this case, the dialogue device 1 stores summaries of the user's past utterances in the vector database. The acquisition unit 51 can also acquire the summary and relevance related to the current utterance from the vector database where the user's past summaries are stored.

[0086] (Numerical values, etc.) The numerical values ​​and thresholds (for example, arbitrary numerical values) used in the above embodiment are merely examples and can be changed as desired.

[0087] [Hardware configuration] Furthermore, the dialogue device 1 according to Embodiments 1 to 6 described above is realized by a computer 1000 having a configuration such as that shown in Figure 13. The following explanation will use the dialogue device 1 as an example. Figure 13 is a hardware configuration diagram showing an example of a computer that realizes the functions of the dialogue device 1 of Embodiments 1 to 6. The computer 1000 has a CPU 1100, RAM 1200, ROM 1300, HDD 1400, communication interface (I / F) 1500, input / output interface (I / F) 1600, and media interface (I / F) 1700.

[0088] The CPU 1100 operates based on programs stored in the ROM 1300 or HDD 1400, controlling various components. The ROM 1300 stores boot programs executed by the CPU 1100 when the computer 1000 starts up, as well as programs that depend on the computer 1000's hardware.

[0089] The HDD1400 stores programs executed by the CPU1100, as well as data used by such programs. The communication interface1500 receives data from other devices via a predetermined communication network and sends it to the CPU1100, and transmits data generated by the CPU1100 to other devices via the predetermined communication network.

[0090] The CPU 1100 controls output devices such as displays and printers, and input devices such as keyboards and mice, via the input / output interface 1600. The CPU 1100 acquires data from input devices via the input / output interface 1600. The CPU 1100 also outputs the generated data to output devices via the input / output interface 1600.

[0091] The media interface 1700 reads a program or data stored in the recording medium 1800 and provides it to the CPU 1100 via the RAM 1200. The CPU 1100 loads the program from the recording medium 1800 onto the RAM 1200 via the media interface 1700 and executes the loaded program. The recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or PD (Phase Change Rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape, a magnetic recording medium, or a semiconductor memory.

[0092] For example, when computer 1000 functions as an interactive device 1 according to Embodiments 1 to 6, the CPU 1100 of computer 1000 realizes the functions included in the control unit 5 of the interactive device 1 by executing programs loaded on RAM 1200. The CPU 1100 of computer 1000 reads and executes these programs from the recording medium 1800, but as another example, these programs may be obtained from other devices via a predetermined communication network.

[0093] 〔others〕 Furthermore, among the processes described in each of the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by known methods. In addition, the processing procedures, specific names, and information including various data and parameters shown in the above document and drawings can be changed at will unless otherwise specified. For example, the various information shown in each figure is not limited to the information shown.

[0094] Furthermore, the components of each illustrated device are functionally conceptual and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions.

[0095] Furthermore, the above embodiments can be combined as appropriate, provided that the processing content is not contradictory.

[0096] Although some embodiments of the present invention have been described in detail above with reference to the drawings, these are illustrative examples, and the present disclosure can be implemented in various other forms with modifications and improvements based on the knowledge of those skilled in the art, starting with the embodiments described in the disclosure section.

[0097] Furthermore, the terms "section, module, unit" mentioned above can be replaced with "means" or "circuit," etc. For example, the acquisition unit can be replaced with acquisition means or acquisition circuit. [Explanation of Symbols]

[0098] 1. Dialogue device 2 Communications Department 3. Voice input section 4 Output section 5. Control Unit 51 Acquisition Department 52 Creation Section 6 Memory section 61 Utterance Data 62 First Trained Language Model 63. Second pre-trained language model

Claims

1. When a user's utterance is detected, an acquisition unit obtains a summary related to the current utterance from utterance data that contains at least summaries created in relation to the user's past utterances, using a first trained language model. If the acquisition unit is able to acquire the relevant summary, it creates a prompt to input the relevant summary into the current utterance into the second trained language model; if the acquisition unit is unable to acquire the relevant summary, it creates the prompt based on the current utterance. A dialogue device characterized by comprising the following features.

2. The dialogue device according to claim 1, characterized in that the acquisition unit calculates the degree of relevance between the current utterance and the related summary using the first trained language model, and acquires the related summary if the degree of relevance is equal to or greater than a threshold.

3. The dialogue device according to claim 1 or 2, characterized in that, if the acquisition unit has a plurality of related summaries for the utterance data, it acquires the latest related summary from among the plurality of related summaries.

4. The aforementioned speech data includes information about the speech environment in which the user previously made speech, The dialogue device according to claim 1, characterized in that the acquisition unit acquires the relevant summaries which are common to the information regarding the speech environment of the current utterance and the information regarding the speech environment in which utterances were made in the past.

5. A method of dialogue performed by a dialogue device, When a user's utterance is detected, the process includes an acquisition step of obtaining a summary related to the current utterance from utterance data that contains at least summaries created in relation to the user's past utterances, using a first trained language model; If the relevant summary can be obtained through the acquisition step, a prompt is created to input the relevant summary into the second trained language model, and if the relevant summary cannot be obtained through the acquisition step, a creation step is made to create the prompt based on the current utterance. A dialogue method characterized by including

6. A procedure for obtaining a summary related to the current utterance using a first trained language model from speech data in which at least summaries created in relation to the user's past utterances are recorded when a user's utterance is detected, If the relevant summary can be obtained by the acquisition procedure, a prompt is created to input the relevant summary into the second trained language model, and if the relevant summary cannot be obtained by the acquisition procedure, a creation step is made to create the prompt based on the current utterance. An interactive program characterized by causing a computer to execute something.

Images