Dialogue system, dialogue program and dialogue method

Abstract

To smoothly continue a dialogue even if a problem occurs in communication with a language model or in the language model in a dialogue system that uses a language model to have a dialogue with a user. [Solution] The dialogue system (100) is constructed by machine learning and comprises a first language model (M1) that stores previously input input voice data and previously output output voice data as a dialogue history and outputs output voice data by referring to the dialogue history, a second language model (M2) that outputs output voice data by referring to a dialogue history input from outside, a data input unit (232) that inputs the input voice data to the first language model and inputs the input voice data and the dialogue history to the second language model, and an output control unit (233) that acquires the output voice data output by the first language model or the output voice data output by the second language model in accordance with predetermined output rules and outputs it to the output unit.

Description

[Technical Field]

[0001] The present disclosure relates to a dialogue system, a dialogue program, and a dialogue method. [Background technology]

[0002] In recent years, various technologies have been proposed for interacting with users using AI. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2025-133621 Summary of the Invention [Means for solving the problem]

[0004] A dialogue system according to one aspect of the present disclosure includes an acquisition unit that acquires input speech data including a user's utterance; a first language model that, when the input speech data is input, outputs output speech data corresponding to the utterance, the first language model retaining previously input input speech data and previously output output speech data as a dialogue history and outputting the output speech data by referring to the dialogue history; a second language model that, when the input speech data is input, outputs output speech data corresponding to the utterance, the second language model outputting the output speech data by referring to the dialogue history input from outside; a data input unit that inputs the input speech data to the first language model and inputs the input speech data and the dialogue history to the second language model; and an output control unit that acquires either the output speech data output by the first language model or the output speech data output by the second language model in accordance with a predetermined output rule and causes an output unit to output the acquired output speech data.

[0005] A dialogue program according to another aspect of the present disclosure causes a processor to execute an acquisition process for acquiring input speech data including a user's utterance; a data input process for inputting the input speech data into a first language model that, when the input speech data is input, outputs output speech data corresponding to the utterance, the first language model retaining previously input input speech data and previously output output speech data as a dialogue history and outputting the output speech data by referring to the dialogue history; and a data input process for inputting the input speech data and the dialogue history into a second language model that, when the input speech data is input, outputs output speech data corresponding to the utterance by referring to the dialogue history input from outside; and an output control process for acquiring either the output speech data output by the first language model or the output speech data output by the second language model in accordance with a predetermined output rule, and outputting the acquired output speech data to an output unit.

[0006] A dialogue method according to another aspect of the present disclosure includes an acquisition step in which a processor acquires input speech data including a user's utterance; a generation step in which the processor inputs the input speech data into a first language model that outputs output speech data corresponding to the utterance when the input speech data is input, the first language model retaining previously input input speech data and previously output output speech data as a dialogue history and outputting the output speech data by referring to the dialogue history, and causing the first language model to output the output speech data, and a second language model that outputs output speech data corresponding to the utterance when the input speech data is input, and outputting the output speech data by referring to the dialogue history input from outside, the second language model inputting the input speech data and the dialogue history into the second language model and causing the second language model to output the output speech data; and an output control step in which the processor acquires either the output speech data output by the first language model or the output speech data output by the second language model in accordance with a predetermined output rule, and outputs the acquired output speech data to an output unit. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a block diagram illustrating an example of a functional configuration of a dialogue system according to the present disclosure. [Figure 2] FIG. 2 is a block diagram showing an example of the functional configuration of a dialogue management server included in the system. [Figure 3] 10 is a flowchart showing an example of the flow of processing executed by the system. [Figure 4] 10 is a flowchart showing another example of the flow of the process executed by the system. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, an embodiment of the present disclosure will be described in detail with reference to the drawings. For ease of understanding, the background and problems of the present disclosure will be described first, and then the details of the present disclosure will be described.

[0009] In recent years, various technologies for using AI to have a dialogue with a user have been proposed. For example, Patent Document 1 describes a question-and-answer data output method that acquires historical data in which conversation content regarding an inquiry about a product or service is transcribed or historical data of a text-based conversation, inputs the acquired historical data into a language model, acquires question-and-answer data in which questions and answers to the questions are associated from the language model, and executes a process of outputting the acquired question-and-answer data.

[0010] However, in conventional dialogue technologies, due to problems with the communication path between the dialogue management device and the device having the language model, transmission of the response output by the language model to the dialogue management device may be delayed. This may result in a delay in transmission of the response from the dialogue management device to the user terminal. Furthermore, the language model may output a response indicating an error. In this case, the dialogue management device must take measures such as resending a response request to the device having the language model. This may result in a delay in transmission of the response from the dialogue management device to the user terminal, even if there is no problem with the communication path. As a result, it may take a long time from when the user inputs voice until they receive a response, which may make the dialogue between the user and the language model awkward.

[0011] <Dialogue system overview> The dialogue system according to the present disclosure comprises: an acquisition unit that acquires input voice data including a user's speech; a first language model that outputs output speech data corresponding to the utterance when the input speech data is input, the first language model storing previously input input speech data and previously output output speech data as a dialogue history, and outputting the output speech data by referring to the dialogue history; a second language model that outputs output speech data corresponding to the utterance when the input speech data is input, the second language model outputting the output speech data by referring to the dialogue history input from outside; a data input unit that inputs the input speech data to the first language model and inputs the input speech data and the dialogue history to the second language model; an output control unit that acquires either output speech data output by the first language model or output speech data output by the second language model according to a predetermined output rule, and causes an output unit to output the acquired output speech data; Equipped with.

[0012] According to the above configuration, output speech data serving as a response to the content of input speech data is output not only by the first language model but also by the second language model. Then, either the output speech data output by the first language model or the output speech data output by the second language model is output to the output unit. Therefore, even if the transmission of the output speech data output by either the first language model or the second language model is delayed or if the response output by either model indicates an error, the output speech data of the other model can be output to the output unit without delay. Furthermore, if the second language model is configured similarly to the first language model, the randomness inherent in the language models will cause the contents of the dialogue histories held by the first language model and the second language model to differ. As a result, when a dialogue between one model and a user switches to a dialogue between the other model and the user, the content of the output speech data output by the other model may be inconsistent with the previous dialogue. However, with the above configuration, unlike the first language model, the second language model outputs output speech data by referring to an externally input dialogue history, i.e., a dialogue history with the same content as the dialogue history held by the first language model. Therefore, even if a dialogue between one model and a user switches to a dialogue between the other model and the user, the content of the output speech data output by the other model is consistent with the previous dialogue. In other words, with the above configuration, the dialogue can be continued smoothly even if a problem occurs in communication with or within the language model.

[0013] Hereinafter, a dialogue system 100 according to an embodiment of the present disclosure will be described with reference to the drawings.

[0014] <Configuration of the dialogue system 100> FIG. 1 is a block diagram showing an example of the functional configuration of a dialogue system 100 according to this embodiment. As shown in FIG. 1, the dialogue system 100 includes a user terminal 1, a dialogue management server 2, a first processing server 3, and a second processing server 4. The user terminal 1 and the dialogue management server 2 are connected to each other via a network N1. The dialogue management server 2 and the first processing server 3 are connected to each other via a network N2. The dialogue management server 2 and the second processing server 4 are connected to each other via a network N3. The networks N1, N2, and N3 may be mobile communication systems such as 3G, 4G, 5G, and 6G, LTE (Long Term Evolution), Wi-Fi (registered trademark), an in-house LAN, etc. The networks N1, N2, and N3 may be different networks or the same network. Note that while FIG. 2 illustrates an example in which the user terminal 1 and the dialogue management server 2 are separate devices, at least two of the user terminal 1, the dialogue management server 2, the first processing server 3, and the second processing server 4 may be integrated into a single information processing terminal device without using the network N.

[0015] [User device 1] The user terminal 1 is an information processing device operated by a user who uses the service. The user terminal 1 receives input operations from the user who uses the service and operates the calculation unit 23 of the dialogue management server 2. The user terminal 1 may be, for example, a desktop personal computer (PC), a laptop PC, a mobile terminal such as a smartphone or tablet, an HMD (Head Mount Display), or a wearable terminal such as a wristwatch terminal. The user terminal 1 according to this embodiment includes a voice input unit 11, a terminal communication unit 12, and a voice output unit 13.

[0016] (Audio input unit 11) The user's speech is input to the speech input unit 11. The speech input unit 11 then converts the input user's speech into input speech data. The input speech data is speech data containing the user's voice. The speech input unit 11 can be configured with various microphones, etc.

[0017] (Terminal communication unit 12) The terminal communication unit 12 communicates with the dialogue management server 2. For example, the terminal communication unit 12 transmits input voice data converted by the voice input unit 11 to the dialogue management server 2. The terminal communication unit 12 also receives output voice data from the dialogue management server 2. The terminal communication unit 12 according to this embodiment is configured with a wireless communication module, a terminal for wired connection to the dialogue management server 2, etc.

[0018] (Audio output unit 13) The audio output unit 13 outputs audio based on the output audio data received by the terminal communication unit 12. The audio output unit 13 can be configured with various speakers that output audio.

[0019] [First Processing Server 3] The first processing server includes a first communication unit 31 , a first storage unit 32 , and a first calculation unit 33 .

[0020] [First Communication Department 31] The first communication unit 31 communicates with the dialogue management server 2. For example, the first communication unit 31 receives input voice data from the dialogue management server 2. The first communication unit 31 also transmits output voice data to the dialogue management server 2. The first communication unit 31 according to this embodiment is configured with a wireless communication module, a terminal for wired connection to the dialogue management server 2, etc.

[0021] [First storage section 32] The first storage unit 32 stores a first processing program 321. The first storage unit 32 also stores a first language model M1.

[0022] (First Processing Program 321) The first processing program 321 is a program for causing the first calculation unit 33 of the first processing server 3 to function as each of the functional units 331 to 333.

[0023] (First Language Model M1) The first language model M1 outputs output speech data corresponding to an utterance when input speech data is input. The first language model M1 is a stateful language model that stores previously input input speech data and previously output output speech data as a dialogue history and outputs output speech data by referring to the dialogue history. The first language model M1 according to this embodiment is a multimodal model. The first language model M1 can be constructed by various types of machine learning, etc. The first language model M1 may be a generative AI. The first language model M1 according to this embodiment has a voice activity detection (VAD) function. That is, the first language model M1 detects portions of a user's utterance from the input speech data and outputs output speech data corresponding to the portions of the utterance. The first language model M1 may be a cascade model, similar to the second language model M2 described later.

[0024] [First calculation unit 33] The first calculation unit 33 includes a first acquisition unit 331 , a first generation unit 332 , and a first output control unit 333 .

[0025] (First Acquisition Department 331) The first acquisition unit 331 acquires the input voice data received by the first communication unit 31 .

[0026] (First generation unit 332) The first generation unit 332 inputs input speech data to the first language model M1, and acquires output speech data output by the first language model M1.

[0027] (First output control unit 333) When the first language model outputs the output voice data, the first output control unit 333 controls the first communication unit 31. As a result, the first communication unit 31 transmits the output voice data to the dialogue management server 2.

[0028] [Second processing server 4] The second processing server 4 includes a second communication unit 41 configured similarly to the first communication unit 31 , a second storage unit 42 , and a second calculation unit 43 .

[0029] [Second storage unit 42] The second storage unit 42 stores a second processing program 421. The second storage unit 42 also stores a second language model M2.

[0030] (Second Processing Program 421) The second processing program 421 is a program for causing the second calculation unit 43 of the second processing server 4 to function as each of the functional units 431 to 433.

[0031] (Second Language Model M2) The second language model M2 outputs output speech data corresponding to the utterance when input speech data is input. The second language model M2 also outputs the output speech data by referencing a dialogue history input from outside. The second language model according to this embodiment is a stateless language model that does not retain a dialogue history. That is, the second language model according to this embodiment is a language model that does not retain output speech data that it has previously output. The second language model M2 according to this embodiment is a cascade model having a speech recognition model, a large-scale language model, and a speech synthesis model. The speech recognition model converts the content of an utterance included in input speech data into input text. The large-scale language model is a model that, when input text is input, generates output text corresponding to the content of the input text. The speech synthesis model converts the content of the output text into output speech data. Like the first language model M1, the second language model M2 can be constructed using various machine learning techniques, etc. Note that the second language model M2 may be a generative AI. Like the first language model, the second language model M2 according to this embodiment also has a VAD function. The second language model M2 may be a multimodal model, similar to the first language model M1.

[0032] [Second calculation unit 43] The second calculation unit 43 includes a second acquisition unit 431 , a second generation unit 432 , and a second output control unit 433 .

[0033] (Second acquisition part 431) The second acquisition unit 431 acquires the input voice data that the second communication unit 41 receives from the dialogue management server 2. In addition, the second acquisition unit 431 acquires the dialogue history that the second communication unit 41 receives from the dialogue management server 2.

[0034] (Second generation unit 432) The second generation unit 432 inputs the input voice data and the dialogue history to the second language model M2, and acquires the output voice data output by the second language model M2.

[0035] (Second output control unit 433) When the first language model M1 outputs the output voice data, the second output control unit 433 controls the second communication unit 41. As a result, the second communication unit 41 transmits the output voice data to the dialogue management server 2.

[0036] [Dialogue management server 2] 2 is a block diagram showing an example of the functional configuration of the dialogue management server 2. As shown in FIG. 2, the dialogue management server 2 includes a communication unit 21, a storage unit 22, and a calculation unit 23.

[0037] [Communications Department 21] The communication unit 21 communicates with the user terminal 1. The communication unit 21 is configured by, for example, a wireless communication module that communicates with the user terminal 1, a terminal for connecting to the user terminal 1 by wire, and the like.

[0038] [Storage unit 22] The storage unit 22 stores a dialogue program 221. A database 222 is also built in the storage unit 22 according to this embodiment. The storage unit 22 according to this embodiment is configured with a flash memory, a hard disk drive (HDD), a magneto-optical disk, or the like. The storage unit 22 may be configured to be able to store various calculation results of the calculation unit 23. The database 222 may also be built in another device that communicates with the dialogue management server 2.

[0039] (Dialogue Program 221) The dialogue program 221 is a program for causing the calculation unit 23 of the dialogue management server 2 to function as each functional unit.

[0040] (Database 222) The database 222 stores the dialogue history. The dialogue history is stored in a form that allows the second language model M2 to grasp the order of utterances (outputs) of the contents of dialogues (past input voice data and past output voice data) that have taken place in the past between the user and the dialogue system 100.

[0041] [Calculation unit 23] The calculation unit 23 is composed of a processor and a memory. The processor executes instructions written in the dialogue program 221. The processor is composed of an arithmetic unit, a register, and / or a peripheral circuit. The memory temporarily stores the dialogue program 221 and data processed by the dialogue program 221, etc., and is, for example, a volatile memory such as a DRAM (Dynamic Random Access Memory). The calculation unit 23 includes an acquisition unit 231, a data input unit 232, and an output control unit 233. The calculation unit 23 according to this embodiment further includes a history management unit 234.

[0042] (Acquisition part 231) The acquisition unit 231 acquires input voice data including a user's speech. The acquisition unit 231 according to this embodiment acquires the input voice data received by the communication unit 21.

[0043] (Data input unit 232) The data input unit 232 inputs input speech data to the first language model M1, and inputs the input speech data and the dialogue history to the second language model M2. The data input unit 232 according to this embodiment causes the communication unit 21 to transmit the input speech data to the first language model M1, and transmits the input speech data and the dialogue history to the second language model M2. When at least one of the first language model M1 and the second language model M2 is stored in the storage unit 22, the data input unit 232 directly inputs the input speech data to at least one of the first language model M1 and the second language model M2. The data input unit 232 according to this embodiment inputs the input speech data and the dialogue history stored in the database 222 to the second language model M2.

[0044] (output control unit 233) The output control unit 233 acquires either the output speech data output by the first language model or the output speech data output by the second language model according to a predetermined output rule and outputs the acquired output speech data to the output unit. The output control unit 233 according to this embodiment has the following output rules set: "normally acquire and output the output speech data output by the first language model" and "acquire and output the output speech data output by the second language model when a predetermined condition is met." Therefore, the output control unit 233 according to this embodiment outputs the output speech data output by the second language model to the output unit when a predetermined condition is met. The "predetermined condition" includes, for example, a condition in which the first language model does not output output speech data even after a predetermined time has elapsed since the second language model output the output speech data, or a condition in which the first language model outputs information indicating an error. The output rule set in the output control unit 233 may be, for example, "acquire and output the output speech data output by the first language model or the output speech data output by the second language model, whichever is higher in quality." The "output unit" that the output control unit 233 according to this embodiment causes to output the output speech data is the communication unit 21. That is, the output control unit 233 according to this embodiment causes the communication unit 21 to transmit the output voice data. In addition, when the user terminal 1 is configured integrally with the dialogue management server 2, the output unit may be the voice output unit 13.

[0045] (History Management Unit 234) The history management unit 234 accumulates the input voice data acquired by the acquisition unit 231 and the output voice data output by the output control unit 233 in the database 222 as part of the dialogue history (updates the dialogue history). The history management unit 234 according to this embodiment accumulates the input voice data in the database 222 every time the data input unit 232 inputs the input voice data to the first language model and the second language model. Furthermore, the history management unit 234 according to this embodiment accumulates the output voice data in the database 222 every time the output control unit 233 outputs the output voice data to the output unit.

[0046] <Processing example of the dialogue system 100> Next, a description will be given of the flow of processing executed by the dialogue system 100. Fig. 3 is a flowchart showing an example of the flow of processing executed by the dialogue system 100 according to this embodiment.

[0047] As shown in Fig. 3, in the dialogue system 100 according to this embodiment, the user terminal 1 executes a reception process A1, a transmission process A2, an acquisition process A3, and an output process A4. Furthermore, in the dialogue system 100 according to this embodiment, the dialogue management server 2 executes an acquisition process B1, a data input process B2, an output control process B3, and a history management process B4. Furthermore, in the dialogue system 100 according to this embodiment, the first processing server 3 executes a first acquisition process C1, a first generation process C2, and a first output control process C3. Furthermore, in the dialogue system 100 according to this embodiment, the second processing server 4 executes a second acquisition process D1, a second generation process D2, and a second output control process D3.

[0048] (Reception process A1) First, the user terminal 1 executes a reception process A1. In the reception process A1, the voice input unit 11 of the user terminal 1 receives input of a user's utterance (for example, "I would like to make a flight reservation"). The input utterance is converted into input voice data including the user's utterance.

[0049] (Transmission process A2) After the reception process A1, the user terminal 1 executes a transmission process A2. In the transmission process A2, the terminal communication unit 12 of the user terminal 1 transmits the input voice data to the dialogue management server 2.

[0050] (Acquisition process B1) After the transmission process A2, the dialogue management server 2 executes the acquisition process B1. In the acquisition process B1, the acquisition unit 231 of the dialogue management server 2 acquires input voice data. The acquisition of the input voice data by the acquisition unit 231 corresponds to the acquisition step in the dialogue method.

[0051] (Data entry process B2) After the acquisition process B1, the dialogue management server 2 executes a data input process B2. In the data input process B2, the data input unit 232 of the dialogue management server 2 inputs input speech data to the first language model M1 (B21). In the data input process B2 according to this embodiment, the data input unit 232 causes the communication unit 21 to transmit the input speech data. Also, in the data input process B2, the data input unit 232 inputs the input speech data and the dialogue history to the second language model M2 (B22). In the data input process B2 according to this embodiment, the data input unit 232 causes the communication unit 21 to transmit the input speech data and the dialogue history. The transmission of the input speech data and the dialogue history to the second processing server 4 may be performed in parallel with the transmission of the input speech data to the first processing server 3, or may be slightly earlier or later than the transmission of the input speech data to the first processing server 3. The data input unit 232 inputs the input speech data to the first language model and the second language model, respectively, and causes the first language model and the second language model to generate output speech data, respectively, which corresponds to the generation step in the dialogue method.

[0052] When the dialogue management server 2 executes the data input process B2, the first acquisition unit 331 of the first processing server 3 receives the input voice data and acquires the input voice data (C1). Then, the first generation unit 332 of the first processing server 3 inputs the input voice data to the first language model M1 and acquires the output voice data (e.g., "Is this a domestic flight okay?") output by the first language model M1 (C2). Then, the first communication unit 31 of the first processing server 3 transmits the output voice data to the dialogue management server 2 (C3). Also, the second acquisition unit 431 of the second processing server 4 receives the input voice data and the dialogue history and acquires the input voice data and the dialogue history (D1). Then, the second generation unit 432 of the second processing server 4 inputs the input voice data and the dialogue history to the second language model M2 and acquires the output voice data (e.g., "Could you please tell me where you're leaving from?") output by the second language model M2 (D2). Then, the second communication unit 41 of the second processing server 4 transmits the output voice data to the dialogue management server 2 (D3).

[0053] (Output control process B3) After the data input process B2, the dialogue management server 2 executes an output control process B3. In the output control process B3, the output control unit 233 of the dialogue management server 2 acquires either the output speech data output by the first language model or the output speech data output by the second language model in accordance with a predetermined output rule (B31). As described above, the output control unit 233 according to this embodiment is set with an output rule that "usually acquires and outputs the output speech data output by the first language model." Therefore, as shown in FIG. 3, when output speech data is transmitted without any problems from the first processing server 3 and the second processing server 4, the output control unit 233 outputs the acquired output speech data of the first language model M1 to the output unit (causes the communication unit 21 to transmit it: B32). In addition, the output control unit 233 according to this embodiment is also set with an output rule that "when a predetermined condition is met, acquires and outputs the output speech data output by the second language model." 4, if there is a delay in the output voice data from the first processing server 3 or if the first processing server 3 outputs information indicating a voice error, the output control unit 233 causes the output unit to output the acquired output voice data of the second language model M2 (causes the communication unit 21 to transmit it: B33). The output control unit 233 acquiring any output voice data and causing the output unit to output it corresponds to the output control step in the dialogue method.

[0054] (History management process B4) After the acquisition process B1 and the output control process B3, the dialogue management server 2 executes a history management process B4. In the history management process B4, the history management unit 234 of the dialogue management server 2 stores the input voice data and the output voice data output in the output control process B3 in the database 222 as part of the dialogue history. Storing the input voice data and the output voice data in the database 222 corresponds to the storage step in the dialogue method. Note that the history management process B4 may be executed after the acquisition process B1 or the output control process B3. That is, in the history management process B4, the history management unit 234 may store the input voice data and the output voice data together in the database 222.

[0055] (Acquisition process A3) After the output control process B3, the user terminal 1 executes an acquisition process A3. In the acquisition process A3, the user terminal 1 acquires output audio data.

[0056] (Output process A4) After the acquisition process A3, the user terminal 1 executes the output process A4. In the output process A4, the voice output unit 13 of the user terminal 1 outputs voice (a response to the user's utterance) based on the output voice data.

[0057] <Modification of the dialogue system 100> The present disclosure is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims.

[0058] For example, the first language model according to the above embodiment holds its own dialogue history. However, the first language model M1 may be configured to be able to replace the dialogue history it holds with a dialogue history input from outside. Furthermore, the second language model M2 according to the above embodiment is a stateless language model that does not hold a dialogue history. However, the second language model M2 may be configured to be able to hold a dialogue history. In this way, it is possible to switch the main and secondary roles of a language model that is primarily used (the first language model M1 in this embodiment) and a language model that is not primarily used (the second language model M2 in this embodiment). In addition, by doing this, when the first language model M1 that has been removed from the dialogue with the user returns to the dialogue, it is possible to output output speech data that reflects the content of the dialogue that was taking place up until recently.

[0059] Furthermore, the dialogue system 100 according to the above embodiment includes only the first language model M1 and the second language model M2 as language models. However, the dialogue system 100 may include third to n-th language models in addition to the first language model M1 and the second language model M2. When it becomes impossible to obtain output speech data from the first language model M1 or the second language model M2, the dialogue system 100 may be configured to obtain output speech data from the third to n-th language models. When the dialogue system 100 includes the third to n-th language models, if the first language model does not output output speech data even after a predetermined time has elapsed since the second language model output the output speech data, the output control unit 233 may be configured to obtain the output speech data output by the language model that is fastest in processing (generation of output speech data) among the second to n-th language models, and output the data to the output unit.

[0060] Furthermore, the history management unit 234 according to the above embodiment accumulates the input voice data and the output voice data directly in the database as a dialogue history. However, the history management unit 234 may be configured to manage the length of the dialogue history (dialogue time, data amount, etc.) and summarize the contents of the dialogue history when the length reaches a predetermined value. The history management unit may then input the summarized dialogue history into the second language model M2.

[0061] In the dialogue system 100 according to the above embodiment, the first language model M1 is stored in the first processing server, and the second language model M2 is stored in the second processing server. However, the first language model M1 and the second language model M2 may be stored together in one processing server, or at least one of the first language model M1 and the second language model M2 may be stored in the dialogue management server 2.

[0062] <Effects of the dialogue system 100> The dialogue system 100 described above outputs the output voice data not only to the first language model M1 but also to the second language model M2. Then, either the output voice data output by the first language model M1 or the output voice data output by the second language model M2 is output to the output unit. Therefore, even if the transmission of the output voice data output by either the first language model M1 or the second language model M2 is delayed, or the answer output by either model indicates an error, the output voice data of the other model can be output to the output unit without delay. Also, in the dialogue system 100 described above, unlike the first language model M1, the second language model M2 outputs the output voice data by referring to the dialogue history input from the outside, that is, the dialogue history having the same content as the dialogue history held by the first language model M1. Therefore, even when the dialogue that has been conducted between one model and the user so far is switched to the dialogue between the other model and the user, the content of the output voice data output by the other model is consistent with the previous dialogue. That is, according to the dialogue system 100, even if there is communication with the language model or a problem occurs in the language model, the dialogue can be smoothly continued.

[0063] <Supplementary Article 1 on AI: General Explanation> As used in this specification, generative AI refers to artificial intelligence (including AGI or ASI) that includes generative AI (generative system AI), generative AI, language models (LLM / SLM), GPT (registered trademark), Gemini (registered trademark), Claude (registered trademark), Llama (registered trademark), and other language models, and uses deep learning (deep neural network) technologies such as transformers, self-attention, and autoregressive networks to generate content such as text, images, audio, and video. The extended technologies of generative AI include frameworks such as Retrieval-Augmented Generation (RAG), Memory-Augmented Generation, hybrid search using vector databases, chunking / chunk processing, knowledge graph linkage, entity linking, AutoGen, AOG, and LangChain. The performance improvement technologies of generative AI include fine-tuning by RLHF / RLAIF, PEFT, LoRA, etc., distillation, quantization, weight sharing, continuous learning, federated learning, in-context learning, etc. Also, the operating environment of generative AI can be any environment of on-premises / cloud / edge (on-device), and parallel / distributed learning and inference using GPUs, TPUs, NPUs, IPUs, ASICs, and FPGAs are also possible.

[0064] <Supplementary Article 2 on AI: Use Cases> Generative AI can be applied to a very wide range of application areas, such as when multiple (multi) AIs cooperate, collaborate, and work together to form an AI agent team for call center support, automatic FAQ answering by chatbots, AI assistants, translation / summarization / meeting minutes generation, programming support (code generation, debugging), data analysis, system development, financial and legal document review, research, diagnosis, drug discovery, design optimization, supply chain management, education, games, the metaverse, advertising / creative / marketing, e-commerce / business transactions (recommendations), threat intelligence, image / video / music generation, robotics, smart factories, smart cities, traffic control, autonomous driving, and IoT device control. The use cases of generative AI include both internal processing types used in systems at the backend and chatbot types provided to users at the frontend.

[0065] <Supplementary Article 3 on AI: Worldview and Problem Solving of Collaboration between AIs> Two or more (multi) AIs are executed in a chain while collaborating, cooperating, and coordinating with each other. If necessary, an orchestrator AI or orchestration AI that serves as a command center commands two or more (multi) AIs, enabling AI agents as a joint team to solve various tasks and problems. On the other hand, as issues with the collaboration, cooperation, and coordination between two or more (multi) AIs, risks such as hallucination, fake, bias, discrimination, data leakage, misdelivery, theft, privacy (personal information) protection, and copyright infringement become apparent. Therefore, technologies for the management and monitoring (logging, monitoring) of multiple (multi) AIs are used to ensure security, governance, responsibility, auditing, reliability, and compliance.

[0066] <Supplementary Article 4 on AI: Future Developments Related to AI> In the future, the management and operation of multiple (multi) AIs will require infrastructure such as high-performance GPUs, quantum computers, HPC clusters, distributed / parallel processing, load balancers, CDNs, multi-access edge computing (MEC), and fog computing. And generative AI agents are expected to acquire autonomy while balancing reliability and economy (cost), and to develop to the AGI or ASI level beyond singularity. This disclosure encompasses these future technological trends and application possibilities and is not limited to each embodiment described in this specification.

[0067] <Example of Realization by Software> The functions of each device (hereinafter referred to as "device") constituting the dialogue system 100 can be realized by a dialogue program 221 for causing a computer to function as the device, and by a dialogue program 221 for causing a computer to function as each control block (especially each part included in the arithmetic unit 23) of the device.

[0068] In this case, the device includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory) as hardware for executing the dialogue program 221. The dialogue program 221 is executed by the control device and storage device, thereby realizing the functions described in each of the above embodiments.

[0069] The dialogue program 221 may be non-transitory and may be recorded on one or more computer-readable recording media. The recording media may or may not be included in the device. In the latter case, the dialogue program 221 may be supplied to the device via any wired or wireless transmission medium.

[0070] In addition, some or all of the functions of each of the control blocks can be realized by logic circuits. For example, integrated circuits in which logic circuits that function as each of the control blocks are formed are also included in the scope of the present disclosure. In addition, the functions of each of the control blocks can also be realized by, for example, a quantum computer.

[0071] Furthermore, each process described in each of the above embodiments may be executed by AI (Artificial Intelligence). In this case, the AI ​​may run on the control device or on another device (for example, an edge computer or a cloud server).

[0072] <Summary> The present disclosure describes at least the following aspects.

[0073] The dialogue system according to aspect 1 of the present disclosure includes an acquisition unit that acquires input speech data including a user's utterance; a first language model that, when the input speech data is input, outputs output speech data corresponding to the utterance, the first language model retaining previously input input speech data and previously output output speech data as a dialogue history and outputting the output speech data by referring to the dialogue history; a second language model that, when the input speech data is input, outputs output speech data corresponding to the utterance, the second language model outputting the output speech data by referring to the dialogue history input from outside; a data input unit that inputs the input speech data to the first language model and inputs the input speech data and the dialogue history to the second language model; and an output control unit that acquires either the output speech data output by the first language model or the output speech data output by the second language model according to a predetermined output rule and causes an output unit to output the acquired output speech data.

[0074] In the dialogue system according to aspect 2 of the present disclosure, in aspect 1, the second language model may be a language model that does not retain output speech data that it has output in the past.

[0075] A dialogue system according to aspect 3 of the present disclosure may be, in aspect 1 or 2, wherein the first language model is a multimodal model, and the second language model is a cascade model having a speech recognition model that converts the content of the utterance contained in the input voice data into input text, a large-scale language model that generates output text corresponding to the content of the input text when the input text is input, and a speech synthesis model that converts the content of the output text into the output voice data.

[0076] The dialogue system according to aspect 4 of the present disclosure may be configured such that, in any of aspects 1 to 3, if the first language model does not output the output speech data even after a predetermined time has elapsed since the second language model output the output speech data, or if the first language model outputs information indicating an error, the output control unit outputs the output speech data output by the second language model.

[0077] The dialogue system according to aspect 5 of the present disclosure may be configured in any one of aspects 1 to 4, further comprising a history management unit that stores the input voice data and the output voice data in the database as part of the dialogue history, and the data input unit inputs the input voice data and the dialogue history stored in the database into the second language model.

[0078] The dialogue system according to aspect 6 of the present disclosure may be configured such that, in any of aspects 1 to 5, the first language model and the second language model each detect a portion of the user's speech from the input speech data and output the output speech data corresponding to the portion of the speech.

[0079] A dialogue program according to aspect 7 of the present disclosure causes a processor to execute an acquisition process for acquiring input speech data including a user's utterance; a data input process for inputting the input speech data into a first language model that, when the input speech data is input, outputs output speech data corresponding to the utterance, the first language model retaining previously input input speech data and previously output output speech data as a dialogue history and outputting the output speech data by referring to the dialogue history; and a data input process for inputting the input speech data and the dialogue history into a second language model that, when the input speech data is input, outputs output speech data corresponding to the utterance, the second language model outputting the output speech data by referring to the dialogue history input from outside; and an output control process for acquiring either the output speech data output by the first language model or the output speech data output by the second language model in accordance with a predetermined output rule, and outputting the acquired output speech data to an output unit.

[0080] An interaction method according to aspect 8 of the present disclosure includes an acquisition step in which a processor acquires input speech data including a user's utterance; a generation step in which the processor inputs the input speech data into a first language model that, when the input speech data is input, outputs output speech data corresponding to the utterance, the first language model retaining previously input input speech data and previously output output speech data as a dialogue history and outputting the output speech data by referring to the dialogue history, causing the first language model to output the output speech data, and a generation step in which the processor inputs the input speech data and the dialogue history into a second language model that, when the input speech data is input, outputs output speech data corresponding to the utterance, the second language model outputting the output speech data by referring to the dialogue history input from outside, causing the second language model to output the output speech data; and an output control step in which the processor acquires either the output speech data output by the first language model or the output speech data output by the second language model in accordance with a predetermined output rule, and outputs the acquired output speech data to an output unit.

[0081] (Additional notes) This disclosure will enable smooth voice communication with language models (generative AI) and will serve as an innovative technological foundation for AI businesses, thereby contributing to the achievement of Goal 9 of the Sustainable Development Goals (SDGs), "Build resilient infrastructure, promote inclusive and sustainable industrialization, and promote innovation and resilience." [Explanation of symbols]

[0082] 100 Dialogue Systems 1. User terminal 11 Audio input section 12 Terminal communication unit 13 Output section 2. Dialogue Management Server 21 Communications Department 22 Memory section 221 Dialogue Program 23 Arithmetic section 231 Acquisition Department 232 Data Entry Section 233 Output control section 234 History Management Department 3. First processing server M1 First Language Model 4 Second processing server M2 Second Language Model

Claims

1. an acquisition unit that acquires input voice data including a user's speech; a first language model that outputs output speech data corresponding to the utterance when the input speech data is input, the first language model storing previously input input speech data and previously output output speech data as a dialogue history, and outputting the output speech data by referring to the dialogue history; a second language model that outputs output speech data corresponding to the utterance when the input speech data is input, the second language model outputting the output speech data by referring to the dialogue history input from outside; a data input unit that inputs the input speech data to the first language model and inputs the input speech data and the dialogue history to the second language model; an output control unit that acquires either output speech data output by the first language model or output speech data output by the second language model according to a predetermined output rule, and causes an output unit to output the acquired output speech data; Equipped with Dialogue system.

2. the second language model is a language model that does not store output speech data that it has output in the past. The dialogue system according to claim 1 .

3. the first language model is a multimodal model; The second language model is a speech recognition model that converts the content of the utterance included in the input speech data into input text; a large-scale language model that, when inputted with the input text, generates an output text corresponding to the content of the input text; a speech synthesis model that converts the content of the output text into the output speech data; This is a cascade model with 3. A dialogue system according to claim 1 or 2.

4. The output control unit When the first language model does not output the output speech data even after a predetermined time has elapsed since the second language model output the output speech data, Or, if the first language model outputs information indicating an error, outputting the output speech data output by the second language model to the output unit; The dialogue system according to claim 1 .

5. a history management unit that stores the input voice data and the output voice data in a database as part of the dialogue history; the data input unit inputs the input speech data and the dialogue history stored in the database into the second language model; The dialogue system according to claim 1 .

6. The first language model and the second language model are detecting a portion of the user's speech from the input speech data; outputting the output voice data corresponding to the part of the utterance; The dialogue system according to claim 1 .

7. The processor an acquisition process for acquiring input voice data including a user's speech; a data input process in which the input speech data is input into a first language model that outputs output speech data corresponding to the utterance when the input speech data is input, the first language model holding previously input input speech data and previously output output speech data as a dialogue history and outputting the output speech data by referring to the dialogue history, and a second language model that outputs output speech data corresponding to the utterance when the input speech data is input, the second language model outputting the output speech data by referring to the dialogue history input from outside; an output control process of acquiring either output speech data output by the first language model or output speech data output by the second language model according to a predetermined output rule, and causing an output unit to output the acquired output speech data; Execute Interactive program.

8. an acquiring step in which a processor acquires input speech data including a user's speech; a generation step in which the processor inputs the input speech data into a first language model that outputs output speech data corresponding to the utterance when the input speech data is input, the first language model holding previously inputted input speech data and previously outputted output speech data as a dialogue history and outputting the output speech data by referring to the dialogue history, and causes the first language model to output the output speech data, and a second language model that outputs output speech data corresponding to the utterance when the input speech data is input, the second language model inputting the input speech data and the dialogue history into a second language model that outputs the output speech data by referring to the dialogue history input from outside, and causes the second language model to output the output speech data; an output control step in which the processor acquires either output speech data output by the first language model or output speech data output by the second language model according to a predetermined output rule, and causes an output unit to output the acquired output speech data; Including, How to interact.

Images