Dialog system, terminal, and program

Abstract

To provide a dialog system, a terminal, and a program allowing a user to continue a dialog even while on the move.SOLUTION: A dialog system 100 comprises a terminal 1 used by a user P1 on the move, and the terminal checks a state of communication with a dialog server 4, according to a result of checking of the communication state, switches between processing performed by a local dialog unit having a dialog with the user P1 by using a knowledge unit having specific knowledge, and processing performed by a remote dialog unit having a dialog with the user P1 through the dialog server 4 by using a language model 6a having learned a plurality of documents and details of dialogs, when switching from the processing performed by the local dialog unit to the processing performed by the remote dialog unit, and transmits, to the dialog server 4, dialog history data that is the details of a dialog between the user P1 and the other party having a dialog with the user P1, in a terminal history storage unit that stores the dialog history data. The dialog server 4 registers the dialog history data received from the terminal 1 in a server history storage unit that stores the dialog history data in association with terminal identification information.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an interaction system, a terminal, and a program.

Background Art

[0002] Conventionally, in a tour by a sightseeing bus, a crew member on the sightseeing bus guides the passengers in the bus about the scenery visible through the window, for example. Currently, tours are diversifying, and services that provide participants with virtual tours via the Internet have emerged (see, for example, Patent Document 1). Thus, even in the current situation where various services are provided, there are still many participants who want to visit tourist destinations in person as in the past. However, due to a shortage of personnel, etc., there may be a situation where it is impossible to secure a guide such as a crew member. In such a case, for example, it is possible to use a technique for providing content according to the position of a vehicle (see, for example, Patent Document 2).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the content according to the position of a vehicle described in Patent Document 2, only one-way display of content is performed, resulting in low satisfaction. Therefore, a system that enables interaction is desired, and a mechanism that can continue the interaction even during movement is required.

[0005] Therefore, an object of the present invention is to provide an interaction system, a terminal, and a program that enable continuous interaction even during movement.

Means for Solving the Problem

[0006] The present invention solves the above problems by the following means. A first invention is an interactive system including a terminal used by a user in motion and a server, wherein the terminal includes a knowledge unit having specific knowledge, a local history storage unit that stores dialogue history data which is the content of dialogue between the user and the other party with whom the user conducts dialogue, local dialogue means for conducting dialogue with the user using the knowledge unit, remote dialogue means for conducting dialogue with the user using a language model that has learned a plurality of documents and dialogue content via the server, communication status confirmation means for confirming the communication status with the server, dialogue switching means for switching between the process by the local dialogue means and the process by the remote dialogue means according to the confirmation result of the communication status by the communication status confirmation means, and history transmission means for transmitting the dialogue history data in the local history storage unit to the server when the dialogue switching means switches from the process by the local dialogue means to the process by the remote dialogue means, and the server includes a server history storage unit that stores the dialogue history data in association with the user, and history registration means for registering the dialogue history data received from the terminal in the server history storage unit. A second invention is an interactive system according to the first invention, wherein the history transmission means of the terminal transmits the dialogue history data generated at least during the process by the local dialogue means to the server. A third invention is an interactive system according to the first or second invention, wherein the dialogue switching means of the terminal switches to the process by the local dialogue means when no response data which is the response of the dialogue is received from the server within a certain period during the process by the remote dialogue means. The fourth invention is a dialogue system according to any one of the first to third inventions, wherein the server includes a speech reception unit that receives speech data of the user from the terminal, and inputs the speech data received by the speech reception unit and the dialogue history data of the user stored in the server history storage unit into the language model, and a dialogue processing unit that transmits the response data output by the language model to the terminal. The fifth invention is a dialogue system according to the fourth invention, wherein the terminal includes a position acquisition unit that acquires position information at a predetermined time interval, and a position transmission unit that transmits the position information acquired by the position acquisition unit during the process by the remote dialogue unit to the server, the server includes a position reception unit that receives the position information from the terminal, and the dialogue processing unit of the server further inputs the position information received by the position reception unit into the language model. The sixth invention is a dialogue system according to any one of the first to third inventions, wherein the server includes a speech reception unit that receives speech data of the user from the terminal, an information output unit that outputs the speech data received by the speech reception unit and the dialogue history data of the user stored in the server history storage unit to the other party terminal, and a response transmission unit that transmits the response data transmitted from the other party terminal to the terminal. The seventh invention is a dialogue system according to the sixth invention, wherein the terminal includes a position acquisition unit that acquires position information at a predetermined time interval, and a position transmission unit that transmits the position information acquired by the position acquisition unit during the process by the remote dialogue unit to the server, the server includes a position reception unit that receives the position information from the terminal, and the information output unit of the server further outputs the position information received by the position reception unit to the other party terminal. The eighth invention is a dialogue system according to any one of the fourth to seventh inventions, wherein the speech data and the response data are voice data, and the terminal includes a voice input unit that acquires the speech data and a voice output unit that outputs the response data. The ninth invention is a dialogue system according to any one of the first to eighth inventions, wherein the terminal includes a display unit that outputs a character corresponding to the other party, and a character processing unit that operates the character in response to a response from the other party. The tenth invention is a terminal used by a user for conducting a dialogue with a server, the terminal including: a knowledge unit having specific knowledge; a local history storage unit that stores dialogue history data which is the content of the dialogue between the user and the other party with whom the user conducts the dialogue; a local dialogue unit that conducts a dialogue with the user using the knowledge unit; a remote dialogue unit that conducts a dialogue with the user using a language model learned from a plurality of documents and dialogue content via the server; a communication status confirmation unit that checks the communication status with the server; a dialogue switching unit that switches between the processing by the local dialogue unit and the processing by the remote dialogue unit according to the communication status confirmation result by the communication status confirmation unit; and a history transmission unit that transmits the dialogue history data in the local history storage unit to the server when the dialogue switching unit switches from the processing by the local dialogue unit to the processing by the remote dialogue unit. The eleventh invention is a program for causing a computer used by a user to function as: a knowledge unit having specific knowledge; a local history storage unit that stores dialogue history data which is the content of the dialogue between the user and the other party with whom the user conducts the dialogue; a local dialogue unit that conducts a dialogue with the user using the knowledge unit; a remote dialogue unit that conducts a dialogue with the user using a language model learned from a plurality of documents and dialogue content via the server; a communication status confirmation unit that checks the communication status with the server; a dialogue switching unit that switches between the processing by the local dialogue unit and the processing by the remote dialogue unit according to the communication status confirmation result by the communication status confirmation unit; and a history transmission unit that transmits the dialogue history data in the local history storage unit to the server when the dialogue switching unit switches from the processing by the local dialogue unit to the processing by the remote dialogue unit, when the computer conducts a dialogue with the user via the server.

Advantages of the Invention

[0007] According to the present invention, it is possible to provide a dialogue system, a terminal, and a program that enable continuous dialogue even while moving.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Modes for Carrying Out the Invention

[0009] Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. Note that this is merely an example, and the technical scope of the present invention is not limited thereto. (First Embodiment) <Overall Configuration of Dialogue System 100> FIG. 1 is an overall configuration diagram of a dialogue system 100 according to the first embodiment. FIG. 2 is a functional block diagram of the terminal 1 according to the first embodiment. FIG. 3 is a functional block diagram of the dialogue server 4 according to the first embodiment. The dialogue system 100 shown in FIG. 1 includes a terminal 1, a dialogue server 4 (server), and a language model server 6 (language model). The terminal 1, the dialogue server 4, and the language model server 6 are communicably connected via a communication network N and a base station R.

[0010] The dialogue system 100 is, for example, a virtual guide system that enables a user P1 to have a dialogue with a virtual partner using a terminal 1 installed in the vehicle interior SP1 of a bus. On the display 27 (display unit) of the terminal 1, a virtual character imitating, for example, a bus guide is displayed as a virtual partner for the dialogue. In the dialogue system 100, when the terminal 1 is connected to the communication network N, the dialogue server 4 uses the language model server 6 to output guidance according to the position of the bus to the terminal 1 and also makes a response to the speech of the user P1. On the other hand, when the terminal 1 is not connected to the communication network N, the terminal 1 makes a response to the speech of the user P1.

[0011] In this way, the dialogue system 100 is a system that enables continuous dialogue regardless of the communication state during the movement of the terminal 1. In the following, the dialogue system 100 will be described assuming its use in the vehicle interior SP1 of a bus, but it is not limited thereto. Also, in the following description, the passenger who speaks will be described as the user P1. The user P1 is not limited to a specific single passenger.

[0012] <Terminal 1> The terminal 1 is installed, for example, in the in-vehicle SP1 of a bus and is a device used by passengers on the bus including user P1. In FIG. 1, the display 27 is installed on the ceiling of the in-vehicle SP1, and a plurality of passengers can view one display 27. Here, an example will be described where the terminal 1 includes one or more displays 27. However, the terminal 1 is not limited thereto. For example, a screen may be attached to the in-vehicle SP1, and an image may be projected onto the screen instead of the display 27. Also, a display may be provided for each seat in the in-vehicle SP1. Further, the terminal 1 may be a portable terminal such as a smartphone or a tablet owned by each passenger, or may be smart glasses. Furthermore, the terminal 1 may be, for example, a general personal computer (PC) or the like.

[0013] As shown in FIG. 2, the terminal 1 includes a control unit 10, a storage unit 20, a microphone 25 (voice input unit), a speaker 26 (voice output unit), a display 27, a GPS (Global Positioning System) receiver 28, and a communication interface unit 29. The control unit 10 is a CPU (Central Processing Unit) that controls the entire terminal 1. The control unit 10 appropriately reads and executes the OS (Operating System) and various application programs stored in the storage unit 20, and cooperates with the above-described hardware to execute various functions.

[0014] Before explaining the control unit 10, the storage unit 20 will be explained. The storage unit 20 is a storage device such as a hard disk or a semiconductor memory element for storing programs, data, etc. necessary for the operation of the terminal 1. The storage unit 20 includes a program storage unit 21, a knowledge unit 22, and a terminal history storage unit 23 (local history storage unit). The program storage unit 21 is a storage area for storing programs. The program storage unit 21 stores an interactive program 21a (program). The dialogue program 21a is a program for executing various functions of the control unit 10 described below, for example. Note that the various functions of the control unit 10 may be executed by a single dialogue program 21a, or may be executed by a plurality of programs for each function.

[0015] The knowledge unit 22 is, for example, a storage area that stores knowledge about the place to be guided. The knowledge unit 22 may be, for example, a rule-based one that stores pairs of questions and answers, or a learning model that has learned a document about the place to be guided. When the knowledge unit 22 is a learning model, since the learning model is stored in the terminal 1, it has a smaller model size compared to the language model 6a described later. The terminal history storage unit 23 is a storage area that stores the utterance content input to the terminal 1 and the dialogue content including the responses output by the dialogue server 4 and the terminal 1 in chronological order. The terminal history storage unit 23 stores, for example, the dialogue content during the start to end of sightseeing using a vehicle or until arrival at the destination as dialogue history data.

[0016] Next, the control unit 10 will be described. The control unit 10 shown in FIG. 1 includes a position acquisition unit 11, a communication connection confirmation unit 12, a dialogue switching unit 13, a remote dialogue unit 14, a local dialogue unit 15, a character processing unit 16, a history transmission unit 17, and a history registration unit 18. The position acquisition unit 11 has a function as a position acquisition means. The position acquisition unit 11 acquires position information via the GPS reception unit 28 at a predetermined time interval. The communication connection confirmation unit 12 has a function as a communication status confirmation means. The communication connection confirmation unit 12 confirms the communication status with the dialogue server 4. When connected to the communication network N, the communication connection confirmation unit 12 confirms that the communication status is communicable. Also, when not connected to the communication network N, the communication connection confirmation unit 12 confirms that the communication status is incommunicable. Note that the communication connection confirmation unit 12 may confirm that the communication status is incommunicable when connected to the communication network N but with weak radio waves.

[0017] The dialogue switching unit 13 has a function as dialogue switching means. The dialogue switching unit 13 switches between the processing by the local dialogue unit 15 and the processing by the remote dialogue unit 14 according to the confirmation result of the communication status by the communication connection confirmation unit 12. More specifically, when the communication status by the communication connection confirmation unit 12 is communicable, the dialogue switching unit 13 performs the processing by the remote dialogue unit 14, and switches the processing to perform the processing by the local dialogue unit 15 only when the communication status by the communication connection confirmation unit 12 is incommunicable. In addition, when the dialogue switching unit 13 does not receive reply data, which is a response to the dialogue, from the dialogue server 4 within a certain period during the processing by the remote dialogue unit 14, it switches to perform the processing by the local dialogue unit 15.

[0018] The remote dialogue unit 14 has functions as remote dialogue means and position transmission means. The remote dialogue unit 14 conducts a dialogue with the user P1 via the dialogue server 4. More specifically, the remote dialogue unit 14 acquires the utterance of the user P1 as utterance data via the microphone 25, and transmits the utterance data to the dialogue server 4. In addition, the remote dialogue unit 14 transmits the position information acquired by the position acquisition unit 11 to the dialogue server 4. Further, the remote dialogue unit 14 outputs the reply data received from the dialogue server 4 via the speaker 26.

[0019] The local dialogue unit 15 has a function as local dialogue means. The local dialogue unit 15 conducts a dialogue with the user P1 using the knowledge unit 22. More specifically, the local dialogue unit 15 acquires the utterance of the user P1 as utterance data via the microphone 25, converts the utterance data into text using, for example, a known speech recognition technology, and inputs the text to the knowledge unit 22. Then, the local dialogue unit 15 converts the text related to the reply output from the knowledge unit 22 into reply data in the form of speech using a known technology, and outputs the reply data via the speaker 26. The character processing unit 16 has a function as character processing means. The character processing unit 16 outputs a character as an interlocutor to the display 27. Then, the character processing unit 16 operates the character in response to the reply of the other party.

[0020] The history transmission unit 17 has a function as history transmission means. When the dialogue switching unit 13 switches from the process by the local dialogue unit 15 to the process by the remote dialogue unit 14, the history transmission unit 17 transmits the dialogue history data in the terminal history storage unit 23 to the dialogue server 4. At that time, the history transmission unit 17 transmits the dialogue history data generated at least during the process by the local dialogue unit 15 to the dialogue server 4. The history registration unit 18 registers the dialogue content between the user P1 and the other party as dialogue history data in the terminal history storage unit 23. Details of these functions will be described later.

[0021] The microphone 25 is an audio input device used to acquire the speech of the user P1. The microphone 25 is provided, for example, near the display 27. The speech content of the user P1 can be acquired by the control unit 10 as audio data via the microphone 25. The speaker 26 is an audio output device that outputs the speech content and reply content of the interlocutor. The speaker 26 is provided, for example, near the display 27. The speaker 26 outputs the audio data output by the control unit 10 as audio. The display 27 is a display device composed of, for example, a liquid crystal panel or the like. The GPS reception unit 28 measures the current position information (latitude and longitude) of the terminal 1 by using, for example, radio waves received from satellites. The communication interface unit 29 is an interface for communicating with the dialogue server 4 via the base station R and the communication network N.

[0022] <Dialogue server 4> When communicatively connected to the terminal 1, the dialogue server 4 is a server that performs response based on the speech data received from the terminal 1 and dialogue processing based on the location information received from the terminal 1. As shown in FIG. 3, the dialogue server 4 includes a control unit 40, a storage unit 50, and a communication interface unit 59. The control unit 40 is a CPU that controls the entire dialogue server 4. By appropriately reading and executing the OS and various application programs stored in the storage unit 50, the control unit 40 cooperates with the above-described hardware to execute various functions.

[0023] The control unit 40 includes a location reception unit 41, a speech reception unit 42, a dialogue processing unit 43, a history reception unit 47, and a history registration unit 48. The location reception unit 41 has a function as location reception means. The location reception unit 41 receives location information from the terminal 1. The speech reception unit 42 has a function as speech reception means. The speech reception unit 42 receives speech data from the terminal 1. The dialogue processing unit 43 has a function as dialogue processing means. The dialogue processing unit 43 converts the received speech data into text using, for example, a known speech recognition technique, and transmits it to the language model server 6 together with the dialogue history data stored in the server history storage unit 53 and the location information received by the location reception unit 41. Then, the dialogue processing unit 43 receives the response content from the language model server 6, converts it into response data in the form of speech using a known technique, and transmits it to the terminal 1.

[0024] The history reception unit 47 receives the dialogue history data transmitted by the terminal 1. The history registration unit 48 has a function as history registration means. The history registration unit 48 registers the dialogue history data received by the history reception unit 47 in the server history storage unit 53 in association with, for example, terminal identification information (user) for identifying the terminal 1. Further, the history registration unit 48 registers the speech content and response content obtained by the dialogue processing unit 43 in the server history storage unit 53 as dialogue history data in association with the terminal identification information.

[0025] The storage unit 50 is a storage device such as a hard disk or a semiconductor memory device for storing programs, data, etc. necessary for the operation of the dialogue server 4. The storage unit 50 includes a program storage unit 51 and a server history storage unit 53. The program storage unit 51 is a storage area for storing programs. The program storage unit 51 stores various programs including programs for executing various functions of the control unit 40. The server history storage unit 53 is a storage area for storing the dialogue content between the dialogue server 4 and the terminal 1 and the dialogue history data received from the terminal 1 in chronological order in association with the terminal identification information. The communication interface unit 59 is an interface for performing communication between the terminal 1 and the language model server 6 via the communication network N.

[0026] Note that there is no limit to the number of hardware components that make up the dialogue server 4. It may be composed of one or more as needed. Also, the hardware of the dialogue server 4 may include various servers such as a web server, a DB (database) server, and an application server as needed, and may be composed of one server, or each may be composed of separate servers. Further, the dialogue server 4 may be, for example, a cloud. A computer refers to an information processing device equipped with a control unit, a storage device, etc. The terminal 1 and the dialogue server 4 are each information processing devices equipped with a control unit, a storage unit, etc., and are included in the concept of a computer.

[0027] <Language model server 6> The language model server 6 shown in FIG. 1 is a server equipped with a language model 6a that has learned a plurality of documents and conversation contents. The language model 6a is a deep learning model, which is a huge neural network that has learned language patterns from a vast amount of text data and enables tasks related to text, such as text generation and summarization, to be performed with high accuracy. For example, it is a large language model (LLM). Also, the language model 6a may be a generative AI (generative AI). Note that since the language model 6a requires large-scale calculation instructions, the language model server 6 may be, for example, a cloud.

[0028] The base station R is a base station for wireless communication and relays communication between the terminal 1 and the dialogue server 4. The base station R is, for example, a wireless LAN (Local Area Network) or a wireless station of a mobile phone carrier. The communication network N is a network between various servers and between various servers and the base station R. The communication network N is, for example, a communication network such as an Internet line, regardless of whether it is wired or wireless.

[0029] <Description of the process> Next, the process performed by the dialogue system 100 will be described. FIG. 4 is a diagram showing a dialogue example in the dialogue system 100 according to the first embodiment. FIG. 5 is a diagram for explaining the relationship between the dialogue and the history in the dialogue system 100 according to the first embodiment. As shown in FIG. 4, when the guidance starts, first, the dialogue R1 based on the location information is output from the terminal 1. Since the communication status of the terminal 1 allows communication, remote dialogue processing is performed. Therefore, the location information is transmitted from the terminal 1 to the dialogue server 4, and the dialogue R1 based on the location information is transmitted from the dialogue server 4 to the terminal 1, so that the dialogue R1 is output to the terminal 1. The dialogue R1 is output from the language model 6a by inputting the location information into the language model 6a. Note that the dialogue R1 at the start of the guidance may be obtained from the storage unit 50 and output the dialogue R1 corresponding to a predetermined position when the location information is at a predetermined position, for example.

[0030] When the user P1 who heard it asks the question Q1, the communication status of the terminal 1 still allows communication. Therefore, the terminal 1 performs remote dialogue processing. The terminal 1 transmits the question Q1 to the dialogue server 4, and the dialogue server 4 outputs the reply A1 to the terminal 1. The reply A1 is output from the language model 6a by inputting the question Q1 and the dialogue R1 as the dialogue history data into the language model 6a. Next, when the user P1 asks the question Q2, the communication status of the terminal 1 is not communicable. Therefore, the terminal 1 performs local dialogue processing. Specifically, the terminal 1 uses the knowledge unit 22 to obtain and output the reply A2 to the question Q2.

[0031] After that, when the user P1 asks the question Q3, the communication status of the terminal 1 allows communication. Therefore, the terminal 1 performs remote dialogue processing. The terminal 1 transmits the question Q3 and the question Q2 and the reply A2, which are the dialogue history of the local dialogue processing, as the dialogue history data to the dialogue server 4, and the dialogue server 4 outputs the reply A3 to the terminal 1. The reply A3 is output from the language model 6a by inputting the question Q3 and the dialogue from the question Q1 to the reply A2 as the dialogue history data into the language model 6a. FIG. 5 shows a mode of generating response A3. In the server history storage unit 53, the dialogue history data of the male in the local dialogue process is also registered. Therefore, the response A3 is generated using the dialogue history data in the local dialogue process performed only on the terminal 1.

[0032] Next, the above-described process will be described based on a flowchart. First, the process in the terminal 1 will be described. FIG. 6 is a flowchart showing the dialogue process of the terminal 1 according to the first embodiment. FIG. 7 is a flowchart showing the local process of the terminal 1 according to the first embodiment. In step S (hereinafter, "step S" will be simply referred to as "S") 11 of FIG. 6, the position acquisition unit 11 of the terminal 1 acquires the position information of the terminal 1 via the GPS reception unit 28. The position acquisition unit 11 can acquire the position information at a predetermined time interval, for example, every minute.

[0033] In S12, the communication connection confirmation unit 12 checks the communication status and determines whether the communication status is communicable. When the communication status is communicable (S12: YES), the communication connection confirmation unit 12 transfers the process to S13, and the dialogue switching unit 13 switches to the process by the remote dialogue unit 14. On the other hand, when the communication status is not communicable (S12: NO), the communication connection confirmation unit 12 transfers the process to S19, and the dialogue switching unit 13 switches to the process by the local dialogue unit 15. In S13, the remote dialogue unit 14 determines whether it has received the utterance of the user P1. When the remote dialogue unit 14 has received the utterance of the user P1 (S13: YES), the remote dialogue unit 14 transfers the process to S14. On the other hand, when the remote dialogue unit 14 has not received the utterance of the user P1 (S13: NO), the remote dialogue unit 14 transfers the process to S15.

[0034] In S14, the remote dialogue unit 14 transmits the position information acquired in the process of S11 and the utterance data to the dialogue server 4. Thereafter, the control unit 10 transfers the process to S16. In S15, the remote dialogue unit 14 transmits the position information acquired in the process of S11 to the dialogue server 4. In S16, the dialogue switching unit 13 determines whether it has received the response data transmitted by the dialogue server 4 within a certain period. Here, the certain period is the elapsed time with the time point when transmitted to the dialogue server 4 as the start time, for example, 10 seconds or the like. If the response data is received within the certain period (S16: YES), the dialogue switching unit 13 moves the process to S17. On the other hand, if the response data is not received within the certain period (S16: NO), the dialogue switching unit 13 moves the process to S19. The case where the response data is not received within the certain period may be, for example, the case where the communication status becomes unavailable during the process. Therefore, if the response data is not received within the certain period, the dialogue switching unit 13 switches to the process by the local dialogue unit 15.

[0035] In S17, the history registration unit 18 registers the dialogue history data consisting of the utterance and the response in the terminal history storage unit 23. In S18, the remote dialogue unit 14 outputs the received response data to the speaker 26. By doing so, the user P1 can listen to the response to his own utterance. During the execution of this process, the character processing unit 16 outputs characters to the display 27 of the terminal 1. Also, while the character processing unit 16 outputs the response data to the speaker 26, it performs an operation process of moving the mouth as if the character is speaking. After that, the control unit 10 moves the process to S11.

[0036] On the other hand, in S19, the local dialogue unit 15 performs local processing. Here, the local processing will be described with reference to FIG. 7. In S31 of FIG. 7, the local dialogue unit 15 determines whether it has received the speech of user P1. If it has received the speech of user P1 (S31: YES), the local dialogue unit 15 transfers the process to S32. On the other hand, if it has not received the speech of user P1 (S31: NO), the local dialogue unit 15 transfers the process to S35. Note that when transitioning to this process due to NO in the process of S16 in FIG. 6, the process proceeds as if the speech of user P1 has been received (S31: YES).

[0037] In S32, the local dialogue unit 15 inputs the speech data into the knowledge unit 22. At that time, the local dialogue unit 15 converts the speech data from voice to text and then inputs it into the knowledge unit 22. In S33, the local dialogue unit 15 converts the response output by the knowledge unit 22 from text to voice and then outputs it via the speaker 26. In S34, the history registration unit 18 registers the dialogue history data consisting of the speech and the response in the terminal history storage unit 23.

[0038] In S35, the communication connection confirmation unit 12 checks the communication status with the dialogue server 4 and determines whether the communication status is communicable. If the communication status is communicable (S35: YES), the communication connection confirmation unit 12 transfers the process to S36, and the dialogue switching unit 13 switches to the process by the remote dialogue unit 14. On the other hand, if the communication status is not communicable (S35: NO), the communication connection confirmation unit 12 transfers the process to S11 in FIG. 6. In S36, the history transmission unit 17 transmits the dialogue history data registered in the terminal history storage unit 23 at least in the local dialogue process to the dialogue server 4. After that, the control unit 10 transfers the process to S11 in FIG. 6.

[0039] Next, the process of the dialogue server 4 will be described. FIG. 8 is a flowchart showing the server process of the dialogue server 4 according to the first embodiment. The processing of the dialogue server 4 starts when the terminal 1 transmits information to the dialogue server 4 (see S14 and S15 in FIG. 6 and S36 in FIG. 7), and the dialogue server 4 receives the information. In S41 of FIG. 8, the position receiving unit 41 determines whether it has received position information from the terminal 1. If it has received the position information (S41: YES), the control unit 40 moves the process to S42. On the other hand, if it has not received the position information (S41: NO), the control unit 40 moves the process to S46.

[0040] In S42, the speech receiving unit 42 determines whether it has received speech data from the terminal 1. If it has received the speech data (S42: YES), the control unit 40 moves the process to S43. On the other hand, if it has not received the speech data (S42: NO), the control unit 40 moves the process to S44. In S43, the dialogue processing unit 43 transmits the received speech data, position information, and the dialogue history data associated with the terminal identification information of the terminal 1 to the language model server 6. Here, the speech data is converted into text and then transmitted to the language model server 6. Also, the position information, for example, is converted from latitude and longitude to an address and then transmitted to the language model server 6. In the language model server 6, the received data is input into the language model 6a. Then, the control unit 40 moves the process to S45.

[0041] In S44, the dialogue processing unit 43 transmits the received position information and the dialogue history data associated with the terminal identification information of the terminal 1 to the language model server 6. In the language model server 6, the received data is input into the language model 6a. In S45, the dialogue processing unit 43 transmits the response data output by the language model 6a and received from the language model server 6 to the terminal 1. Here, the dialogue processing unit 43 converts the response content received from the language model server 6 into speech as the response data and then transmits it to the terminal 1. Then, the control unit 40 ends this process.

[0042] On the other hand, in S46, the history reception unit 47 determines whether or not it has received the dialogue history data from the terminal 1. If it has received the dialogue history data (S46: YES), the history reception unit 47 transfers the process to S47. On the other hand, if it has not received the dialogue history data (S46: NO), the control unit 40 ends this process. In S47, the history registration unit 48 registers the dialogue history data in the server history storage unit 53 in association with the terminal identification information. Then, the control unit 40 ends this process.

[0043] As described above, according to the first embodiment, the dialogue system 100 has the following effects. (1) The terminal 1 includes a terminal history storage unit 23 that stores dialogue history data, which is the content of the dialogue between the user P1 and the other party with whom the user P1 has a dialogue; a local dialogue unit 15 that conducts a dialogue with the user P1 using a knowledge unit 22 having specific knowledge; a remote dialogue unit 14 that conducts a dialogue with the user P1 via the dialogue server 4; a communication connection confirmation unit 12 that checks the communication status with the dialogue server 4; a dialogue switching unit 13 that switches between the process by the local dialogue unit 15 and the process by the remote dialogue unit 14 according to the communication status confirmation result by the communication connection confirmation unit 12; and a history transmission unit 17 that transmits the dialogue history data in the terminal history storage unit 23 to the dialogue server 4 when the dialogue switching unit 13 switches from the process by the local dialogue unit 15 to the process by the remote dialogue unit 14. The dialogue server 4 registers the dialogue history data received from the terminal 1 in the server history storage unit 53 in association with the terminal identification information.

[0044] Therefore, in the terminal 1, according to the communication status confirmation result, it is possible to switch whether to conduct a dialogue with the user P1 by local dialogue processing using the knowledge unit 22 in the terminal 1 or by remote dialogue processing via the dialogue server 4. Also, when switching from local dialogue processing to remote dialogue processing, since the dialogue history data in the local dialogue processing is transmitted to the dialogue server 4, the dialogue server 4 can also have the history related to the dialogue conducted in the terminal 1 while not being connected to the terminal 1.

[0045] (2) The history transmission unit 17 of terminal 1 transmits the dialogue history data generated at least during the processing by the local dialogue unit 15 to the dialogue server 4. Therefore, since the dialogue history generated by the processing of the local dialogue unit 15 by user P1 is transmitted to the dialogue server 4, the dialogue server 4 can have the minimum dialogue history data related to the dialogue between user P1 and the other party performed in chronological order.

[0046] (3) When the dialogue switching unit 13 of terminal 1 does not receive reply data, which is a response to the dialogue, from the dialogue server 4 within a certain period during the processing by the remote dialogue unit 14, it switches to the processing by the local dialogue unit 15. Therefore, when user P1 does not have to wait for a reply and there is no reply from the dialogue server 4, the dialogue can be continued by switching to the processing by the local dialogue unit 15.

[0047] (4) The dialogue server 4 includes a speech reception unit 42 that receives the speech data of user P1 from terminal 1, inputs the speech data received by the speech reception unit 42 and the dialogue history data of user P1 stored in the server history storage unit 53 into a language model 6a that has learned a plurality of documents and dialogue contents, and a dialogue processing unit 43 that transmits the reply data output by the language model 6a to terminal 1. Therefore, the reply to the speech of user P1 can be generated by the language model 6a from the speech and the dialogue history up to that point. As a result, a seamless dialogue without a sense of incongruity can be provided to user P1.

[0048] (5) Terminal 1 includes a position acquisition unit 11 that acquires position information at predetermined time intervals. The remote dialogue unit 14 transmits the position information acquired by the position acquisition unit 11 during the processing by the remote dialogue unit 14 to the dialogue server 4. The dialogue server 4 includes a position reception unit 41 that receives the position information from terminal 1. The dialogue processing unit 43 of the dialogue server 4 further inputs the position information into the language model 6a. Therefore, the response to the utterance of user P1 can be generated by the language model 6a from the utterance, the previous dialogue history, and the location information. As a result, a response that also takes into account the location information of user P1 can be provided to user P1.

[0049] (6) The utterance data and the response data are voice data, and the terminal 1 includes a microphone 25 that acquires the utterance data and a speaker 26 that outputs the response data. Therefore, user P1 can conduct a dialogue by voice.

[0050] (7) The terminal 1 includes a display 27 that outputs a character corresponding to the other party and a character processing unit 16 that operates the character in response to the response of the other party. Therefore, user P1 can feel as if having a dialogue with the character. Also, by having the character perform an operation corresponding to the response, the operation of the character can be made natural.

[0051] (Second Embodiment) In the second embodiment, a description will be given of a case where an operator conducts a dialogue with the user of the terminal instead of using a language model. In the following description, parts that perform the same functions as those in the above-described first embodiment are denoted by the same reference numerals or reference numerals with the same suffix at the end, and duplicate descriptions are appropriately omitted.

[0052] <Overall Configuration of Dialogue System 200> FIG. 9 is an overall configuration diagram of a dialogue system 200 according to the second embodiment. FIG. 10 is a functional block diagram of the dialogue server 204 and the operator terminal 207 according to the second embodiment. The dialogue system 200 shown in FIG. 9 includes a terminal 1, a dialogue server 204, and an operator terminal 207 (the other party terminal). The terminal 1, the dialogue server 204, and the operator terminal 207 are communicably connected via a communication network N and a base station R.

[0053] The dialogue system 200 is a remote guidance system that enables a user P1 to have a dialogue with an operator P2 who is indoors at a place such as home, using a terminal 1 installed in an in-vehicle SP1 of a bus, for example. In the dialogue system 100, when the terminal 1 is connected to the communication network N, the dialogue server 204 mediates the transmission of speech data to the operator terminal 207. Since the operator P2 transmits response data from the operator terminal 207 to the dialogue server 204, the response data is transmitted from the dialogue server 204 to the terminal 1. Also, when the terminal 1 is not connected to the communication network N, the terminal 1 processes the speech of the user P1 and gives a response to the speech of the user P1.

[0054] <Dialogue server 204> The dialogue server 204 shown in Fig. 10(A) includes a control unit 240, a storage unit 250, and a communication interface unit 59. The control unit 240 includes a position reception unit 41, a speech reception unit 42, an information transmission unit 244, a response reception processing unit 245, a history reception unit 47, and a history registration unit 48. The information transmission unit 244 has a function as information output means. The information transmission unit 244 transmits the speech data and position information received from the terminal 1 and the dialogue history data associated with the terminal identification information of the terminal 1 stored in the server history storage unit 53 to the operator terminal 207. The response reception processing unit 245 has a function as response transmission means. The response reception processing unit 245 receives response data from the operator terminal 207 and transmits it to the terminal 1.

[0055] The storage unit 250 includes a program storage unit 251 and a server history storage unit 53. The program storage unit 251 is a storage area that stores various programs including programs for executing various functions of the control unit 240.

[0056] <Operator terminal 207> The operator terminal 207 shown in FIG. 10(B) includes a control unit 270, a storage unit 275, an input unit 276, a display 277, a headset 278, and a communication interface unit 279. The control unit 270 includes an information reception unit 271, an information output unit 272, and a response reception processing unit 273. The information reception unit 271 receives speech data, location information, and conversation history data from the dialogue server 204. The information output unit 272 outputs various types of information received by the information reception unit 271 to the display 277 and the headset 278. For example, the information output unit 272 outputs the speech data as sound to the headset 278 and outputs the conversation history data as characters in chronological order to the display 277, for example. Also, the information output unit 272 may output map information indicating the location information on a map to the display 277, for example. The response reception processing unit 273 receives the response of the operator P2 via the headset 278 and transmits the response data to the dialogue server 204.

[0057] The storage unit 275 is a storage area that stores various programs and the like in addition to the program for performing the processing of the control unit 270. The input unit 276 is an input device such as a keyboard or a mouse, for example. The display 277 is a display device composed of a liquid crystal panel or the like, for example. The headset 278 is a device in which a microphone and headphones are integrated and worn on the head by the operator P2, for example. The communication interface unit 279 is an interface for performing communication with the dialogue server 204 via the communication network N.

[0058] <Explanation of processing> Next, the processing performed by the dialogue system 200 will be described. The processing in the terminal 1 is the same as that in the first embodiment (FIGS. 6 and 7). The processing of the dialogue server 204 will be described. FIG. 11 is a flowchart showing the server processing of the dialogue server 204 according to the second embodiment. The processing of the dialogue server 204 starts when the terminal 1 transmits information to the dialogue server 204 (see S14 and S15 in FIG. 6 and S36 in FIG. 7) and the dialogue server 204 receives the information, or when the dialogue server 204 receives information from the operator terminal 207.

[0059] In S241 of FIG. 11, the position receiving unit 41 determines whether it has received position information from the terminal 1. If it has received the position information (S241: YES), the control unit 240 moves the process to S242. On the other hand, if it has not received the position information (S241: NO), the control unit 240 moves the process to S243. In S242, the information transmitting unit 244 transmits the position information received from the terminal 1 to the operator terminal 207. Here, the information transmitting unit 244 may specify the address from the position information received from the terminal 1 and transmit the position information as map information indicating the position information on a map including the address to the operator terminal 207. Further, the position information may include, for example, the previously acquired information as a trajectory. By doing so, the operator P2 can know the position of the vehicle including the moving direction of the vehicle, making it easier to provide guidance.

[0060] In S243, the speech receiving unit 42 determines whether it has received speech data from the terminal 1. If it has received the speech data (S243: YES), the control unit 240 moves the process to S244. On the other hand, if it has not received the speech data (S243: NO), the control unit 240 moves the process to S245. In S244, the information transmitting unit 244 transmits the speech data received from the terminal 1 and the dialogue history data associated with the terminal identification information of the terminal 1 stored in the server history storage unit 53 to the operator terminal 207. By this process, the information receiving unit 271 of the operator terminal 207 receives various information, and the information output unit 272 outputs the received various information. Therefore, the operator P2 can refer to the output information and reply to the speech data.

[0061] In S245, the reply reception processing unit 245 determines whether reply data has been received from the operator terminal 207. If reply data has been received from the operator terminal 207 (S245: YES), the control unit 240 transfers the process to S246. On the other hand, if reply data has not been received from the operator terminal 207 (S245: NO), the control unit 240 transfers the process to S247. In S246, the reply reception processing unit 245 transmits the reply data to terminal 1. The processes of S247 and S248 are the same as the processes of S46 and S47 in the first embodiment (FIG. 8).

[0062] As described above, according to the second embodiment, the dialogue system 200 has the following effects. (1) The dialogue server 204 includes a speech reception unit 42 that receives speech data of user P1 from terminal 1, an information transmission unit 244 that outputs the speech data received by the speech reception unit 42 and the dialogue history data of user P1 stored in the server history storage unit 53 to the operator terminal 207, and a reply reception processing unit 245 that transmits the reply data transmitted from the operator terminal 207 to terminal 1. Therefore, through the mediation of the dialogue server 204, user P1 of terminal 1 and operator P2 of the operator terminal 207 can have a dialogue. In addition, since the dialogue history data stored in the server history storage unit 53 is transmitted to the operator terminal 207, the operator P2 can proceed with the dialogue while checking the dialogue history performed on terminal 1 by local dialogue processing.

[0063] (2) Terminal 1 includes a position acquisition unit 11 that acquires position information at predetermined time intervals. The remote dialogue unit 14 transmits the position information acquired by the position acquisition unit 11 during the processing by the remote dialogue unit 14 to the dialogue server 204. The dialogue server 204 includes a position reception unit 41 that receives the position information from terminal 1, and the information transmission unit 244 further outputs the position information received by the position reception unit 41 to the operator terminal 207. Therefore, since the operator P2 can check the location information of the terminal 1, responses and conversations can be made taking into account the location information.

[0064] As described above, embodiments of the present invention have been described. However, the present invention is not limited to the above-described embodiments. Also, the effects described in the embodiments are merely an enumeration of the most suitable effects resulting from the present invention, and the effects of the present invention are not limited to those described in the embodiments. Note that although the above-described embodiments and the modified forms described later can be used in appropriate combinations, detailed descriptions thereof are omitted.

[0065] (Modified Forms) (1) In each of the embodiments, an example in which the dialogue system is used in a sightseeing bus has been described, but the present invention is not limited to this. As long as it includes an in-vehicle terminal and a dialogue server and performs dialogue using a language model or by an operator, for example, it can be used in a car navigation system or for other purposes. Further, it may be a terminal in a train, or the terminal may be, for example, a portable terminal carried by a user.

[0066] (2) In each of the embodiments, an example in which the dialogue server performs the conversion from voice data to text and the conversion from text to voice data has been described, but the present invention is not limited to this. The terminal may perform the conversion processing between voice and text.

[0067] (3) In each of the embodiments, an example in which the dialogue is performed by voice has been described, but the present invention is not limited to this. At least one of the dialogues may be performed by text. In that case, if the terminal performs input in text, an input unit such as a keyboard may be provided instead of a microphone. Also, if the dialogue server is in text, the text may be output to a display.

[0068] (4) In each of the embodiments, an example in which an image of a person imitating a guide is displayed as a character on a display has been described, but the present invention is not limited to this. An image of another character may be used, or an image of a character may not be used.

[0069] (5) In the first embodiment, the processing using the language model possessed by the language model server was described as an example, but it is not limited to this. The dialogue server may possess a language model.

Explanation of Signs

[0070] 1 Terminal 4,204 Dialogue Server 6 Language Model Server 6a Language Model 10,40,240,270 Control Unit 11 Position Acquisition Unit 12 Communication Connection Confirmation Unit 13 Dialogue Switching Unit 14 Remote Dialogue Unit 15 Local Dialogue Unit 16 Character Processing Unit 17 History Transmission Unit 18,48 History Registration Unit 20,50,250,275 Storage Unit 21a Dialogue Program 22 Knowledge Unit 23 Terminal History Storage Unit 25 Microphone 26 Speaker 27,277 Display 28 GPS Receiver 41 Position Reception Unit 42 Utterance Reception Unit 43 Dialogue Processing Unit 47 History Reception Unit 53 Server History Storage Unit 100,200 Dialogue System 207 Operator Terminal 244 Information Transmission Unit 245 Response Reception Processing Unit 271 Information Reception Unit 272 Information Output Unit 273 Response Reception Processing Unit 276 Input Unit 278 Headset N Communication Network P1 User P2 Operator SP1 Inside the vehicle SP2 Inside the room R Base station

Claims

1. A dialogue system comprising a terminal used by a moving user and a server, wherein the terminal has a knowledge part with specific knowledge, a local history storage part for storing dialogue history data which is the content of the dialogue between the user and the other party with whom the user has a dialogue, local dialogue means for conducting a dialogue with the user using the knowledge part, remote dialogue means for conducting a dialogue with the user using a language model that has learned a plurality of documents and dialogue content via the server, communication status confirmation means for checking the communication status with the server, dialogue switching means for switching between the processing by the local dialogue means and the processing by the remote dialogue means according to the confirmation result of the communication status by the communication status confirmation means, history transmission means for transmitting the dialogue history data in the local history storage part to the server when the dialogue switching means switches from the processing by the local dialogue means to the processing by the remote dialogue means, and is provided with wherein the server has a server history storage part for storing the dialogue history data in association with the user, and history registration means for registering the dialogue history data received from the terminal in the server history storage part, A dialogue system.

2. In the dialogue system according to Claim 1, the history transmission means of the terminal transmits the dialogue history data generated at least during the processing by the local dialogue means to the server. A dialogue system.

3. In the dialogue system according to Claim 1, when the dialogue switching means of the terminal does not receive reply data which is a response to the dialogue from the server within a certain period during the processing by the remote dialogue means, it switches to the processing by the local dialogue means. A dialogue system.

4. In the dialogue system according to Claim 1, the server has speech reception means for receiving the speech data of the user from the terminal, dialogue processing means for inputting the speech data received by the speech reception means and the dialogue history data of the user stored in the server history storage part into the language model, and transmitting the reply data output by the language model to the terminal, A dialogue system.

5. In the dialogue system according to Claim 4, the terminal has position acquisition means for acquiring position information at a predetermined time interval, A position transmission means for transmitting the position information acquired by the position acquisition means during the processing by the remote interaction means to the server, comprising: The server includes a position reception means for receiving the position information from the terminal, The dialogue processing means of the server inputs the position information received by the position reception means into the language model, a dialogue system.

6. In the dialogue system according to claim 1, The server, A speech reception means for receiving the speech data of the user from the terminal, An information output means for outputting the speech data received by the speech reception means and the dialogue history data of the user stored in the server history storage unit to the other party terminal, A reply transmission means for transmitting the reply data transmitted from the other party terminal to the terminal, comprising a dialogue system.

7. In the dialogue system according to claim 6, The terminal, A position acquisition means for acquiring position information at a predetermined time interval, A position transmission means for transmitting the position information acquired by the position acquisition means during the processing by the remote interaction means to the server, comprising: The server, A position reception means for receiving the position information from the terminal, The information output means of the server outputs the position information received by the position reception means to the other party terminal, a dialogue system.

8. In the dialogue system according to any one of claims 4 to 7, The speech data and the reply data are voice data, The terminal, A voice input unit for acquiring the speech data, A voice output unit for outputting the reply data, comprising a dialogue system.

9. In the dialogue system according to claim 8, The terminal, A display unit for outputting a character corresponding to the other party, A character processing means for operating the character in response to the reply of the other party, comprising a dialogue system.

10. A terminal used by the user for interacting with the user via a server, A knowledge section having specific knowledge, A local history storage unit for storing dialogue history data which is the content of the dialogue between the user and the other party with whom the user interacts, Local dialogue means for conducting a dialogue with the user using the knowledge section, Remote dialogue means for conducting a dialogue with the user using a language model learned from a plurality of documents and dialogue contents via the server, Communication status confirmation means for confirming the communication status with the server, Dialogue switching means for switching between the processing by the local dialogue means and the processing by the remote dialogue means according to the communication status confirmation result by the communication status confirmation means; History transmission means for transmitting the dialogue history data in the local history storage unit to the server when the dialogue switching means switches from the processing by the local dialogue means to the processing by the remote dialogue means; A terminal comprising the same.

11. A computer used by the user for conducting a dialogue with the user via a server, A knowledge section having specific knowledge; A local history storage unit for storing dialogue history data which is the content of the dialogue between the user and the party with whom the user conducts a dialogue; Comprising: The computer is configured to: Local dialogue means for conducting a dialogue with the user using the knowledge section; Remote dialogue means for conducting a dialogue with the user using a language model that has learned a plurality of documents and dialogue contents via the server; Communication status confirmation means for confirming the communication status with the server; Dialogue switching means for switching between the processing by the local dialogue means and the processing by the remote dialogue means according to the communication status confirmation result by the communication status confirmation means; History transmission means for transmitting the dialogue history data in the local history storage unit to the server when the dialogue switching means switches from the processing by the local dialogue means to the processing by the remote dialogue means; A program for causing the same to function.

Images