Information processing device, dialogue system, dialogue control method, and program

Abstract

Enable the system to output a response regarding the end of a conversation based on the user's utterance. [Solution] The information processing device includes an input unit that receives input from a user, an evaluation unit that evaluates the user's willingness to continue the dialogue based on the input, and a response unit that outputs a response regarding the termination of the dialogue based on the evaluation result of the evaluation unit.

Description

[Technical Field]

[0001] The present invention relates to an information processing apparatus, a dialogue system, a dialogue control method, and a program. [Background Art]

[0002] There are dialogue systems that automatically respond to user utterances. Additionally, an information providing system is known that estimates a user's participation attitude from the user's utterance content, and constructs a conversational sentence that arouses the user's interest based on the estimated user participation attitude (see, for example, Patent Document 1). [Summary of the Invention] [Problem to be Solved by the Invention]

[0003] Conventional dialogue systems have a problem in that even when the user gets bored with the dialogue, the dialogue system cannot perceive this and continues the dialogue.

[0004] An embodiment of the present disclosure has been made in view of the above problems, and enables a dialogue system that automatically responds to user utterances to output a response regarding the end of a dialogue based on the user's utterance. [Means for Solving the Problem]

[0005] To solve the above problems, an information processing apparatus according to an embodiment includes: an input unit that receives an input from a user; an evaluation unit that evaluates the user's willingness to continue the dialogue based on the input; and a response unit that outputs a response regarding the end of the dialogue based on the evaluation result of the evaluation unit. [Effects of the Invention]

[0006] According to an embodiment of the present disclosure, in a dialogue system that automatically responds to user utterances, a response regarding the end of a dialogue can be output based on the user's utterance. [Brief Description of the Drawings]

[0007] [Figure 1] This figure shows an example of the system configuration of a dialogue system according to one embodiment. [Figure 2] This figure shows an example of a computer hardware configuration according to one embodiment. [Figure 3] This figure shows an example of the hardware configuration of a terminal device according to one embodiment. [Figure 4] This figure shows an example of the functional configuration of a dialogue system according to one embodiment. [Figure 5] This is a diagram illustrating user information according to one embodiment. [Figure 6] This is a sequence diagram showing an example of the processing of the dialogue system according to the first embodiment. [Figure 7] This is a flowchart showing an example of dialogue processing according to the first embodiment. [Figure 8] This figure shows an example of a dialogue history according to the first embodiment. [Figure 9] This is a sequence diagram showing an example of the processing of the dialogue system according to the second embodiment. [Figure 10] This is a sequence diagram showing an example of the dialogue introduction process according to the third embodiment. [Figure 11] Figure (1) shows an example of a prompt according to the third embodiment. [Figure 12] This figure shows an example of a dialogue introduction screen according to the third embodiment. [Figure 13] This is a diagram illustrating the image acquisition process according to the third embodiment. [Figure 14] Figure (2) shows an example of a prompt according to the third embodiment. [Figure 15] This is a flowchart showing an example of the evaluation process according to the fourth embodiment. [Figure 16] This is a diagram illustrating the evaluation by the number of utterance characters according to the fourth embodiment. [Figure 17] This is a flowchart showing an example of the evaluation process according to the fifth embodiment. MODE FOR CARRYING OUT THE INVENTION

[0008] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0009] <Overview> There exist dialogue systems that automatically generate responses to user utterances. In such dialogue systems, a technology is known that returns linguistically natural responses by passing the user's utterance content to an LLM (Large Language Model) and instructing it to generate a response.

[0010] However, conventional dialogue systems have a problem that even when the user gets bored with the conversation, the dialogue system cannot detect this and continues the conversation, leading to a decrease in user satisfaction.

[0011] The dialogue system according to the present embodiment is assumed to be used for reminiscence therapy in nursing care facilities as one of its applications. In reminiscence therapy, it is known that presenting photos familiar to the user plays an important role. Presenting photos during a conversation can appeal to the user's visual sense and attract their attention. Furthermore, the use of photos is also effective when presenting topics, and is expected to be utilized as a means to deepen the user's understanding and interest.

[0012] In the conventional technology, there is a problem that it is not possible to output a response regarding the end of a dialogue based on the user's utterance in consideration of the progress of the dialogue, for example, as the dialogue time on a certain topic becomes longer, the possibility that the user gets bored becomes higher.

[0013] Therefore, in order to enable control of the end of a dialogue in consideration of the progress of the dialogue in a dialogue system that automatically responds to a user's utterance, the dialogue system according to the present embodiment has a configuration as shown in, for example, FIGS. 1 to 4. <System Configuration> FIG. 1 is a diagram showing an example of the system configuration of a dialog system according to an embodiment. The dialog system 1 includes, for example, a server device 100 connected to a communication network N such as the Internet and a LAN (Local Area Network), and a terminal device 10.

[0014] The server device (information processing device) 100 is, for example, an information processing device having a computer configuration, or a system including a plurality of computers, and is capable of communicating with the terminal device 10 and an external system 110 via the communication network N. The server device 100 provides a dialog service that automatically responds to utterances from a user 11 using the terminal device 10 by executing a predetermined program on a computer included in the server device 100. Note that the server device 100 is an example of the information processing device according to the present embodiment.

[0015] The external system 110 includes an LLM 111 provided from outside the dialog system 1, a map search service, and the like. The LLM 111 and the map search service 112 may be provided by the same provider or different providers, respectively. The LLM 111 is a large language model constructed using a large amount of text data and deep learning technology. The LLM 111 can execute various natural language processing tasks such as, for example, text reading, understanding, generation, translation, summarization, and response. In the present embodiment, it is assumed that an existing LLM 111 such as ChatGPT is used as-is without modification, for example.

[0016] The map search service 112 is a service that enables searching of, for example, map information, facility information, route information, and the like via the communication network N. In the present embodiment, it is assumed that an existing map search service such as Google Maps (registered trademark) is used as-is without modification, for example.

[0017] Terminal device 10 is an information terminal used by user 11, such as a PC (Personal Computer), tablet device, or smartphone. Terminal device 10 can communicate with server device 100 via communication network N. User 11 can use the conversational services provided by server device 100 using terminal device 10.

[0018] <Hardware Configuration> (Computer hardware configuration) The server device 100 has, for example, the hardware configuration of a computer 200 as shown in Figure 2. Alternatively, the server device 100 is composed of multiple computers 200. The terminal device 10 has, for example, the hardware configuration of the terminal device 10 described later in Figure 3, or the hardware configuration of a computer 200 as shown in Figure 2.

[0019] Figure 2 shows an example of the hardware configuration of a computer according to one embodiment. The computer 200 includes, for example, a CPU (Central Processing Unit) 201, ROM (Read Only Memory) 202, RAM (Random Access Memory) 203, HD (Hard Disk) 204, HDD (Hard Disk Drive) controller 205, display 206, external device connection I / F (Interface) 207, network I / F 208, keyboard 209, pointing device 210, DVD-RW (Digital Versatile Disk Rewritable) drive 212, media I / F 214, and bus line 215, as shown in Figure 2. The computer 200 may also have other processors, such as a GPU (Graphics Processing Unit).

[0020] Furthermore, if the computer 200 is the terminal device 10, the computer 200 is further equipped with a microphone 221, a speaker 222, an audio input / output interface 223, a CMOS (Complementary Metal Oxide Semiconductor) sensor 224, and an image sensor interface 225, etc.

[0021] Of these components, the CPU 201 controls the overall operation of the computer 200. The ROM 202 stores programs used to start the computer 200, such as the IPL (Initial Program Loader). The RAM 203 is used, for example, as a work area for the CPU 201. The HD 204 stores programs such as the OS (Operating System), applications, device drivers, and various data. The HDD controller 205 controls the reading or writing of various data to the HD 204, for example, according to the control of the CPU 201. Note that the HD 204 and the HDD controller 205 are examples of storage devices.

[0022] Display 206 displays various information, such as a cursor, menu, window, text, or image. Note that display 206 may be located outside the computer 200. External device connection I / F 207 is an interface for connecting various external devices to the computer 200. Network I / F 208 is an interface for connecting the computer 200 to a communication network N and communicating with other devices.

[0023] The keyboard 209 is a type of input device equipped with multiple keys for inputting characters, numbers, and various instructions. The pointing device 210 is a type of input device for selecting and executing various instructions, selecting processing targets, moving the cursor, and so on. The keyboard 209 and the pointing device 210 may be located outside the computer 200.

[0024] The DVD-RW drive 212 controls the reading or writing of various data to the DVD-RW 211, which is an example of a removable recording medium. Note that the DVD-RW 211 is not limited to DVD-RW; it may be any other removable recording medium. The media interface 214 controls the reading or writing (storage) of data to the media 213, such as flash memory. The bus line 215 includes an address bus, a data bus, and various control signals for electrically connecting the above components.

[0025] Microphone 221 is a built-in circuit that converts sound into electrical signals. Speaker 222 is a built-in circuit that converts electrical signals into physical vibrations to produce sounds such as music and speech. Audio input / output interface 223 is a circuit that processes the input and output of audio signals between microphone 221 and speaker 222 according to the control of CPU 201.

[0026] The CMOS sensor 224 is a type of built-in imaging means that captures an image of a subject (e.g., a self-portrait) and obtains image data according to the control of the CPU 201. The computer 200 may have other imaging means, such as a CCD (Charge Coupled Device) sensor, instead of the CMOS sensor 224. The image sensor interface 225 is a circuit that controls the driving of the CMOS sensor 224.

[0027] (An example of terminal device hardware configuration) Figure 3 shows an example of the hardware configuration of a terminal device according to one embodiment. Here, we will describe an example of the hardware configuration of the terminal device 10 when the terminal device 10 is an information terminal such as a smartphone or a tablet device.

[0028] In the example shown in Figure 3, the terminal device 10 includes a CPU 301, ROM 302, RAM 303, storage device 304, CMOS sensor 305, image sensor I / F 306, acceleration / direction sensor 307, media I / F 309, and GPS (Global Positioning System) receiver 310.

[0029] Of these, the CPU 301 controls the operation of the entire terminal device 10 by executing a predetermined program. The ROM 302 stores programs used to start the CPU 301, such as an IPL. The RAM 303 is used as the work area for the CPU 301. The storage device 304 is a large-capacity storage device that stores programs such as the OS and applications, and various data, and is implemented by, for example, an SSD (Solid State Drive) or flash ROM.

[0030] The CMOS sensor 305 is a type of built-in imaging means that captures an image of a subject (mainly a self-portrait) and obtains image data according to the control of the CPU 301. The terminal device 10 may have an imaging means such as a CCD sensor instead of the CMOS sensor 305. The image sensor interface 306 is a circuit that controls the driving of the CMOS sensor 305. The acceleration / direction sensor 307 is a type of sensor such as an electronic magnetic compass, gyrocompass, or acceleration sensor that detects the Earth's magnetic field. The media interface 309 controls the reading or writing (storage) of data to or from the media (storage medium) 308, such as flash memory. The GPS receiver 310 receives GPS signals (positioning signals) from GPS satellites.

[0031] Furthermore, the terminal device 10 includes a long-range communication circuit 311, an antenna 311a for the long-range communication circuit 311, a CMOS sensor 312, an image sensor interface 313, a microphone 314, a speaker 315, an audio input / output interface 316, a display 317, an external device connection interface 318, a short-range communication circuit 319, an antenna 319a for the short-range communication circuit 319, and a touch panel 320.

[0032] Of these, the long-distance communication circuit 311 is a circuit that communicates with other devices, for example, via the communication network 2. The CMOS sensor 312 is a type of built-in imaging means that captures an image of a subject and obtains image data according to the control of the CPU 301. The image sensor interface 313 is a circuit that controls the driving of the CMOS sensor 312. The microphone 314 is a built-in circuit that converts sound into electrical signals. The speaker 315 is a built-in circuit that converts electrical signals into physical vibrations to produce sounds such as music and speech. The sound input / output interface 316 is a circuit that processes the input and output of sound wave signals between the microphone 314 and the speaker 315 according to the control of the CPU 301.

[0033] The display 317 is a type of display means, such as a liquid crystal or organic EL (electroluminescence), that displays images of the subject and various icons. The external device connection I / F 318 is an interface for connecting various external devices. The short-range communication circuit 319 includes a circuit for performing short-range wireless communication. The touch panel 320 is a type of input means that allows the user to operate the terminal device 10 by pressing the display 317.

[0034] Furthermore, the terminal device 10 is equipped with a bus line 621. The bus line 621 includes an address bus, a data bus, and the like for electrically connecting each component, such as the CPU 301 shown in Figure 6.

[0035] Note that the hardware configuration of the terminal device 10 shown in Figure 3 is just one example. The terminal device 10 may have a different hardware configuration as long as it has a computer, communication circuitry, display, microphone, and speaker, etc.

[0036] <Functional Configuration> Figure 4 shows an example of the functional configuration of a dialogue system according to one embodiment.

[0037] (Server device functional configuration) The server device 100 implements various functional configurations, such as those shown in Figure 4, by executing predetermined programs stored in a storage medium using the CPU 201 provided in the server device 100. In the example in Figure 4, the server device 100 has various functional configurations such as a user information acquisition unit 401, a presentation information generation unit 402, an information presentation unit 403, an input unit 404, a speech recognition unit 405, a response generation unit 406, an output unit 407, an evaluation unit 408, a response control unit 409, an update unit 410, a storage unit 411, and a user information DB (Database) 412. At least some of the above functional configurations may be implemented by hardware.

[0038] The user information acquisition unit 401 executes a user information acquisition process to obtain user information, which is information about users who use the terminal device 10, from the user information DB 412 or the like.

[0039] Figure 5 is a diagram illustrating user information according to one embodiment. User information 500 includes, for example, user identification information, basic user information, and system usage status information as data types. User identification information is information for identifying a user and includes, for example, information such as username or user ID.

[0040] User basic information is the primary information about the user. User basic information is used to estimate topics that the user is likely to be interested in. User basic information includes, for example, information such as background, relationships, and personality. Background includes, for example, one or more pieces of information such as the user's age, birthday, zodiac sign, place of origin, place of residence, occupation, position, career, and experience. Relationships include, for example, one or more pieces of information such as the user's family, spouse, friends, and roommates. Personality includes, for example, one or more pieces of information such as the user's character, physical characteristics, possessions, knowledge, likes, dislikes, habits, special skills, opinions, desires, and topics of interest.

[0041] System usage status is information about the user's history of using the dialogue system. System usage status includes information such as the date and time of use, dialogue history, and a summary of the dialogue history. The date and time of use is information indicating the date and time the user used dialogue system 1. The dialogue history is the history of the dialogue that took place between the user and dialogue system 1. The dialogue history summary is a summary of the dialogue history summarized using LLM111.

[0042] In this embodiment, user information 500 for each user utilizing the dialogue system 1 is pre-registered in the user information DB 412. Preferably, when the dialogue with the user ends, the server device 100 updates the user information 500 based on the content of the dialogue.

[0043] Furthermore, the dialogue system 1 according to this embodiment is intended for use in nursing care facilities as one of its intended applications.

[0044] The presentation information generation unit 402 performs a presentation information generation process that generates information on topics of interest to the user (topics related to the user information) based on the user information acquired by the user information acquisition unit 401. For example, the presentation information generation unit 402 inputs the user information into the LLM 111 to generate utterances on topics likely to interest the user, which are presented to the user as an introduction to a conversation. Preferably, the presentation information generation unit 402 acquires images related to topics likely to interest the user from the map search service 112 or the LLM 111, etc., and generates presentation information that includes utterances on topics likely to interest the user and images.

[0045] As a specific example, the presentation information generation unit 402 searches for facilities that the user might be interested in within the user's region of origin, based on information about the user's place of origin and interests, and generates presentation information that includes images of the facilities and spoken content related to those facilities.

[0046] When the information presentation unit 403 initiates a conversation with the user, it performs an information presentation process to present information to the user that is likely to be of interest to the user, which has been generated by the presentation information generation unit 402. For example, the information presentation unit 403 transmits presentation information, which includes an image of a facility that the user is likely to be of interest to, and spoken content related to that facility, generated by the presentation information generation unit 402, to the terminal device 10 for display.

[0047] The input unit 404 performs input processing to receive input from the user. For example, the input unit 404 receives the user's spoken voice, the user's video including the spoken voice, or the spoken text representing the user's speech, etc., transmitted by the terminal device 10.

[0048] The speech recognition unit 405 performs speech recognition processing on the user's spoken voice and performs speech recognition processing to obtain the spoken text, which is the user's spoken voice converted into text. For example, when the input unit 40 receives the user's spoken voice or the user's video, the speech recognition unit 405 performs speech recognition processing on the input user's spoken voice or the user's spoken voice included in the user's video.

[0049] The response generation unit 406 performs a response generation process to generate a response to the utterance text (hereinafter simply referred to as "user utterance") which is a text representation of the user's utterance. The response generation unit uses LLM111 to generate a response to the utterance text. For example, the response generation unit 406 inputs the user's utterance into an existing LLM111 such as ChatGPT, instructs it to generate a response to the user's utterance, and retrieves the response generated by LLM111.

[0050] The output unit 407 performs output processing to output a response in response to user input input to the input unit 404. For example, the output unit 407 transmits the response to the user's utterance, generated by the response generation unit 406, to the terminal device 10.

[0051] The output unit 407 may convert the response (text data) to the user's utterance, generated by the response generation unit 406, into speech using known speech synthesis technology, and transmit the converted speech response to the terminal device 10. Furthermore, if the output unit 407 wants the terminal device 10 to display a dialogue agent that interacts with the user, it may draw the dialogue agent and transmit the drawn dialogue agent to the terminal device 10. The terminal device 10 may perform the processing such as converting the utterance text into speech and drawing the dialogue agent.

[0052] The evaluation unit 408 performs an evaluation process to evaluate the user's willingness to continue the conversation based on the input received by the input unit 404. For example, the evaluation unit 408 evaluates whether the user is willing to continue the conversation (whether they are bored or not) based on the user's utterances received by the input unit 404 in response to the response sent by the output unit 407 to the terminal device 10. Specific examples of the evaluation process performed by the evaluation unit 408 will be described later with examples of multiple embodiments. Willingness to continue the conversation refers to the user's desire to continue the conversation with the conversational agent.

[0053] The response control unit 409 (an example of a response unit) executes a response control process (response processing) that outputs a response regarding the termination of the dialogue based on the evaluation result of the evaluation unit 408. For example, the response control unit 409 determines whether or not to terminate the current dialogue based on the evaluation result of the evaluation unit 408 and a threshold that changes as the dialogue progresses. For example, if the evaluation unit 408 evaluates that the user is bored with the current dialogue, the response control unit 409 determines whether or not to terminate the current dialogue and outputs a response regarding the termination of the dialogue. For example, the response control unit 409 outputs a message to terminate the dialogue, or a message suggesting to the user that the dialogue be terminated. A message to terminate the dialogue might be "It's time to end the dialogue now," and a message suggesting to the user that the dialogue be terminated might be "Shall we end the dialogue now?". Note that the message to terminate the dialogue and the message suggesting to the user that the dialogue be terminated are examples of responses that suggest to the user that the dialogue be terminated.

[0054] The response control unit 409 may determine whether or not to change the topic of the current dialogue based on the evaluation result from the evaluation unit 408 and a threshold that changes as the dialogue progresses.

[0055] The update unit 410 performs an update process to update user information based on the interaction history when the interaction with the user ends. For example, when the interaction with the user ends, the update unit 410 updates the system usage status of user information 500 (usage date and time, interaction history, and summary of the interaction history, etc.) as shown in Figure 5. Preferably, the update unit 410 also updates the basic user information of user information 500 as shown in Figure 5 based on the interaction history.

[0056] The storage unit 411 is implemented by, for example, storage devices such as HD204 and HDD controller 205, and a program executed by CPU201, and stores various data (or information) such as user utterances, responses to utterances, and dialogue history. The storage unit 411 may be located outside the server device 100.

[0057] The user information DB 412 is a database that stores user information 500 of users who use the dialogue system 1. The user information DB 412 is implemented by storage devices such as HD 204 and HDD controller 205, and a program executed by CPU 201. The user information DB 412 stores, for example, the user information 500 described in Figure 5, for each user. The user information DB 412 may be located outside the server device 100.

[0058] (Functional configuration of terminal devices) Here, we will assume that the terminal device 10 has the hardware configuration shown in Figure 3, and the following explanation will be provided accordingly.

[0059] The terminal device 10 implements various functional configurations, such as those shown in Figure 4, by executing a predetermined program stored in a storage medium using the CPU 301 provided in the terminal device 10. In the example in Figure 4, the terminal device 10 has various functional configurations such as an input control unit 421, a transmission unit 422, a reception unit 423, and an output control unit 424. At least some of the above functional configurations may be implemented by hardware.

[0060] The input control unit 421 performs input control processing to control user input. User input includes, for example, spoken text input to the touch panel 320, or spoken voice input to the microphone 314. User input may also be, for example, video input including an image of the user captured by the CMOS sensor 305 and image sensor I / F 306, and the user's spoken voice acquired by the microphone 314. Voice input may also be from a wireless microphone that can communicate using the short-range communication circuit 319, an external microphone connected to the external device connection I / F 318, or an external camera.

[0061] The transmission unit 422 performs a transmission process to send user input to the server device (information processing device) 100. For example, the transmission unit 422 transmits user speech, such as user speech text, user speech audio, or user image, acquired by the input control unit 421, to the server device 100.

[0062] The receiving unit 423 performs a receiving process to receive from the server device 100 the response to the user's utterance that the transmitting unit 422 has sent to the server device 100.

[0063] The output control unit 424 performs output control processing to control the output of the response to the user. For example, the output control unit 424 outputs the response audio included in the response received by the receiving unit 423 using the sound input / output interface 316 and the speaker 315, etc. The output control unit 424 may also output using, for example, a wireless earphone or wireless speaker that can communicate via the short-range communication circuit 319.

[0064] Furthermore, if the response received by the receiving unit 423 includes response text and an image of the dialogue agent, the output control unit 424 will display a display screen showing the response text and the dialogue agent on the display 317 or the like.

[0065] Note that the functional configuration of the dialogue system 1 shown in Figure 4 is just one example. For example, each functional configuration of the server device 100 in Figure 4 may also be possessed by the terminal device 10. In this case, the terminal device 10 becomes the information processing device according to this embodiment. Furthermore, the terminal device 10 may have some of the functional configurations of the server device 100 in Figure 4. Moreover, each functional configuration of the server device 100 in Figure 4 may be distributed and provided on multiple information processing devices.

[0066] <Processing flow> Next, the processing flow of the dialogue control method according to this embodiment will be described.

[0067] [First Embodiment] (Processing of the dialogue system) Figure 6 is a sequence diagram showing an example of the processing of the dialogue system according to the first embodiment. This processing shows, for example, an example of the processing performed by the dialogue system 1 described in Figures 1 to 5.

[0068] In step S601, when the user logs in to the server device 100 using the terminal device 10, the dialogue system 1 executes the processes from step S602 onward.

[0069] In step S602, the user information acquisition unit 401 of the server device 100 acquires the user information 500 of the logged-in user from the user information DB 412. The server device 100 may also acquire the system usage status from the user information 500 as shown in Figure 5 and present the user with a greeting message, for example, "[Username], hello. Last time we talked about [topic of our previous conversation]."

[0070] In steps S603 to S605, the information generation unit 402 of the server device 100 creates topics that are likely to interest the user and outputs them to the terminal device 10.

[0071] For example, in step S603, the information generation unit 402 uses information such as the user's place of origin, place of residence, and topics of interest, as shown in Figure 5, to search for facilities that might interest the user using the map search service 112. Specifically, based on the user information, the unit searches for facilities that might interest the user using the map search service 112, using the API (Application Programming Interface) of the map search service 112.

[0072] In step S604, the information generation unit 402 obtains facilities and images that are likely to interest the user from the map search service 112.

[0073] In step S605, the information presentation unit 403 of the server device 100 presents the topic and images of the facility acquired by the presentation information generation unit 402 to the terminal device 10 as an introduction to the dialogue.

[0074] In step S606, the server device 100 performs a dialogue process to interact with the user using the terminal device 10. For example, the server device 100 performs a dialogue process as shown in Figure 7.

[0075] Figure 7 is a flowchart illustrating an example of dialogue processing according to the first embodiment. This process shows, for example, an example of dialogue processing performed by the server device 100 in step S606 of Figure 6.

[0076] In step S701, the input control unit 421 of the terminal device 10 receives the user's utterance as input, and the transmission unit 422 transmits the user's utterance to the server device 100. The server device 100 receives the user's utterance from the terminal device 10, and once the input unit 404 accepts the user's utterance input, the server device 100 executes the processing from step S702 onward. If the user's utterance input is voice data, the speech recognition unit 405 converts the user's utterance into text at this point.

[0077] In step S702, the response generation unit 406 instructs the LLM 111 to generate a response to the user's utterance. The instruction includes the content of the user's utterance and at least a portion of the user's basic information. The response generation unit 406 also generates a response to the user's utterance by acquiring the response generated by the LLM 111.

[0078] In step S703, the output unit 407 outputs the response generated by the response generation unit 406 to the terminal device 10.

[0079] In step S704, the server device 100 stores the dialogue content from steps S701 to S704. For example, the input unit 404 stores the user's utterances input from the terminal device 10 in the dialogue history 800 as shown in Figure 8. The output unit 407 also stores the responses output to the terminal device 10 in the dialogue history 800.

[0080] In step S705, the evaluation unit 408 performs an evaluation process to assess whether the user is getting bored with the conversation, based on the conversation history 800. The specific details of the evaluation process will be described in detail in the fourth and fifth embodiments.

[0081] In step S706, the response control unit 409 determines whether the user is bored with the conversation based on the evaluation result from the evaluation unit 408. If the user is not bored with the conversation, the response control unit 409 returns to step S701 and continues the conversation. On the other hand, if the user is bored with the conversation, the response control unit 409 moves the process to step S707.

[0082] When the process moves to step S707, the response control unit 409 suggests to the user that the conversation be ended. For example, the response control unit 409 outputs a message to the terminal device 10 via the output unit 407 that concludes the conversation and suggests that it be ended. For example, the response control unit 409 outputs the message, "Shall we end the conversation now?" as a message suggesting that the conversation be ended.

[0083] In step S708, the response control unit 409 decides whether or not to terminate the dialogue. For example, if the user wishes to end the dialogue, the response control unit 409 decides to terminate the dialogue, and if the user wishes to continue the dialogue, it decides not to terminate the dialogue. If it decides not to terminate the dialogue, the response control unit 409 returns to step S701 and continues the dialogue. On the other hand, if it decides to terminate the dialogue, the response control unit 409 terminates the dialogue process shown in Figure 7.

[0084] Now, let's return to Figure 6 and continue the explanation of the sequence diagram. When the interactive process in step S606 is completed, the server device 100 executes the process in step S607.

[0085] In step S607, the update unit 410 updates the user information. For example, the update unit 410 adds information such as the date and time of use and the dialogue history included in the system usage status of the user information 500, as shown in Figure 5, to the user information 500. The update unit 410 may also change or delete the user information.

[0086] (supplement) When using the dialogue system 1 according to this embodiment for caregiving purposes, the following procedure may be added as a function to check the psychological state of the person receiving care. 1) Create a "Caregiver Confirmation Items" item in the "System Usage Status" field of User Information 500, as shown in Figure 5. 2) At the end of the conversation, the update unit 410 determines, for example, using LLM111 or text search, whether there is any negative content in the user's statements or whether the same topic as in past conversations is being repeated, and records the determination result in "Items to be confirmed by the caregiver".

[0087] Furthermore, a system for creating care records from the content of conversations with the user is known (see, for example, Patent Document 4). The technology disclosed in Patent Document 4 focuses on the user's physical ailments (e.g., difficulty sleeping, frequent trips to the toilet, etc.) and conducts conversations based on a scenario that guides the user to fill in predetermined care record items. On the other hand, the methods described in 1) and 2) above are expected to have the effect of detecting the user's psychological ailments from natural casual conversations.

[0088] <Example of dialogue history> Figure 8 shows an example of a dialogue history according to the first embodiment. This dialogue history 800 shows an example of a dialogue history when the dialogue system 1 and the user interact in the processing flow of the dialogue system described in Figures 6 and 7.

[0089] It should be assumed that user information 500 contains the following information: Name: Saburo Suzuki, Place of Origin: Shinagawa Ward, Tokyo, Hobbies: Reading, etc.

[0090] In the dialogue history 800 in Figure 8, history 801 corresponds to the processing in step S605 in Figure 6, where the dialogue system 1 provides the user with topics that are likely to interest the user, along with images.

[0091] Furthermore, history entries 802-804 correspond to the dialogue processing shown in Figure 7. Dialogue system 1 continues the dialogue in history entry 802, and if it determines in history entry 803 that the user has grown tired of the dialogue, it concludes the dialogue in history entry 804 and suggests ending the dialogue to the user.

[0092] [Second Embodiment] Figure 9 is a sequence diagram showing an example of the processing of the dialogue system according to the second embodiment. This processing shows another example of the processing performed by the dialogue system 1 described in Figures 1 to 5. Since the basic processing content is the same as that of the dialogue system according to the first embodiment described in Figure 6, a detailed explanation of the processing content similar to that of the first embodiment is omitted here.

[0093] In step S901, when the user logs in to the server device 100 using the terminal device 10, the dialogue system 1 executes the processes from step S902 onward.

[0094] In step S902, the user information acquisition unit 401 of the server device 100 acquires the user information 500 of the logged-in user from the user information DB 412.

[0095] In step S903, the information generation unit 402 of the server device 100 inputs, for example, the dialogue history or a summary of the dialogue history included in the user information 500 as shown in Figure 5 to the LLM 111 and instructs the LLM 111 to extract the topic of the previous dialogue.

[0096] In step S904, the presentation information generation unit 402 obtains the topic of the previous conversation generated by LLM111.

[0097] In step S905, the information display unit 403 of the server device 100 outputs a greeting to the terminal device 10 that includes the topic from the previous conversation as an introduction to the dialogue. For example, the information display unit 403 presents the user with a greeting such as, "Hello, [username]. Last time we talked about [topic from the previous conversation]."

[0098] In step S906, the server device 100 executes the dialogue process described in Figure 7, for example. After the dialogue process in step S906 is completed, the server device 100 executes the processes from step S907 onward.

[0099] In step S907, the update unit 410 inputs, for example, the dialogue history 800 shown in Figure 8 to the LLM 111 and instructs it to generate a summary of the dialogue history. Then, in step S908, the update unit 410 retrieves the summary of the dialogue history generated by the LLM 111.

[0100] In step S909, the update unit 410 inputs, for example, the dialogue history 800 shown in Figure 8 to the LLM 111 and instructs it to extract new user information. For example, the update unit 410 instructs the LLM 111 to extract unregistered information from the dialogue history 800 among the information included in the user basic information of the user information 500 shown in Figure 5. Also in step S910, the update unit 410 acquires the new user information extracted by the LLM 111.

[0101] In step S911, the update unit 410 updates the user information. For example, the update unit 410 adds information such as the date and time of use and the dialogue history included in the system usage status of the user information 500, as shown in Figure 5, to the user information 500.

[0102] Furthermore, the update unit 410 adds the summary of the dialogue history obtained in step S908 to the summary of the dialogue history included in the system usage status of the user information 500. This summary of the dialogue history is used, for example, when the user logs in next time. When incorporating the content of the previous dialogue into the system's utterances, incorporating a summary of the dialogue history into the prompt saves tokens and makes the prompt content more concise, rather than incorporating the entire dialogue history into the prompt, thus improving the response accuracy of the LLM 111. In addition, when the dialogue system 1 according to this embodiment is used for caregiving purposes, the summarization of the dialogue history makes it easier for caregivers to understand the system usage status of the person being cared for.

[0103] Furthermore, the update unit 410 adds the new user information acquired in step S910 to the user basic information of the user information 500. Preferably, the response generation unit 406 generates a response using the user basic information, which is a more concise summary of the user's information. This allows the response generation unit 406 to continue referencing information the user has spoken in past conversations while keeping the number of tokens down.

[0104] For example, if the entire conversation history, which continues to grow with each use, is incorporated into the prompt, token consumption will increase and the conciseness of the response will decrease. On the other hand, if only the content of the most recent few conversations is incorporated into the prompt, the system will not be able to understand what the user said before that. To address these problems, response generation that incorporates the user's basic information into the prompt extracts and uses only the essential points of the previous conversation, thus keeping the number of tokens down while maintaining information consistency.

[0105] For example, as shown in Figure 5, the user information 500 allows for the extraction of a wide range of information from the conversation, such as the user's hobbies, interests, experiences, and desires, which can then be flexibly used to respond to various topics.

[0106] In the future, the server device 100 may be able to devise original attributes based on the user's basic information. For example, for a user whose hobby is baseball, it may be possible to add more detailed attributes such as "favorite baseball team" or "baseball experience and memories" to better cater to individual hobbies and interests.

[0107] [Third Embodiment] Figure 10 is a sequence diagram showing an example of the dialogue introduction process according to the third embodiment. This process shows, for example, a more specific example of the process (dialogue introduction process) described in steps S601 to S605 of Figure 6.

[0108] In step S1001, when the user logs in to the server device 100 using the terminal device 10, the dialogue system 1 executes the processes from step S1002 onward.

[0109] In step S1002, the user information acquisition unit 401 of the server device 100 acquires the user information 500 of the logged-in user from the user information DB 412.

[0110] If the necessary information for user information 500 is missing, the server device 100 executes the processes in steps S1003 and S1004. The necessary information includes, for example, (A) place of origin or residence, (B) hobbies, favorite things, or topics of interest.

[0111] In steps S1003 and S1004, the user information acquisition unit 401 of the server device 100 obtains the necessary user information by sending a question to the terminal device 10 to supplement the missing information and receiving the answer sent by the terminal device 10. If the necessary user information is already registered in the user information 500, the user information acquisition unit 401 does not execute the processes in steps S1003 and S1004.

[0112] In step S1005, the information generation unit 402 of the server device 100 instructs the LLM 111 to output facilities that the user might be interested in, and the basis for those facilities. For example, the information generation unit 402 sends a prompt 1100, as shown in Figure 11, to the LLM 111 along with user information. This prompt 1100 instructs the LLM 111 to output facilities related to the user's hobbies or things they like, and the user information that supports those selections. In response, the LLM 111 outputs information about facilities that the user might be interested in (e.g., soccer field, curry restaurant, etc.) and user information that supports the facility selection (e.g., "The user's hobby is soccer," "The user likes curry," etc.) in JSON (JavaScript Object Notation) format.

[0113] In step S1006, the information generation unit 402 of the server device 100 acquires information about facilities that the user is likely to be interested in, output by the LLM 111, and user information that serves as the basis for facility selection.

[0114] In step S1007, the information generation unit 402 of the server device 100 requests the map search service 112 to search for facilities. For example, the information generation unit 402 uses the API provided by the map search service 112 to request a search for "place of origin + facilities that the user might be interested in".

[0115] In step S1008, the presentation information generation unit 402 of the server device 100 acquires the search results. Preferably, the presentation information generation unit 402 excludes facilities whose addresses do not include the place of origin from the search results.

[0116] In step S1009, the information generation unit 402 of the server device 100 selects one facility to present to the user from the search results. For example, the information generation unit 402 extracts the top 10 facilities with the most reviews from the facilities included in the search results, randomly selects one facility from the extracted facilities, and obtains an image of that facility. The number of facilities to be extracted can be any number.

[0117] In step S1010, the presentation information generation unit 402 of the server device 100 instructs the LLM 111 to generate a dialogue introductory sentence. For example, the presentation information generation unit 402 instructs the LLM 111 to generate a dialogue introductory sentence, including the following information in the prompt. - Instruct the system to respond to a user's statement, "My place of origin is [place name]." • Facility and supporting information obtained in Step S1006. • The facility selected in step S1009 must be located in the user's region of origin.

[0118] In step S1011, the presentation information generation unit 402 of the server device 100 acquires the dialogue introduction text generated by LLM111.

[0119] In step S1012, the information display unit 403 of the server device 100 transmits to the terminal device 10 an interactive introduction screen containing the image of the facility selected in step S1009 and the interactive introduction text acquired in step S1011. As a result, the terminal device 10 displays an interactive introduction screen 1200, for example, as shown in Figure 12.

[0120] Figure 12 shows an example of a dialogue introduction screen according to the third embodiment. In the example in Figure 12, the dialogue introduction screen 1200 displays a dialogue agent 1201 that interacts with the user, a dialogue introduction statement obtained in step S1011 of Figure 10, and an image 1203 of the facility selected in step S1009 of Figure 10. Note that since a single image 1203 may not allow the user to visualize the facility, it is desirable to display multiple different images.

[0121] By providing users with images of facilities that are familiar to them and of interest, such an introductory dialogue screen 1200 can enhance the effectiveness of reminiscence therapy.

[0122] Since acquiring images takes time, in order to reduce the user's perceived waiting time, the server device 100 should, for example, speak a phrase such as "[User's hometown]" once the topic has been decided, as shown in Figure 13.

[0123] Figure 13 shows an example of image acquisition processing according to the third embodiment. In Figure 13, the dialogue system 1 searches for ramen restaurants in Yokohama City, and at time t0, the topic is determined to be ramen restaurants in Yokohama City. The dialogue system 1 also takes time t0 to t2 to acquire images of the search results (ramen restaurants in Yokohama City). In this case, at time t0, the dialogue system 1 outputs an utterance such as, "So you're from Yokohama City?". This reduces the user's perceived waiting time from t0 to t2 to t1 to t2.

[0124] Now, let's return to Figure 10 and continue the explanation of the sequence diagram. In step S1013, when the user speaks to the dialogue introduction screen, the dialogue system 1 executes the processes from step S1014 onwards.

[0125] In step S1014, the evaluation unit 408 of the server device 100 instructs the LLM 111 to determine whether the user's utterance regarding the presented topic is positive or negative. For example, the evaluation unit 408 uses a prompt 1400 as shown in Figure 14 to cause the LLM 111 to determine whether the user's utterance is a positive or negative response. In step S1015, the evaluation unit 408 obtains the determination result from the LLM 111.

[0126] In step S1016, the response generation unit 406 determines the dialogue strategy depending on whether the user's utterance is a positive or negative response. For example, if the user's response is positive (e.g., I know it, I've been there, etc.), the response generation unit 406 uses LLM 111 to generate questions that delve deeper into the presented topic. For example, the response generation unit 406 generates questions such as, "So you've been there. What kind of ramen is your favorite at XX?"

[0127] On the other hand, if the user's utterance is negative (e.g., I don't know, I'm not interested, etc.), LLM111 acknowledges the user's negative attitude towards the topic and generates a response that presents a different related topic. For example, the response generation unit 406 generates a question such as, "I see. So, do you have any other ramen restaurant recommendations?"

[0128] The above process allows the dialogue system 1 to provide topics tailored to the user's interests, thereby offering the following benefits: By providing images of facilities that are familiar and interesting to the user, conversations can become more engaging, and the effectiveness of reminiscence therapy can be enhanced. By excluding facilities whose address data does not contain the user's place of origin from search results, it is possible to prevent facilities that are not located in the user's region from being incorrectly displayed. By initially narrowing down the search results to a predetermined number of items with a large number of reviews, it is possible to prevent the display of facilities that are not very familiar to residents, or facilities attached to them, such as parking lots. By randomly selecting facilities to be presented as topics, it is possible to prevent the exact same facilities or topics from being repeated each time, thus adding freshness and diversity to the conversation.

[0129] <Supplement> Here, we will provide supplementary explanations regarding the first to third embodiments.

[0130] (Regarding topics that will interest users) In this embodiment, "facilities located within the user's region that are likely to interest the user" are presented as "topics that will pique the user's interest." For example, if the user likes soccer and curry, the dialogue system 1 will suggest facilities such as a soccer field or a curry restaurant based on that information. By providing topics tailored to the user in this way, the user is expected to feel that their conversation partner is interested in them and considerate, leading to greater satisfaction. The reason for suggesting facilities that match the user's hobbies and preferences, rather than general places like tourist spots, is that the latter are thought to be more likely to pique personal interest and keep the conversation flowing smoothly. Even if the suggested facility itself does not pique the user's interest, the direction of the conversation is guided in line with the user's hobbies and preferences, making it easier to continue the conversation.

[0131] (Regarding the acquisition of user information) The dialogue system 1 retrieves the following user information from the user information DB 412, for example: Place of origin or place of residence • Other information This information is obtained from the "Basic User Information" in the User Information 500 mentioned above. From this, at least one of the following is selected as "Place of Origin or Place of Residence." The remaining information is designated as "Other Information." "Other Information" is used in later steps to predict facilities that the user might be interested in. This includes information that is not necessary when selecting a facility (e.g., occupation, dislikes, etc.), but this information is excluded at a later stage, thus maintaining the accuracy of topic setting.

[0132] (Regarding methods for predicting topics that users might be interested in) Dialogue system 1 uses LLM111 to predict facilities that the user might be interested in. For example, dialogue system 1 provides LLM111 with "other information" obtained from user information 500, and then narrows down the user information to focus on with a prompt such as "Please show me facilities that are particularly related to the user's hobbies, favorite foods, and topics of interest." This prevents dialogue system 1 from showing facilities that are unrelated to the user's interests.

[0133] In this case, it is desirable to exclude the user's occupational information from the reference information used when selecting a topic. For example, while LLM111 is likely to answer that stationery stores are related facilities to the occupation of an office worker, not all office workers are interested in stationery stores. This is because it is difficult to accurately narrow down the range of the user's interests from occupational information. Similarly, it is desirable to exclude information such as age and dislikes from the reference information used when selecting a topic among "other information." The dialogue system 1 achieves topic setting based on the user's interests by prioritizing information on hobbies and interests rather than directly referring to occupational information.

[0134] (Regarding methods for searching for and selecting topics) The dialogue system 1 uses the map search service 112 to search for facilities using, for example, the user's place of origin and facilities that the user might be interested in as search terms. With this method, the search results may include, for example, facilities located in municipalities surrounding the specified area, facilities that are not very familiar to residents, and facilities attached to the facility (e.g., parking lots). Therefore, it is desirable for the dialogue system 1 to narrow down the results to present only facilities that the user might be interested in by performing the following processing. Topics that the user might be interested in (topics related to user information) are, for example, search results that meet certain conditions from the search results obtained using user information. • Each search result checks whether the address data contains the string of the place of origin, and facilities that do not match are excluded, as they are considered not to exist within the specified area. • We compare the number of reviews for each search result and select only the top 10 with the most reviews. This is because facilities and amenities that are not familiar to residents are likely to have fewer reviews. - One facility is randomly selected from the 10 narrowed-down facilities to be presented to the user as a topic. This prevents the same facility from being presented as a topic every time a user with limited user information interacts with the system. For example, if the only user information registered is "I'm from Yokohama City, Kanagawa Prefecture, and I like ramen," presenting the top-ranked facility by number of reviews every time would likely cause the system to repeatedly present topics about the same ramen shop in Yokohama City.

[0135] (Regarding the generation of dialogue introductions) When the topic to be presented to the user (the facility in the search results) and its image are ready, the dialogue system 1 generates an introductory dialogue sentence using LLM111. At this time, it is desirable that the prompt instructing LLM111 to generate the introductory dialogue sentence include the following information. The request is to consider a response to a user's statement, "My hometown is [place name]." • A set of "facilities that the user is likely to be interested in" and "user information that serves as the basis for listing those facilities," used in the facility search. • The facilities selected in the search results must be located in the user's region of origin. For example, for a user with prior information such as "I'm from Yokohama City, Kanagawa Prefecture, and I like ramen," the system might generate an introductory sentence like, "You mentioned you like ramen; have you ever been to the ramen shop called XX? I hear it's a great place to enjoy delicious ramen, I recommend it." This allows the user to feel that "they've presented a topic that was thoughtfully chosen for me," or "they're interested in the things I like."

[0136] (Measures to provide topics that are more in line with user interests) In the future, by storing the following information in the user information DB412 at the end of a conversation, it is conceivable that this data could be used to more accurately understand the user's interests and effectively suggest topics that will pique their interest. • Topics presented (facility name, user information referenced) • The image that was presented • Search terms used when searching for topics How many turns did the conversation on that topic last? • How users reacted to the topic (e.g., positive reactions such as "You know a lot about this," or negative reactions such as "Not that again?") etc.

[0137] <Evaluation process> Next, we will explain specific examples of evaluation processes that assess the user's willingness to continue the conversation (whether they are bored with the conversation or not, etc.).

[0138] [Fourth Embodiment] Figure 15 is a flowchart illustrating an example of an evaluation process according to the fourth embodiment. This process shows, for example, an example of a process in step S705 of Figure 7 in which the server device 100 evaluates whether the user is bored with the conversation (whether or not they have the desire to continue the conversation). Here, the state of "the user is bored with the conversation" refers to a state in which the user has no desire to continue the conversation with the conversation system 1 (desire to continue the conversation) (or a state in which it is presumed that the user has no desire to continue the conversation). For example, this refers to a state in which the user appears bored with the utterances of the conversation system 1, or appears to want to end the conversation.

[0139] In step S1501, the evaluation unit 408 of the server device 100 acquires the dialogue history between the user and the dialogue system 1.

[0140] In step S1502, the evaluation unit 408 uses the LLM 111 to evaluate the user's boredom. For example, the evaluation unit 408 inputs a prompt containing the dialogue history into the LLM 111 to instruct the user to evaluate whether they are bored with the dialogue in response to the dialogue system 1's utterances. As a specific example, the evaluation unit 408 outputs -1 point if the user is bored, and +1 point if the user is not bored, to the LLM 111. This point is an example of a first evaluation result obtained by inputting the dialogue history (content of the dialogue) into a large-scale language model to evaluate the user's willingness to continue the dialogue (first willingness to continue the dialogue). The point may be a positive value when the user is bored and a negative value when the user is not bored.

[0141] This process incorporates past dialogue history into the prompts, enabling the prediction of boredom by considering the context and the user's speaking style. For example, consider the case where the user says, "I read a novel." In this case, if the previous utterance from Dialogue System 1 was, "What kind of book did you read?", that is, if the user's utterance is in line with the preceding context, it can be inferred that the user is likely not bored. On the other hand, if the previous utterance from Dialogue System 1 was, "Do you have a favorite drink?", the user's utterance is an inconsistent response, and therefore it can be inferred that the user is likely bored.

[0142] In step S1503, the evaluation unit 408 evaluates the user's boredom based on the number of characters spoken. For example, the evaluation unit 408 monitors whether the average number of characters spoken over several utterances, including the most recent utterance, is decreasing, thereby inferring boredom from the perspective of whether the user is speaking less.

[0143] For example, it is known that when children get bored of playing, they tend to speak less, become expressionless, act more slowly, and become preoccupied with other things (see, for example, Non-Patent Document 1). It is thought that similar characteristics appear when adults get bored of a conversation. In this embodiment, we focused on "whether the number of words is decreasing," which can be judged solely from the utterance without using special equipment such as a camera, and devised a method to monitor the trend in the number of characters in the user's utterances. In this method, we determine whether the user's number of words is decreasing by comparing the average number of characters in each utterance. However, instead of estimating boredom based solely on the number of characters in the current utterance, we estimate it using the average number of characters including the previous utterance to mitigate the influence of sudden long utterances or short interjections.

[0144] Figure 16 is a diagram illustrating the evaluation based on the number of utterance characters according to the fourth embodiment. The evaluation unit 408 obtains the number of utterance characters 1601 for each user utterance from the utterance history, calculates the average number of characters for the most recent five user utterances, and stores it in list 1602. This list 1602 is updated each time the user utters, and the average value of the most recent five utterance character counts is successively added.

[0145] Next, the evaluation unit 408 checks whether the value of the number of utterances in list 1602 is continuously decreasing. If the value of the number of utterances is continuously decreasing, the evaluation unit 408 determines that the user is speaking less and infers that the user is getting bored with the conversation. For example, if the user is speaking less, the evaluation unit 408 assigns -1 point, and if the user is not speaking less, it assigns +1 point. This point is an example of a second evaluation result that evaluates the user's willingness to continue the conversation (second willingness to continue the conversation) based on the number of utterances the user has spoken. The point may be a positive value when the user is speaking less and a negative value when the user is not speaking less.

[0146] Note that the "boredom" evaluation in steps S1502 and S1503 is just one example of evaluating whether or not the user is willing to continue the conversation. For example, the evaluation unit 408 may evaluate whether or not the user is willing to continue the conversation based on other indicators, such as whether the user is bored or irritated.

[0147] In step S1504, the evaluation unit 408 adds the points evaluated in steps S1502 and S1503.

[0148] In step S1505, the evaluation unit 408 aggregates the points from the most recent predetermined number of times (for example, 5 times). For example, the evaluation unit 408 creates data such as [0, 2, -2, -2, 2] in order from the oldest points.

[0149] In step S1506, the evaluation unit 408 calculates evaluation points by weighting the points collected a predetermined number of times. For example, the evaluation unit 408 weights newer utterances more importantly. For example, the evaluation unit 408 calculates evaluation points by adding the points a predetermined number of times, weighting them in order from oldest to newest, such as [0.5, 0.75, 1, 1.5, 1.75], with newer points having a larger weight.

[0150] In step S1507, the evaluation unit 408 determines whether the calculated evaluation points are below a threshold (for example, "-3"). In this specific example, a larger negative evaluation point indicates a more negative evaluation (e.g., the user is bored or talking less), so the unit determines whether it is below the threshold. If a larger positive evaluation point indicates a more negative evaluation (e.g., the user is bored or talking less), then the unit determines whether it is above the threshold.

[0151] Preferably, this threshold changes dynamically based on time information associated with the progress of the dialogue, such as the total number of utterances (or the duration of the dialogue) or the duration of the same topic. For example, if the same topic continues for a long time, the threshold is lowered to make it easier to determine that the user is bored. This prevents the user from getting bored by unintentionally continuing the same topic indefinitely. On the other hand, if the user wants to continue talking about the same topic, the conversation will continue naturally if the user actively speaks. Lowering the threshold means changing the threshold so that it is easier to determine that the user is bored. If a large negative evaluation point indicates a negative evaluation (such as the user being bored or speaking less), the threshold is changed in the positive direction, and if a large positive evaluation point indicates a negative evaluation, the threshold is changed in the negative direction. The number of utterances indicates the number of exchanges of responses between the user and the system. When the system makes an initial utterance and the user responds to that utterance, the number of utterances becomes 1, and when the system responds to the user's response, the number of utterances becomes 2.

[0152] If the evaluation points are below the threshold, the evaluation unit 408 proceeds to step S1508 and evaluates that the user is bored with the conversation. On the other hand, if the evaluation points are not below the threshold, the evaluation unit 408 proceeds to step S1509 and determines that the user is not bored with the conversation.

[0153] Thus, in this embodiment, user boredom is determined by using two methods in combination: one that uses LLM111 to estimate user boredom, and another that monitors the progression of the number of characters spoken by the user. The two methods are combined to mutually compensate for the disadvantages of each method and to realize a system with fewer false positives.

[0154] Regarding the disadvantages of using LLM to predict user boredom, for example, if a large amount of training data is collected and trained, as in the technology disclosed in Patent Document 3, it is considered possible to determine user boredom with relatively high accuracy using a machine learning model. However, when implementing this at low cost by reusing the existing LLM111, there is a disadvantage that the accuracy of the determination will be lower compared to a model specifically designed for boredom determination.

[0155] One disadvantage of monitoring the user's utterance length is that, for example, if the user makes a short response such as an interjection, the utterance length decreases, which can easily lead to the system misinterpreting the user as having lost interest, even if the user intended to continue the conversation.

[0156] In this embodiment, two simple and low-cost methods were used to determine boredom, but in the future, the accuracy of the determination may be further improved by adding the following methods in combination. This would allow the dialogue system 1 to capture user information from more diverse perspectives and make it less likely to miss signs of boredom. • Inference from facial expressions and posture: Dialogue system 1 uses camera footage to analyze the user's facial expressions, face orientation, and changes in posture. For example, if the user looks away from the screen or their posture deteriorates, dialogue system 1 can determine that they are likely to be bored. • Voice emotion recognition: Dialogue system 1 analyzes emotions from the user's speech and considers the likelihood of boredom high if negative emotions are consistently observed. It is believed that signs of boredom can be grasped not only from the content of speech but also from the manner of speaking, as the tone and tempo of the voice show a consistent trend.

[0157] The evaluation process according to the fourth embodiment can be expected to have the following effects, for example. By using two methods to estimate boredom, if one method fails to estimate well, the other method can compensate, reducing the likelihood of misjudging boredom. • By adding up "points" from the most recent predetermined number of conversations, in other words, by making a determination of boredom that takes time information into account, it is possible to prevent the system from immediately determining that the user is bored when they, for example, give a short nod of agreement. Since there is no need to create a new model, it can be easily implemented using the existing LLM111 without incurring additional costs. • The weights and thresholds used to determine user boredom can be set to any value. This makes it easier to understand the basis for boredom detection during debugging, for example, and allows for fine-tuning as needed. Furthermore, it is designed to be flexible and not dependent on specific settings, allowing it to accommodate diverse user characteristics and speaking styles.

[0158] [Fifth Embodiment] Figure 17 is a flowchart showing an example of the evaluation process according to the fifth embodiment. This process shows another example of the process in step S705 of Figure 7, where the server device 100 evaluates whether the user is bored with the conversation (whether or not they are willing to continue the conversation). Note that steps S1501 to S1506 of the process in Figure 17 are the same as the evaluation process according to the fourth embodiment described in Figure 15, so their explanation is omitted here.

[0159] In step S1701, the evaluation unit 408 obtains a threshold value corresponding to the number of utterance turns. For example, as shown in Figure 17, the server device 100 pre-stores correspondence information 1700, which shows the correspondence between the number of utterance turns and the threshold value, in the storage unit 411 or the like. This allows the evaluation unit 408 to obtain the threshold value corresponding to the current number of utterance turns from the correspondence information 1700.

[0160] As shown in Figure 17, the corresponding information 1700 is set so that the threshold increases as the number of utterances increases, making it easier to determine that the speaker has lost interest. This threshold is a concrete example of a threshold that changes as the dialogue progresses.

[0161] In step S1702, the evaluation unit 408 determines whether the calculated evaluation points are below the threshold obtained in step S1701. If the evaluation points are below the threshold, the evaluation unit 408 proceeds to step S1703 and evaluates that the user is bored with the conversation. On the other hand, if the evaluation points are not below the threshold, the evaluation unit 408 proceeds to step S1704 and determines that the user is not bored with the conversation.

[0162] In the example shown in Figure 17, the threshold is determined by the number of utterances. However, in the future, a function may be added to automatically adjust parameters such as thresholds or weights according to the user's dialogue pattern or speaking style. For example, the server device 100 can store thresholds or weights corresponding to the user's dialogue tendencies in a database and use parameters optimized for each user to determine boredom, thereby determining the user's level of boredom with greater accuracy. By individualizing the parameters in this way, flexible responses tailored to each user's preferences can be expected.

[0163] Furthermore, regarding the method of determining boredom while considering the progress of the conversation, this was achieved in this case by lowering the boredom threshold based on time information, but it could also be achieved by changing the weighting value when adding boredom points based on time information. In other words, the weighting value could be a value that changes with the progress of the conversation. In this case, by increasing the weighting value when the same topic continues for a long time, the accumulated points will more easily exceed the threshold, and as a result, it will be easier to determine that the person has become bored.

[0164] (Strategies for continuing the conversation with users who have become bored) In the future, if it is determined that a user has lost interest in a conversation, instead of immediately ending the conversation, attempts may be made to continue the conversation in the following ways: 1) Providing news tailored to interests Dialogue system 1 may attempt to keep the conversation going by adding freshness to it by obtaining the latest news and topics that the user might be interested in from external sources and providing them as themes for the conversation. 2) Presenting new topics using related images When suggesting a new topic, dialogue system 1 may attempt to continue the conversation by visually attracting the user's attention and encouraging their interest in the conversation by displaying related images. 3) Generating topics using real-time video Dialogue system 1 acquires video footage of the user's surroundings using a camera and generates topics from the events happening in the environment. This enables natural conversations based on things that are actually happening around the user. For example, by commenting on the weather or clothing, the system can make the user feel more comfortable and encourage them to continue the conversation, thereby encouraging them to continue. For example, dialogue system 1 might suggest topics such as, "It's getting a little brighter around here. Has the rain stopped?" or "You're wearing a red shirt today, it suits you very well. Where did you buy it?"

[0165] Furthermore, the dialogue system 1 may reduce the "points" for evaluating boredom by a certain percentage during the first few turns after changing topics, temporarily loosening the threshold for determining boredom, thereby reducing the impact of boredom from the previous topic. This method makes it easier to evaluate interest in the new topic and increases the likelihood that the user will become actively involved in the dialogue again.

[0166] As described above, according to this embodiment, in a dialogue system 1 that automatically responds to user utterances, it becomes possible to control the response while taking into account the progress of the dialogue.

[0167] Furthermore, the following effects can be expected from the dialogue system 1 according to this embodiment. • By searching for and presenting topics based on user information, it is possible to provide topics that match the user's interests. • By presenting related images along with the topic, it is possible to attract the user's attention and prevent the conversation from becoming monotonous. Regarding the expression of boredom, instead of using uniform standards for all people or user-specific settings, boredom is determined by the changes in the user's verbal responses during each conversation. This allows for the estimation of boredom to take into account not only the user's personality but also their mood and physical condition at the time. • When estimating boredom, time information such as "the content of the last few utterances," "the total number of utterances in the dialogue (dialogue duration)," and "the duration of the same topic" is taken into consideration, allowing for boredom estimation that takes into account the progress of the dialogue. For example, measures can be taken to prevent the dialogue on the same topic from lasting longer than necessary. • Because it uses the existing LLM111 instead of a dedicated LLM for estimating boredom, and relies only on text information or user behavior, it can be implemented at a low cost.

[0168] <Supplement> Each function of the embodiments described above can be realized by one or more processing circuits. Hereinafter, "processing circuit" as used herein includes processors programmed to execute each function by software, such as processors implemented by electronic circuits, as well as devices such as ASICs (Application Specific Integrated Circuits), DSPs (digital signal processors), FPGAs (field programmable gate arrays), and conventional circuit modules designed to execute each function described above.

[0169] Furthermore, the apparatus described in each embodiment represents only one of several computing environments for carrying out the embodiments disclosed herein. In one embodiment, the server apparatus 100 includes multiple computing devices, such as a server cluster. The multiple computing devices are configured to communicate with each other via any type of communication link, including a network or shared memory, and perform the processing disclosed herein.

[0170] Furthermore, each functional configuration of the server device 100 may be combined into a single server device, or it may be divided among multiple devices. In addition, at least some of the functional configurations of the server device 100 may be provided by the terminal device 10.

[0171] <Note> This specification discloses the following information processing device, dialogue system, dialogue control method, and program. (Note 1) An input unit that receives input from the user, Based on the input, an evaluation unit evaluates the user's willingness to continue the conversation, A response unit outputs a response regarding the termination of the dialogue based on the evaluation results of the evaluation unit, An information processing device having (Note 2) The information processing device described in Appendix 1, wherein the response regarding the termination of the aforementioned dialogue is a response that proposes to the user that the dialogue be terminated. (Note 3) The information processing apparatus according to Appendix 1 or Appendix 2, wherein the response unit outputs a response regarding the termination of the dialogue based on the evaluation result and a threshold that changes as the dialogue progresses. (Note 4) The information processing device described in Appendix 3, wherein the threshold value changes based on the duration of the same topic or the number of utterances on the same topic. (Note 5) The evaluation unit described above, The content of the aforementioned dialogue was input into a large-scale language model to evaluate the user's first willingness to continue the dialogue, and the first evaluation result was obtained from this input. A second evaluation result that assesses the user's second willingness to continue the conversation based on the number of characters spoken by the user, An information processing device according to any one of the appendices 1 to 4, which evaluates the user's willingness to continue the dialogue based on the above. (Note 6) A user information acquisition unit that acquires the user's information, A presentation information generation unit generates information related to topics associated with the user's information based on the user's information, When the aforementioned dialogue is initiated, an information presentation unit presents the user with information related to the aforementioned topic, Having, An information processing device as described in any of the appendices 1 to 5. (Note 7) The aforementioned topic includes information about facilities related to the user information, as described in Appendix 5 of the information processing device. (Note 8) The information processing apparatus according to Appendix 6 or Appendix 7, comprising an update unit that updates the user's information based on the content of the dialogue after the dialogue with the user has ended. (Note 9) A dialogue system including a terminal device used by a user and an information processing device that can communicate with the terminal device via a communication network, The aforementioned information processing device is An input unit that receives input from the user via the terminal device, Based on the input, an evaluation unit evaluates the user's willingness to continue the conversation, A response unit outputs a response regarding the termination of the dialogue based on the evaluation results of the evaluation unit, It has, The aforementioned terminal device is A transmission unit that transmits the input from the user to the information processing device, A receiving unit that receives the response from the information processing device, An output control unit that controls the output of the response to the user, Having, Dialogue system. (Note 10) Computers Input processing that accepts input from the user, Based on the input, an evaluation process is performed to evaluate the user's willingness to continue the conversation. Based on the evaluation results of the aforementioned evaluation process, a response process outputs a response regarding the termination of the dialogue. An interactive control method for executing [this]. (Note 11) On the computer, Input processing that accepts input from the user, Based on the input, an evaluation process is performed to evaluate the user's willingness to continue the conversation. Based on the evaluation results of the aforementioned evaluation process, a response process outputs a response regarding the termination of the dialogue. A program that executes something.

[0172] Although embodiments of the present invention have been described above, the present invention is not limited to these specific embodiments, and various modifications and applications are possible within the scope of the gist of the present invention as described in the claims. [Explanation of Symbols]

[0173] 1. Dialogue System 10 Terminal devices 100 Server equipment (information processing equipment) 200 Computers 401 User Information Acquisition Unit 402 Presentation information generation unit 403 Information Presentation Department 404 Input Section 407 Output section 408 Evaluation Department 409 Response Control Unit (Example of a Response Unit) 410 Update Department 422 Transmitter 423 Receiving Unit 424 Output Control Unit [Prior art documents] [Patent Documents]

[0174] [Patent Document 1] Japanese Patent Publication No. 2018-91979 [Patent Document 2] International Publication No. 2021 / 064948 [Patent Document 3] Japanese Patent Publication No. 2017-010207 [Patent Document 4] Japanese Patent Publication No. 2021-162928 [Non-patent literature]

[0175] [Non-Patent Document 1] Ayami Yokoyama, et al., "Behavioral Strategies of Human Interaction Robots Based on Estimation of Others' States," 28th Annual Meeting of the Japanese Cognitive Science Society, 2011.<URL: https: / / www.jcss.gr.jp / meetings / JCSS2011 / proceedings / pdf / JCSS2011_P2-17.pdf>

Claims

1. An input unit that receives input from the user, Based on the input, an evaluation unit evaluates the user's willingness to continue the conversation, A response unit outputs a response regarding the termination of the dialogue based on the evaluation results of the evaluation unit, An information processing device having

2. The information processing apparatus according to claim 1, wherein the response regarding the termination of the dialogue is a response in which the user is instructed to terminate the dialogue.

3. The information processing apparatus according to claim 1, wherein the response unit outputs a response regarding the termination of the dialogue based on the evaluation result and a threshold that changes as the dialogue progresses.

4. The information processing apparatus according to claim 3, wherein the threshold value changes based on the duration of the dialogue, the number of utterances in the dialogue, or the duration of the same topic, or the number of utterances in the same topic.

5. The evaluation unit described above, The content of the aforementioned dialogue was input into a large-scale language model to evaluate the user's first willingness to continue the dialogue, and the first evaluation result was obtained from this input. A second evaluation result that assesses the user's second willingness to continue the conversation based on the number of characters spoken by the user, An information processing apparatus according to any one of claims 1 to 4, which evaluates the user's willingness to continue the dialogue based on the above.

6. A user information acquisition unit that acquires the user's information, A presentation information generation unit generates information related to topics associated with the user's information based on the user's information, When the aforementioned dialogue is initiated, an information presentation unit presents the user with information related to the aforementioned topic, Having, The information processing apparatus according to any one of claims 1 to 4.

7. The information processing apparatus according to claim 6, wherein the aforementioned topic includes facility information related to the user information.

8. The information processing apparatus according to claim 6, further comprising an update unit that updates the user's information based on the content of the dialogue after the dialogue with the user has ended.

9. A dialogue system including a terminal device used by a user and an information processing device that can communicate with the terminal device via a communication network, The aforementioned information processing device is An input unit that receives input from the user via the terminal device, Based on the input, an evaluation unit evaluates the user's willingness to continue the conversation, A response unit outputs a response regarding the termination of the dialogue based on the evaluation results of the evaluation unit, It has, The aforementioned terminal device is A transmission unit that transmits the input from the user to the information processing device, A receiving unit that receives the response from the information processing device, An output control unit that controls the output of the response to the user, Having, Dialogue system.

10. Computers Input processing that accepts input from the user, Based on the input, an evaluation process is performed to evaluate the user's willingness to continue the conversation. Based on the evaluation results of the aforementioned evaluation process, a response process outputs a response regarding the termination of the dialogue. An interactive control method for executing [this].

11. On the computer, Input processing that accepts input from the user, Based on the input, an evaluation process is performed to evaluate the user's willingness to continue the conversation. Based on the evaluation results of the aforementioned evaluation process, a response process outputs a response regarding the termination of the dialogue. A program that executes something.

Images