Control device, control program, and control method

Abstract

To provide a control device, a control program, and a control method for appropriately guiding a user by using an avatar.SOLUTION: In an interaction control device that provides a chat service for answering a question of a user who uses a user terminal, a chat screen (150) including an avatar image (152) is displayed on a web screen (100) displayed on a display device of the user terminal, and when a predetermined event occurs in the user terminal, an avatar executes at least one of an action and a speech for prompting an operation of the user according to the predetermined event.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present invention relates to a control device, a control program, and a control method, and more particularly to a control device, a control program, and a control method for controlling a chat in which answers are given to questions from users, for example. [Background technology]

[0002] An example of this type of background art is disclosed in Patent Document 1. In the output device disclosed in Patent Document 1, when a selection or input is made in an input form, a user terminal transmits to the output device an identifier of the input form from which the selection or input was made. When the output device receives the identifier of the input form, it generates options related to the input of the input form from which the user made the selection or input and transmits the generated options to the user terminal. When the user terminal receives options related to the input of the input form, it displays the received options in a dialog box, and when one of the displayed options is selected, it transmits an identifier of the selected option to the output device. When the output device receives the identifier of the option, it generates an answer to the selected option. Furthermore, on the user terminal, along with displaying the dialog box and outputting the answer, a chatbot avatar is displayed to respond to the user. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2022-160217 Summary of the Invention [Problem to be solved by the invention]

[0004] According to the above background art, the chatbot avatar is displayed, but since the avatar is merely a static image, it is merely decorative. Therefore, there is room for improvement.

[0005] Therefore, a primary object of the present invention is to provide a novel control device, control program, and control method.

[0006] Another object of the present invention is to provide a control device, a control program, and a control method that can appropriately guide a user using an avatar. [Means for solving the problem]

[0007] A first invention is a control device that controls a chat that answers users' questions, and includes a question receiving means that receives questions sent from a user terminal used by the user, a response sending means that generates a response to the question received by the question receiving means and sends it to the user terminal, an event occurrence determination means that determines whether a predetermined event has occurred on the user terminal during the chat, and an execution means that, when the event occurrence determination means determines that a predetermined event has occurred, causes an avatar to perform at least one of an action and a speech that prompts the user to perform an operation related to the predetermined event.

[0008] A second invention is dependent on the first invention, and the predetermined event is a situation in which the user should perform an operation or a situation in which it is better for the user to perform an operation.

[0009] A third invention is according to the first or second invention, and the execution means causes the avatar to perform at least one of an action and an utterance that prompts a user operation, and then causes the avatar to further perform at least one of an action and an utterance in response to the user's reaction.

[0010] A fourth invention is a control program executed by a control device that controls a chat in which answers are given to user questions, the control program causing a processor of the control device to execute a question receiving step that causes the processor to receive questions sent from a user terminal used by the user, an answer sending step that causes the processor to generate an answer to the question received in the question receiving step and send it to the user terminal, an event occurrence determination step that causes the processor to determine whether a predetermined event has occurred in the user terminal during the chat, and an execution step that causes an avatar to perform at least one of an action and a speech that prompts the user to perform an operation related to the predetermined event if it is determined in the event occurrence determination step that a predetermined event has occurred.

[0011] A fifth invention is a control method for a control device that controls a chat that answers a user's questions, in which a processor of the control device receives a question sent from a user terminal used by the user, generates an answer to the received question, sends it to the user terminal, determines whether a specified event has occurred on the user terminal during the chat, and, if it determines that the specified event has occurred, causes an avatar to perform at least one of an action and a speech that prompts the user to perform an operation related to the specified event. [Effects of the Invention]

[0012] According to the present invention, it is possible to appropriately guide a user using an avatar.

[0013] The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description of the preferred embodiments with reference to the drawings. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 is a diagram showing a dialogue control system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram showing an example of the electrical configuration of the dialogue control device shown in FIG. [Figure 3]FIG. 3 is a block diagram showing an example of the electrical configuration of the user terminal shown in FIG. [Figure 4] Figure 4(A) is a diagram showing a first example of a web screen displayed on the display device of the user terminal shown in Figure 1, and Figure 4(B) is a diagram showing a second example of a web screen displayed on the display device of the user terminal shown in Figure 1. [Figure 5] FIG. 5 is a diagram showing a first example of a chat screen displayed on the display device of the user terminal shown in FIG. [Figure 6] FIG. 6 is a diagram showing a second example of the chat screen displayed on the display device of the user terminal shown in FIG. [Figure 7] Figure 7(A) is a diagram showing a third example of a web screen displayed on the display device of the user terminal shown in Figure 1, and Figure 7(B) is a diagram showing a third example of a chat screen displayed on the display device of the user terminal shown in Figure 1. [Figure 8] FIG. 8 is a diagram showing a fourth example of the chat screen displayed on the display device of the user terminal shown in FIG. [Figure 9] FIG. 9 is a diagram showing an example of a memory map of the RAM of the dialogue control device shown in FIG. [Figure 10] FIG. 10 is a diagram showing an example of a memory map of the RAM of the user terminal shown in FIG. [Figure 11] FIG. 11 is a flowchart showing a first part of an example of the dialogue control process of the CPU of the dialogue control device shown in FIG. [Figure 12] FIG. 12 is a flowchart showing a second part of an example of the dialogue control process of the CPU of the dialogue control device shown in FIG. 2, and follows FIG. [Figure 13] FIG. 13 is a flowchart showing a third part of an example of the dialogue control process of the CPU of the dialogue control device shown in FIG. 2, and follows FIG. [Figure 14] FIG. 14 is a flowchart showing a first part of an example of chat processing by the CPU of the user terminal shown in FIG. [Figure 15] FIG. 15 is a flowchart showing a second part of an example of the chat process by the CPU of the user terminal shown in FIG. 3, and follows FIG. [Figure 16] FIG. 16 is a flowchart showing a third part of an example of the chat process by the CPU of the user terminal shown in FIG. 3, and follows FIG. [Figure 17] FIG. 17 is a flowchart showing a fourth part of an example of the chat process by the CPU of the user terminal shown in FIG. 3, and follows FIG. [Figure 18] FIG. 18 is a flowchart showing a part of the dialogue control process of the CPU of the dialogue control device of another embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0015] Referring to FIG. 1, a dialogue control system 10 of the embodiment includes a dialogue control device 12, which is communicatively connected to a user terminal 16 via a network 14.

[0016] In this embodiment, one dialogue control device 12 and one user terminal 16 are shown, but in reality, multiple dialogue control devices 12 and multiple user terminals 16 may be connected to the network 14, and a connection is established between the dialogue control device 12 that provides a chat service on a website accessed by the user terminal 16 and the user terminal 16. Although not shown in the figure, a server that operates the website is also connected to the network 14. However, the dialogue control device 12 may operate the website.

[0017] The dialogue control device 12 is an information processing device, such as a PC or a server. The dialogue control device 12 provides a chat service and executes an automatic dialogue program 302f (i.e., a chatbot) that utilizes artificial intelligence and responds in text to utterances (or statements) entered by a user, i.e., questions (see FIG. 9). However, responses to questions can also be output as voice.

[0018] The network 14 is composed of an IP network (or IP network) including the Internet, and an access network (or access network) for accessing this IP network. The access network may be a public telephone network, a mobile phone network, a wired LAN, a wireless LAN, or a CATV (Cable Television) network.

[0019] The user terminal 16 is an information processing device, and examples thereof include a notebook PC, a desktop PC, a smartphone, and a tablet PC. The user terminal 16 is owned and used by a user. However, the user terminal 16 may also be a terminal installed in a store to which the dialogue control system 10 is applied and used by the user.

[0020] In this embodiment, a chat screen 150 (see FIG. 5 ) including an image (hereinafter, “avatar image”) 152 corresponding to the avatar of the chatbot is displayed on the display device 30 of the user terminal 16, and the user interacts with the chatbot via the chat screen 150. In other words, the user uses a chat service or an interaction service provided by the interaction control device 12.

[0021] Fig. 2 is a block diagram showing an example of the electrical configuration of the dialogue control device 12 shown in Fig. 1. As shown in Fig. 2, the dialogue control device 12 includes a CPU 20, which is connected to a RAM 22, a communication interface (hereinafter referred to as "communication I / F") 24, and an input / output interface (hereinafter referred to as "input / output I / F") 26 via an internal bus.

[0022] The CPU 20 is responsible for overall control of the dialogue control device 12. However, instead of the CPU 20, a SoC (System-on-a-chip) including multiple functions such as a CPU function and a GPU (Graphics Processing Unit) function may be provided.

[0023] The RAM 22 is a main storage device and is used as a work area or buffer area for the CPU 20. Although not shown, the dialogue control device 12 is provided with a HDD and a ROM as auxiliary storage devices. However, a non-volatile memory such as an SSD may be used instead of or in addition to the HDD.

[0024] The communication I / F 24 is a wired interface for transmitting and receiving control signals and data to and from an external computer such as a user terminal 16 via the network 14 under the control of the CPU 20. However, the communication I / F 24 may also be a wireless interface for connecting to a wireless LAN.

[0025] An input device 28 and a display device 30 are connected to the input / output I / F 26. The input device 28 is a keyboard, a computer mouse, and a microphone. A touch panel may also be provided. The display device 30 is, for example, a liquid crystal display device.

[0026] Furthermore, the input / output I / F 26 outputs operation data (or operation information) input from the input device 28 to the CPU 20, and also outputs image data generated by the CPU 20 to the display device 30, causing a screen or image corresponding to the image data to be displayed on the display device 30. However, the operation data may also be audio data from a microphone.

[0027] The electrical configuration of the dialogue control device 12 shown in FIG. 2 is an example and need not be limited to this.

[0028] Fig. 3 is a block diagram showing the electrical configuration of the user terminal 16 shown in Fig. 1. As shown in Fig. 3, the user terminal 16 includes a CPU 50, which is connected to a RAM 52, a communication I / F 54, and an input / output I / F 56 via an internal bus.

[0029] The CPU 50 is responsible for overall control of the user terminal 16. However, instead of the CPU 50, an SoC including multiple functions such as a CPU function and a GPU function may be provided.

[0030] The RAM 52 is a main storage device and is used as a work area or buffer area for the CPU 50. Although not shown in the figure, the user terminal 16 is provided with a HDD and a ROM as auxiliary storage devices. However, a non-volatile memory such as an SSD may be used instead of or in addition to the HDD.

[0031] The communication I / F 54 is a wired interface for transmitting and receiving control signals and data via the network 14 to and from external computers such as a server that operates a website and the dialogue control device 12 under the control of the CPU 50. However, the communication I / F 54 can also be a wireless interface for connecting to a wireless LAN.

[0032] An input device 58 and a display device 60 are connected to the input / output I / F 56 .

[0033] The input device 58 is a keyboard, a computer mouse, and a microphone. A touch panel may also be provided. However, when a smartphone or a tablet PC is used as the user terminal 16, the input device 58 is a touch panel, hardware buttons, and a microphone. The display device 60 is, for example, a liquid crystal display device.

[0034] Furthermore, the input / output I / F 56 outputs operation data (or operation information) input from the input device 58 to the CPU 50, and also outputs image data generated by the CPU 50 to the display device 60, causing the display device 60 to display a screen or image corresponding to the image data. However, image data received from an external computer (for example, a website server and the dialogue control device 12) may also be output by the CPU 50. The image data received from the dialogue control device 12 is image data of a chat screen. The image received from the website server is image data of a web screen. The operation data may also be audio data from a microphone.

[0035] The electrical configuration of the user terminal 16 shown in FIG. 3 is an example and need not be limited to this.

[0036] Furthermore, if the user terminal 16 is a smartphone, it is equipped with a call circuit for making calls via a mobile phone network, or via a mobile phone network and a public telephone network; however, in this embodiment, such calls are not made, and therefore are not shown.

[0037] The operation of the dialogue control system 10 of this embodiment will be described below. In this dialogue control system 10, when a user accesses a website such as a general company site or an electronic commerce site (hereinafter referred to as an "EC site") on the user terminal 16, the user terminal 16 performs (or starts) chat processing when a chat request is made or when it is detected that no operation has been performed for more than a first predetermined time (for example, 60 seconds).

[0038] FIG. 4(A) shows an example of a web screen 100 displayed on the display device 60 of the user terminal 16, and FIG. 4(B) shows another example of a web screen 100 displayed on the display device 60 of the user terminal 16.

[0039] As shown in Figure 4(A), web screen 100 is an example of a web page of an e-commerce site for buying and selling products, displaying photos of multiple products with information and / or descriptions of each product displayed below the product. However, since web screen 100 itself is not essential to the present invention, in this specification, text strings such as "Product A," "Product B," "Product C," and "Product D" are displayed instead of product photos, and information and / or descriptions of each product are indicated by multiple dots. The same applies to Figure 4(B).

[0040] The user can refer to the web screen 100 to select and purchase a desired product and quantity. However, the purchase is not necessarily required. Since it is already well known that a product and quantity can be selected and a purchase process can be performed on the web screen 100, a detailed description thereof will be omitted.

[0041] The web screen 100 also displays a button image 102. The button image 102 is an icon for using a chat service provided by the dialogue control device 12. When the button image 102 is turned on, a connection state between the user terminal 16 and the dialogue control device 12 is established, and as shown in FIG. 4(B), the display device 60 of the user terminal 16 displays the web screen 100 with a chat window (hereinafter referred to as "chat screen") 150 displayed in the foreground. However, the chat screen 150 may be displayed instead of the web screen 100.

[0042] The operation of the chat window is controlled using JavaScript ("JAVASCRIPT" is a registered trademark) that runs on the web browser of the user terminal 16. The operation of the chat window includes operations such as displaying a question entered by the user on the chat screen 150, displaying a response from the dialogue control device 12 on the chat screen 150, and executing processing in response to an operation on a link or button image displayed on the chat screen 150.

[0043] However, as described above, even if the button image 102 is not turned on, chat processing can also be started if the time without any user operation continues for a first predetermined time or longer while the web screen 100 is displayed, because there is a possibility that the user may not understand the operation on the web screen 100.

[0044] Fig. 5 is a diagram showing an example of the specific content of chat screen 150 shown in Fig. 4(B). As shown in Fig. 5, chat screen 150 includes two display areas 150a and 150b arranged vertically. Avatar image 152, button image 154, and button image 156 are displayed in display area 150a, and text display frame 160, text display frame 162, and button image 164 are displayed in display area 150b.

[0045] As described above, the avatar image 152 is an image of the chatbot's avatar. The avatar is an alter ego of the person who is chatting with the user (in this embodiment, the chatbot or the operator of the dialogue control device 12). The button image 154 is an icon for calling a human responder, i.e., the operator of the dialogue control device 12. The button image 156 is an icon for closing the chat screen 150, i.e., for ending the chat.

[0046] The text display frame 160 is an area or region for displaying the text of the avatar's answer and the text of the avatar's message. The text display frame 162 is an area or region for displaying the text of the user's question. The button image 164 is an icon for the user to send a question to the chatbot, i.e., the dialogue control device 12.

[0047] When the chat screen 150 is first displayed, a message such as "Please enter your question" is displayed in the text display frame 160. When the user enters a question, the content is displayed in the text display frame 162, and when the user turns on the button image 164, the entered question is sent to the dialogue control device 12. In response, the dialogue control device 12 generates a response and sends it to the user terminal 16. The text of the response from the dialogue control device 12 is displayed in the text display frame 160. If the user has any further questions, the above process is repeated.

[0048] The questions are drawn in chronological order, and when the latest question is input, part of the question text, including the text of the latest question, is displayed in the text display frame 162, while the remaining part is scrolled and hidden from the text display frame 162. Similarly, the answers are drawn in chronological order, and when the text of the latest answer is displayed, part of the answer text, including the text of the latest answer, is displayed in the text display frame 162, while the remaining part is scrolled and hidden from the text display frame 162.

[0049] On the other hand, the dialogue control device 12 receives a question sent from the user terminal 16, selects an answer to the question using artificial intelligence, and sends the selected answer to the user terminal 16 (that is, returns it).

[0050] In this embodiment, when a predetermined event occurs, the avatar performs actions and speaks to guide the user so that the user can perform an operation corresponding to the predetermined event. The predetermined event is a situation in which the user should or would prefer to operate. Specifically, a situation in which the user should operate corresponds to the situation in which the user has not input a question for a second predetermined time (e.g., 30 seconds) or more, or a situation in which a dialog box is displayed asking the user whether or not to allow access, and a situation in which the user would prefer to operate corresponds to the situation in which a link is displayed in the answer.

[0051] If the user does not input a question for a second predetermined time or longer, the avatar makes an action and speaks to guide (or prompt) the user to input (or operate) a question. Figure 6 shows an example of chat screen 150 when the avatar makes an action and speaks to guide (or prompt) the user to input a question.

[0052] As shown in FIG. 6, the avatar performs a waving motion with its right hand, and the text of a message such as "If you have anything to ask, please enter it" is displayed in the text display frame 160. However, instead of the waving motion with its right hand, the avatar may perform a pointing motion at the text display frame 162. That is, on the chat screen 150 shown in FIG. 6, the avatar performs a pointing motion toward the bottom of the screen. Also, in this embodiment, the avatar's speech is performed by displaying the speech content, i.e., the text of the message, but it may also be output by the avatar's voice. The same applies to the avatar's speech below.

[0053] The dialogue control device 12 detects that the user has not input a question for a second predetermined time or longer when a state in which no key operation is performed on the user terminal 16 continues for a second predetermined time or longer. Specifically, when a key operation to input a question is performed, the user terminal 16 notifies the dialogue control device 12 that a question has been input. The dialogue control device 12 detects (or determines) that the user has not input a question for a second predetermined time or longer when the dialogue control device 12 does not receive a notification that a question has been input even once for a second predetermined time or longer after the chat started, when the dialogue control device 12 does not receive a notification that a question has been input for a second predetermined time or longer after sending an answer to the user terminal 16, or when the dialogue control device 12 once received a notification that a question has been input but then did not receive a notification that a question has been input for a second predetermined time or longer in a state in which no question has been received.

[0054] The process of detecting whether a key operation has been performed, i.e., whether a key is being input, is also controlled using JavaScript ("JAVASCRIPT" is a registered trademark) that runs on the web browser of the user terminal 16. In this embodiment, JavaScript code for detecting whether keyboard input is being performed is embedded in the web page.

[0055] When a dialog box is displayed requesting the user to grant or deny access, the avatar makes movements and speaks to guide (or prompt) the user to grant or deny access. Fig. 7(A) is a diagram showing an example of web screen 100 on which, in addition to chat screen 150, a dialog box 200 requesting permission to use the microphone is displayed, and Fig. 7(B) is a diagram showing an example of chat screen 150 when the avatar makes movements and speaks to guide the user to grant or deny access.

[0056] As shown in FIG. 7(B), the avatar performs an action of pointing in the direction in which the dialog is displayed, and the text of a message such as "Please allow the use of XXX" is displayed in the text display frame 160.

[0057] The permission for access refers to the permission for use of any of the microphone, camera, and location information. For example, if a user inputs a question by voice, the use of the microphone must be permitted. Furthermore, if an image of the user is to be transmitted to an external computer (which may be the dialogue control device 12), such as a server that operates the website of the web screen 100, at the request of a service provided by the web screen 100 or a certain application, the use of the camera must be permitted. Furthermore, if a user's location information is to be transmitted to an external computer (which may be the dialogue control device 12), such as a server that operates the website of the web screen 100, at the request of a service provided by the web screen 100 or another application, the use of the GPS function and the transmission of location information must be permitted.

[0058] When confirming whether or not access is permitted, a dialog box requesting whether or not access is permitted is displayed, and the text of the message corresponding to the microphone, camera, or location information for which access is being requested is displayed in text display frame 160. In other words, depending on the type of access being requested, the above XXX will contain the character string for microphone, camera, or location information.

[0059] The display of the dialog and the type of accessibility are detected when the dialog is displayed on the user terminal 16. The process of detecting the display of the dialog is also controlled using JavaScript ("JAVASCRIPT" is a registered trademark) that runs on the web browser of the user terminal 16. In this embodiment, JavaScript code for detecting the display of the dialog and the type of accessibility is embedded in the web page.

[0060] 7(A) and 7(B), the avatar points in the direction in which the dialog is displayed, and thus the direction of the position in which the dialog is displayed is calculated from the position in which avatar image 152 is displayed. As an example, the position in which avatar image 152 is displayed is the center of the avatar (the center of the abdomen or chest) on web screen 100, and the position in which the dialog is displayed is the center of the dialog on web screen 100.

[0061] However, since it is not necessary to strictly indicate the direction in which the dialog is displayed, the avatar points upward, downward, left, right, diagonally upward to the left, diagonally upward to the right, diagonally downward to the left, or diagonally downward to the right. The diagonal angle is 45 degrees or 60 degrees with respect to the horizontal direction (or left-right direction) of the chat screen 150.

[0062] When the user indicates permission by turning on the permission button image, the microphone is turned on, and although not shown, the avatar performs an action in response to the user's reaction. In this example, the avatar performs an action of gratitude for the permission and speaks a message of gratitude. For example, the gratitude action is a deep bow. The message of gratitude is, "Thank you for allowing me to use the microphone." The same applies when permission is given to use the camera or location information.

[0063] When a link is displayed in the reply, the avatar makes an action and speaks to guide (or urge) the user to access the link. Figure 8 shows an example of chat screen 150 when making an action and speaking to guide the user to access the link.

[0064] As shown in Fig. 8, the avatar performs a bowing motion, and a message text such as "Please open the link" is displayed below the link in text display frame 160. However, instead of the bowing motion, the avatar may perform a pointing motion toward text display frame 160. That is, on chat screen 150 as shown in Fig. 8, the avatar performs a pointing motion toward the bottom of the screen.

[0065] The display of a link in an answer is detected when the answer displayed in the text display frame 160 of the user terminal 16 includes a link.

[0066] When a user clicks to open a link, the linked web page opens in a window separate from the web screen 100, and although not shown, the avatar performs an action in response to the user's reaction to the click. For example, the avatar's reaction action is to smile and wave its right hand.

[0067] In this embodiment, after image data of the initial chat screen 150 is sent from the dialogue control device 12 at the start of the chat, the user terminal 16 generates or updates the image data of the chat screen 150 and displays the generated or updated image data on the display device 60, but it is also possible to generate or update the image data of the chat screen 150 on the dialogue control device 12 side and send it to the user terminal 16.

[0068] In this way, when interacting with the chatbot, i.e., during a chat, avatar image 152 is displayed on chat screen 150, and further, when a predetermined event occurs during the chat, such as the user not inputting a question for a second predetermined time or more, a dialog box prompting the user for access permission is displayed, or a link is displayed in a response, the avatar is made to perform an action and speak in accordance with the predetermined event that has occurred, so that the avatar can guide the user's operations. This allows the user to feel as if they are interacting with the avatar.

[0069] Fig. 9 shows an example of a memory map 300 of the RAM 22 built into the dialogue control device 12 shown in Fig. 2. As shown in Fig. 9, the RAM 22 includes a program storage area 302 and a data storage area 304. The program storage area 302 stores a dialogue control processing program executed by the dialogue control device 12 of this embodiment.

[0070] The dialogue control processing programs executed by the dialogue control device 12 include a main processing program 302a, an operation detection program 302b, a communication program 302c, an image generation program 302d, an image output program 302e, an automatic dialogue program 302f, and an avatar control program 302g.

[0071] The main processing program 302a is a program for executing the main routine of the dialogue control processing of the dialogue control device 12 of this embodiment.

[0072] The operation detection program 302b is a program for detecting operation data 304a input from the input device 28 in accordance with an operator's operation, and storing the data in the data storage area 304.

[0073] The communication program 302c is a program for communicating (sending and receiving data, etc.) with an external device, in this embodiment, the user terminal 16.

[0074] Image generation program 302d is a program for generating image data corresponding to all or part of a screen or image to be displayed on display device 30, using image generation data 304b.

[0075] The image output program 302e is a program for outputting image data generated according to the image generation program 302d to the display device 30. Therefore, a screen or image corresponding to the image data is displayed on the display device 30.

[0076] The automatic dialogue program 302f is a chatbot, and is a program for generating answer data 304e to question data 304d received from the user terminal 16 based on the talk script data 304c, and transmitting the generated answer data 304e to the user terminal 16. When receiving the question data 304d or transmitting the answer data 304e, the communication program 302c is also executed.

[0077] Avatar control program 302g is a program for determining the speech and action of an avatar corresponding to a predetermined event that has occurred, and transmitting message data 304f and animation data 304g for the avatar that will perform the determined speech and action to user terminal 16. When transmitting message data 304f and animation data 304g, communication program 302c is also executed.

[0078] Although not shown, the program storage area 302 stores the operating system and middleware of the dialogue control device 12, as well as a browser and other application programs other than the dialogue control program of this application.

[0079] The data storage area 304 stores operation data 304a, image generation data 304b, talk script data 304c, question data 304d, answer data 304e, message data 304f, animation data 304g, access information data 304h, etc. The data storage area 304 also includes a link display flag 304i.

[0080] The operation data 304a is data of the operation of the operator detected in accordance with the operation detection program 302b. The operation data 304a is deleted from the data storage area 304 after being used for processing by the CPU 20.

[0081] The image generation data 304b is data for generating a screen or image to be displayed on the display device 30 of the dialogue control device 12.

[0082] The talk script data 304c is data about a talk script for generating answers to user questions. As an example, the talk script is generated by machine learning answers given by a human operator to user questions.

[0083] The question data 304d is text data about a question that is input and sent by the user on the chat screen 150 displayed on the display device 60 of the user terminal 16 and received by the dialogue control device 12.

[0084] The answer data 304e is text data of an answer generated based on the talk script data 304c in response to the question data 304d.

[0085] The message data 304f is message data about the avatar's utterances determined in response to a predetermined event that occurred during the chat and the avatar's utterances determined in response to the operator's reaction. The message data about each utterance of the avatar is stored or registered in advance in an auxiliary storage device such as a HDD or ROM, and the data about the determined avatar's utterances is read from the auxiliary storage device into the RAM 22.

[0086] However, if the specified event that occurs during the chat is the display of a dialog box requesting whether or not access is permitted, message data including the type of access permitted or not contained in the access information data 304h is also read from the auxiliary storage device into RAM 22.

[0087] The animation data 304g is animation data about the avatar's movements determined in response to a predetermined event that occurs during the chat and the avatar's movements determined in response to the operator's reaction. The animation data about each avatar's movement is stored or registered in advance in an auxiliary storage device such as a HDD or ROM, and the data about the determined avatar's movement is read from the auxiliary storage device into the RAM 22.

[0088] However, if the predetermined event occurring during the chat is the display of a dialog box requesting access permission, animation data for a pointing action in the direction indicated by the direction data included in access information data 304h is also loaded from the auxiliary storage device into RAM 22. In this embodiment, the avatar points up, down, left, right, diagonally upward to the left, diagonally upward to the right, diagonally downward to the left, or diagonally downward to the right, and therefore animation data for the pointing action in each direction is prepared. The direction of the display position of dialog 200 as seen from avatar image 152 is classified as up, down, left, right, diagonally upward to the left, diagonally upward to the right, diagonally downward to the left, or diagonally downward to the right, and the avatar's action is determined accordingly. When the display position of dialog 200 is viewed from the position of avatar image 152 as the center, if the direction is between 11 and 1 o'clock, it is classified as upward, if the direction is between 1 and 2 o'clock, it is classified as diagonally upward to the right, if the direction is between 2 and 4 o'clock, it is classified as right, if the direction is between 4 and 5 o'clock, it is classified as diagonally downward to the right, if the direction is between 5 and 7 o'clock, it is classified as downward, if the direction is between 7 and 8 o'clock, it is classified as diagonally downward to the left, if the direction is between 8 and 10 o'clock, it is classified as left, and if the direction is between 10 and 11 o'clock, it is classified as diagonally upward to the left. However, the same numbers are classified into one of the directions.

[0089] The access information data 304h is data regarding the type of access permission and the direction of the display position of the dialog 200 as viewed from the avatar image 152 when a dialog requesting whether or not to allow access to the microphone, camera, or location information is displayed on the user terminal 16 as a specified event.

[0090] The link display flag 304i is a flag that determines whether the answer data 304e contains information for moving to another web page, i.e., a link. If the answer data 304e contains a link, the link display flag 304i is turned on, and if it is notified that the link has been opened or that opening of the link has been refused, the link display flag 304i is turned off.

[0091] Although not shown in the figure, the data storage area 304 stores other data required to execute the dialogue control process, and is provided with a timer (counter) and flags required to execute the dialogue control process.

[0092] Fig. 10 shows an example of a memory map 400 of the RAM 52 built into the user terminal 16. As shown in Fig. 10, the RAM 52 includes a program storage area 402 and a data storage area 404. The program storage area 402 stores an information processing program executed by the user terminal 16 of this embodiment.

[0093] The information processing programs executed on the user terminal 16 include a main processing program 402a, an operation detection program 402b, a communication program 402c, an image generation program 402d, an image output program 402e, a browser program 402f, and an access information generation program 402g.

[0094] The main processing program 402a is a program for executing the main routine of information processing of the user terminal 16 of this embodiment. However, the information processing includes chat processing.

[0095] The operation detection program 402b is a program for detecting operation data 404a input from the input device 58 in accordance with a user's operation and storing the data in the data storage area 404.

[0096] The communication program 402c is a program for communicating with external devices, in this embodiment, the website server and the dialogue control device 12.

[0097] Image generation program 402d is a program for using image generation data 404b to generate image data corresponding to a screen or image to be displayed on display device 60. However, when generating image data corresponding to chat screen 150 such as those shown in Figures 4(B), 5, 6, 7(B), and 8, any one or more of question data 404c, answer data 404d, message data 404e, and animation data 404f are used as necessary.

[0098] The image output program 402e is a program for outputting image data generated in accordance with the image generation program 402d to the display device 60. Therefore, a screen or image corresponding to the image data is displayed on the display device 60.

[0099] Browser program 402f is a program for executing processes related to displaying a web page, i.e., web screen 100, scrolling web screen 100, moving web pages, and processes corresponding to operations on web screen 100. Browser program 402f is also a program for executing chat processes, i.e., processes for displaying and hiding a chat window, i.e., chat screen 150, and processes corresponding to the operation of the chat window.

[0100] Access information generation program 402g is a program for generating data regarding the type of accessibility and the direction of the display position of dialog 200 as viewed from avatar image 152 when checking whether or not access to a microphone, camera, or location information is possible.

[0101] Although not shown, the program storage area 402 stores the operating system and middleware of the user terminal 16, as well as other application programs other than the information processing program of the present application.

[0102] Data storage area 404 stores operation data 404a, image generation data 404b, question data 404c, answer data 404d, message data 404e, animation data 404f, access information data 404g, etc. Data storage area 404 also stores an accessibility flag 404h and a link display flag 404i.

[0103] The operation data 404a is data of a user's operation detected in accordance with the operation detection program 402b. The operation data 404a is deleted from the data storage area 404 after being used for processing by the CPU 50.

[0104] Image generation data 404b is data used to generate a screen or image to be displayed on display device 60 of user terminal 16.

[0105] The question data 404c is text data of a question entered by a user on the chat screen 150, and is stored in chronological order.

[0106] The response data 404d is text data about responses generated and transmitted by the dialogue control device 12 and received by the user terminal 16, and is stored in chronological order.

[0107] The message data 404e is text data of a message that guides the user in response to a predetermined event that has occurred or a message that corresponds to the user's reaction to the guidance, transmitted from the dialogue control device 12. The message data 404e is deleted from the data storage area 404 when the predetermined event that has occurred disappears or when the user operates in accordance with the guidance.

[0108] The animation data 404f is animation data about an action that guides the user in response to a predetermined event that is occurring or an action that corresponds to the user's reaction to the guidance, transmitted from the dialogue control device 12. The animation data 404f is deleted from the data storage area 404 when the predetermined event that is occurring disappears or when the animation has been played once.

[0109] Access information data 404g is generated in accordance with access information generating program 402g and is data regarding the type of accessibility and the direction of the display position of dialog 200 as viewed from avatar 152.

[0110] The access permission flag 404h is a flag for determining whether access to the microphone, camera, or location information is permitted. If access to the microphone, camera, or location information is permitted, the access permission flag 404h is turned on. If access to the microphone, camera, or location information is denied, the access permission flag 404h is turned off.

[0111] The link display flag 404i is a flag for determining whether or not an instruction to open a link has been given. If an instruction to open a link has not been given, the link display flag 404i is turned on. If an instruction to open a link has been given, the link display flag 404i is turned off.

[0112] Although not shown, the data storage area 404 stores other data required to execute information processing, and is provided with a timer (counter) and flags required to execute information processing.

[0113] Fig. 11 is a flow diagram showing an example of the dialogue control process of the CPU 20 of the dialogue control device 12 shown in Fig. 2. When an instruction to start a chat is given on the user terminal 16, the dialogue control device 12 is notified of the execution of the chat, and the CPU 20 starts the dialogue control process.

[0114] As shown in FIG. 11, when the CPU 20 starts the dialogue control process, in step S1, it establishes a connection with the user terminal 16 that has notified it of the intention to start a chat, and in step S3, it determines whether question data 304d has been received from the user terminal 16.

[0115] If step S3 is "NO", that is, if the question data 304d has not been received from the user terminal 16, the process proceeds to step S9. On the other hand, if step S3 is "YES", that is, if the question data 304d has been received from the user terminal 16, then in step S5, answer data 304e is generated using the talk script data 304c, and in step S7, the answer data 304e generated in step S5 is sent to the user terminal 16, and the process proceeds to step S9.

[0116] In step S9, it is determined whether or not a user operation is required. In this step S9, CPU 20 determines whether or not the user has not input a question for a second predetermined time (for example, 30 seconds) or longer. The details are as described above.

[0117] If step S9 is "NO," that is, if user operation is not required, the process proceeds to step S17 shown in Fig. 12. On the other hand, if step S9 is "YES," that is, if user operation is required, then in step S11, message data 304f of a question operation request (or input request) is sent to the user terminal 16. Here, when an event that requires a question to be input occurs, the CPU 20 determines an utterance for the avatar in accordance with this event, reads message data 304f of the determined utterance from the auxiliary storage device, and sends it to the user terminal 16. The same applies hereinafter to the case where message data 304f in accordance with an event is sent to the user terminal 16.

[0118] In the next step S13, animation data 304g of the action requesting the operation of the question is transmitted to the user terminal 16. Here, when an event requiring input of a question occurs, the CPU 20 determines an action of the avatar in accordance with this event, reads out animation data 304g of the determined action from the auxiliary storage device, and transmits it to the user terminal 16. The same applies to the subsequent cases where animation data 304g in accordance with an event is transmitted to the user terminal 16.

[0119] Then, in step S15, it is determined whether the chat has ended. Here, the CPU 20 determines whether a notification of the chat end has been received from the user terminal 16. If the answer is "NO" in step S15, that is, if the chat has not ended, the process returns to step S3. On the other hand, if the answer is "YES" in step S15, that is, if the chat has ended, the dialogue control process ends.

[0120] 12, in step S17, it is determined whether or not access information has been received. In this step, the CPU 20 determines whether or not access information data 304h has been received from the user terminal 16.

[0121] If step S17 is "NO", that is, if access information has not been received, the process proceeds to step S35 shown in Fig. 13. On the other hand, if step S17 is "YES", that is, if access information has been received, then in step S19, message data 304f requesting whether or not access is permitted is transmitted to the user terminal 16. In the next step S21, animation data 304g of the operation requesting whether or not access is permitted is transmitted to the user terminal 16.

[0122] Next, in step S23, it is determined whether permission data has been received. If the determination in step S23 is "YES," that is, if permission data has been received, then in step S25, message data 304f expressing gratitude for the access permission is transmitted to the user terminal 16. Here, the CPU 20 reads the message data 304f expressing gratitude for the access permission from the auxiliary storage device and transmits it to the user terminal 16. The same applies to the subsequent cases where message data 304f corresponding to the user's reaction is transmitted to the user terminal 16.

[0123] In the next step S27, animation data 304g of the user making a gesture of gratitude for the access permission is transmitted to the user terminal 16. Here, the CPU 20 reads out the animation data 304g of the user making a gesture of gratitude for the access permission from the auxiliary storage device and transmits it to the user terminal 16. The same applies to the subsequent cases where animation data 304g corresponding to the user's reaction is transmitted to the user terminal 16.

[0124] If step S23 returns "NO," that is, if permission data has not been received, then in step S29 it is determined whether denial data has been received. If step S29 returns "NO," that is, if denial data has not been received, then the process also returns to step S23. On the other hand, if step S29 returns "YES," that is, if denial data has been received, then in step S31 message data 304f regarding the access denial is sent to the user terminal 16. Here, the CPU 20 reads message data 304f regarding the access denial from the auxiliary storage device and sends it to the user terminal 16.

[0125] Then, in step S33, it is determined whether or not the chat has ended. If "NO" in step S33, the process returns to step S3. On the other hand, if "YES" in step S33, the dialogue control process ends.

[0126] As shown in Fig. 13, in step S35, it is determined whether or not a link is to be displayed. Here, the CPU 20 determines whether or not a link is included in the answer generated in step S5.

[0127] If "NO" in step S35, that is, if it is not a link display, proceed to step S51. On the other hand, if "YES" in step S35, that is, if it is a link display, it is determined in step S37 whether the link display flag 304i is on.

[0128] If "YES" in step S37, that is, if the link display flag 304i is on, a request to open the link has already been made, so the process proceeds to step S45. On the other hand, if "NO" in step S37, that is, if the link display flag 304i is off, the link display flag 304i is turned on in step S39.

[0129] Next, in step S41, message data 304f requesting an operation to open the link is sent to the user terminal 16, and in step S43, animation data 304g of the operation requesting an operation to open the link is sent to the user terminal 16, and the process proceeds to step S45.

[0130] In step S45, it is determined whether a link click notification has been received. If the result in step S45 is "NO," that is, if a link click notification has not been received, the process proceeds to step S51. On the other hand, if the result in step S45 is "YES," that is, if a link click notification has been received, the link display flag 304i is turned off in step S47, and animation data 304g of the reaction action to the link click is transmitted to the user terminal 16 in step S49, and the process proceeds to step S51.

[0131] Then, in step S51, it is determined whether or not the chat has ended. If "NO" in step S51, the process returns to step S3. On the other hand, if "YES" in step S51, the dialogue control process ends.

[0132] 14 to 17 are flow diagrams showing an example of chat processing by the CPU 50 of the user terminal 16 shown in Fig. 3. When the user turns on the button image 102 on the web screen 100 or when the CPU 50 automatically determines that a chat should be started, the CPU 50 starts chat processing.

[0133] It should be noted that since it is already well known that browser functions are executed to perform browser processing, i.e., displaying web screen 100 and performing processing on web screen 100 in response to user operations, flow diagrams and explanations thereof will be omitted.

[0134] 14, when the chat process is started, the CPU 50 establishes a connection with the dialogue control device 12 in step S101, and displays the chat screen 150 in step S103. At the beginning of the chat process, the animation data 404f has not been received, and therefore a still avatar image 152 is displayed on the chat screen 150. However, an avatar image 152 in which the avatar introduces itself or greets the user may also be displayed.

[0135] In the next step S105, it is determined whether a question is to be sent. Here, it is determined whether the button image 164 has been turned on. If "NO" in step S105, that is, if a question is not to be sent, the process proceeds to step S109. On the other hand, if "YES" in step S105, that is, if a question is to be sent, the question data 404c is sent to the dialogue control device 12 in step S107, and the process proceeds to step S109.

[0136] In step S109, it is determined whether a question has been input. Here, the CPU 50 determines whether a key operation has been performed. If "NO" in step S109, that is, if a question has not been input, the process proceeds to step S115 shown in Fig. 15. On the other hand, if "YES" in step S109, that is, if a question has been input, in step S111, the text of the question is displayed in the text display frame 162 in response to the key operation, and in step S113, the dialogue control device 12 is notified that a question has been input, and the process proceeds to step S115.

[0137] In step S115 shown in Fig. 15, it is determined whether or not a dialogue asking whether or not access is permitted is displayed. If "NO" in step S115, that is, if a dialogue asking whether or not access is permitted is not displayed, the process proceeds to step S135 shown in Fig. 16. On the other hand, if "YES" in step S115, that is, if a dialogue asking whether or not access is permitted is displayed, it is determined in step S117 whether or not the access permission flag 404h is on.

[0138] If the result is “NO” in step S117, that is, if accessibility flag 404h is off, accessibility flag 404h is turned on in step S119, access information is generated in step S121, the access information is transmitted to dialogue control device 12 in step S123, and the process proceeds to step S135. In step S121, CPU 50 determines whether the accessibility is being requested from the microphone, the camera, or location information, calculates the direction of the display position of the accessibility dialog as viewed from the display position of avatar image 152, generates access information data 404g, and transmits the generated access information data 404g to dialogue control device 12 in step S123.

[0139] On the other hand, if the answer is "YES" in step S117, that is, if the access permission flag 404h is on, then in step S125 it is determined whether or not an instruction for permission or denial has been given. Here, the CPU 50 determines whether or not a button image for permission or denial (block) provided in the dialog has been turned on.

[0140] If the result in step S125 is "NO," that is, if neither the Allow nor Deny button image provided in the dialog is turned on, the process proceeds to step S135. On the other hand, if the result in step S125 is "YES," that is, if the Allow or Deny button image provided in the dialog is turned on, the access permission flag 404h is turned on in step S127, and it is determined in step S129 whether access permission has been detected. That is, the CPU 50 determines whether the Allow button image in the dialog is turned on.

[0141] If "YES" in step S129, that is, if access permission is detected, permission data is sent to the dialogue control device 12 in step S131, and the process proceeds to step S135. On the other hand, if "NO" in step S129, that is, if access denial is detected, denial data is sent to the dialogue control device 12 in step S133, and the process proceeds to step S135.

[0142] 16, it is determined whether or not a link is displayed. Here, the CPU 50 determines whether or not a link is included in the answer corresponding to the answer data 404d received from the dialogue control device 12.

[0143] If "NO" in step S135, that is, if a link is not displayed, proceed to step S147 shown in Fig. 17. On the other hand, if "YES" in step S135, that is, if a link is displayed, it is determined in step S137 whether the link display flag 404i is on.

[0144] If the determination in step S137 is "NO", that is, if the link display flag 404i is off, the link display flag 404i is turned on in step S139, and the process proceeds to step S147. On the other hand, if the determination in step S137 is "YES", that is, if the link display flag 404i is on, it is determined in step S141 whether or not a click on a link has been detected.

[0145] If step S141 is "NO", that is, if a link click has not been detected, the process proceeds to step S147. On the other hand, if step S141 is "YES", that is, if a link click has been detected, the link display flag 404i is turned off in step S143, and the link click is notified to the dialogue control device 12 in step S145, and the process proceeds to step S147.

[0146] Although not shown in the drawings or described in detail, when a link is clicked, a screen of the linked web page is displayed separately from web screen 100.

[0147] 17, in step S147, it is determined whether or not answer data 404d has been received. If "NO" in step S147, that is, if answer data 404d has not been received, the process proceeds to step S151. On the other hand, if "YES" in step S147, that is, if answer data 404d has been received, the answer is displayed in step S149, and the process proceeds to step S151. In step S149, CPU 50 displays the text of the answer corresponding to answer data 404d in text display frame 160 on chat screen 150.

[0148] In step S151, it is determined whether message data 404e has been received. If "NO" in step S151, that is, if message data 404e has not been received, the process proceeds to step S155. On the other hand, if "YES" in step S151, that is, if message data 404e has been received, the process displays the message in step S153 and proceeds to step S155. In step S153, CPU 50 displays the text of the message corresponding to message data 404e in text display frame 160 on chat screen 150. Therefore, the avatar's speech guides the user's operation in response to a predetermined event that has occurred.

[0149] In step S155, it is determined whether or not animation data 404f has been received. If "NO" in step S155, that is, if animation data 404f has not been received, the process proceeds to step S159. On the other hand, if "YES" in step S155, that is, if animation data 404f has been received, the process plays animation in step S157 and proceeds to step S159. In step S157, CPU 50 starts playing animation data 404f and displays the played animation in display area 150a. Therefore, the movement of the avatar guides the user's operation in response to a predetermined event that is occurring.

[0150] In step S159, it is determined whether the chat has ended. Here, the CPU 50 determines whether the button image 156 has been turned on. If the answer is "NO" in step S159, that is, if the chat has not ended, the process returns to step S105. On the other hand, if the answer is "YES" in step S159, that is, if the chat has ended, in step S161, the dialogue control device 12 is notified of the end of the chat, and the chat process is terminated. In other words, the chat screen 150 displayed in front of the web screen 100 is hidden, and the button image 102 is displayed.

[0151] According to this embodiment, when a predetermined event occurs during a chat, the avatar is made to perform an action and speak in response to the predetermined event, so that the avatar can be used to appropriately guide the user, allowing the user to feel as if they are conversing with the avatar.

[0152] In this embodiment, animation data of an avatar that performs an action in response to a predetermined event is transmitted to the user terminal and played back on the user terminal, but it is also possible to play back the animation data on the dialogue control device, transmit the played back data to the user terminal, and display the played back data on the user terminal.

[0153] In this embodiment, answers to user questions are generated based on talk scripts generated by machine learning, but this is not limited to this. Instead of talk scripts generated by machine learning, anticipated Q&As may be prepared in advance, and answers to user questions may be generated using the Q&As. Also, answers to user questions may be generated using a large-scale language model.

[0154] Furthermore, in this embodiment, the answers to the user's questions are generated by a chatbot, but the answers may also be generated by a human operator. In this case, the avatar can be made to act in accordance with the operator's actions unless a predetermined event occurs, such as the user not inputting a question for a predetermined time or more, a dialog box prompting the user for access permission or disallowance is displayed, or a link is displayed in the answer. If a predetermined event occurs, the avatar can be made to perform an action corresponding to the predetermined event regardless of the operator's actions.

[0155] Specifically, the CPU 20 of the dialogue control device 12 executes a process that is a partial modification of the dialogue control process shown in Figures 11 to 13. The partial modification of the dialogue control process is shown in Figure 18. For convenience of illustration, steps S11 and S13 are omitted from the illustration.

[0156] To put it simply, in a process that is a partial modification of the dialogue control process, steps S2a and S2b are provided between steps S1 and S3, step S5 is replaced by step S5a, and step S7 is replaced by step S7a. Also, if the answer is "NO" in each of steps S15, S33, and S51, the process returns to step S2a.

[0157] In step S2a, the CPU 20 generates motion data of the avatar. The motion data of the avatar is generated based on a captured image of the operator. Therefore, the dialogue control device 12 is provided with a camera for capturing images of the operator. As an example, the motion data can be generated using an image processing library such as MediaPipe Holistic. In another example, the motion data can be generated using the method disclosed in Japanese Patent Application Laid-Open No. 2021-56940.

[0158] As an example, the movement data is information (data) about the position and orientation of each part of the body for controlling the avatar's gestures, facial orientation, and facial expressions. Specifically, the movement data is information about the positions of each part of the face, namely the eyebrows, eyes (upper eyelids, lower eyelids), nose, and mouth (upper lip, lower lip), the positions of the head, hands, arms, shoulders, waist, knees, and ankles, and the orientation of the neck (or face), wrists, and ankles. However, if the avatar's abdomen and below is not displayed, information about the positions and orientations of the waist, knees, and ankles does not need to be included in the movement data.

[0159] Returning to FIG. 18 , in step 2b, the action data generated in step S2a is transmitted to user terminal 16. Although not shown, user terminal 16 receives the action data and controls the action of the avatar. Therefore, an avatar image 152 that moves in accordance with the operator's action is displayed on chat screen 150. If step S3 is “YES,” an answer is generated in step S5a in accordance with the operator's operation. In step S5a, an answer is generated in accordance with the operator's key operation. In the following step S7a, answer data is transmitted to user terminal 16 in accordance with the operator's operation, and the process proceeds to step S9. Note that if there is no operation by the operator, CPU 20 proceeds to step S9 without transmitting the answer data. Other processes are as described in the embodiment. However, except when a predetermined event occurs, a still avatar image 152 may be displayed, as in the above-described embodiment. In addition, operation data is sent to the user terminal 16, and the operation of the avatar is controlled by the user terminal 16. Alternatively, data of an avatar image in which the operation of the avatar is controlled by the dialogue control device 12 may be generated and sent to the user terminal 16, and the avatar image may be displayed on the user terminal 16.

[0160] Furthermore, the order of the steps in the flowchart shown in this embodiment can be changed if the same results are obtained.

[0161] Furthermore, the various screens and specific numerical values ​​shown in this specification and drawings are merely examples and can be changed as needed. [Explanation of symbols]

[0162] 10...Dialogue control system 12...Dialogue control device 14...Network 16...User terminal 18...Large-scale language models 20, 50...CPU 22, 52...RAM 24, 54...Communication I / F 26, 56... Input / output I / F 28, 58...input device 30, 60…display device

Claims

1. A control device for controlling a chat for answering questions from users, a question receiving means for receiving the question transmitted from the user terminal used by the user; an answer sending means for generating an answer to the question received by the question receiving means and sending the answer to the user terminal; an event occurrence determination means for determining whether a predetermined event has occurred on the user terminal during the chat; and A control device comprising: an execution means for causing the avatar to perform at least one of an action and an utterance that prompts a user to perform an operation related to the specified event when the event occurrence determination means determines that the specified event has occurred.

2. The control device according to claim 1 , wherein the predetermined event is a situation in which the user should perform an operation or a situation in which it is better for the user to perform an operation.

3. The control device according to claim 1 or 2, wherein the execution means causes the avatar to further perform at least one of an action and a speech in response to the user's response after the avatar has performed at least one of an action and a speech that prompts the user to operate.

4. A control program executed by a control device that controls a chat that answers questions from users, a processor of the control device; a question receiving step of receiving the question transmitted from the user terminal used by the user; an answer sending step of generating an answer to the question received in the question receiving step and sending the answer to the user terminal; an event occurrence determination step for determining whether a predetermined event has occurred on the user terminal during the chat; and A control program that, when it is determined in the event occurrence determination step that the specified event has occurred, executes an execution step in which the avatar performs at least one of an action and a speech that prompts a user to perform an operation related to the specified event.

5. A control method for a control device that controls a chat that answers questions from users, comprising: The processor of the control device receiving the question transmitted from a user terminal used by the user; generating an answer to the received question and transmitting the answer to the user terminal; determining whether a predetermined event occurs at the user terminal during the chat; and A control method that, when it is determined that the predetermined event has occurred, causes the avatar to perform at least one of an action and an utterance that prompts a user to perform an operation related to the predetermined event.

Images