Information processing device, information processing system, control program, and control method

Abstract

To provide an information processing apparatus, an information processing system, a control program, and a control method that improve security without complicating an authentication method.SOLUTION: An information processing apparatus executes first authentication processing using an operation ID and a password entered by an operator of an operator terminal when the operator receives a predetermined service, and if the first authentication is successful, uses an image 302 of a dialogue agent included in a pattern authentication screen to make a plurality of statements or questions to the operator or an image 304 of an avatar of the operator. In response to this, the operator speaks or controls an action of the avatar image. The information processing apparatus further executes second authentication processing using these operation patterns and a reaction pattern including prosody information generated from a speech of the operator and an emotion of the operator at the time of the speech.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing system, a control program, and a control method, and more particularly to an information processing device, an information processing system, a control program, and a control method for authenticating a user in two stages, for example. [Background technology]

[0002] An example of this type of conventional information processing device is disclosed in Patent Document 1. In the login authentication system disclosed in Patent Document 1, a server determines whether a user ID and login password entered into a user terminal are registered. The server also compares image data captured using a camera on the user terminal with pre-registered facial photo data to obtain a numerical value indicating the possibility that the two are the same person, and determines whether the numerical value indicating the possibility that the two are the same person is equal to or greater than a predetermined value. If both predetermined requirements are met, the server permits login. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2020-201595 Summary of the Invention [Problem to be solved by the invention]

[0004] In the above-mentioned Patent Document 1, authentication is performed using facial photograph data, thereby increasing the security level compared to a configuration that only requires input of an ID and password. However, there is a risk that a third party may be able to successfully authenticate using facial photograph data by, for example, taking a previously obtained photograph of the user's face with a camera. Another method involves inputting the answer to a secret question in addition to inputting an ID and password, but this may be guessed by a third party. For this reason, it is conceivable to input a passcode using a meaningless character string instead of inputting the answer to the secret question, but this is difficult for the user to remember and is inconvenient to use. Therefore, there is room for improvement in authentication methods.

[0005] Therefore, a primary object of the present invention is to provide a novel information processing device, information processing system, control program, and control method.

[0006] Another object of the present invention is to provide an information processing device, an information processing system, a control program, and a control method that can improve security without complicating the authentication method. [Means for solving the problem]

[0007] A first invention is an information processing device that provides a predetermined service on a website, comprising: a first authentication means for authenticating an ID and a password inputted into an operator terminal of an operator who uses the predetermined service; a second authentication means for authenticating a response pattern of the operator to a dialogue agent when the authentication by the first authentication means is successful; and an authorization means for authorizing the operator terminal to start providing the predetermined service when the authentication by the second authentication means is successful; and the second authentication means performs authentication in accordance with a degree of coincidence of a response pattern including prosodic information of the operator's voice when the operator responds to each of the plurality of utterance contents from the dialogue agent to the operator. the predetermined service is a service that uses a robot as an avatar of an operator, and further comprises a UI display means for displaying on the operator terminal a UI screen for operating the robot, and an authentication screen display means for displaying on the operator terminal an authentication screen including an image of a dialogue agent and an avatar drawn by computer graphics in a virtual space when performing authentication processing by the second authentication means, wherein when the operator responds by voice in response to each of a plurality of speech contents, the operator causes the image of the avatar drawn by computer graphics to operate using the same screen as the UI screen, and the response pattern includes the operation content of the operator that causes the image of the avatar to operate; It is an information processing device.

[0008] A second invention is dependent on the first invention, and further comprises a receiving means for receiving a captured image from the operator terminal, and the response pattern includes an emotion of the operator estimated from a facial image of the operator included in the captured image.

[0009] A third invention is according to the first or second invention, and the response pattern includes a speech content of the operator recognized from the operator's voice.

[0011] No. 4 The invention is an information processing system comprising an information processing device that provides a predetermined service on a website and an operator terminal of an operator who uses the predetermined service, wherein the information processing device comprises a first authentication means for authenticating an ID and a password inputted into the operator terminal, a second authentication means for authenticating a response pattern of the operator to a dialogue agent when authentication by the first authentication means is successful, and an authorization means for authorizing the operator terminal to start providing the predetermined service when authentication by the second authentication means is successful, and the second authentication means performs authentication in accordance with the degree of coincidence of a response pattern including prosodic information of the operator's voice when the operator responds to each of the plurality of utterance contents from the dialogue agent to the operator. the predetermined service is a service that uses a robot as an avatar of an operator, and further comprises a UI display means for displaying on the operator terminal a UI screen for operating the robot, and an authentication screen display means for displaying on the operator terminal an authentication screen including an image of a dialogue agent and an avatar drawn by computer graphics in a virtual space when performing authentication processing by the second authentication means, wherein when the operator responds by voice in response to each of a plurality of speech contents, the operator causes the image of the avatar drawn by computer graphics to operate using the same screen as the UI screen, and the response pattern includes the operation content of the operator that causes the image of the avatar to operate; It is an information processing system.

[0012] No. 5 The invention is an information processing program for an information processing device that provides a predetermined service on a website, and causes a processor of the information processing device to execute a first authentication step of authenticating an ID and password entered on an operator terminal of an operator who uses the predetermined service, a second authentication step of authenticating a response pattern of the operator to a dialogue agent if the authentication is successful in the first authentication step, and an authorization step of authorizing the operator terminal to start providing the predetermined service if the authentication is successful in the second authentication step, and the second authentication step performs authentication in accordance with a degree of coincidence of a response pattern including prosodic information of the operator's voice when the operator responds to each of the plurality of utterance contents from the dialogue agent to the operator. the predetermined service is a service that uses a robot as an avatar of an operator, and the processor is further caused to execute a UI display step of displaying, on the operator terminal, a UI screen for operating the robot, and an authentication screen display step of displaying, on the operator terminal, an authentication screen including an image of a dialogue agent and an avatar rendered in computer graphics in a virtual space, when performing authentication processing by the second authentication step, and when the operator responds by voice in response to each of a plurality of utterance contents, the operator causes the image of the avatar rendered in computer graphics to operate using the same screen as the UI screen, and the response pattern includes the operation content of the operator that causes the image of the avatar to operate; It is an information processing program.

[0013] No. 6 The invention is an information processing method of an information processing device that provides a predetermined service on a website, and includes: (a) a step of authenticating an ID and a password entered at an operator terminal of an operator who uses the predetermined service; (b) a step of authenticating a response pattern of the operator to a dialogue agent if the authentication is successful in step (a); and (c) a step of permitting the operator terminal to start providing the predetermined service if the authentication is successful in step (b), wherein step (c) performs authentication in accordance with a degree of coincidence of a response pattern including prosodic information of the operator's voice when the operator responds to each of the plurality of utterance contents from the dialogue agent to the operator. the predetermined service is a service that uses a robot as an avatar of an operator, and further includes (d) a step of displaying on the operator terminal a UI screen that operates the robot, and (e) a step of displaying on the operator terminal an authentication screen that includes an image of a dialogue agent and an avatar drawn by computer graphics in a virtual space when performing the authentication process by step (b), wherein when the operator responds by voice in response to each of the multiple utterance contents, the operator operates the image of the avatar drawn by computer graphics using the same screen as the UI screen, and the response pattern includes the operation content of the operator that operates the image of the avatar; It is an information processing method. [Effects of the Invention]

[0014] According to the present invention, security can be increased without complicating the authentication operation.

[0015] The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description of the preferred embodiments with reference to the drawings. [Brief explanation of the drawings]

[0016] [Figure 1] FIG. 1 is a diagram showing an information processing system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram showing the electrical configuration of the server shown in FIG. [Figure 3] FIG. 3 is a block diagram showing the electrical configuration of the operator terminal shown in FIG. [Figure 4] FIG. 4 is a diagram showing an example of an ID and password registration screen displayed on the operator terminal. [Figure 5] FIG. 5 is a diagram showing an example of a pattern registration screen displayed on the operator terminal. [Figure 6]FIG. 6 is a diagram showing an example of a UI screen displayed on the operator terminal. [Figure 7] FIG. 7 is a diagram showing an example of an ID and password authentication screen displayed on the operator terminal. [Figure 8] FIG. 8 is a diagram showing an example of a pattern authentication screen displayed on the operator terminal. [Figure 9] FIG. 9 is a diagram showing an example of a memory map of the RAM of the server shown in FIG. [Figure 10] FIG. 10 is a diagram showing an example of a memory map of the RAM of the operator terminal shown in FIG. [Figure 11] FIG. 11 is a flowchart showing a part of an example of the registration process of the CPU of the server shown in FIG. [Figure 12] FIG. 12 is a flowchart showing another part of an example of the registration process by the CPU of the server shown in FIG. 2, and follows FIG. [Figure 13] FIG. 13 is a flowchart showing a part of an example of authentication processing by the CPU of the server shown in FIG. [Figure 14] FIG. 14 is a flowchart showing another part of an example of the authentication process by the CPU of the server shown in FIG. 2, and follows FIG. DETAILED DESCRIPTION OF THE INVENTION

[0017] Referring to FIG. 1, an information processing system 10 of the first embodiment includes a server 12, which is communicatively connected to an operator terminal 16 and a robot 18 via a network 14.

[0018] The operator terminal 16 is operated by an operator and is mainly used to control the robot 18. In this embodiment, the operator is a person (service user) who uses a predetermined service using an avatar provided by the server 12. The robot 18 functions as an avatar for the operator.

[0019] If the robot 18 is equipped with a display device, the display device may display an image of the operator's face (or an image of the upper body or the whole body), or an image of a character corresponding to the operator.

[0020] As an example of a predetermined service, a robot 18 placed in a museum, art gallery, aquarium, science museum, zoo, or the like functions as an avatar for a remote operator, and the remote operator tours the museum, art gallery, aquarium, science museum, zoo, or the like through the robot 18. In this case, a guide present at the museum, art gallery, aquarium, science museum, zoo, or the like moves around the museum or zoo with the robot 18. The operator also listens to the guide's explanations and converses with the guide through the robot 18. In other words, the guide is the operator's conversation partner.

[0021] Another example of a predetermined service is to have a robot 18 placed in a movie theater, concert hall, or theater function as an avatar for a remote operator, who watches a movie, concert, live performance, or play at the movie theater, concert hall, or theater through the robot 18. In this case, the robot 18 is fixedly placed in a seat or the like, and there is no conversation partner. However, a guide may be attached to the robot 18 as a conversation partner.

[0022] 1 shows one operator terminal 16 and one robot 18, in reality, a plurality of operator terminals 16 and a plurality of robots 18 are provided. The following description will be given using one operator terminal 16 and one robot 18, but the same content applies to other operator terminals 16 and other robots 18.

[0023] The server 12 is an example of an information processing device, and a general-purpose server may be used. In this embodiment, the server 12 operates a website that provides a predetermined service using an avatar. The server 12 is connected to an operator database (hereinafter referred to as "operator DB") 12a. The operator DB 12a stores information (hereinafter sometimes referred to as "authentication information") for authenticating operators (or service users) who use a predetermined service using an avatar provided by the server 12.

[0024] The operator DB 12a may be provided in a HDD built into the server 12, or may be provided so as to be communicable via the network .

[0025] The network 14 is composed of an IP network (or IP network) including the Internet, and an access network (or access network) for accessing this IP network. The access network may be a public telephone network, a mobile phone network, a wired LAN, a wireless LAN, or a CATV (Cable Television) network.

[0026] The operator terminal 16 is an information processing device other than the server 12, and is, for example, a desktop PC having a browser function. In other embodiments, the operator terminal 16 may be a general-purpose terminal such as a smartphone, a tablet PC, or a notebook PC.

[0027] The robot 18 is an autonomous robot, and commercially available stationary or mobile robots can be used. As an example, a stationary robot such as Sota™ manufactured and sold by Vstone Co., Ltd. can be used. Furthermore, as a mobile robot, Robovie™ developed by the applicant or the Double 3 robot sold by Double Robotics, Inc. can be used.

[0028] Fig. 2 is a block diagram showing the electrical configuration of the server 12 shown in Fig. 1. As shown in Fig. 2, the server 12 includes a CPU 20, which is connected to a RAM 22, a communication interface (hereinafter referred to as "communication I / F") 24, and an input / output interface (hereinafter referred to as "input / output I / F") 26 via an internal bus.

[0029] The CPU 20 is responsible for overall control of the server 12. However, an SoC (System-on-a-chip) including multiple functions such as a CPU function and a GPU (Graphics Processing Unit) function may be provided instead of the CPU 20. The RAM 22 is the main storage device of the server 12 and is used as a work area and a buffer area for the CPU 20.

[0030] Although not shown, the server 12 also includes auxiliary storage devices such as an HDD and a ROM. However, instead of or in addition to the HDD and the ROM, a non-volatile memory such as an SSD may be used.

[0031] The communication I / F 24 has a wired interface for transmitting and receiving control signals and data to and from external computers such as the operator terminal 16 and the robot 18 via the network 14 under the control of the CPU 20. However, a wireless interface such as wireless LAN or Bluetooth (registered trademark) can also be used as the communication I / F 24.

[0032] An input device 28 and a display device 30 are connected to the input / output I / F 26. A keyboard and a computer mouse are used as the input device 28. The display device 30 is an LCD or an organic EL display.

[0033] The input / output I / F 26 outputs operation data (or operation information) input from the input device 28 to the CPU 20, and also outputs image data generated by the CPU 20 to the display device 30, causing the display device 30 to display a screen corresponding to the image data.

[0034] The electrical configuration of the server 12 shown in FIG. 2 is an example and does not need to be limited to this.

[0035] Fig. 3 is a block diagram showing the electrical configuration of the operator terminal 16 shown in Fig. 1. As shown in Fig. 3, the operator terminal 16 includes a CPU 50, which is connected to a RAM 52, a communication I / F 54, and an input / output I / F 56 via an internal bus.

[0036] The CPU 50 is responsible for overall control of the operator terminal 16. However, an SoC including multiple functions such as a CPU function and a GPU function may be provided instead of the CPU 50. The RAM 52 is the main memory device of the operator terminal 16, and is used as a work area and a buffer area for the CPU 50.

[0037] Although not shown, the operator terminal 16 also includes auxiliary storage devices such as a HDD and a ROM. However, instead of or in addition to the HDD and the ROM, a non-volatile memory such as an SSD may be used.

[0038] The communication I / F 54 has a wired interface for transmitting and receiving control signals and data to and from external computers such as the server 12 and the robot 18 via the network 14 under the control of the CPU 50. However, a wireless interface such as a wireless LAN or Bluetooth (registered trademark) can also be used as the communication I / F 54.

[0039] An input device 58, a display device 60, a microphone 62, and a speaker 64 are connected to the input / output I / F 56. A keyboard and a computer mouse are used as the input device 58. A touch panel may also be provided. The display device 60 is an LCD or an organic EL display.

[0040] However, when a smartphone is used as the operator terminal 16, the input device 58 is a touch panel and hardware buttons. The touch panel is a general-purpose touch panel, and any type of touch panel can be used, such as a capacitance type, an electromagnetic induction type, a resistive film type, or an infrared type.

[0041] The input / output I / F 56 converts the operator's voice detected by the microphone 62 into digital voice data and outputs it to the CPU 50, and also converts the voice data output by the CPU 50 into an analog voice signal and outputs it from the speaker 64. However, in the embodiment, the voice data output from the CPU 50 is voice data received from the server 12 or the robot 18. Furthermore, the input / output I / F 56 outputs operation data (or operation information) input from the input device 58 to the CPU 50, and outputs image data generated by the CPU 50 to the display device 60, causing a screen or image corresponding to the image data to be displayed on the display device 60. However, there are also cases where image data received from an external computer (for example, the server 12 or the robot 18) is output by the CPU 50.

[0042] The operator terminal 16 also includes a sensor I / F 66 and a camera 68. The CPU 50 is connected to the camera 68 via the bus and the sensor I / F 66. The camera 68 is a camera that uses an imaging element such as a CCD or CMOS.

[0043] The electrical configuration of the operator terminal 16 shown in FIG. 3 is an example and does not need to be limited to this.

[0044] Furthermore, if the operator terminal 16 is a smartphone, it is equipped with a call circuit for making calls via a mobile phone network, or via a mobile phone network and a public telephone network; however, in this embodiment, such calls are not made, and therefore are not shown.

[0045] In such an information processing system 10, an operator, i.e., a service user, uses the operator terminal 16 to launch a browser, access a website for a predetermined service operated by the server 12, select a menu on the top screen, and register an operator ID and password. Figure 4 shows an example of an ID and password registration screen 100 displayed on the display device 60 of the operator terminal 16.

[0046] As shown in FIG. 4, the ID and password registration screen 100 has display areas 102 and 104 arranged vertically, and below the display area 104, buttons 110 and 112 arranged horizontally.

[0047] Display area 102 is an area for displaying the input operator ID. Display area 104 is an area for displaying the input password. However, to prevent others from seeing the password, the password may be displayed replaced with a symbol such as an asterisk.

[0048] As an example, the operator ID and password are arbitrarily determined by the operator using available letters, numbers, and symbols.

[0049] The button 110 is a button for registering an operator ID and a password for a predetermined service. When the button 110 is turned on, the operator ID and the password are stored (registered) as first authentication information in the operator DB 12a connected to the server 12.

[0050] Button 112 is a button for canceling the registration of the operator ID and password to a predetermined service. When button 112 is turned on, the registration of the operator ID and password to the predetermined service is canceled, the ID and password registration screen 100 is hidden, and the screen returns to the top screen.

[0051] As described above, when an operator ID and password are registered with a predetermined service as the first piece of authentication information, a response pattern is registered as the second piece of authentication information. When registering a response pattern, the pattern registration screen 150 shown in Fig. 5 is displayed on the display device 60 of the operator terminal 16, and the UI screen 200 shown in Fig. 6 is also displayed. The operator interacts with the human or first robot image 152 (302) facing the robot 18 by inputting voice and operating the UI screen 200.

[0052] 5 displays a first robot image 152 and a second robot image 154. The first robot image 152 is a virtual interactive agent. The second robot image 154 is a virtual avatar corresponding to the robot 18 shown in FIG. 1. The second robot image 154 does not need to have the same appearance as the robot 18, but has an appearance that allows it to perform the same actions as the robot 18.

[0053] The movements and speech of the first robot image 152 are automatically controlled by the CPU 20 of the server 12, and the movements and speech of the second robot image 154 are controlled according to the operation of the operator.

[0054] 6 is a screen for operating the robot 18, and is a screen for operating the second robot image 154, i.e., a virtual avatar, when registering a response pattern. However, as will be described later, the UI screen 200 is also a screen for operating the second robot image 304, i.e., a virtual avatar, when authenticating a response pattern.

[0055] The UI screen 200 has display areas 202, 204, and 206. The display area 202 is an area for displaying an image (camera image) taken by a camera provided on the robot 18. Therefore, nothing is displayed during registration and authentication.

[0056] The display area 204 displays a plurality of buttons for inputting commands (hereinafter referred to as "movement commands") that control the movement of the robot 18. In this embodiment, the display area 204 displays buttons 210, 212, 214, 216, and 218. The button 210 is a button for moving the robot 18 forward. The button 212 is a button for making the robot 18 turn left or rotate left. The button 214 is a button for making the robot 18 turn right or rotate right. The button 216 is a button for stopping the robot 18. The button 218 is a button for making the robot 18 move backward.

[0057] Although detailed explanation will be omitted, when button 212 is turned on while button 210 or button 218 is turned on, the robot 18 turns left while moving forward or backward. When button 214 is turned on while button 210 or button 218 is turned on, the robot 18 turns right while moving forward or backward. When button 212 is turned on while the robot 18 is stopped, the robot 18 changes direction of movement counterclockwise. When button 214 is turned on while the robot 18 is stopped, the robot 18 changes direction of movement clockwise.

[0058] However, in this embodiment, the movement of the second robot image 154 is not controlled during registration (authentication), and therefore the buttons 210-218 are not turned on. In another example, the movement of the second robot image 154 may be controlled during registration. In this case, the second robot image 154 is moved in the virtual space in the same way as the robot 18 is moved in the real world. This is also true during authentication.

[0059] When registering a response pattern (the same applies to authentication), a dialogue agent image rendered in computer graphics (CG), i.e., the first robot image 152, makes several predetermined utterances or questions, and the operator responds and interacts through an avatar, i.e., the second robot image 154, by speaking or operating the UI screen 200. In other words, when registering a response pattern (the same applies to authentication), an avatar image rendered in CG is displayed on the pattern registration screen 150. At this time, the server 12 stores (registers) the operator's operation patterns and reaction patterns in response to each utterance or question as second authentication information in the operator DB 12a.

[0060] In this embodiment, the dialogue agent makes some statements or questions by outputting synthetic voice, but the content of the statements or questions may also be displayed in text using subtitles or speech bubbles. Alternatively, both synthetic voice output and text display may be performed. As will be described later, the operator's voice is used to generate reaction patterns. Therefore, if the dialogue agent's statements or questions are only displayed in text, it is necessary to guide the operator to speak on the pattern registration screen 150 (the same applies to the pattern authentication screen 300, which will be described later).

[0061] However, the operation pattern of the operator means the content of the operator's utterances and the content of the operator's operations (movements of the second robot image 154) in response to statements or questions from the first robot image 152. However, since the operator may not operate the second robot image 154, the content of the operator's operations (movements of the second robot image 154) may not be included in the operation pattern. Alternatively, the absence of an operation may be included in the operation pattern.

[0062] The operator's response pattern is an involuntary response of the operator when the operator speaks. Specifically, the involuntary response is prosodic information of the operator's voice and the operator's emotion.

[0063] The prosodic information of the operator's voice can be generated using a method using deep learning or the technology disclosed in Japanese Patent Application Laid-Open No. 2008-15361. Japanese Patent Application Laid-Open No. 2008-15361 discloses that the prosodic information includes information on duration, pitch, energy (power), and the like, and further includes information on intonation, tone, and rhythm. However, this information is time-series data. Furthermore, it is not necessary to use all of this prosodic information for authentication; it is sufficient to use one or more of these pieces of information for authentication.

[0064] However, the prosodic information may be generated by another computer communicatively connected to the server 12. In this case, when the server 12 receives the voice data, it transmits the received voice data to the other computer and receives the prosodic information from the other computer. Alternatively, the operator terminal 16 may generate the prosodic information from the voice data of the operator and transmit the prosodic information to the server 12.

[0065] The operator's emotions are estimated from the captured image, i.e., the captured facial image of the operator, transmitted from the operator terminal 16. As an example, emotions are classified into five categories: joy, fear, sadness, disgust, and anger. Known techniques can be used to estimate human emotions from facial images. For example, the technologies disclosed in "Kobayashi, Hiroshi, Hara, Fumio: Recognition of basic human facial expressions using neural networks, Transactions of the Society of Instrument and Control Engineers, Vol. 29, No. 1, 112 / 118 (1993)," "Koyanaka, Yosuke, Homma, Tsuneyasu, Sakai, Masao, Abe, Kenichi: Facial expression recognition using neural networks, Bulletin of the Tohoku University School of Medicine and Health Sciences 13(1):23-32, 2004," and "Nishime, Taiki, Endo, Satoshi, Touma, Aiko, Yamada, Koji, Akamine, Yuhei: Acquisition of facial expressions and analysis of facial features using convolutional neural networks, Transactions of the Japanese Society for Artificial Intelligence, Vol. 32, No. 5, FZ (2017)" can be used.

[0066] In addition, in another known technology, the technology disclosed in JP 2020-163660 A can also be used as a method for estimating human emotions based on feature points extracted from a facial image.

[0067] In another example, human emotions can also be estimated from speech. Known technologies can be used to estimate human emotions from speech. For example, the technologies disclosed in JP 2021-12285 A and "Mori, Daiki: Understanding Emotions and Attitudes from Speech," Journal of the Institute of Electronics, Information and Communication Engineers, Vol. 101, No. 9, 2018, can be used.

[0068] However, when using a known technique for estimating emotions based on facial images or voice, the necessary circuit components and data are provided in the server 12. Alternatively, a device for estimating emotions based on facial images or voice (hereinafter referred to as an "estimation device") may be provided on the cloud, and the server 12 may transmit facial images or voice to the estimation device and receive emotion estimation results from the estimation device. Alternatively, the operator's emotion may be estimated by the operator terminal 16, and the estimation results may be transmitted to the server 12.

[0069] Furthermore, the content of the operator's speech is detected by the server 12 by performing voice recognition on the voice data transmitted from the operator terminal 16. However, the voice recognition function may be performed by another computer communicably connected to the server 12. In this case, when the server 12 receives voice data, it transmits the received voice data to the other computer and receives the voice recognition processing result (text sentence) from the other computer. Alternatively, the operator terminal 16 may perform voice recognition on the operator's voice data and transmit the voice recognition processing result to the server 12.

[0070] The operation content of the operator is the type of operation command or button (here, button 220, 222 or 224 shown in FIG. 6) transmitted from operator terminal 16.

[0071] For example, the dialogue agent utters "Hello." That is, the voice data of the dialogue agent saying "Hello" is transmitted from the server 12 to the operator terminal 16 and output from the speaker 64. The same applies to the following cases where the dialogue agent utters. At this time, the emotion of the operator is estimated to be joy.

[0072] In response to this, the operator utters "Good evening." Furthermore, in response to the operator's operation, the operator's avatar raises its right hand. That is, the operator turns on button 224 on the UI screen 200 and transmits the "raise right hand" action command to the server 12. Accordingly, the server 12 causes the second robot image 154 on the pattern registration screen 150 to raise its right hand.

[0073] Next, the conversational agent utters, "What did you have for breakfast?" In response, the operator's avatar nods in response to the operator's actions. The operator then utters, "Natto." At this point, the operator's emotion is estimated to be happiness.

[0074] Next, the conversational agent utters, "Huh? Please say it again." In response, the operator utters again, "It's natto." At this point, the operator's emotion is estimated to be disgust.

[0075] As described above, some utterances or inquiries from the dialogue agent are predetermined, and the operator's operation patterns and reaction patterns to each utterance or inquiry from the dialogue agent are stored as response patterns in the operator DB 12a. However, the response patterns are stored in association with the operator ID of the operator. The response patterns stored in the operator DB 12a (hereinafter referred to as "registered response patterns") are used as second authentication information in authentication processing when a predetermined service is provided using an avatar, i.e., a robot 18.

[0076] Furthermore, when the operator, i.e., the service user, uses the operator terminal 16 to start a browser, access a website of a predetermined service operated by the server 12, select a menu on the top screen, and select to use an avatar (robot 18), an authentication process for the operator is executed before the avatar is used. The authentication process includes a first authentication process performed using first authentication information and a second authentication process performed using second authentication information.

[0077] When the server 12 starts the authentication process, it first authenticates the operator ID and password. That is, the first authentication process is executed. Fig. 7 is a diagram showing an example of an ID / password authentication screen 250 displayed on the display device 60 of the operator terminal 16. The ID / password authentication screen 250 and the first authentication process will be described below, but the same content as that described in the registration process above will be briefly described.

[0078] As shown in FIG. 7, ID and password authentication screen 250 has display areas 252 and 254 arranged vertically, and below display area 254, buttons 260 and 262 arranged horizontally.

[0079] Display area 252 is an area for displaying the input operator ID. Display area 254 is an area for displaying the input password. However, to prevent others from seeing the password, the password may be displayed replaced with a symbol such as an asterisk.

[0080] Button 260 is a button for authenticating the operator ID and password. When button 260 is turned on, the server 12 refers to the first authentication information stored in the operator DB 12a and determines whether the operator ID and password are correct. If the operator ID and password are correct, the first authentication is successful. On the other hand, if the operator ID and / or password are incorrect, the first authentication fails. If the authentication fails, the operator ID and password are entered again on the ID / password authentication screen 250.

[0081] Button 262 is a button for canceling authentication of the operator ID and password. When button 262 is turned on, the authentication process of the operator ID and password is canceled, the ID and password authentication screen 250 is hidden, and the screen returns to the top screen.

[0082] As described above, if the first authentication process is successful, the server 12 executes the second authentication process. When the second authentication process is executed, the pattern authentication screen 300 shown in Fig. 8 is displayed on the display device 60 of the operator terminal 16, and the UI screen 200 shown in Fig. 6 is also displayed. The pattern authentication screen 300 and the second authentication process will be described below, but the same content as that explained in the above registration process will be briefly explained.

[0083] In the second authentication process, the operator also interacts with the interactive agent (here, the first robot image 302) by voice input and operation of the UI screen 200. This is the same as when the second registration process is performed, so a duplicated explanation will be omitted.

[0084] 8 displays a first robot image 302 and a second robot image 304. The first robot image 302 is the same as the first robot image 152 displayed on the pattern registration screen 150, and the second robot image 304 is the same as the second robot image 154 displayed on the pattern registration screen 150.

[0085] The movements and speech of the first robot image 302 are automatically controlled by the CPU 20 of the server 12, and the movements and speech of the second robot image 154 are controlled according to the operation of the operator.

[0086] When authenticating a dialogue pattern, the first robot image 302 makes some predetermined statements or questions, and the operator responds and interacts through the second robot image 304 by speaking or operating the UI screen 200. However, some of the statements or questions are the same as those in the second registration process.

[0087] In the second authentication process, the operator's response pattern to several utterances or questions (hereinafter referred to as the "authentication response pattern") is acquired in the same manner as in the two registration processes. Then, if it is determined that the authentication response pattern matches highly with the registered response pattern stored in the operator DB 12a, the second authentication is successful. On the other hand, if it is determined that the authentication response pattern matches poorly or does not match with the registered response pattern, the second authentication fails. However, the registered response pattern associated with the operator ID of the operator is used to determine the match with the operator's authentication response pattern.

[0088] However, if the dialogue patterns included in the authentication response pattern and the registration response pattern do not match, it is determined that the authentication response pattern and the registration response pattern do not match to a high degree or do not match at all.

[0089] Furthermore, if the operator's emotions do not match among the response patterns included in the authentication response pattern and the registration response pattern, it is determined that the authentication response pattern does not match the registration response pattern.

[0090] Furthermore, the degree of match of the prosodic information of the response patterns included in the authentication response pattern and the enrollment response pattern is detected by, for example, deep learning, DP matching, or Hidden Markov Model (HMM). If the degree of match of any one of the prosodic information generated from the speech of the operator in response to multiple utterances or questions is less than a predetermined percentage (for example, 80%), it is determined that the authentication response pattern does not match the enrollment response pattern.

[0091] In other words, the authentication response pattern and the registered response pattern are determined to have a high degree of match when the speech content in response to multiple utterances or questions contained in the authentication response pattern and the registered response pattern all match, the emotions of the operators all match, and the degree of match of all prosodic information is a predetermined percentage or more.

[0092] However, in this embodiment, if the degree of match of any one of the pieces of prosodic information generated from the speech of the operator is less than a predetermined rate (e.g., 80%), it is determined that the authentication response pattern does not match the registration response pattern, but this is not limited to this. In another example, if the degree of match of any two or more pieces of prosodic information generated from the speech of the operator is equal to or greater than a predetermined rate (e.g., 80%), it may be determined that the degree of match of the authentication response pattern and the registration response pattern is high. Furthermore, if the degree of match of any one of the pieces of prosodic information generated from the speech of the operator is very high (e.g., more than 90%), it may be determined that the degree of match of the authentication response pattern and the registration response pattern is high.

[0093] If the second authentication is also successful, the server 12 permits the start of a session between the operator terminal 16 and the robot 18. Therefore, a connection state is established between the operator terminal 16 and the robot 18, and the robot 18 functions as an avatar of the operator of the operator terminal 16. At this time, the operator's voice is transmitted to the robot 18 and output from a speaker of the robot 18. In addition, the operator operates the UI screen 200 displayed on the display device 60 of the operator terminal 16, and an action command corresponding to the operation is transmitted to the robot 18, and the robot 18 moves in accordance with the action command or performs an action other than movement.

[0094] On the other hand, if the first authentication is not successful or the second authentication is not successful, the server 12 rejects the start of a session between the operator terminal 16 and the robot 18.

[0095] Fig. 9 shows an example of a memory map 400 of the RAM 22 built into the server 12. As shown in Fig. 9, the RAM 22 includes a program storage area 402 and a data storage area 404. The information processing program of this embodiment is stored in the program storage area 402.

[0096] The information processing programs include a communication program 402a, a registration program 402b, an authentication program 402c, a voice recognition program 402d, a prosodic information generation program 402e, and an emotion estimation program 402f.

[0097] The communication program 402a is a program for communicating (transmitting and receiving data) with an external device, in this embodiment, the operator terminal 16, via wire or wirelessly.

[0098] The registration program 402b is a program for executing a first registration process for registering an operator ID and a password and a second registration process for registering a response pattern in accordance with the operation of the operator of the operator terminal 16.

[0099] The authentication program 402c is a program for executing a first authentication process for authenticating an operator ID and a password and a second authentication process for authenticating a response pattern in accordance with an operation by an operator of the operator terminal 16.

[0100] The voice recognition program 402d is a program for recognizing the voice (voice data) of the operator.

[0101] The prosodic information generating program 402e is a program for generating prosodic information of the operator's voice (voice data).

[0102] The feeling estimation program 402f is a program for estimating the feeling of the operator based on a facial image of the operator.

[0103] Although not shown, the program storage area 402 also stores middleware such as the operating system of the server 12, programs for providing predetermined services, and programs for generating image data such as display image data 404a.

[0104] The data storage area 404 also stores display image data 404a, interactive agent voice data 404b, operator voice data 404c, operator image data 404d, action command data 404e, registration ID / password data 404f, registration response pattern data 404g, authentication ID / password data 404h, and authentication response pattern data 404i.

[0105] The display image data 404a is image data for displaying various screens (100, 150, 200, 250, 300, etc.), and includes image data of the first robot image 152, 302, which is an image of an interactive agent drawn using CG, and the second robot image 154, 304, which is an image of an operator's avatar drawn using CG.

[0106] The dialogue agent voice data 404b is data of synthesized voices for a plurality of utterances and inquiries prepared in advance when a dialogue pattern is registered and authenticated.

[0107] The operator voice data 404c is voice data of the operator received from the operator terminal 16.

[0108] The operator image data 404d is photographed image data received from the operator terminal 16.

[0109] The operation command data 404e is the operation command data included in the operation data 504a received from the operator terminal 16.

[0110] The registration ID and password data 404f is data of the operator ID and password determined (or input) by the operation data received from the operator terminal 16 during the first registration process.

[0111] The registered response pattern data 404g is data on response patterns in which operation patterns and reaction patterns received from the operator terminal 16 during the second registration process are written in chronological order.

[0112] The authentication ID and password data 404h is data of the operator ID and password determined (or input) by the operation data received from the operator terminal 16 during the first authentication process.

[0113] The authentication response pattern data 404i is data on response patterns in which operation patterns and reaction patterns received from the operator terminal 16 during the second authentication process are written in chronological order.

[0114] Although not shown, the data storage area 404 stores other data required to execute information processing, and is provided with a timer (counter) and flags required to execute information processing.

[0115] Fig. 10 shows an example of a memory map 500 of the RAM 52 built into the operator terminal 16. As shown in Fig. 10, the RAM 52 includes a program storage area 502 and a data storage area 504. The control program of this embodiment is stored in the program storage area 502.

[0116] The control programs include a communication program 502a, an operation detection program 502b, a photographed image acquisition program 502c, an image display program 502d, a sound detection program 502e, and a sound output program 502f.

[0117] The communication program 502a is a program for communicating (sending and receiving data) with external devices, in this embodiment, the server 12 and the robot 18, via wire or wirelessly.

[0118] The operation detection program 502b is a program for detecting operation data input from the input device 58 in accordance with the operator's operation and storing the data in the data storage area 504. However, when a button provided on each screen (100, 150, 200, 250, 300) is operated, data of operation information (including an action command) assigned to the operated button is detected as operation data.

[0119] The photographed image acquisition program 502c is a program for acquiring photographed image data photographed by the camera 68.

[0120] The image display program 502d is a program for outputting the display image data 504b received from the server 12 and / or the display image data generated by the operator terminal 16 to the display device 60.

[0121] The sound detection program 502e is a program for detecting the operator's voice input from the microphone 62 and temporarily storing in the data storage area 504 operator voice data 504d corresponding to the detected voice.

[0122] The sound output program 502f is a program for outputting the dialogue agent voice data 504c received from the server 12. However, during operation of the robot 18, the sound output program 502f is also a program for outputting the dialogue partner's voice data received from the robot 18.

[0123] Although not shown, the program storage area 502 also stores middleware such as the operating system of the operator terminal 16 and a program for executing the browser function.

[0124] The data storage area 504 also stores operation data 504a, display image data 504b, dialogue agent voice data 504c, operator voice data 504d, operator image data 504e, and the like.

[0125] The operation data 504a is operation data detected according to the operation detection program 502b. The operation data 504a is deleted after being transmitted to the server 12 or used for processing by the operator terminal 16.

[0126] The display image data 504b is image data for displaying various screens (100, 150, 200, 250, 300, etc.) received from the server 12, and includes image data of the first robot images 152, 302, which are images of a conversational agent drawn using CG, and the second robot images 154, 304, which are images of the operator's avatar drawn using CG.

[0127] The dialogue agent voice data 504c is data of synthesized voices for a plurality of utterances and inquiries prepared in advance when a dialogue pattern received from the server 12 is registered and authenticated. The dialogue agent voice data 504c is deleted after being output from the speaker 64.

[0128] The operator voice data 504d is voice data detected by the microphone 62, and is deleted after being transmitted to the server 12.

[0129] The operator image data 504e is photographed image data acquired by the camera 68, and is deleted after being transmitted to the server 12.

[0130] Although not shown, the data storage area 504 stores other data required to execute the control process, and is provided with a timer (counter) and flags required to execute the control process.

[0131] 11 and 12 are flow diagrams showing the registration processing of the CPU 20 of the server 12. Although not shown in the drawings, the CPU 20 executes processing to receive the operation data 504a, the operator voice data 504d, and the operator image data 504e transmitted from the operator terminal 16 in parallel with the registration processing.

[0132] Although not shown in the figure, the CPU 50 of the operator terminal 16 detects operation data 504a, detects operator voice data 504d, and acquires operator image data 504e, and executes processing to transmit each piece of data to the server 12, and executes processing to receive display image data 404a and / or interactive agent voice data 404b transmitted from the server 12.

[0133] The CPU 20 of the server 12 executes the registration process for each operator terminal 16 that is accessing the website operated by the server 12.

[0134] The same applies to the authentication process (see FIGS. 13 and 14) described below.

[0135] 11, when the CPU 20 of the server 12 starts the registration process, in step S1, it transmits display image data for the ID and password registration screen 100 as shown in FIG. 4 to the operator terminal 16. Therefore, the ID and password registration screen 100 is displayed on the display device 60 of the operator terminal 16.

[0136] In the next step S3, it is determined whether or not an instruction to register an operator ID and password has been issued. If the answer is "NO" in step S3, that is, if an instruction to register an operator ID and password has not been issued, the process returns to step S3. Although not shown, when characters or the like of an operator ID or password are entered in response to an operation by the operator, the characters or the like are stored and are displayed in display area 102 or display area 104 of ID / password registration screen 100.

[0137] On the other hand, if "YES" in step S3, that is, if there is an instruction to register an operator ID and password, in step S5, the operator ID and password, that is, the registration ID / password data 404f, are registered in the operator DB 12a.

[0138] Next, in step S7, the display image data of the pattern registration screen 150 as shown in Fig. 5 is sent to the operator terminal 16. In the next step S9, an initial value is set to a variable n (n = 1), where n is a variable for identifying a statement or question to be uttered by the dialogue agent in the response pattern registration process.

[0139] 12, in the next step S11, the interactive agent is caused to utter the nth utterance or question. Here, the CPU 20 transmits the interactive agent voice data 404b for the nth utterance or question to the operator terminal 16. Therefore, the voice of the nth utterance or question, i.e., the voice of the first robot image 152, is output from the speaker 64 of the operator terminal 16.

[0140] In the next step S13, it is determined whether or not there is a response. Here, it is determined whether or not the data received from the operator terminal 16 includes the operator voice data 504d, i.e., the operator voice data 404c, or the operation data 504a, i.e., the operation command data 404e.

[0141] If step S13 is "NO", that is, if there is no response, the process returns to step S13. On the other hand, if step S13 is "YES", that is, if there is a response, in step S15, the avatar image, i.e., the second robot image 154, is made to speak according to the operator voice data 404c, and in step S17, the avatar image, i.e., the second robot image 154 is made to move according to the movement command.

[0142] However, if the operator voice data 504d has not been received, step S15 is skipped, and if the operation data 504a has not been received, step S17 is skipped.

[0143] In the next step S19, the registered response pattern data 404g is generated (updated). Here, the CPU 20 generates (updates) data describing the operator's response to a statement or question from the dialogue agent, in the order of the utterance or question, including operation patterns and reaction patterns. As described above, the operation patterns are the operator's utterance content and the operator's operation content. Furthermore, the reaction patterns are prosodic information generated from the operator's voice and emotions estimated from the operator's facial image.

[0144] As described above, in step S19, the CPU 20 performs speech recognition on the received operator voice data 504d, i.e., the operator voice data 404c, and converts the operator's speech into text. The CPU 20 also generates prosodic information for the received operator voice data 504d, i.e., the operator voice data 404c. Furthermore, the CPU 20 estimates the operator's emotions based on the operator image data 504e, i.e., the operator image data 404d, received together with the operator voice data 504d.

[0145] Then, in step S21, it is determined whether the variable n is at its maximum value, that is, the CPU 20 determines whether the dialogue agent has uttered all of the multiple statements or questions.

[0146] If step S21 is "NO," that is, if variable n is not the maximum value, variable n is incremented by 1 in step S23, and the process returns to step S11. Therefore, the dialogue agent utters the next statement or question.

[0147] On the other hand, if the answer is "YES" in step S21, that is, if the variable n is the maximum value, in step S25, the registered response pattern data 404g is linked to the operator ID registered in step S5 and registered in the operator DB 12a, and the registration process is terminated.

[0148] Note that steps S1 to S5 shown in FIG. 11 correspond to the first registration process, and steps S7 to S25 shown in FIGS. 11 and 12 correspond to the second registration process.

[0149] 13 and 14 are flow diagrams showing the authentication process of the CPU 20 of the server 12. The authentication process will be described below, but the same content as the above-mentioned registration process will be briefly described.

[0150] 13, when the CPU 20 of the server 12 starts the authentication process, in step S51, it transmits display image data for the ID and password authentication screen 250 as shown in Fig. 7 to the operator terminal 16. Therefore, the ID and password authentication screen 250 is displayed on the display device 60 of the operator terminal 16.

[0151] In the next step S53, it is determined whether or not an instruction to authenticate the operator ID and password has been issued. If the answer is "NO" in step S53, that is, if an instruction to authenticate the operator ID and password has not been issued, the process returns to step S53. Although not shown, when characters or the like of the operator ID or password are input in response to an operation by the operator, the characters or the like are stored and are displayed in display area 252 or display area 254 of ID / password authentication screen 250.

[0152] On the other hand, if the answer is "YES" in step S53, that is, if an instruction to authenticate the operator ID and password has been issued, the CPU 20 authenticates the operator ID and password, i.e., the authentication ID / password data 404h, in step S55. Here, the CPU 20 determines whether the authentication ID / password data 404h matches the registered ID / password data 404f registered in the operator DB 12a.

[0153] Then, in step S57, it is determined whether the authentication is successful. If the answer is "NO" in step S57, that is, if the authentication is unsuccessful, the process returns to step S51. In other words, the operator is prompted to re-enter the operator ID and password. However, if the authentication fails a predetermined number of times (for example, three times), the CPU 20 may refuse to start a session with the robot 18.

[0154] On the other hand, if "YES" in step S57, that is, if the authentication is successful, in step S59, display image data of the pattern authentication screen 300 as shown in Fig. 8 is transmitted to the operator terminal 16. In the next step S61, an initial value is set to a variable n (n=1).

[0155] 14, in the next step S63, the dialogue agent is caused to utter the n-th utterance or question. That is, the voice of the n-th utterance or question, i.e., the voice of the first robot image 302, is output from the speaker 64 of the operator terminal 16.

[0156] In the next step S65, it is determined whether or not there is a response. If "NO" in step S65, the process returns to step S65. On the other hand, if "YES" in step S65, in step S67, the avatar image, i.e., second robot image 304, is made to speak in accordance with the operator voice data 404c, and in step S69, the avatar image, i.e., second robot image 304, is made to move in accordance with the movement command.

[0157] However, if the operator voice data 504d has not been received, step S67 is skipped, and if the operation data 504a has not been received, step S69 is skipped.

[0158] In the next step S71, the authentication response pattern data 404i is generated (updated). Here, the CPU 20 generates (updates) data describing the operation patterns and reaction patterns of the operator's responses to utterances or questions from the dialogue agent in the order of the utterances or questions.

[0159] Then, in step S73, it is determined whether the variable n is the maximum value. If "NO" in step S73, in step S75, the variable n is incremented by 1, and the process returns to step S63.

[0160] On the other hand, if the result in step S73 is "YES," the response pattern is authenticated in step S77. Here, CPU 20 determines whether there is a high degree of match between authentication response pattern data 404i and registered response pattern data 404g that is associated with the operator ID determined to have been successfully authenticated in step S57 and registered (stored) in operator DB 12a. The method for this determination is as described above.

[0161] Then, in step S79, it is determined whether the authentication is successful. If the answer is "NO" in step S79, that is, if the authentication is unsuccessful, in step S81, the start of a session is denied to the operator terminal 16, and the authentication process ends. On the other hand, if the answer is "YES" in step S79, that is, if the authentication is successful, in step S83, the start of a session is permitted to the operator terminal 16, and the authentication process ends.

[0162] Note that steps S51 to S57 shown in FIG. 13 correspond to the first authentication process, and steps S59 to S79 ​​shown in FIGS. 13 and 14 correspond to the second authentication process.

[0163] Furthermore, when the start of the session is permitted, the operator terminal 16 closes the pattern authentication screen 300, establishes a connection with the robot 18, and controls the robot 18 in accordance with the operator's operation. However, the operator operates the UI screen 200 to make the robot 18 operate (including move). Furthermore, when the operator speaks, the operator's voice is output through the robot 18, and the robot 18 speaks. However, the operator's voice can also be converted into another voice and output from the robot 18.

[0164] According to this embodiment, when two-step authentication is performed, if the operator ID and password are successfully authenticated, the operator operates an avatar image and interacts with an image of a dialogue agent, thereby authenticating not only the operator's operation pattern but also the operator's reaction pattern during operation, so there is no need to complicate the operation. Furthermore, since information that is difficult to steal, such as reaction patterns, is used for authentication, security can be improved.

[0165] In this embodiment, the image of the avatar displayed on the authentication screen for receiving a predetermined service using a robot that functions as the operator's avatar can be operated using a UI screen for operating the robot, and after successful authentication, the robot can be operated using the same UI screen. This simplifies the operator's operations, making it easier to use the predetermined service.

[0166] In this embodiment, the operator's emotions among the reaction patterns are estimated based on the facial image of the operator when he / she is speaking, but they may also be estimated based on the facial image of the operator when the dialogue agent makes a statement or asks the operator a question.

[0167] In this embodiment, the second authentication process for authenticating a response pattern authenticates all of the information, including the operator's speech content, the operator's operation content, the prosodic information of the operator's voice, and the operator's emotions. However, this is not limited to this. It is also possible to authenticate only a portion of this information, i.e., one, two, or three pieces of information. In this case, the information to be authenticated may be changed each time the second authentication process is executed.

[0168] In this embodiment, the operation pattern includes the operator's operation of moving the avatar image to authenticate the operator, i.e., the service user, who receives a predetermined service using a robot. However, the operation pattern does not need to include the operator's operation when authenticating a service user who receives other predetermined services that do not use a robot. For example, the other predetermined services may be online banking or credit card services. In this case, there is no need to display a pattern authentication screen in the authentication process for the dialogue pattern, and the dialogue agent may be a human.

[0169] Furthermore, if the same results are obtained from the steps in the flow charts shown in the above embodiments, the order in which the steps are processed can be changed.

[0170] Furthermore, the various screens and specific values ​​given in the above embodiments are merely examples and can be changed as needed. [Explanation of symbols]

[0171] 10. Information Processing Systems 12...server 14...Network 16...Operator terminal 18...Robot 20, 50...CPU 22, 52...RAM 24, 54...Communication I / F 26, 56... Input / output I / F 28, 58...input device 30, 60…display device 62 ...Mike 64...Speaker 66 ...Sensor I / F 68...Camera

Claims

1. An information processing device that provides a predetermined service on a website, a first authentication means for authenticating an ID and a password inputted into an operator terminal of an operator who uses the predetermined service; a second authentication means for authenticating a response pattern of the operator to a dialogue agent when authentication by the first authentication means is successful; and a permission unit that permits the operator terminal to start providing the predetermined service when the authentication by the second authentication unit is successful, the second authentication means, when the dialogue agent utters a plurality of utterance contents to the operator, authenticates the operator according to a degree of coincidence of a response pattern including prosodic information of the operator's voice when the operator responds to each of the plurality of utterance contents by voice; the predetermined service is a service that uses a robot as an avatar of the operator, a UI display means for displaying a UI screen for operating the robot on the operator terminal; and further comprising an authentication screen display means for displaying, on the operator terminal, an authentication screen including an image of the interactive agent and the avatar rendered in a virtual space by computer graphics when the authentication process is performed by the second authentication means; When the operator responds by voice in response to each of the plurality of utterance contents, the operator operates the image of the avatar drawn by the computer graphics using the same screen as the UI screen; The information processing device, wherein the response pattern includes the operation content of the operator that causes the image of the avatar to move.

2. further comprising a receiving means for receiving a captured image from the operator terminal, The information processing apparatus according to claim 1 , wherein the response pattern includes an emotion of the operator estimated from a facial image of the operator.

3. The information processing apparatus according to claim 1 , wherein the response pattern includes a speech content of the operator recognized from the operator's voice.

4. An information processing system comprising: an information processing device that provides a predetermined service on a website; and an operator terminal of an operator who uses the predetermined service, The information processing device includes: a first authentication means for authenticating an ID and a password inputted into the operator terminal; a second authentication means for authenticating a response pattern of the operator to a dialogue agent when authentication by the first authentication means is successful; and a permission unit that permits the operator terminal to start providing the predetermined service when the authentication by the second authentication unit is successful, the second authentication means, when the dialogue agent utters a plurality of utterance contents to the operator, authenticates the operator according to a degree of coincidence of a response pattern including prosodic information of the operator's voice when the operator responds to each of the plurality of utterance contents by voice; the predetermined service is a service that uses a robot as an avatar of the operator, a UI display means for displaying a UI screen for operating the robot on the operator terminal; and further comprising an authentication screen display means for displaying, on the operator terminal, an authentication screen including an image of the interactive agent and the avatar rendered in a virtual space by computer graphics when the authentication process is performed by the second authentication means; When the operator responds by voice in response to each of the plurality of utterance contents, the operator operates the image of the avatar drawn by the computer graphics using the same screen as the UI screen; An information processing system, wherein the response pattern includes an operation content of the operator that moves the image of the avatar.

5. An information processing program for an information processing device that provides a predetermined service on a website, A processor of the information processing device a first authentication step of authenticating an ID and a password inputted into an operator terminal of an operator who uses the predetermined service; a second authentication step of authenticating a response pattern of the operator to a dialogue agent when authentication is successful in the first authentication step; and executing a permission step of permitting the operator terminal to start providing the predetermined service when the authentication is successful in the second authentication step; the second authentication step performs authentication in accordance with a degree of coincidence of a response pattern including prosodic information of the operator's voice when the operator responds to each of the plurality of utterance contents by voice, when the dialogue agent utters a plurality of utterance contents to the operator; the predetermined service is a service that uses a robot as an avatar of the operator, the processor, a UI display step of displaying a UI screen for operating the robot on the operator terminal; and when performing the authentication process by the second authentication step, further executing an authentication screen display step of displaying on the operator terminal an authentication screen including an image of the dialogue agent and the avatar rendered in a virtual space by computer graphics, When the operator responds by voice in response to each of the plurality of utterance contents, the operator operates the image of the avatar drawn by the computer graphics using the same screen as the UI screen; An information processing program, wherein the response pattern includes the operation content of the operator that causes the image of the avatar to move.

6. An information processing method for an information processing device that provides a predetermined service on a website, comprising: (a) authenticating an ID and a password entered at an operator terminal of an operator who uses the predetermined service; (b) if the authentication is successful in the step (a), authenticating a response pattern of the operator to the dialogue agent; and (c) permitting the operator terminal to start providing the predetermined service if the authentication is successful in the step (b); the step (c) performs authentication in accordance with a degree of coincidence of a response pattern including prosodic information of the operator's voice when the operator responds to each of the plurality of utterance contents by voice, when the dialogue agent utters a plurality of utterance contents to the operator; the predetermined service is a service that uses a robot as an avatar of the operator, (d) displaying a UI screen for operating the robot on the operator terminal; and (e) when executing the authentication process in step (b), further comprising the step of displaying on the operator terminal an authentication screen including an image of the dialogue agent and the avatar rendered in a virtual space by computer graphics; When the operator responds by voice in response to each of the plurality of utterance contents, the operator operates the image of the avatar drawn by the computer graphics using the same screen as the UI screen; An information processing method, wherein the response pattern includes an operation content of the operator that moves the image of the avatar.

Images