Information processing device, information processing system, program, and information processing method

Abstract

To provide an information processing apparatus that improves the convenience of a dialogue service using a dialogue apparatus used by each of a plurality of users located remotely from each other.SOLUTION: An information processing apparatus 10 supports dialogue using dialogue apparatuses used by a plurality of users located remotely from each other. The information processing apparatus 10 includes a control unit 13 configured to: estimate, during execution of a remote dialogue, a parameter of each interlocutor with respect to a topic of the remote dialogue based on a video of each interlocutor; display an alert, on a first dialogue apparatus 20 of a first user, the alert indicating each interlocutor whose estimated parameter is less than a threshold; and when the first user gazes at and speaks to an interlocutor displayed with the alert among a plurality of interlocutors as second users displayed on a screen of the first dialogue apparatus 20, transmit first audio data with a high volume to a second dialogue apparatus 30 of the interlocutor and transmit second audio data with a low volume to a second dialogue apparatus 30 of other interlocutors than the interlocutor.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an information processing system, a program, and an information processing method. [Background technology]

[0002] Conventionally, there is known a technique for supporting a dialogue using a dialogue device used by each of a plurality of users located remotely from each other. For example, Patent Document 1 discloses a communication device that can easily recognize whether a listener understands the contents of a remote conference or the like when the remote conference or the like is held by communication between a plurality of communication terminals. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2009-267621 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the conventional technology described in Patent Document 1, when a user wants to call out to a specific user as a conversation partner, the voice of the user calling out is output to other users at the same volume as the voice output to the specific user. Therefore, there is a possibility that the call to the specific user will not reach the specific user. As a result, the convenience of a conversation service using a conversation device used by each of multiple users located remotely from each other has decreased.

[0005] The present disclosure aims to provide an information processing device, an information processing system, a program, and an information processing method that improve the convenience of interactive services using interactive devices used by multiple users who are located remotely from each other. [Means for solving the problem]

[0006] An information processing device according to one embodiment of the present disclosure is an information processing device that supports a dialogue using a dialogue device used by each of multiple users who are located remotely from each other, and is equipped with a control unit that, during a remote dialogue, estimates parameters of each dialogue partner for a topic of the remote dialogue based on video of each dialogue partner, displays an alert on the dialogue device of a first user indicating each dialogue partner for which the estimated parameters are below a threshold, and, when the first user turns his or her gaze toward a dialogue partner for which the alert is displayed and speaks, among multiple dialogue partners as a second user displayed on the screen of the dialogue device, transmits first audio data at a high volume to the dialogue device of the dialogue partner and transmits second audio data at a low volume to the dialogue device of a dialogue partner other than the first dialogue partner.

[0007] A program according to one embodiment of the present disclosure causes an information processing device that supports dialogue using dialogue devices used by multiple users located remotely from each other to perform operations including: estimating parameters of each dialogue partner for a topic of the remote dialogue based on video of each dialogue partner during the remote dialogue; displaying an alert on the dialogue device of a first user indicating each dialogue partner for which the estimated parameters are below a threshold; and, when the first user turns his or her gaze toward and speaks to a dialogue partner for which the alert is displayed, among multiple dialogue partners as a second user displayed on the screen of the dialogue device, transmitting first audio data at a high volume to the dialogue device of the dialogue partner and transmitting second audio data at a low volume to the dialogue device of a dialogue partner other than the first dialogue partner.

[0008] An information processing method according to one embodiment of the present disclosure is an information processing method for supporting a dialogue using a dialogue device used by each of a plurality of users located remotely from each other, which includes estimating, during a remote dialogue, parameters of each dialogue partner for a topic of the remote dialogue based on an image of each dialogue partner, displaying an alert on the dialogue device of a first user indicating each dialogue partner for which the estimated parameters are below a threshold, and when the first user turns his / her gaze toward and speaks to a dialogue partner for which the alert is displayed, among a plurality of dialogue partners as a second user displayed on a screen of the dialogue device, transmitting first audio data at a high volume to the dialogue device of the dialogue partner and transmitting second audio data at a low volume to the dialogue device of a dialogue partner other than the first dialogue partner. Effect of the Invention

[0009] According to an information processing device, an information processing system, a program, and an information processing method according to one embodiment of the present disclosure, the convenience of an interactive service using an interactive device used by each of multiple users who are located remotely from each other is improved. [Brief description of the drawings]

[0010] [Figure 1] 1 is a configuration diagram illustrating a configuration of an information processing system including an information processing device according to an embodiment of the present disclosure. [Diagram 2] 2 is a functional block diagram showing a schematic configuration of each of the information processing device, the first dialogue device, and the second dialogue device in FIG. 1. [Diagram 3] 1. FIG. 4 is a sequence diagram illustrating a first example of an information processing method executed by the information processing system of FIG. [Figure 4] 1. FIG. 4 is a sequence diagram illustrating a second example of an information processing method executed by the information processing system of FIG. [Diagram 5] 4 is a schematic diagram showing an example of an image displayed on a screen of an output unit of a first interaction device of a first user; FIG. [Figure 6] 13 is a schematic diagram showing an example of an image displayed on a screen of an output unit of a second interaction device of a second user. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings.

[0012] Fig. 1 is a configuration diagram showing a configuration of an information processing system 1 including an information processing device 10 according to an embodiment of the present disclosure. An overview of the information processing system 1 including the information processing device 10 according to an embodiment of the present disclosure will be mainly described with reference to Fig. 1. The information processing system 1 has one first dialogue device 20 and multiple second dialogue devices 30 in addition to the information processing device 10.

[0013] For ease of explanation, only one second dialogue device 30 is illustrated in Fig. 1, but the information processing system 1 has two or more second dialogue devices 30. For ease of explanation, only one information processing device 10 is illustrated in Fig. 1, but the information processing system 1 may have two or more information processing devices 10. Each of the information processing device 10, the one first dialogue device 20, and the multiple second dialogue devices 30 is communicatively connected to a network 40 including a mobile communication network and the Internet.

[0014] As an overview of one embodiment, an information processing device 10 supports a dialogue using a dialogue device used by each of a plurality of users located remotely from each other. In the present disclosure, the "dialogue device" includes a first dialogue device 20 or a second dialogue device 30.

[0015] During a remote dialogue, the information processing device 10 estimates parameters of each dialogue partner for a topic of the remote dialogue based on the video of each dialogue partner. In the present disclosure, the "topic" includes, for example, the content being explained in a class. The "parameter" includes, for example, at least one of the level of understanding, the level of interest, and the level of concentration. The information processing device 10 causes the dialogue device of the first user to display an alert indicating each dialogue partner whose estimated parameter is less than a threshold value.

[0016] When a first user turns his / her gaze to a dialogue partner for which an alert is displayed among multiple dialogue partners as second users displayed on a screen of the dialogue device and speaks, the information processing device 10 transmits first voice data at a loud volume to the dialogue device of the dialogue partner. The information processing device 10 transmits second voice data at a low volume to dialogue devices of dialogue partners other than the dialogue partner.

[0017] In the present disclosure, the "first user" includes, for example, a teacher who is the topic of remote dialogue. The "second user" includes, for example, a student who is a dialogue partner with the teacher. The "first voice data" includes data set so that the volume of the voice data when the user speaks is increased when the content of the speech is output as voice from another user's dialogue device. The "second voice data" includes data set so that the volume of the voice data when the user speaks is decreased when the content of the speech is output as voice from another user's dialogue device.

[0018] A user, for example, installs his / her own dialogue device and uses a Web application for remote dialogue that cooperates with the information processing device 10 to dialogue with another user while viewing the screen of the dialogue device. For example, images of a plurality of second users are displayed on the screen of the dialogue device of the first user. For example, images of the first user and images of the other second users are displayed on the screen of the dialogue device of the second user. The dialogue device of the user transmits the contents of the user's speech as voice data to the dialogue device of the other user via the information processing device 10. The dialogue device of the user outputs the contents of the speech of the other user as voice based on the voice data received from the dialogue device of the other user.

[0019] The information processing device 10 is one or more server devices that can communicate with each other. The information processing device 10 is not limited to these, and may be any general-purpose electronic device such as a PC (Personal Computer) or a smartphone, or may be another electronic device dedicated to the information processing system 1.

[0020] The first dialogue device 20 is a general-purpose electronic device such as a PC, a tablet PC, or a smartphone. The first dialogue device 20 is an electronic device used by a first user. The first dialogue device 20 is not limited to these, and may be one or more server devices that can communicate with each other and are used by the first user, or may be another electronic device dedicated to the information processing system 1.

[0021] The second dialogue device 30 is a general-purpose electronic device such as a PC, a tablet PC, or a smartphone. The second dialogue device 30 is an electronic device used by a second user. The second dialogue device 30 is not limited to these, and may be one or more server devices that can communicate with each other and are used by the second user, or may be another electronic device dedicated to the information processing system 1.

[0022] Fig. 2 is a functional block diagram showing a schematic configuration of each of the information processing device 10, the first dialogue device 20, and the second dialogue device 30 in Fig. 1. With reference to Fig. 2, an example of the configuration of each of the information processing device 10, the first dialogue device 20, and the second dialogue device 30 included in the information processing system 1 will be mainly described.

[0023] As shown in FIG. 2, the information processing device 10 includes a communication unit 11, a storage unit 12, and a control unit 13.

[0024] The communication unit 11 includes a communication module that connects to the network 40. For example, the communication unit 11 includes a communication module that supports mobile communication standards or Internet standards such as 4G (4th Generation) and 5G (5th Generation). In one embodiment, the information processing device 10 is connected to the network 40 via the communication unit 11. The communication unit 11 transmits and receives various information via the network 40.

[0025] The storage unit 12 is, for example, but not limited to, a semiconductor memory, a magnetic memory, or an optical memory. The storage unit 12 functions as a main storage device, an auxiliary storage device, or a cache memory. The storage unit 12 stores any information used in the operation of the information processing device 10. The storage unit 12 stores system programs, application programs, and various information received or transmitted by the communication unit 11. The information stored in the storage unit 12 can be updated with information received from the network 40 via the communication unit 11.

[0026] The control unit 13 includes one or more processors. In one embodiment, the "processor" is a general-purpose processor or a dedicated processor specialized for a specific process, but is not limited thereto. The control unit 13 is communicably connected to each component of the information processing device 10, and controls the operation of the information processing device 10 as a whole.

[0027] The following mainly describes the configuration of the first dialogue device 20 included in the information processing system 1. As shown in Fig. 2, the first dialogue device 20 has a communication unit 21, a storage unit 22, an acquisition unit 23, an input unit 24, an output unit 25, and a control unit 26.

[0028] The communication unit 21 includes a communication module that connects to the network 40. For example, the communication unit 21 includes a communication module that supports mobile communication standards or Internet standards such as 4G and 5G. In one embodiment, the first interaction device 20 is connected to the network 40 via the communication unit 21. The communication unit 21 transmits and receives various information via the network 40.

[0029] The storage unit 22 is, for example, but not limited to, a semiconductor memory, a magnetic memory, or an optical memory. The storage unit 22 functions as a main storage device, an auxiliary storage device, or a cache memory. The storage unit 22 stores any information used in the operation of the first dialogue device 20. The storage unit 22 stores system programs, application programs, and various information received or transmitted by the communication unit 21. The information stored in the storage unit 22 can be updated with information received from the network 40 via the communication unit 21.

[0030] The acquisition unit 23 includes any imaging device capable of capturing an image of the first user speaking. For example, the acquisition unit 23 includes a Web camera. The acquisition unit 23 captures an image of the first user performing a remote conversation using the first conversation device 20. The acquisition unit 23 constantly captures an image of the first user while the remote conversation is being performed.

[0031] The input unit 24 includes one or more input interfaces that detect a user input and acquire input information based on a user operation. For example, the input unit 24 includes physical keys, capacitive keys, a touch screen that is integrated with the display of the output unit 25, and a microphone that accepts voice input.

[0032] The output unit 25 includes one or more output interfaces that output information to notify the user. For example, the output unit 25 includes a display that outputs information as a video, and a speaker, earphones, headphones, etc. that output information as an audio.

[0033] The control unit 26 includes one or more processors. The control unit 26 is communicably connected to each component of the first interaction device 20, and controls the operation of the first interaction device 20 as a whole.

[0034] The following mainly describes the configuration of the second dialogue device 30 included in the information processing system 1. As shown in Fig. 2, the second dialogue device 30 has a communication unit 31, a storage unit 32, an acquisition unit 33, an input unit 34, an output unit 35, and a control unit 36.

[0035] The communication unit 31 includes a communication module that connects to the network 40. For example, the communication unit 31 includes a communication module that supports mobile communication standards or Internet standards such as 4G and 5G. In one embodiment, the second interaction device 30 is connected to the network 40 via the communication unit 31. The communication unit 31 transmits and receives various information via the network 40.

[0036] The storage unit 32 is, for example, but not limited to, a semiconductor memory, a magnetic memory, or an optical memory. The storage unit 32 functions as a main storage device, an auxiliary storage device, or a cache memory. The storage unit 32 stores any information used in the operation of the second dialogue device 30. The storage unit 32 stores system programs, application programs, and various information received or transmitted by the communication unit 31. The information stored in the storage unit 32 can be updated with information received from the network 40 via the communication unit 31.

[0037] The acquisition unit 33 includes any imaging device capable of capturing an image of the second user speaking. For example, the acquisition unit 33 includes a Web camera. The acquisition unit 33 captures an image of the second user performing a remote conversation using the second conversation device 30. The acquisition unit 33 constantly captures an image of the second user while the remote conversation is being performed.

[0038] The input unit 34 includes one or more input interfaces that detect a user input and obtain input information based on a user operation. For example, the input unit 34 includes physical keys, capacitive keys, a touch screen that is integrated with the display of the output unit 35, and a microphone that accepts voice input.

[0039] The output unit 35 includes one or more output interfaces that output information to notify the user. For example, the output unit 35 includes a display that outputs information as a video, and a speaker, earphones, headphones, etc. that output information as an audio.

[0040] The control unit 36 ​​includes one or more processors. The control unit 36 ​​is communicably connected to each component of the second dialogue device 30, and controls the operation of the second dialogue device 30 as a whole.

[0041] Fig. 3 is a sequence diagram for explaining a first example of an information processing method executed by the information processing system 1 of Fig. 1. With reference to Fig. 3, the first example of the information processing method executed by the information processing system 1 of Fig. 1 will be mainly explained. The sequence diagram shown in Fig. 3 mainly shows a basic processing flow of the information processing method when a first user using the first dialogue device 20 speaks to a second user using the second dialogue device 30. Although Fig. 3 shows only two second dialogue devices 30 as an example, the number of second dialogue devices 30 included in the information processing system 1 may be three or more.

[0042] In step S100, the control unit 36 ​​of the second dialogue device 30 used by the one second user acquires an image of the one second user engaging in a remote dialogue, using an imaging device included in the acquisition unit 33. At this time, when the one second user speaks, the control unit 36 ​​acquires the contents of the one second user's speech as audio data together with the image, using a microphone included in the input unit 34.

[0043] In step S101, the control unit 36 ​​of the second dialogue device 30 used by the one second user transmits the video acquired in step S100 to the information processing device 10 via the communication unit 31 and the network 40. At this time, the control unit 36 ​​transmits audio data based on the speech content of the one second user to the information processing device 10 together with the video as necessary. The control unit 13 of the information processing device 10 receives the video acquired by the second dialogue device 30 used by the one second user from the second dialogue device 30 via the network 40 and the communication unit 11. At this time, the control unit 13 receives audio data based on the speech content of the one second user from the second dialogue device 30 together with the video as necessary.

[0044] In step S102, the control unit 36 ​​of the second dialogue device 30 used by the other second user acquires an image of the other second user with whom the remote dialogue is being conducted, using an imaging device included in the acquisition unit 33. At this time, when the other second user speaks, the control unit 36 ​​acquires the contents of the other second user's speech as audio data together with the image, using a microphone included in the input unit 34.

[0045] In step S103, the control unit 36 ​​of the second dialogue device 30 used by the other second user transmits the video acquired in step S102 to the information processing device 10 via the communication unit 31 and the network 40. At this time, the control unit 36 ​​transmits audio data based on the speech content of the other second user together with the video to the information processing device 10 as necessary. The control unit 13 of the information processing device 10 receives the video acquired by the second dialogue device 30 used by the other second user from the second dialogue device 30 via the network 40 and the communication unit 11. At this time, the control unit 13 receives audio data based on the speech content of the other second user together with the video from the second dialogue device 30 as necessary.

[0046] The control unit 13 of the information processing device 10 transmits the video and audio data of the second user acquired in steps S101 and S103 to the first dialogue device 20 via the communication unit 11 and the network 40. As a result, the control unit 13 causes the output unit 25 of the first dialogue device 20 to output the video and audio data of the second user acquired in steps S101 and S103.

[0047] In step S104, the control unit 13 of the information processing device 10 estimates parameters of each conversation partner for the topic of the remote conversation based on the video of each conversation partner, i.e., the video of the second user acquired in steps S101 and S103 during the remote conversation. More specifically, the control unit 13 estimates the parameters of the second user based on a technique such as image analysis from the facial expression and gestures of the second user shown in the video. At this time, the control unit 13 may estimate the parameters of the second user using a learning model constructed by learning parameters corresponding to the user's facial expression and gestures based on learning data in which the user's facial expression and gestures are associated with parameters.

[0048] In step S105, the control unit 13 of the information processing device 10 generates alert information regarding an alert indicating each dialogue partner whose parameter estimated in step S104 is less than the threshold value.

[0049] In step S106, the control unit 13 of the information processing device 10 transmits the alert information generated in step S105 to the first dialogue device 20 via the communication unit 11 and the network 40. The control unit 26 of the first dialogue device 20 receives the alert information generated by the information processing device 10 from the information processing device 10 via the network 40 and the communication unit 21. As a result, the control unit 13 of the information processing device 10 causes the first dialogue device 20 of the first user to display an alert indicating each dialogue partner whose parameter estimated in step S104 is less than the threshold value. The control unit 26 of the first dialogue device 20 displays, on the output unit 25, an alert indicating each dialogue partner whose parameter estimated in step S104 is less than the threshold value, based on the alert information acquired from the information processing device 10.

[0050] In step S107, the control unit 26 of the first dialogue device 20 acquires, using the acquisition unit 23, gaze information when the first user turns his / her gaze toward the dialogue partner for which the alert was displayed in step S106 and speaks, among multiple dialogue partners as the second user displayed on the screen of the output unit 25 of the first dialogue device 20. At this time, the control unit 26 acquires the speech content of the first user as voice data together with the gaze information, using a microphone included in the input unit 24.

[0051] In step S108, the control unit 26 of the first dialogue device 20 transmits the gaze information and voice data acquired in step S107 to the information processing device 10 via the communication unit 21 and the network 40. The control unit 13 of the information processing device 10 receives the gaze information and voice data acquired by the first dialogue device 20 from the first dialogue device 20 via the network 40 and the communication unit 11.

[0052] In step S109, the control unit 13 of the information processing device 10 determines, based on the gaze information acquired in step S108, that the first user has spoken while directing his / her gaze toward the dialogue partner for which the alert was displayed in step S106, among multiple dialogue partners as the second user displayed on the screen of the first dialogue device 20. When the control unit 13 determines in this way, it generates first voice data for outputting the speech content of the first user directed toward the dialogue partner at a high volume on the second dialogue device 30 of the dialogue partner, based on the voice data acquired in step S108.

[0053] In step S110, the control unit 13 of the information processing device 10 generates second voice data based on the voice data acquired in step S108 in response to the determination process in step S109, for outputting the speech content of the first user directed to the above-mentioned dialogue partner at a low volume on the second dialogue device 30 of the dialogue partner other than the first dialogue partner.

[0054] In step S111, the control unit 13 of the information processing device 10 transmits the first voice data of a high volume generated in step S109 to the second dialogue device 30 of the dialogue partner. More specifically, the control unit 13 transmits the first voice data generated in step S109 to the second dialogue device 30 of the second user, who is the dialogue partner to whom the alert was displayed in step S106 and to whom the first user turned his / her gaze and spoke, via the communication unit 11 and the network 40. The control unit 36 ​​of the second dialogue device 30 receives the first voice data generated in the information processing device 10 from the information processing device 10 via the network 40 and the communication unit 31.

[0055] As described above, when the first user turns his / her gaze toward the dialogue partner for which the alert was displayed in step S106 and speaks among multiple dialogue partners as the second user displayed on the screen of the first dialogue device 20, the control unit 13 of the information processing device 10 transmits first voice data at a high volume to the second dialogue device 30 of the dialogue partner. The control unit 36 ​​of the second dialogue device 30 outputs the content of the first user's speech to the second user from the speaker of the output unit 35 or the like at a high volume based on the acquired first voice data.

[0056] In step S112, the control unit 13 of the information processing device 10 transmits the second voice data with a low volume generated in step S110 to the second dialogue device 30 of the dialogue partner other than the dialogue partner mentioned above. More specifically, the control unit 13 transmits the second voice data generated in step S110 to the second dialogue device 30 of the second user as the dialogue partner other than the dialogue partner to whom the first user directed his / her gaze and spoke, via the communication unit 11 and the network 40. The control unit 36 ​​of the second dialogue device 30 receives the second voice data generated in the information processing device 10 from the information processing device 10 via the network 40 and the communication unit 31.

[0057] As a result, the control unit 13 of the information processing device 10 transmits the second voice data at a low volume to the second dialogue device 30 of the dialogue partner other than the dialogue partner to whom the alert was displayed in step S106 and to whom the first user turned his / her gaze and spoke, among the multiple dialogue partners as the second user displayed on the screen of the first dialogue device 20. The control unit 36 ​​of the second dialogue device 30 outputs the contents of the speech of the first user to the second user at a low volume from the speaker of the output unit 35 or the like based on the acquired second voice data.

[0058] Fig. 4 is a sequence diagram for explaining a second example of the information processing method executed by the information processing system 1 of Fig. 1. With reference to Fig. 4, the second example of the information processing method executed by the information processing system 1 of Fig. 1 will be mainly explained. The sequence diagram shown in Fig. 4 mainly shows a basic processing flow of the information processing method when a second user using the second dialogue device 30 speaks to a first user using the first dialogue device 20. Although Fig. 4 shows only two second dialogue devices 30 as an example, the number of second dialogue devices 30 included in the information processing system 1 may be three or more.

[0059] In step S200, the control unit 26 of the first interaction device 20 used by the first user acquires an image of the first user engaging in a remote interaction, using an imaging device included in the acquisition unit 23. At this time, when the first user speaks, the control unit 26 acquires the contents of the first user's speech as audio data together with the image, using a microphone included in the input unit 24.

[0060] In step S201, the control unit 26 of the first dialogue device 20 used by the first user transmits the video acquired in step S200 to the information processing device 10 via the communication unit 21 and the network 40. At this time, the control unit 26 transmits audio data based on the speech content of the first user together with the video to the information processing device 10 as necessary. The control unit 13 of the information processing device 10 receives the video acquired by the first dialogue device 20 used by the first user from the first dialogue device 20 via the network 40 and the communication unit 11. At this time, the control unit 13 receives audio data based on the speech content of the first user together with the video from the first dialogue device 20 as necessary.

[0061] In step S202, the control unit 13 of the information processing device 10 transmits the video and audio data of the first user acquired in step S201 to the multiple second dialogue devices 30 via the communication unit 11 and the network 40. As a result, the control unit 13 causes the output unit 35 of each of the multiple second dialogue devices 30 to output the video and audio data of the first user acquired in step S201.

[0062] In step S203, the control unit 36 ​​of the second dialogue device 30 acquires, using the acquisition unit 33, gaze information when the second user speaks while directing his / her gaze at the first user displayed on the screen of the output unit 35 of the second dialogue device 30. At this time, the control unit 36 ​​acquires the speech content of the second user as voice data together with the gaze information using a microphone included in the input unit 34.

[0063] In step S204, the control unit 36 ​​of the second dialogue device 30 transmits the gaze information and voice data acquired in step S203 to the information processing device 10 via the communication unit 31 and the network 40. The control unit 13 of the information processing device 10 receives the gaze information and voice data acquired by the second dialogue device 30 from the second dialogue device 30 via the network 40 and the communication unit 11.

[0064] In step S205, the control unit 13 of the information processing device 10 determines, based on the gaze information acquired in step S204, that the second user has spoken while directing his / her gaze at the first user displayed on the screen of the second dialogue device 30. When the control unit 13 determines in this way, it generates first voice data for outputting, at a high volume, the speech content directed at the first user corresponding to the dialogue partner of the second user on the first dialogue device 20 of the first user, based on the voice data acquired in step S204.

[0065] In step S206, the control unit 13 of the information processing device 10 generates second voice data based on the voice data acquired in step S204 in response to the determination process in step S205, for outputting the speech content directed to the first user corresponding to the dialogue partner of the second user at a low volume on the second dialogue device 30 of the dialogue partner other than the first user.

[0066] In step S207, the control unit 13 of the information processing device 10 transmits the first voice data of the high volume generated in step S205 to the first dialogue device 20 of the first user as the dialogue partner. More specifically, the control unit 13 transmits the first voice data generated in step S205 to the first dialogue device 20 of the first user as the dialogue partner to which the second user turns his / her gaze and speaks, via the communication unit 11 and the network 40. The control unit 26 of the first dialogue device 20 receives the first voice data generated in the information processing device 10 from the information processing device 10 via the network 40 and the communication unit 21.

[0067] As described above, when the second user speaks while directing his / her gaze at the first user displayed on the screen of the second dialogue device 30, the control unit 13 of the information processing device 10 transmits the first voice data at a loud volume to the first dialogue device 20 of the first user. The control unit 26 of the first dialogue device 20 outputs the contents of the speech of the second user from the speaker of the output unit 25 or the like at a loud volume to the first user based on the acquired first voice data.

[0068] In step S208, the control unit 13 of the information processing device 10 transmits the second voice data with a low volume generated in step S206 to the second dialogue device 30 of a user other than the first user as the dialogue partner and the second user who has spoken. More specifically, the control unit 13 transmits the second voice data generated in step S206 to the second dialogue device 30 of a user other than the first user as the dialogue partner to whom the second user turns his / her gaze and speaks, via the communication unit 11 and the network 40. The control unit 36 ​​of the second dialogue device 30 receives the second voice data generated in the information processing device 10 from the information processing device 10 via the network 40 and the communication unit 31.

[0069] As described above, when the second user speaks while directing his / her gaze at the first user displayed on the screen of the second dialogue device 30, the control unit 13 of the information processing device 10 transmits second voice data at a low volume to the second dialogue device 30 of a user other than the first user and the second user who spoke. The control unit 36 ​​of the second dialogue device 30 outputs the content of the second user's speech at a low volume from a speaker of the output unit 35 to the second user based on the acquired second voice data.

[0070] FIG. 5 is a schematic diagram showing an example of an image displayed on the screen of the output section 25 of the first interaction device 20 of the first user.

[0071] The control unit 26 of the first dialogue device 20 outputs the video and audio data of the second user acquired by using the second dialogue device 30, using the output unit 25. More specifically, the control unit 26 displays the videos of the multiple second users on the screen of the output unit 25. The control unit 26 outputs the audio of the multiple second users from the earphones of the output unit 25.

[0072] At this time, the control unit 26 displays an alert indicating each dialogue partner whose estimated parameter is less than the threshold on the screen of the output unit 25 based on the alert information acquired from the information processing device 10. More specifically, the control unit 13 of the information processing device 10 displays the alert by highlighting the dialogue partners displayed on the screen of the output unit 25 of the first dialogue device 20 of the first user.

[0073] For example, the control unit 26 displays a first alert U1 indicating the second user A whose estimated understanding level is less than a threshold on the screen of the output unit 25. The control unit 26 displays the first alert U1 by highlighting the second user A by surrounding the second user A displayed on the screen of the output unit 25 with a frame.

[0074] For example, the control unit 26 displays a second alert U2 indicating second user B and second user C whose estimated interest levels are less than a threshold on the screen of the output unit 25. The control unit 26 displays the second alert U2 by highlighting each of the second user B and second user C by surrounding each of the second user B and second user C displayed on the screen of the output unit 25 with a frame different from that of the first alert U1.

[0075] The control unit 26 acquires, using the acquisition unit 23, gaze information when the first user turns his / her gaze toward the second user A on whom the first alert U1 is displayed on the screen of the output unit 25 and speaks. At this time, the control unit 26 acquires the content of the first user's speech as audio data together with the gaze information using a microphone included in the input unit 24.

[0076] The control unit 36 ​​of the second dialogue device 30 of the second user A outputs the contents uttered by the first user from the speaker of the output unit 35 or the like to the second user A at a high volume based on the first voice data generated by the information processing device 10 and acquired from the information processing device 10. The control unit 36 ​​of the second dialogue device 30 of a second user other than the second user A outputs the contents uttered by the first user from the speaker of the output unit 35 or the like to the second user at a low volume based on the second voice data generated by the information processing device 10 and acquired from the information processing device 10.

[0077] The control unit 26 acquires, using the acquisition unit 23, gaze information when the first user speaks while directing his / her gaze at the second user B on whom the second alert U2 is displayed on the screen of the output unit 25. At this time, the control unit 26 acquires the content of the first user's speech together with the gaze information as audio data using a microphone included in the input unit 24.

[0078] The control unit 36 ​​of the second dialogue device 30 of the second user B outputs the contents uttered by the first user from the speaker of the output unit 35 or the like to the second user B at a high volume based on the first voice data generated by the information processing device 10 and acquired from the information processing device 10. The control unit 36 ​​of the second dialogue device 30 of the second user other than the second user B outputs the contents uttered by the first user from the speaker of the output unit 35 or the like to the second user at a low volume based on the second voice data generated by the information processing device 10 and acquired from the information processing device 10.

[0079] The above-described processing contents in the information processing system 1 also apply to the second user C for whom the second alert U2 is displayed on the screen of the output unit 25.

[0080] FIG. 6 is a schematic diagram showing an example of an image displayed on the screen of the output section 35 of the second dialogue device 30 of the second user.

[0081] The control unit 36 ​​of the second dialogue device 30 outputs the video and audio data of the first user acquired using the first dialogue device 20, using the output unit 35. More specifically, the control unit 36 ​​displays the video of the first user on the screen of the output unit 35. The control unit 36 ​​outputs the audio of the first user from the earphones of the output unit 35.

[0082] The control unit 36 ​​of the second dialogue device 30 outputs the video and audio data of the other second users acquired using the other second dialogue device 30 using the output unit 35. More specifically, the control unit 36 ​​displays the videos of the multiple other second users on the screen of the output unit 35. The control unit 36 ​​outputs the audio of the multiple other second users from the earphones of the output unit 35.

[0083] The control unit 36 ​​acquires, using the acquisition unit 33, gaze information when the second user speaks while directing his / her gaze at the first user D displayed on the screen of the output unit 35. At this time, the control unit 36 ​​acquires the speech content of the second user as audio data together with the gaze information using a microphone included in the input unit 34.

[0084] The control unit 26 of the first dialogue device 20 of the first user D outputs the contents of the utterance by the second user from a speaker of the output unit 25 or the like to the first user D at a high volume based on the first voice data generated by the information processing device 10 and acquired from the information processing device 10. The control unit 36 ​​of the second dialogue device 30 of a user other than the first user D and the second user who spoke outputs the contents of the utterance by the second user from a speaker of the output unit 35 or the like to the second user at a low volume based on the second voice data generated by the information processing device 10 and acquired from the information processing device 10.

[0085] According to the embodiment described above, the convenience of the dialogue service using the dialogue device used by each of a plurality of users located remotely from each other is improved. The information processing device 10 causes the first dialogue device 20 of the first user to display an alert indicating each dialogue partner whose estimated parameter is less than a threshold value. This allows the first user to visually recognize each dialogue partner whose parameter for the topic in the remote dialogue is low. Therefore, the first user can provide a more detailed explanation to a dialogue partner who has a low level of understanding of the topic in order to improve the level of understanding. The first user can also provide an appropriate follow-up to a dialogue partner who has a low level of interest or concentration in the topic in order to draw more attention to the topic.

[0086] The information processing device 10 transmits first voice data at a high volume to the second dialogue device 30 of the dialogue partner on which the first user directs his / her gaze and speaks, and on which an alert is displayed. On the other hand, the information processing device 10 transmits second voice data at a low volume to the second dialogue device 30 of the dialogue partner other than the dialogue partner. As a result, the voice uttered by the first user is output at a relatively high volume in the second dialogue device 30 of the dialogue partner who is the first user's attention target, while being output at a relatively low volume in the second dialogue device 30 of the dialogue partner other than the attention target. Therefore, for example, when the first user is a teacher, the first user can selectively call out to dialogue partners with low parameters.

[0087] When the second user turns his / her gaze toward the first user and speaks, the information processing device 10 transmits the first voice data at a high volume to the first dialogue device 20 of the first user. The information processing device 10 transmits the second voice data at a low volume to the second dialogue device 30 of a user other than the first user and the second user who spoke. As a result, the voice uttered by the second user is output at a relatively high volume in the first dialogue device 20 of the first user who is the second user's attention target, while being output at a relatively low volume in the second dialogue device 30 of the dialogue partner other than the attention target. Therefore, for example, when the second user is a student, the second user can make it difficult for other students to understand the content of the question by asking a question while paying attention to the teacher on the screen of the second dialogue device 30 of the second user. Therefore, the information processing device 10 can actively encourage the second user to ask a question.

[0088] When the first user includes a teacher who is a topic of remote dialogue and the second user includes a student who is a dialogue partner with the teacher, the information processing system 1 can be used for one-to-many online lesson services.

[0089] The information processing device 10 displays the alert by highlighting the dialogue partner displayed on the screen of the first dialogue device 20 of the first user. This improves the visibility of the alert indicating the predetermined second user. The first user can easily recognize the predetermined second user for whom the alert has been displayed on the screen of his / her first dialogue device 20.

[0090] Since the parameters include at least one of the level of understanding, level of interest, and level of concentration of each conversation partner regarding the topic of the remote conversation, the information processing device 10 can identify a specific second user whose level of understanding, level of interest, and level of concentration is below a threshold value based on at least one of the level of understanding, level of interest, and level of concentration.

[0091] Although the present disclosure has been described based on the drawings and examples, it should be noted that those skilled in the art can make various modifications and alterations based on the present disclosure. Therefore, it should be noted that these modifications and alterations are included in the scope of the present disclosure. For example, the functions included in each configuration or each step can be rearranged so as not to be logically inconsistent, and multiple configurations or steps can be combined or divided into one.

[0092] For example, at least a part of the processing operations executed in the information processing device 10 in the above-mentioned embodiment may be executed in the first dialogue device 20 or the second dialogue device 30. For example, instead of the information processing device 10, the first dialogue device 20 or the second dialogue device 30 itself may execute the above-mentioned series of processing operations related to the information processing device 10. At least a part of the processing operations executed in the first dialogue device 20 or the second dialogue device 30 may be executed in the information processing device 10.

[0093] For example, a general-purpose electronic device such as a smartphone or a computer can be configured to function as the information processing device 10 according to the above-described embodiment. Specifically, a program describing the processing contents for realizing each function of the information processing device 10 according to the embodiment is stored in the memory of the electronic device, and the program is read and executed by a processor of the electronic device. Therefore, the disclosure according to one embodiment can also be realized as a program executable by a processor.

[0094] Alternatively, the disclosure according to an embodiment may be realized as a non-transitory computer-readable medium storing a program executable by one or more processors to cause the information processing device 10 according to the embodiment to execute each function. It should be understood that these are also included in the scope of the present disclosure.

[0095] In the above embodiment, the information processing device 10 transmits the first voice data to the first dialogue device 20 when the second user speaks while directing his / her gaze at the first user, and transmits the second voice data to the second dialogue device 30 of a user other than the first user and the second user who spoke, but is not limited thereto. The information processing device 10 does not need to execute such a process. That is, the information processing system 1 may output the voice of the second user at a similar volume to the dialogue device of a user other than the second user who spoke, even when the second user speaks while directing his / her gaze at the first user.

[0096] In the above embodiment, the first user is described as including a teacher for the topic of the remote dialogue, but is not limited thereto. The first user may include a student who is a dialogue partner with the teacher, or any other user having a relationship other than the relationship between teacher and student. For example, the first user may include a user who is participating online in any conference such as a business conference.

[0097] In the above embodiment, it has been described that the first user who uses the information processing system 1 is one person and the information processing system 1 has only one first dialogue device 20, but this is not limiting. There may be a plurality of first users who use the information processing system 1, and the information processing system 1 may have a plurality of first dialogue devices 20.

[0098] In the above embodiment, the second user is described as including a student who is a conversation partner with a teacher, but is not limited thereto. The second user may include a teacher for the topic of the remote conversation, or any other user having a relationship other than the relationship between a teacher and a student. For example, the second user may include a user who is participating online in any conference such as a business conference.

[0099] In the above embodiment, it has been described that there are multiple second users who use the information processing system 1 and the information processing system 1 has multiple second dialogue devices 30, but this is not limiting. There may be only one second user who uses the information processing system 1, and the information processing system 1 may have only one second dialogue device 30.

[0100] In the above embodiment, the information processing system 1 is described as being used for a service of online classes by one teacher and multiple students, but is not limited thereto. The information processing system 1 may be used for a service of online classes by one teacher and one student, or may be used for a service of online classes by multiple teachers and one or multiple students. In addition, the information processing system 1 may be used for a service of one-to-one, one-to-multiple, multiple-to-one, or multiple-to-multiple online conferences between a first user and a second user.

[0101] In the above embodiment, the information processing device 10 displays an alert by highlighting the dialogue partner displayed on the screen of the first dialogue device 20 of the first user, but the present invention is not limited to this. The information processing device 10 may display an alert by displaying the dialogue partner displayed on the screen of the first dialogue device 20 of the first user by any display means other than highlighting.

[0102] In the above embodiment, the parameters include at least one of the understanding, interest, and concentration of each conversation partner on the topic of the remote conversation, but are not limited thereto. The parameters may include any other index that can objectively grasp the state of each conversation partner on the topic of the remote conversation as a numerical value.

[0103] Some of the embodiments of the present disclosure will be described below as examples. However, it should be noted that the embodiments of the present disclosure are not limited to these examples. [Appendix 1] An information processing device that supports a dialogue using a dialogue device used by each of a plurality of users located remotely from each other, comprising: a control unit that, during a remote conversation, estimates parameters of each conversation partner for a topic of the remote conversation based on a video of each conversation partner, displays an alert on the conversation device of a first user indicating each conversation partner for which the estimated parameters are less than a threshold, and transmits first voice data at a high volume to the conversation device of the conversation partner and transmits second voice data at a low volume to the conversation device of a conversation partner other than the conversation partner when the first user turns his / her gaze toward the conversation partner for which the alert is displayed and speaks among a plurality of conversation partners as a second user displayed on a screen of the conversation device; Information processing device. [Appendix 2] 2. The information processing device according to claim 1, When the second user speaks while directing his / her gaze at the first user, the control unit transmits the first voice data to the interactive device of the first user, and transmits the second voice data to the interactive device of the user other than the first user and the second user who spoke. Information processing device. [Appendix 3] The information processing device according to claim 1 or 2, the first user includes a teacher for the topic of the remote dialogue; The second user includes a student as the conversation partner with the teacher. Information processing device. [Appendix 4] An information processing device according to any one of claims 1 to 3, The control unit displays the alert by highlighting the dialogue partner displayed on the screen of the dialogue device of the first user. Information processing device. [Appendix 5] An information processing device according to any one of claims 1 to 4, The parameters include at least one of the level of understanding, the level of interest, and the level of concentration of each of the conversation partners on the topic of the remote conversation; Information processing device. [Appendix 6] An information processing device according to any one of Supplementary Notes 1 to 5; said interactive device being used by each of a plurality of said users; Equipped with Information processing system. [Appendix 7] An information processing device that supports a dialogue using a dialogue device used by each of a plurality of users located remotely from each other, comprising: During a remote conversation, a parameter of each conversation partner for a topic of the remote conversation is estimated based on an image of each conversation partner, and an alert is displayed on the conversation device of a first user indicating each conversation partner for which the estimated parameter is less than a threshold value, and when the first user turns his / her gaze toward a conversation partner for which the alert is displayed and speaks among a plurality of conversation partners as a second user displayed on a screen of the conversation device, an operation is executed including transmitting a first voice data at a high volume to the conversation device of the conversation partner and transmitting a second voice data at a low volume to the conversation device of a conversation partner other than the conversation partner. program. [Appendix 8] 1. An information processing method for supporting a dialogue using a dialogue device used by each of a plurality of users remotely located from each other, comprising: During a remote conversation, estimating parameters of each conversation partner for a topic of the remote conversation based on a video of each conversation partner, displaying an alert on the conversation device of a first user indicating each conversation partner for which the estimated parameters are less than a threshold, and when the first user turns his / her gaze toward a conversation partner for which the alert is displayed among a plurality of conversation partners as a second user displayed on a screen of the conversation device and speaks, transmitting first voice data at a high volume to the conversation device of the conversation partner and transmitting second voice data at a low volume to the conversation device of a conversation partner other than the conversation partner. Information processing methods. [Explanation of symbols]

[0104] 1. Information Processing Systems 10. Information processing device 11 Communications Department 12 Storage section 13 Control section 20 First dialogue device 21 Communications Department 22 Memory section 23 Acquisition Department 24 Input section 25 Output section 26 Control Unit 30 Second dialogue device 31 Communications Department 32 Storage section 33 Acquisition Department 34 Input section 35 Output section 36 Control Unit 40 Network A, B, C Secondary User D First User U1 First Alert U2 2nd Alert

Claims

1. An information processing device that supports a dialogue using a dialogue device used by each of a plurality of users located remotely from each other, comprising: a control unit that, during a remote conversation, estimates parameters of each conversation partner for a topic of the remote conversation based on a video of each conversation partner, displays an alert on the conversation device of a first user indicating each conversation partner for which the estimated parameters are less than a threshold, and transmits first voice data at a high volume to the conversation device of the conversation partner and transmits second voice data at a low volume to the conversation device of a conversation partner other than the conversation partner when the first user turns his / her gaze toward the conversation partner for which the alert is displayed and speaks among a plurality of conversation partners as a second user displayed on a screen of the conversation device. Information processing device.

2. 2. The information processing device according to claim 1, When the second user speaks while directing his / her gaze at the first user, the control unit transmits the first voice data to the dialogue device of the first user, and transmits the second voice data to the dialogue device of the user other than the first user and the second user who spoke. Information processing device.

3. 3. The information processing device according to claim 1, The interaction device includes a first interaction device used by the first user and a second interaction device used by the second user; The control unit estimates the parameters of a plurality of dialogue partners as the second user using the second dialogue device; displaying an alert on the first interaction device indicating the interaction partner as the second user whose parameter is less than a threshold; the first user includes a teacher for the topic of the remote dialogue; The second user includes a student as the conversation partner with the teacher. Information processing device.

4. 3. The information processing device according to claim 1, The control unit displays the alert by highlighting the dialogue partner displayed on the screen of the dialogue device of the first user. Information processing device.

5. 3. The information processing device according to claim 1, The parameters include at least one of the level of understanding, the level of interest, and the level of concentration of each of the conversation partners on the topic of the remote conversation; Information processing device.

6. An information processing device according to claim 1 or 2; said interactive device being used by each of a plurality of said users; Equipped with Information processing system.

7. An information processing device that supports a dialogue using a dialogue device used by each of a plurality of users located remotely from each other, comprising: During a remote conversation, a parameter of each conversation partner for a topic of the remote conversation is estimated based on an image of each conversation partner, and an alert indicating each conversation partner for which the estimated parameter is less than a threshold is displayed on the conversation device of a first user, and when the first user turns his / her gaze toward a conversation partner for which the alert is displayed and speaks among a plurality of conversation partners as a second user displayed on a screen of the conversation device, an operation is executed including transmitting first voice data at a high volume to the conversation device of the conversation partner and transmitting second voice data at a low volume to the conversation device of a conversation partner other than the conversation partner. program.

8. 1. An information processing method for supporting a dialogue using a dialogue device used by each of a plurality of users remotely located from each other, comprising: During a remote conversation, estimating parameters of each conversation partner for a topic of the remote conversation based on a video of each conversation partner, displaying an alert on the conversation device of a first user indicating each conversation partner for which the estimated parameters are less than a threshold, and when the first user turns his / her gaze toward a conversation partner for which the alert is displayed among a plurality of conversation partners as a second user displayed on a screen of the conversation device and speaks, transmitting first voice data at a high volume to the conversation device of the conversation partner and transmitting second voice data at a low volume to the conversation device of a conversation partner other than the conversation partner. Information processing methods.

Images