Information processing system, information processing method, and program

Abstract

To properly grasp the situation of a remote conference.SOLUTION: A ratio information acquisition unit acquires ratio information indicating the utterance ratios of participants in a conference. A concentration degree calculation unit calculates concentration degrees of the participants in the conference. An output control unit outputs information based on the ratio information and the concentration degrees.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to an information processing system, an information processing method, and a program. [Background technology]

[0002] There is known a technology for multiple users to hold a remote conference via a network. The remote conference is held by each participant transmitting audio and video using a microphone and a camera. The remote conference may also be held using avatars in a virtual space (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-71314 Summary of the Invention [Problem to be solved by the invention]

[0004] Unlike meetings where all participants are present in person, remote meetings make it difficult to grasp the level of involvement of each participant, making it difficult to grasp the situation. Therefore, there is a need to be able to properly grasp the situation of remote meetings.

[0005] The present disclosure has been made in view of the above, and aims to provide an information processing system, an information processing method, and a program that are capable of appropriately grasping the status of a remote conference. [Means for solving the problem]

[0006] The information processing system according to the present disclosure includes a percentage information acquisition unit that acquires percentage information indicating the proportion of participants speaking in a conference, a concentration calculation unit that calculates the concentration level of the participants in the conference, and an output control unit that outputs information based on the percentage information and the concentration level.

[0007] The information processing method according to the present disclosure includes steps of acquiring percentage information indicating the proportion of speeches made by participants in a conference, calculating the degree of concentration of the participants in the conference, and outputting information based on the percentage information and the degree of concentration.

[0008] The program of the present disclosure causes a computer to execute the steps of acquiring percentage information indicating the proportion of participants speaking in a meeting, calculating the concentration level of the participants in the meeting, and outputting information based on the percentage information and the concentration level. [Effects of the Invention]

[0009] According to the present disclosure, it is possible to provide an information processing system, an information processing method, and a program that are capable of appropriately grasping the status of a remote conference. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic diagram of an information processing system according to the first embodiment. [Figure 2] FIG. 2 is a block diagram showing an example of the configuration of the information processing system according to the first embodiment. [Figure 3] FIG. 3 is a block diagram conceptually showing an example of the configuration of the information processing means according to the first embodiment. [Figure 4] FIG. 4 is a diagram showing an example of information output destinations and information contents according to the speech rate of the first participant and the concentration level of the participants. [Figure 5] FIG. 5 is a diagram showing the format of a conference when the speech rate of the first participant is equal to or greater than the first threshold. [Figure 6] FIG. 6 is a diagram showing the format of a conference when the speech rate of the first participant is less than the second threshold. [Figure 7] FIG. 7 is a flowchart illustrating the processing flow of the information processing system according to the first embodiment. [Figure 8] FIG. 8 is a schematic diagram of an information processing system according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Note that the present disclosure is not limited to these embodiments, and when there are multiple embodiments, the present disclosure also includes configurations in which the respective embodiments are combined.

[0012] (First embodiment) The information processing system will be described with reference to Fig. 1. Fig. 1 is a schematic diagram of the information processing system according to the first embodiment.

[0013] The information processing system 100 includes multiple information terminals 10 and a server device 30. The information processing system 100 is used in conference communication in which multiple participants P participate online. In other words, the information processing system 100 is used in a remote conference system. Each information terminal 10 communicates with other information terminals 10 using audio and video, or audio only, via a server device 30 that provides an application service that realizes the remote conference. The information terminal 10 is not particularly limited as long as it is an information terminal device used for online communication, and may be, for example, an information terminal such as a desktop or notebook personal computer, a smartphone, or a tablet terminal. The information terminal 10 is not limited to the notebook personal computer illustrated in FIG. 1.

[0014] FIG. 2 is a block diagram showing an example of the configuration of the information processing system according to the first embodiment.

[0015] (Information terminal) Each information terminal 10 has an imaging unit 11, an audio input unit 12, an operation unit 13, a display unit 14, an audio output unit 15, and a control unit 16. The imaging unit 11, the audio input unit 12, the operation unit 13, the display unit 14, and the audio output unit 15 may each be an externally connected device connected to the information terminal 10, or may be a built-in device assembled into the information terminal 10.

[0016] The photographing unit 11 photographs the user of the information terminal 10 and generates video information. The photographing unit 11 outputs the generated video information to the control unit 16. The photographing unit 11 is configured with a photographing device such as a visible light camera, for example.

[0017] The audio input unit 12 collects audio such as the voice of the user of the information terminal 10 and sounds around the user, and generates audio information. The audio input unit 12 transmits the generated audio information to the control unit 16. The audio input unit 12 is configured with a sound collection device such as a microphone.

[0018] The operation unit 13 receives various operations from the user on the information terminal 10 and outputs them as operation signals to the control unit 16. As the operation unit 13, for example, an input device such as a mouse, a keyboard, a touch panel, a button, a lever, a dial, or a switch is used.

[0019] The display unit 14 displays various types of information. Examples of the display unit 14 include a liquid crystal display and an organic EL (Electro-Luminescence) display. The display unit 14 may be, for example, a head-mounted display that is worn on the user's head. The display unit 14 displays an image based on a video signal output from the control unit 16.

[0020] The audio output unit 15 is a device that outputs various types of audio information. The audio output unit 15 may be a speaker or headphones that are externally connected to the control unit 16, or may be a speaker built into a housing that houses the control unit 16. The audio output unit 15 outputs audio based on an audio signal output from the control unit 16.

[0021] The control unit 16 comprehensively controls the operation of the information terminal 10. The control unit 16 includes a communication unit 17, a processing unit 18, and a storage unit 19.

[0022] The communication unit 17 communicates with external devices via wired or wireless means. The communication unit 17 communicates with the server device 30. The communication unit 17 outputs video information, audio information, display control information, etc. received from the server device 30 to the processing unit 18. The video information is information for displaying the video of the participant P output from each information terminal 10 on the display unit 14. The audio information is information for outputting the audio of the participant P output from each information terminal 10 from the audio output unit 15. The display control information is information for displaying various information such as a user interface, text, and images on the display unit 14.

[0023] The processing unit 18 performs various processes. The processing unit 18 controls the operation of various applications running on the information terminal 10. The processing unit 18 includes, for example, a processor such as a central processing unit (CPU) or a micro processing unit (MPU), and a storage device such as a random access memory (RAM) or a read-only memory (ROM). The processing unit 18 may be implemented by an integrated circuit such as an application specific integrated circuit (ASIC) or a field-programmable gate array (FPGA). The processing unit 18 may be implemented by a combination of hardware and software. The processing unit 18 executes a program that controls the operation of the information terminal 10 according to the present disclosure. The processing unit 18 controls the operation of applications used in a remote conference. The processing unit 18 may operate the applications used in a remote conference in a distributed manner in cooperation with the server device 30.

[0024] The processing unit 18 controls the display unit 14 and the audio output unit 15 to realize application screen display and audio display for communication based on the video information, audio information, display control information, etc. acquired from the server device 30 by the communication unit 17. The processing unit 18 also generates transmission data to be used for communication based on the video information from the shooting unit 11, the audio information from the audio input unit 12, and the operation signal from the operation unit 13, etc. The processing unit 18 transmits the generated transmission data to the server device 30 via the communication unit 17.

[0025] In this embodiment, the processing unit 18 acquires operation information of the user of the information terminal 10 (i.e., participant P). The operation information may be any information relating to an action performed by the participant P on the information terminal 10. The operation information according to this embodiment includes information relating to an input operation performed on the operation unit 13 by the participant P to operate the information terminal 10. Furthermore, the operation information includes, among the input operations performed on the operation unit 13, information relating to an operation on a user interface (icons, text input fields, etc.) of an application screen used in the remote conference. The operation information is one indicator of the participant P's concentration on the conference.

[0026] The processing unit 18 may also acquire biometric information of the participant P. The biometric information of the participant P acquired by the processing unit 18 may be any biometric information such as brain waves or pulse rate. The processing unit 18 according to this embodiment acquires brain wave information of the participant P from an brain wave sensor (not shown) attached to the participant P. The biometric information of the participant P acquired by the processing unit 18 may be acquired from a sensor other than a dedicated sensor for acquiring biometric information. For example, the biometric information may be acquired from video information of the participant P captured by the imaging unit 11. The biometric information acquired from the video information may be, for example, the participant P's line of sight, the frequency of opening and closing their eyes (blinking), the participant P's posture, etc. The biometric information is one indicator of the participant P's concentration on the meeting.

[0027] The storage unit 19 stores various types of information. The storage unit 19 stores programs, data, etc. for performing various processes in the processing unit 18. The storage unit 19 is configured with a storage device such as an SSD (Solid State Drive) or an HDD (Hard Disk Drive), for example.

[0028] (Server device) The server device 30 includes a communication unit 31 , a processing unit 32 , and a storage unit 33 .

[0029] The communication unit 31 is a communication unit for performing wired communication or wireless communication. The communication unit 31 performs communication using a communication method such as Wi-Fi or a mobile phone line, and is connected to an intranet or the Internet. The communication unit 31 communicates with the information terminal 10. More specifically, the communication unit 31 receives video information, audio information, operation information, biometric information, etc. of the participant P from the information terminal 10.

[0030] The processing unit 32 controls each unit of the server device 30. The processing unit 32 includes, for example, a processor such as a CPU or an MPU, and a storage device such as a RAM or a ROM. The processing unit 32 may be realized by, for example, an integrated circuit such as an ASIC or an FPGA. The processing unit 32 may be realized by a combination of hardware and software. The processing unit 32 executes a program that controls the operation of the information processing system 100 according to the present disclosure.

[0031] The processing unit 32 controls the communication unit 31 to execute communication between the server device 30 and the information terminal 10. The processing unit 32, for example, controls the communication unit 31 to receive video and audio of the participant P from the information terminal 10. The processing unit 32, for example, controls the communication unit 31 to transmit video and audio of the participant P and video of the shared screen materials to the information terminal 10.

[0032] The storage unit 33 stores information such as the calculation contents and programs of the processing unit 32. The storage unit 33 is configured with a storage device such as an SSD or HDD.

[0033] (Information processing means) 3 is a block diagram conceptually illustrating an example of the configuration of an information processing means according to this embodiment. The information processing means 40 performs information processing to assist in understanding the status of a remote conference. As shown in FIG. 3, the information processing means 40 includes a ratio information acquisition unit 41, a concentration degree calculation unit 42, and an output control unit 43.

[0034] The information processing means 40 is a conceptual information processing means realized by cooperation between either or both of the information terminal 10 (processing unit 18) and the server device 30 (processing unit 32) in the information processing system 100 according to this embodiment. Therefore, either the information terminal 10 or the server device 30 may be referred to as the information processing means 40, and the information processing system 100 including the information terminal 10 and the server device 30 may be referred to as the information processing means 40. Each unit of the information processing means 40 is realized by information processing in either or both of the processing unit 18 and the processing unit 32. In this embodiment, as an example, the processing unit 32 of the server device 30 executes a program stored in the storage unit 33 to realize the function of the information processing means 40. That is, the program causes the computer (the processor constituting the processing unit 32) to function as a ratio information acquisition unit 41, a concentration degree calculation unit 42, and an output control unit 43.

[0035] The program is not limited to being stored in a storage (storage unit 19, storage unit 33). For example, the program may be distributed to the information processing means via a network. Alternatively, the program may be read from an external recording medium and distributed to the information processing means.

[0036] (Ratio information acquisition section) The proportion information acquisition unit 41 acquires proportion information indicating the proportion of speech made by each participant P in the conference. The speech proportion is the proportion of speech time made by each participant P to the total speech time made in the conference. The speech time is the length of time during which a speech is made. There are no particular limitations on the method for calculating the speech proportion, but for example, the speech proportion may be calculated as the sum of the speech times of the participants P to be calculated divided by the total speech time of all participants P. The total speech time is the total amount of time that any participant P spoke in the conference. The proportion information is an index that indicates how much each participant P participating in the conference spoke throughout the entire conference.

[0037] The ratio information acquisition unit 41 may acquire the ratio information by any method. For example, the ratio information acquisition unit 41 according to this embodiment records the speech time of each participant P during a remote conference and calculates the ratio information based on the recorded speech time information. That is, when each participant P speaks, audio information collected from each participant P is transmitted from the respective information terminals 10 to the server device 30. The server device 30 processes the audio information of each participant P using an application used for the remote conference and transmits the processed audio information to each information terminal 10. As a result, the speeches of the other participants P are output as audio from each information terminal 10. The ratio information acquisition unit 41 records the speech time of each participant P using the information terminal 10 based on the audio information transmitted from the information terminal 10 to the server device 30. Similarly, the ratio information acquisition unit 41 calculates the total speech time of all participants P based on the audio information transmitted from each information terminal 10 to the server device 30. The ratio information acquisition unit 41 acquires the speech ratio of each participant P by dividing the speech time of each participant P by the total speech time. Instead of server device 30 calculating the total speech time of each participant P from the audio information, each information terminal 10 may record the speech time and transmit information on the total speech time to server device 30. That is, processing unit 18 of information terminal 10 may record the speech time of participant P over time based on the audio information transmitted to server device 30 and transmit the recorded total speech time to server device 30. Ratio information acquisition unit 41 acquires the speech time of participant P using information terminal 10 from the numerical information on speech time transmitted from that information terminal 10.

[0038] (Concentration level calculation section) The concentration level calculation unit 42 calculates the concentration level of the participant P in the conference. For each participant P in the conference, the concentration level calculation unit 42 calculates the concentration level of the participant P based on at least one of the operation information of the information terminal 10 and the biometric information of the participant P. The concentration level of the participant P here is an index that indicates the degree of concentration of the participant P in the conference. The higher the concentration level of the participant P, the more the participant P tends to pay attention to what other participants P say and to be more conscious of being involved in the conference. The concentration level calculation unit 42 according to this embodiment acquires the operation information and the biometric information from the information terminal 10, and calculates the concentration level of the participant P based on this information.

[0039] Any method may be used to calculate the concentration level of the participant P, but the concentration level calculation unit 42 according to this embodiment averages the concentration level calculated from the operation information and the concentration level calculated from the biometric information, and calculates this average as the concentration level of the participant P. The concentration level calculation unit 42 may also assign different weights to the concentration level calculated from the operation information and the concentration level calculated from the biometric information, and calculate a weighted average. For example, the weight of the concentration level calculated from the biometric information may be greater than the weight of the concentration level calculated from the operation information.

[0040] Note that any method for calculating the concentration level from the operation information may be used, and in this embodiment, the concentration level calculation unit 42 calculates the concentration level of the participant P based on input frequency information, which is part of the operation information and indicates the number of times that the participant P inputs operation information via the operation unit 13 of the information terminal 10 within a predetermined period of time. For example, the higher the input frequency of the participant P into the operation unit 13 during a conference, the less proactive the participant P is in speaking in the conference and the more likely the participant P is behaving as a listener. In this case, the concentration level calculation unit 42 calculates a higher concentration level the fewer times that the participant P inputs operation information within a predetermined period of time.

[0041] Furthermore, any method for calculating the concentration level from the biometric information may be used. In this embodiment, the concentration level calculation unit 42 calculates the concentration level based on the brain wave information of the participant P in a suitable state, which is stored in advance in a storage unit (the storage unit 19 or the storage unit 33), and the brain wave information acquired by the processing unit 18. The "suitable state" here refers to a state in which the participant P's brain is moderately tense and the participant P is able to concentrate on the meeting. For example, if the participant P feels sleepy, it is considered that the participant P's brain is overly relaxed and unable to concentrate on the meeting. Conversely, if the participant P is in a brain activity state that causes excessive tension, it is considered that the participant P is unable to actively speak up in the meeting or concentrate on what other participants P are saying. For this reason, brain wave information when the participant P is in an appropriate state for the meeting is acquired in advance. The concentration level calculation unit 42 calculates the difference between the brain wave information of the participant P in a suitable state and the biometric information (brain wave information) acquired by the processing unit 18, and calculates the participant P's concentration level based on this difference. In other words, the smaller the calculated difference, the higher the concentration level calculated by the concentration level calculation unit 42.

[0042] (Output control section) The output control unit 43 outputs information based on the ratio information and the concentration level. The output control unit 43 outputs a control signal to notify the participant P of the information on the remote conference application. Upon receiving the control signal from the output control unit 43, the information terminal 10 displays the information on the display unit 14 or outputs the information as synthesized voice or the like by the audio output unit 15. Any information may be output, but in this embodiment, the output control unit 43 outputs information representing the status of the conference. The information output by the output control unit 43 represents, for example, the status of the participant P's concentration level on the conference. The output control unit 43 may also output information to improve the status of the conference.

[0043] The output control unit 43 according to this embodiment outputs information to a participant P determined based on the ratio information, with content determined according to the ratio information and the concentration level. That is, the output control unit 43 determines a participant P who is a target (output destination) to which information is to be output, based on the ratio information acquired by the ratio information acquisition unit 41. The output control unit 43 determines the content of information to be presented to the output destination, based on the ratio information acquired by the ratio information acquisition unit 41 and the concentration level calculated by the concentration level calculation unit 42.

[0044] Here, in determining the output destination of information based on the ratio information, the output control unit 43 classifies the participants P of the conference into a first participant P1 and a second participant P2. In the following, it is assumed that there are at least three participants P in a remote conference. The first participant P1 is the participant P who has the highest speaking ratio among the participants P attending the conference. The second participant P2 is all participants P other than the first participant P1. Typically, a remote conference will have one first participant P1 and N-1 second participants P2 among N participants P. If there are two or more participants who have the same highest speaking ratio, there may be two or more first participants P1.

[0045] The output control unit 43 according to this embodiment changes the output destination of the information to be output depending on the speech rate of the first participant P1, who has the largest speech rate among the participants P. Specifically, when the speech rate of the first participant P1 is equal to or greater than a first threshold Th1, the output control unit 43 outputs information to the first participant P1. When the speech rate of the first participant P1 is less than a second threshold Th2, which is equal to or less than the first threshold Th1, the output control unit 43 outputs information to both the first participant P1 and a second participant P2 other than the first participant P1.

[0046] FIG. 4 is a diagram showing an example of information output destinations and information contents according to the speech rate of the first participant and the concentration level of the participants. FIG. 5 is a diagram showing the format of a conference when the speech rate of the first participant is equal to or greater than a first threshold. FIG. 6 is a diagram showing the format of a conference when the speech rate of the first participant is less than a second threshold. In FIGS. 5 and 6, arrows between participants P conceptually show the speech of participant P. One-way arrows indicate a relationship in which one participant P makes a unilateral statement to another participant P. Two-way arrows indicate a relationship in which one participant P and another participant P make statements to each other.

[0047] When the speech rate of the first participant P1 is equal to or greater than the first threshold Th1, this indicates that the majority of speeches in the conference were made by the first participant P1. Such a conference can be considered a lecture-style conference in which the first participant P1 unilaterally speaks to all the other second participants P2, as shown in FIG. 5 . The first threshold Th1 is preset as a predetermined value higher than, for example, 50%. In this embodiment, the first threshold Th1 is, for example, 60%. In a lecture-style conference, each participant P is clearly divided into a speaker (i.e., the first participant P1) and a listener (i.e., the second participant P2). In a lecture-style conference, the first participant P1, who is the speaker, usually plays the role of controlling the conference. Therefore, when the speech rate of the first participant P1 is equal to or greater than the first threshold Th1, the output control unit 43 outputs information to the first participant P1 and does not output information to the second participant P2. This allows information about the status of the conference to be presented to the first participant P1, who is the conference facilitator.

[0048] A case where the speech rate of the first participant P1 is less than the second threshold Th2 indicates that speech in the conference is not excessively biased toward a specific participant P, and the speech rates among the multiple participants P are within a relatively narrow range. Such a conference can be considered a conference in which each participant P speaks to each other (for convenience, referred to here as a free speech format), as shown in FIG. 6 . The second threshold Th2 is preset as a predetermined value less than the first threshold Th1. In this embodiment, the second threshold Th2 is, for example, 30%. The value of the second threshold Th2 may vary depending on the number of participants P. For example, the larger the number of participants P, the smaller the value of the second threshold Th2. In the free speech format, there is no difference in roles between the first participant P1 and the second participant P2, and it is desirable for each participant P to contribute to livening up the conference. Therefore, when the speech rate of the first participant P1 is less than the second threshold Th2, the output control unit 43 outputs information to both the first participant P1 and the second participant P2. This allows information about the conference status to be presented to each participant P.

[0049] When the speech rate of the first participant P1 is less than the first threshold Th1 and greater than or equal to the second threshold Th2, the conference can be said to be an intermediate format between a lecture format and a free speech format. When the speech rate of the first participant P1 is less than the first threshold Th1 and greater than or equal to the second threshold Th2, it is optional and not particularly limited whether the information is output to the first participant P1 or to all participants P. Furthermore, in one example of this embodiment, the first threshold Th1 and the second threshold Th2 are different, but the first threshold Th1 and the second threshold Th2 may also be the same value. For example, the first threshold Th1 and the second threshold Th2 may both be 60%.

[0050] The output control unit 43 determines the content of information to be presented to the output destination based on a comparison between the concentration level and a threshold. The output control unit 43 may also vary the content of the information to be presented depending on the output destination of the information based on the ratio information. The content of the information to be presented to the output destination is not particularly limited. For example, when the concentration level of the participant P is high overall, the output control unit 43 outputs information indicating that the conference situation is good. For example, when the concentration level of the participant P is low overall, the output control unit 43 outputs information indicating that the conference situation is bad. Specifically, when the concentration level is equal to or greater than a third threshold Th3, the output control unit 43 may output information indicating that the concentration level of the participant P is high and the conference situation is good. The third threshold Th3 is a threshold that distinguishes between a state in which the concentration level of the participant P is high and a state in which it is not high. Furthermore, when the concentration level is less than a fourth threshold Th4, the output control unit 43 may output information indicating that the concentration level of the participant P is low and the conference situation is bad. The fourth threshold Th4 is a threshold that distinguishes between a state in which the concentration level of the participant P is low and a state in which it is not low.

[0051] In addition, when the concentration level of participant P is less than the third threshold Th3 and equal to or greater than the fourth threshold Th4, it can be said that the concentration level is in an intermediate state that cannot be said to be high or low. When the concentration level of participant P is less than the third threshold Th3 and equal to or greater than the fourth threshold Th4, information may be output to the output destination, or information may not be output. When information is output to the output destination, the content of the information to be output is arbitrary and is not particularly limited. The third threshold Th3 and the fourth threshold Th4 may be different or may be the same value.

[0052] The method for comparing the concentration level with the threshold is not particularly limited. For example, the output control unit 43 may calculate a representative value serving as an index of the concentration levels of all participants P from the concentration levels of each participant P calculated by the concentration level calculation unit 42, and compare the calculated representative value with the third threshold Th3 and the fourth threshold Th4. The representative value of the concentration level may be, for example, the average value of the concentration levels of each participant P. Alternatively, for example, the output control unit 43 may compare the concentration level of each participant P with the third threshold Th3 and the fourth threshold Th4 individually, and use the larger of the number of participants P whose concentration levels are equal to or greater than the third threshold Th3 or the number of participants P whose concentration levels are less than the fourth threshold Th4 to determine whether the concentration level of the entire conference is equal to or greater than the third threshold Th3 or less than the fourth threshold Th4.

[0053] (Specific examples of combinations of percentage information values ​​and concentration values) The output control unit 43 varies the content of the information to be output depending on the combination of the value of the ratio information and the value of the concentration level. Examples of the combinations will be described below with reference to FIG.

[0054] The output control unit 43 according to this embodiment outputs first information to the information terminal 10 of the first participant P1, which is the destination of the information, when the speech rate of the first participant P1 is equal to or greater than the first threshold Th1 and the concentration level of the participant P is equal to or greater than the third threshold Th3. The first information is not particularly limited as long as it reflects a state in which the concentration level of the participant P is high, but in this embodiment, it is information indicating that the concentration level is high and the conference situation is good.

[0055] The output control unit 43 according to the present embodiment outputs second information to the information terminal 10 of the first participant P1, which is the destination of the information, when the speech rate of the first participant P1 is equal to or greater than the first threshold Th1 and the level of concentration of the participant P is less than the fourth threshold Th4. The second information is information different from the first information. The second information is not particularly limited as long as it reflects the low level of concentration of the participant P. In the present embodiment, however, the second information is information indicating that the second participant P2 is not concentrating on the speech of the first participant P1 (is not listening carefully to the speech) due to the low level of concentration. Note that in a lecture-style conference in which the speech rate of the first participant P1 is equal to or greater than the first threshold Th1, when the level of concentration of the participant P is low, the output control unit 43 may output supplemental information to all second participants P2 in addition to outputting the second information to the first participant P1. For example, the output control unit 43 may output supplemental information to the information terminal 10 of the second participant P2 to encourage them to concentrate on the speech of the first participant P1. The output control unit 43 may determine whether or not to output auxiliary information intended for the second participant P2 based on a control signal transmitted from the information terminal 10 of the first participant P1 or on pre-setting information.

[0056] In this embodiment, when the speech rate of the first participant P1 is less than the second threshold Th2 and the participant P's concentration level is equal to or greater than the third threshold Th3, the output control unit 43 does not output information because the conference is progressing smoothly. In other words, when the speech rate of the first participant P1 is less than the second threshold Th2 and the participant P's concentration level is equal to or greater than the third threshold Th3, the output control unit 43 outputs no information. This prevents the participant P from lowering their concentration level by presenting unnecessary information to them when they are concentrating in a free-communication conference. Note that when the speech rate of the first participant P1 is less than the second threshold Th2 and the participant P's concentration level is equal to or greater than the third threshold Th3, the output control unit 43 may output third information to the information terminal 10 of each participant P, which is the destination of the information, when the speech rate of the first participant P1 is less than the second threshold Th2 and the participant P's concentration level is equal to or greater than the third threshold Th3. The third information is not particularly limited as long as it indicates that the participant P's concentration level is high. The third information is, for example, information indicating that the concentration of participants P is high and the progress of the conference is good.

[0057] When the speech rate of the first participant P1 is less than the second threshold Th2, the output control unit 43 according to the present embodiment outputs information according to the duration of the participant P's low concentration state. The low concentration state refers to a state in which the participant P's concentration level is less than a fourth threshold Th4. Specifically, the output control unit 43 measures the duration of the state in which the participant P's concentration level is less than the fourth threshold Th4. When the state in which the participant P's concentration level is less than the fourth threshold Th4 continues for a predetermined time or longer, the output control unit 43 outputs fourth information to the information terminals 10 of the participants P, to which the information is output. The fourth information is information different from the first information and the third information. The fourth information is not particularly limited as long as it indicates a state in which the participant P's concentration level is low. In the present embodiment, the fourth information indicates that the participant P's concentration level is low and the meeting is stagnating. The fourth information may also include information recommending a change of topic. In the present embodiment, the output control unit 43 outputs, as the fourth information, information indicating that the meeting is stagnating and information recommending a change of topic. When the concentration of each participant P is low in a free-speech meeting, information indicating that the meeting is stagnating or information recommending changing the topic is output, thereby encouraging the stagnant meeting to be revitalized.

[0058] Note that the output control unit 43 may immediately output the fourth information without considering the duration when the speech rate of the first participant P1 is less than the second threshold Th2 and the concentration level of the participant P is less than the fourth threshold Th4. However, in a free-comment conference, even if the concentration level of each participant P temporarily drops, the situation of the conference is likely to change, such as when a speech by any participant P triggers the conference to become more active. If the condition for information output is that the state in which the concentration level of the participant P remains less than the fourth threshold Th4 for a predetermined period of time or more, it is possible to more accurately determine whether the conference is essentially stagnating, and to output information at an appropriate time.

[0059] (Processing flow) Next, a processing flow of processing for outputting information during a conference according to this embodiment will be described. Fig. 7 is a flowchart illustrating the processing flow of the information processing system according to this embodiment. As shown in Fig. 7, the ratio information acquisition unit 41 acquires ratio information indicating the speech ratio of a participant P in the conference (step S1). The concentration degree calculation unit 42 calculates the concentration degree of the participant P in the conference (step S2).

[0060] After step S2, the output control unit 43 outputs information based on the ratio information and the concentration level. First, the output control unit 43 determines the output destination of the information and the content of the information to be output based on the ratio information and the concentration level (step S3). That is, as shown in FIG. 4, if the ratio information acquired by the ratio information acquisition unit 41 is equal to or greater than the first threshold Th1 and the concentration level calculated by the concentration level calculation unit 42 is equal to or greater than the third threshold Th3, the output control unit 43 determines the output destination of the information to the first participant P1 and sets the content of the information to be output as first information (information indicating that the concentration level is high and the conference situation is good). If the ratio information acquired by the ratio information acquisition unit 41 is equal to or greater than the first threshold Th1 and the concentration level calculated by the concentration level calculation unit 42 is less than the fourth threshold Th4, the output control unit 43 determines the output destination of the information to the first participant P1 and sets the content of the information to be output as second information (information indicating that the concentration level is low and the second participant P2 is not concentrating on the remarks of the first participant P1). When the ratio information acquired by the ratio information acquisition unit 41 is less than the second threshold Th2 and the concentration level calculated by the concentration level calculation unit 42 is equal to or greater than the third threshold Th3, the output control unit 43 does not output information. That is, the output control unit 43 outputs information to all participants P and sets the content of the information to be null. The output control unit 43 may output the third information (information indicating that the concentration level of the participants P is high and the progress of the conference is going well) as the information to be output. When the ratio information acquired by the ratio information acquisition unit 41 is less than the second threshold Th2 and the state in which the concentration level satisfies a predetermined condition (a state in which the concentration level is less than the fourth threshold Th4) continues for a predetermined time or more, the output control unit 43 outputs the information to all participants P and sets the content of the information to be fourth information (information indicating that the concentration level is low and the conference is stagnating, and information recommending changing the topic).

[0061] The output control unit 43 outputs the information determined in step S3 to the information terminal 10 of the participant P, which is the output destination of the information determined in step S3, with the content determined in step S3 (step S4). As a result, the information terminal 10 that has received the information presents the information to the participant P by displaying the information on the display unit 14, outputting audio from the audio output unit 15, etc.

[0062] (effect) As described above, the information processing system 100 according to this embodiment acquires percentage information indicating the proportion of speeches made by the participant P in the conference. In addition, the information processing system 100 calculates the concentration level of the participant P in the conference. Then, the information processing system 100 outputs information based on the percentage information and the concentration level. According to the present disclosure, the participant P in a remote conference via a network can appropriately grasp the status of the remote conference.

[0063] (Second embodiment) Next, a second embodiment will be described. Fig. 8 is a schematic diagram of an information processing system according to the second embodiment. The information processing system 100 according to the first embodiment is a remote conference system that realizes a remote conference via a network using an information terminal 10 such as a personal computer, a smartphone, or a tablet terminal. In contrast, the information processing system 100 according to the second embodiment provides a virtual space V to participants P, enabling the participants P to hold a conference with each other in the virtual space V.

[0064] The information processing system 100 provides a virtual space V to a participant P using an information terminal 10 and a server device 30. The information terminal 10 is an information terminal used by the participant P, and is a head-mounted display in this embodiment. The information terminal 10 provides the virtual space V to the participant P by displaying an image generated by the server device 30.

[0065] The server device 30 generates an image of the virtual space V and manages the information terminals 10 used by each participant P. The server device 30 manages the image of the virtual space V by each information terminal 10 by performing information communication with each information terminal 10. The number of information terminals 10 that perform information communication with the server device 30 may be arbitrary. In other words, the number of information terminals 10 connected to the information processing system 100 is determined according to the number of participants P who access the virtual space V.

[0066] The virtual space V is not a real space (actual space) in which the participant P actually exists, but a space in which the participant P does not exist. The virtual space V is a virtual space that does not actually exist, i.e., VR (Virtual Reality), but may also be a space that reproduces a real place away from the location where the participant P exists, i.e., MR (Mixed Reality). The information terminal 10 provides the virtual space V according to the movement (line of sight) of the participant P in the real space. That is, based on the image of the virtual space V generated by the information processing system 100, the information terminal 10 displays an image of the virtual space V as seen from the avatar U of the participant P of the information terminal 10, assuming that the avatar U exists in the virtual space V. Therefore, the participant P can recognize that he or she exists in the virtual space V. However, the information terminal 10 is not limited to displaying a subjective image, and may also display an image of the virtual space V from any field of view (for example, a bird's-eye view image of the virtual space V).

[0067] There are no particular limitations on the type of space that the virtual space V is constructed and displayed as, but in this embodiment, the virtual space V may be, for example, a conference space where multiple avatars U can gather. The avatars U can move freely within the virtual space V and can converse with each other. Participants P using the information terminal 10 are depicted as avatars U in the virtual space V. Each participant P corresponds uniquely to each avatar U in the virtual space V. The utterances of the participant P picked up by the audio input unit 12 of the information terminal 10 are displayed and output as audio on the information terminal 10 of each participant P as utterances from the avatar U operated by that participant P in the virtual space V. The utterances of the participant P and the avatar U operated by that participant P are the same.

[0068] The ratio information acquisition unit 41 records the speech time of the avatar U in the virtual space V. The ratio information acquisition unit 41 acquires ratio information indicating the speech ratio of the participant P to the conference from the speech time of each avatar U. The concentration level calculation unit 42 calculates the concentration level of each participant P in the conference based on at least one of the operation information of the information terminal 10 and the biometric information of the participant P. The operation information of the information terminal 10 is, for example, information for operating the avatar U in the virtual space V. The operation information of the information terminal 10 may be, for example, information on an operation performed on an input interface (such as a keyboard displayed in the space) virtually displayed in the virtual space V. In this case, the information terminal 10 may include, as the operation unit 13, a motion detection device (such as a hand tracking device) that detects the motion of the participant P. The output control unit 43 outputs information based on the ratio information and the concentration level. Based on the ratio information and the concentration level, the output control unit 43 outputs information with content corresponding to the concentration level to the participant P determined based on the ratio information.

[0069] For example, when the speech rate of the first participant P1 is equal to or greater than the first threshold Th1 and the participant P's concentration level is equal to or greater than the third threshold Th3, the output control unit 43 outputs first information (information indicating that the concentration level is high and the conference situation is good) to the information terminal 10 of the first participant P1, which is the output destination of the information. When the speech rate of the first participant P1 is equal to or greater than the first threshold Th1 and the participant P's concentration level is less than the fourth threshold Th4, the output control unit 43 outputs second information (information indicating that the concentration level is low and the second participant P2 is not concentrating on the speech of the first participant P1) to the information terminal 10 of the first participant P1, which is the output destination of the information. The method of outputting the information is not particularly limited. For example, the information terminal 10 of the first participant P1 may display the first information and the second information as a pop-up on an image (subjective image) of the virtual space V. For example, the information terminal 10 of the first participant P1 may output the first information and the second information as audio, such as synthesized voice. When the destination of the information is the first participant P1, the information to be output is not output from the information terminal 10 of the second participant P2.

[0070] Further, for example, when the speech rate of the first participant P1 is less than the second threshold Th2 and the concentration level of the participant P is equal to or greater than the third threshold Th3, the progress of the conference is favorable and the output control unit 43 determines not to output information. When the speech rate of the first participant P1 is less than the second threshold Th2 and the state in which the concentration level of the participant P is less than the fourth threshold Th4 continues for a predetermined time or more, the output control unit 43 outputs fourth information (information indicating that the concentration level is low and the conference is stagnating, and information recommending changing the topic) to the information terminal 10 of each participant P, which is the output destination of the information. The information terminal 10 of each participant P may, for example, display the fourth information as a pop-up on an image (subjective image) of the virtual space V. The information terminal 10 of the participant P may, for example, output the first information and the second information as audio, such as synthesized voice. If the information is output to the information terminals 10 of all participants P, the displayed information may be displayed individually in the virtual space V in a manner that is visible only to each participant P, or may be posted at a specific location in the virtual space V in a manner that is visible to all avatars U.

[0071] As described above, the information processing system 100 according to the second embodiment acquires percentage information indicating the proportion of speeches made by the participant P in the conference. In addition, the information processing system 100 calculates the concentration level of the participant P in the conference. Then, the information processing system 100 outputs information based on the percentage information and the concentration level. According to the present disclosure, the participant P in a remote conference via a network can appropriately grasp the status of the remote conference.

[0072] Furthermore, the information processing system 100 according to the second embodiment realizes a remote conference in which participants P converse directly with each other via avatars U in a virtual space V. The situation of the remote conference can be properly grasped according to the concentration level of each participant P, which is difficult to grasp from the avatar U.

[0073] (effect) As described above, the information processing system 100 of the present disclosure includes a ratio information acquisition unit 41 that acquires ratio information indicating the speaking ratio of a participant P in a conference, a concentration degree calculation unit 42 that calculates the concentration degree of the participant P in the conference, and an output control unit 43 that outputs information based on the ratio information and the concentration degree. According to the present disclosure, a participant P of a remote conference via a network can appropriately grasp the status of the remote conference.

[0074] Furthermore, according to the present disclosure, the output control unit 43 outputs information to the participant P determined based on the ratio information, with content according to the ratio information and the concentration level. According to the present disclosure, information with appropriate content can be output to the appropriate participant P according to the type of the conference (lecture format, free speech format, etc.) determined from the ratio information.

[0075] Furthermore, according to the present disclosure, the output control unit 43 varies the destination of the information to be output depending on the speech rate of the first participant P1, who has the largest speech rate among the participants P. According to the present disclosure, the form of the conference can be properly understood based on the speech rate of the first participant P1, who has the largest speech rate in the conference. This allows the participants P involved in the progress of the conference to be notified of the status of the conference.

[0076] Furthermore, according to the present disclosure, when the speech rate of the first participant P1 is equal to or greater than a first threshold Th1, the output control unit 43 outputs information to the first participant P1, and when the speech rate of the first participant P1 is less than a second threshold Th2 that is equal to or less than the first threshold Th1, the output control unit 43 outputs information to both the first participant P1 and a second participant P2 other than the first participant P1. According to the present disclosure, information can be provided to appropriate output destinations for both a lecture-style conference in which the first participant P1 has a high speech rate and a free-comment-style conference in which the first participant P1 has a low speech rate.

[0077] The information processing method according to the present disclosure includes the steps of acquiring percentage information indicating a speaking ratio of a participant P in a conference, calculating a concentration level of the participant P in the conference, and outputting information based on the percentage information and the concentration level. According to the present disclosure, the participant P in a remote conference via a network can appropriately grasp the status of the remote conference.

[0078] A program according to the present disclosure causes a computer to execute the steps of acquiring percentage information indicating the speaking ratio of a participant P in a conference, calculating the concentration level of the participant P in the conference, and outputting information based on the percentage information and the concentration level. According to the present disclosure, a participant P in a remote conference via a network can appropriately grasp the status of the remote conference. [Explanation of symbols]

[0079] 10 Information terminals 30 Server device 41 Percentage information acquisition unit 42 Concentration calculation part 43 Output control section 100 Information Processing Systems P participants

Claims

1. a ratio information acquisition unit that acquires ratio information indicating the speaking ratio of participants in a conference; a concentration calculation unit that calculates the concentration of the participants in the conference; an output control unit that outputs information based on the ratio information and the concentration degree; Equipped with Information processing system.

2. the output control unit outputs the information to the participants determined based on the ratio information, with content determined according to the ratio information and the concentration level. The information processing system according to claim 1 .

3. the output control unit varies the output destination of the information to be output depending on the speech rate of a first participant having the largest speech rate among the participants; The information processing system according to claim 2 .

4. The output control unit If the speech rate of the first participant is equal to or greater than a first threshold, output the information to the first participant; When the speech rate of the first participant is less than a second threshold that is equal to or less than the first threshold, output the information to the first participant and a second participant other than the first participant. The information processing system according to claim 3 .

5. acquiring percentage information indicating the speaking percentage of participants in the conference; calculating the degree of concentration of the participants in the conference; outputting information based on the ratio information and the concentration level; Including, Information processing methods.

6. acquiring percentage information indicating the speaking percentage of participants in the conference; calculating the degree of concentration of the participants in the conference; outputting information based on the ratio information and the concentration level; to the computer, program.

Images