Remote conference system, remote conference method, and remote conference program

Abstract

To adjust conferences to make progresses well-balanced between a plurality of groups.SOLUTION: A remote conference system comprises: a group information determination unit 251 which determines group information on a plurality of groups in which a plurality of participants can participate and which are each configured of at least two participants; a conference state acquisition unit 253 which acquires a conference state indicative of the state of a conference in each of the groups; a change unit 255 which changes the group information based on the conference state in each of the groups; and a support unit 255 which reports a change portion from the group information.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a remote conference system, a remote conference method, and a remote conference program, and more particularly to a remote conference system having a breakout room function, a remote conference method executed in the remote conference system, and a remote conference program that causes a computer to execute the remote conference method. [Background technology]

[0002] In recent years, remote conference systems in which multiple participants located at different locations hold a conference via a computer have become widespread. Technologies for supporting the progress of remote conferences have also been developed. For example, Japanese Patent Application Laid-Open Publication No. 2007-325063 describes a remote conference system that connects multiple locations via a network, the remote conference system including a server connected to each location via the network and a display device connected to the server, the server including a recording means for recording data related to speeches made by each participant in the conference, and a first display control means for displaying the data recorded by the recording means at each location on the display device of the conference facilitator, a specific participant, or all participants.

[0003] Meanwhile, remote conference systems are known to have a breakout room function in which participants in a conference are divided into multiple groups and each group holds its own conference. In the remote conference system described in Japanese Patent Laid-Open Publication No. 2007-325063, when the breakout room function is used, the conference can proceed smoothly for each of the multiple groups, but differences in the conference status may occur between the groups. This may result in large differences in the time the conference ends and in large differences in the conference outcomes. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2007-325063 Summary of the Invention [Problem to be solved by the invention]

[0005] An object of the present invention is to provide a remote conference system that can adjust the progress of a conference so that it proceeds in a balanced manner among a plurality of groups.

[0006] Another object of the present invention is to provide a remote conference method that can adjust the progress of the conference so that it proceeds in a balanced manner among a plurality of groups.

[0007] Another object of the present invention is to provide a remote conference program that can be adjusted so that the progress of the conference proceeds in a balanced manner among a plurality of groups. [Means for solving the problem]

[0008] According to one aspect of the present invention, a remote conference system is a remote conference system in which a plurality of participants can participate, and includes: group information determination means for determining group information regarding a plurality of groups, each of which is composed of two or more participants; conference state acquisition means for acquiring a conference state indicating a conference state of each of the plurality of groups; modification means for generating modified information by modifying the group information based on the conference state of each of the plurality of groups; and support means for notifying a portion of the group information that has been modified. The change means determines at least one assistant from among the plurality of participants based on the conference status of each of the plurality of groups, and determines an assistant group from among the plurality of groups that is different from the group to which the assistant belonged before the change based on the conference status of each of the plurality of groups. do.

[0009] According to this aspect, change information is generated that changes the two or more participants constituting each of the multiple groups based on the conference status of each of the multiple groups. Therefore, participants can be changed between groups with good conference status and groups with bad conference status. Furthermore, because the changes from the group information are notified, it is possible to prompt a change to the two or more participants constituting each of the multiple groups so as to improve the conference status of a group with a bad conference status. As a result, it is possible to provide a remote conference system that can improve the conference status of each of the multiple groups and adjust the progress of the conference so that it progresses balanced among the multiple groups.

[0011] Also At least one assistant is selected from among the participants, and an assistant group different from the group to which the assistant belonged before the change is selected from among the groups. Therefore, the group information is changed so that the assistant belongs to the assistant group, making it easy to create change information that changes the group information.

[0012] Preferably, the conference status includes a status index that indicates a larger value the better the status of the two or more participants constituting the group, and the change means determines two target groups with different status indices from among the multiple groups, determines one of the participants belonging to one of the target groups with the larger status index as an assistant, and determines the other target group with the smaller status index as an assistant group.

[0013] According to this aspect, two target groups with different condition indices are determined from multiple groups, and one participant belonging to one target group with a higher condition index is determined as an assistant, and the other target group with a lower condition index is determined as an assistant group. Therefore, a candidate belonging to a group with good participant conditions can be assigned to a group with poor participant conditions. Therefore, assistance can be provided to improve the meeting condition of the group with poor participant conditions.

[0014] Preferably, the condition index includes a first condition index and a second condition index each indicating a different condition, and the modification means generates modification information based on the first condition index during a first period, and generates modification information based on the second condition index during a second period after the first period.

[0015] According to this aspect, the change information is generated based on the first status index during the first period, and the change information is generated based on the second status index during the second period after the first period, so that multiple groups are compared in different states during different periods, thereby making it possible to compare multiple groups in terms of conference states according to the progress of the conference.

[0016] Preferably, the plurality of participants includes a predetermined assistant candidate, and the change means determines one target group from one or more groups to which the assistant candidate belongs, and determines the other target group from one or more groups to which the assistant candidate does not belong.

[0017] According to this aspect, the group to which the candidate belongs is determined from one or more groups to which the assistant candidate belongs, and the assistant group is determined from one or more groups to which the assistant candidate does not belong, which makes it easy to determine the assistant and the assistant group.

[0018] Preferably, the change means determines the group with the largest condition index among one or more groups to which the assistant candidate belongs as one target group, and determines the group with the smallest condition index among one or more groups to which the assistant candidate does not belong as the other target group.

[0019] According to this aspect, the group with the largest status index among one or more groups to which the assistant candidate belongs is determined to be the group to which the assistant belongs, and the group with the smallest status index among one or more groups to which the assistant candidate does not belong is determined to be the assistant group. This makes it easy to determine the assistant and the assistant group.

[0020] Preferably, the change means determines the conference status of each of the plurality of groups at predetermined time intervals.

[0021] According to this aspect, the conference status is determined for each of a plurality of groups at predetermined time intervals, so that the group information can be changed in accordance with the conference status that changes over time.

[0022] Preferably, the change means changes the conference status of any of the plurality of groups. In the first period, the first condition index of any of the plurality of groups is equal to or less than the first threshold value TH1, and in the second period, the second condition index of any of the plurality of groups is equal to or less than the second threshold value TH2. When a predetermined state is reached, change information is generated.

[0023] According to this aspect, change information is generated in response to the conference status of any of the multiple groups becoming a predetermined status, so that group information can be changed in response to the conference status that changes over time.

[0024] Preferably, the system further comprises a history storage means for storing history information including a set of group information and a conference state, and a learning means for learning a predetermined state using the history information as training data.

[0025] According to this aspect, the timing for generating change information is learned using the history information including the pair of group information and the conference status as training data, so that the group information can be changed at an appropriate timing.

[0026] Preferably, the conference state acquisition means acquires the conference state based on images of the voices of the plurality of participants and / or faces of the participants.

[0027] According to this aspect, the conference status can be properly acquired.

[0028] According to another aspect of the present invention, a remote conferencing method is a remote conferencing method executed by a computer that provides a remote conferencing service in which multiple participants can participate, and includes a group information determination step of determining group information regarding multiple groups, each of which is composed of two or more participants; a conference state acquisition step of acquiring a conference state indicating a conference state in each of the multiple groups; a modification step of generating modified information by modifying the group information based on the conference state in each of the multiple groups; and a support step of notifying the user of the modified portion of the group information. The change step includes a step of determining at least one assistant from among the plurality of participants based on the conference state of each of the plurality of groups, and a step of determining an assistant group from among the plurality of groups to which the assistant belongs before the change based on the conference state of each of the plurality of groups. .

[0029] According to this aspect, it is possible to provide a remote conference method that can adjust the progress of the conference so that it proceeds balanced among a plurality of groups.

[0030] According to yet another aspect of the present invention, a remote conference program is a remote conference program executed by a computer that provides a remote conference service in which multiple participants can participate, and causes the computer to execute a group information determination step of determining group information regarding multiple groups, each of which is composed of two or more participants; a conference state acquisition step of acquiring a conference state indicating the conference state of each of the multiple groups; a modification step of generating modified information by modifying the group information based on the conference state of each of the multiple groups; and a support step of notifying the user of the modified portion of the group information. The change step includes a step of determining at least one assistant from among the plurality of participants based on the conference state of each of the plurality of groups, and a step of determining an assistant group from among the plurality of groups to which the assistant belongs before the change based on the conference state of each of the plurality of groups. .

[0031] According to this aspect, it is possible to provide a remote conference program that can be adjusted so that the progress of the conference proceeds balanced among a plurality of groups. . [Brief explanation of the drawings]

[0032] [Figure 1] 1 is a diagram illustrating an example of a system configuration of a remote conference system according to an embodiment of the present invention. [Figure 2]1 is a diagram illustrating an example of an overview of a remote conference system according to an embodiment of the present invention. [Figure 3] FIG. 2 is a block diagram illustrating an example of a hardware configuration of a server. [Figure 4] FIG. 2 is a block diagram showing an example of the hardware configuration of a PC. [Figure 5] FIG. 2 is a diagram illustrating an example of functions of a CPU included in a server according to the present embodiment. [Figure 6] 10 is a flowchart illustrating an example of the flow of a group adjustment process. [Figure 7] 10 is a flowchart illustrating an example of the flow of a conference status acquisition process. [Figure 8] 10 is a flowchart illustrating an example of the flow of a change information generation process. [Figure 9] 10 is a flowchart showing an example of the flow of a change information generation process in the first modified example. DETAILED DESCRIPTION OF THE INVENTION

[0033] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same components are designated by the same reference numerals. The names and functions of these components are also the same. Therefore, detailed description thereof will not be repeated.

[0034] Fig. 1 is a diagram showing an example of a system configuration of a remote conference system according to one embodiment of the present invention. Referring to Fig. 1, remote conference system 1 includes server 200 and personal computers (hereinafter referred to as "PCs") 100-1, 100-2, 100-3, 100-4 to 100-N, where N is a positive integer greater than or equal to 5. Server 200 and PCs 100-1 to 100-N are connected to Internet 5 and are capable of communicating with each other.

[0035] Each of PC100-1 to PC100-N is equipped with a camera, a microphone for collecting sound, and a speaker for outputting sound. Each of PC100-1 to PC100-N is a general computer, and since their main hardware configurations and functions are the same, PC100-1 to PC100-N will be collectively referred to as PC100 here.

[0036] Instead of PCs 100-1 to 100-N, information communication devices such as PDAs (Personal Digital Assistances) or smartphones may be used as long as they are equipped with a camera, microphone, speaker, and communication functions. Furthermore, the network is not limited to the Internet 5, and may be another network as long as server 200 and PCs 100-1 to 100-N can communicate with each other. The network may be, for example, a local area network (LAN) or a wide area network (WAN).

[0037] FIG. 2 is a diagram illustrating an example of an overview of a remote conference system according to an embodiment of the present invention. Referring to FIG. 2, in remote conference system 1, conference participants participate in a conference by operating any of PCs 100-1 to 100-N. In remote conference system 1 according to this embodiment, a case will be described in which the multiple participants participating in the conference include an administrator who hosts the conference and at least one facilitator who assists in the progress of the conference. When the breakout room function is used in remote conference system 1 and a conference is conducted in each of multiple groups, the facilitator belongs to one of the groups and plays a role in assisting the progress of the conference for that group. The administrator, facilitator, and participants are determined in advance, and the participants and facilitators are set in remote conference system 1 by the administrator. Here, a case will be described in which the participants operating PC 100-1 are the administrator and facilitator.

[0038] A program for participating in a conference is installed on each of PCs 100-1 to 100-N, and the conference is held by each of PCs 100-1 to 100-N communicating with server 200. The program installed on each of PCs 100-1 to 100-N may be a dedicated program for communicating with server 200, or may be a general browser program if server 200 provides a web service.

[0039] A remote conference system is realized by executing a remote conference program on server 200. Server 200 communicates with PCs 100-1 to 100-N, and transmits data received from each of PCs 100-1 to 100-N to each of the other PCs 100-1 to 100-N.

[0040] Data transmitted and received between each of PCs 100-1 to 100-N and server 200 includes audio information indicating audio, video information indicating video, and application data. The data transmitted and received between each of PCs 100-1 to 100-N and server 200 may be compressed data or may not be compressed data.

[0041] Server 200 controls the data to be transmitted to each of PCs 100-1 to N. For example, server 200 transmits audio data received from each of PCs 100-1 to N to all of PCs 100-1 to N. Server 200 also aggregates video data received from each of PCs 100-1 to N and transmits the video information to each of PCs 100-1 to N.

[0042] Server 200 determines and transmits the video and application data to be transmitted in response to a request from each of PCs 100-1 to 100-N. Therefore, the images displayed on each of PCs 100-1 to 100-N may be the same or different. Each of PCs 100-1 to 100-N may process and display the video information received from server 200. In this case, the processing of processing the video information on server 200 is not required, thereby reducing the load on server 200.

[0043] Server 200 stores group information about multiple groups A to E, and determines the conference status of each of the multiple groups. Group information is information that identifies multiple participants belonging to each of multiple groups A to E. Server 200 determines the conference status of each of multiple groups A to E based on audio information and video information received from PCs 100-1 to 100-N. Here, a case is shown in which the conference status of group A is determined to be 91%, the conference status of group B is 78%, the conference status of group C is 18%, the conference status of group D is 42%, and the conference status of group E is 36%.

[0044] The server 200 determines and notifies an assistant and an assistant group based on the conference status of each of multiple groups A to E. The server 200 acquires from facilitator information indicating which participant is the facilitator that the groups to which the facilitator belongs are groups D and E. The server 200 determines group D, which has the highest conference status among groups D and E to which the facilitator belongs, as the assistant group, and determines the facilitator belonging to the support group as the assistant. Furthermore, the server 200 determines group C, which has the lowest conference status among groups A to C to which the facilitator does not belong, as the assistant group. The server 200 then notifies the assistant or administrator with a message urging the assistant to move to the assistant group.

[0045] Fig. 3 is a block diagram showing an example of a hardware configuration of a server. Referring to Fig. 3, server 200 is a computer that performs arithmetic processing and includes a central processing unit (CPU) 201 for controlling the entire server 200, a ROM (Read Only Memory) 202 for storing programs to be executed by CPU 201, a RAM (Random Access Memory) 203 used as a work area for CPU 201, an HDD 204 for storing data in a non-volatile manner, a communication unit 205 that connects CPU 201 to Internet 5, a display unit 206 that displays images, an operation unit 207 that accepts operation inputs, and an external storage device 210, each of which is connected to a bus 213.

[0046] Communication unit 205 is an interface for connecting server 200 to Internet 5. Therefore, CPU 201 can communicate with PCs 100-1 to 100-N connected to Internet 5 via communication unit 205.

[0047] The external storage device 210 is equipped with a CD-ROM (Compact Disk Read Only Memory) 211. The CPU 201 controls the external storage device 210 to read data stored in the CD-ROM 211.

[0048] In this embodiment, CPU 201 executes a program stored in ROM 202 or HDD 204. CPU 201 may also control external storage device 210 to read a program to be executed by CPU 201 from CD-ROM 211, store the read program in RAM 203, and execute it.

[0049] Furthermore, the CPU 201 downloads a program from a computer connected to the Internet 5 and stores it in the HDD 204. When a computer connected to the Internet 5 writes a program to the HDD 204, the program is stored in the HDD 204. The CPU 201 may load the program stored in the HDD 204 into the RAM 203 and execute it.

[0050] The recording medium for storing the program to be executed by CPU 201 is not limited to CD-ROM 211, but may be a flexible disk, cassette tape, optical disk (MO (Magnetic Optical Disc) / MD (Mini Disc) / DVD (Digital Versatile Disc)), IC card, optical card, mask ROM, EPROM (Erasable Programmable ROM) or other semiconductor memory. The program here includes not only a program that can be directly executed by CPU 201, but also a source program, a compressed program, an encrypted program, etc.

[0051] Fig. 4 is a block diagram showing an example of the hardware configuration of a PC. Referring to Fig. 4, PC 100 is a computer that performs arithmetic processing and includes a CPU 101 for controlling the entire PC 100, a ROM 102 for storing programs executed by CPU 101, a RAM 103 used as a work area for CPU 101, an HDD 104 for non-volatilely storing data, a communication unit 105 that connects CPU 101 to Internet 5, a display unit 106 for displaying images, an operation unit 107 that accepts operation inputs from participants who are users, a camera 108 that captures images of participants, a speaker 109 that outputs audio, a microphone 110 that collects the audio of the operator, and an external storage device 111, all of which are connected to a bus 112.

[0052] A CD-ROM 111A is attached to the external storage device 111. The CPU 101 controls the external storage device 111 to read data stored on the CD-ROM 111A.

[0053] A module in which at least two of the camera 108, the speaker 109, and the microphone 110 are integrated may be connected to the PC 100. The module includes, for example, a headset in which the speaker 109 and the microphone 110 are integrated.

[0054] Fig. 5 is a diagram showing an example of functions of a CPU included in a server according to the present embodiment. The functions shown in Fig. 5 are realized by CPU 201 included in server 200 as CPU 201 executes a remote conference program stored in ROM 202, HDD 204, or CD-ROM 211. Referring to Fig. 5, CPU 201 includes group information determination unit 251, conference state acquisition unit 253, change unit 255, and support unit 257.

[0055] The group information determination unit 251 determines group information related to multiple groups. The group information is information that defines multiple groups and the participants belonging to each group. Here, the group information includes a participant record that associates, for each participant, participant identification information for identifying that participant with group identification information for identifying the group. A conference participant belongs to one of multiple groups. The multiple groups are determined by an administrator of one of the multiple participants. Here, a case will be described in which the participant operating PC 100-1 is an administrator. The administrator knows the multiple participants in advance and inputs the participant identification information and group identification information for each participant to the remote conference system via PC 100-1. The group information determination unit 251 acquires the group information input to PC 100-1 by the administrator and outputs it to the conference state acquisition unit 253 and the change unit 255. Note that, in this embodiment, a case will be described in which the administrator inputs group information to PC 100-1, but the group information may also be generated by the group information determination unit 251 based on information about the multiple participants. For example, multiple participants may be identified from the meeting invitation and reply emails, and the participants may be classified into groups based on their attributes, such as the organization to which the participant belongs, their work history, and their field of expertise.

[0056] The conference status acquisition unit 253 acquires a conference status indicating the status of the conference in each of the multiple groups. The conference status is determined by the status of each of the multiple participants belonging to the group. The status of the participant is determined based on an image obtained by capturing the participant's voice and / or face. The conference status acquisition unit 253 outputs the conference status of each of the multiple groups to the change unit 255. The conference information is information that indicates a larger value the better the conference status in the group.

[0057] Conference status acquisition unit 253 acquires audio information indicating the voices of the participants and video information obtained by capturing images of the participants' faces from each of PCs 100, 100A to 100N operated by the participants. Each of PCs 100-1 to 100-N transmits to server 200 audio information obtained by converting audio collected by microphone 110 into a digital signal and video information obtained by converting video captured by camera 108 of the participants' faces into a digital signal. Conference status acquisition unit 253 stores the audio information and video information received from each of PCs 100-1 to 100-N by communication unit 205 in HDD 204 in association with the participant. The audio information and video information may be stored in association with PCs 100-1 to 100-N, or may be stored in association with the participant operating PCs 100-1 to 100-N. When a participant operates one of PCs 100-1 to 100-N to log in to the remote conference system, the participant is authenticated and identified by server 200.

[0058] The conference status is indicated by a first status index and a second status index. The first status index and the second status index are indexes with different evaluation criteria. The conference status acquisition unit 253 determines the first status index and the second status index for each group based on the status of each of the multiple participants belonging to the group.

[0059] In this embodiment, the first status index is an index indicating the degree of activity of the conference, and the second status index is an index indicating the degree of satisfaction of each conference participant. The first status index takes a larger value as the degree of activity of the conference increases. The second status index takes a larger value as the degree of satisfaction of each conference participant increases.

[0060] Specifically, the first status index will be described as the speaking rate in a group. The speaking rate is expressed as the proportion of speaking time per unit time. The speaking time is determined from the voice information of the participants. The sum of the speaking rates of each of the multiple participants belonging to each group is used as the first status index of the group. Note that if the speaking rate of one participant is higher than the speaking rate of the other participants, it cannot be said that the group is highly activated. For this reason, the first status index of a group may be a value obtained by adjusting the sum of the speaking rates of each of the multiple participants according to the imbalance in the speaking rates of the multiple participants. For example, the first status index of a group may be a value obtained by dividing the maximum speaking rate of each of the multiple participants (or the average of the top few speaking rates) by the difference between the overall average and the average speaking rate.

[0061] In addition, an example will be described in which the second state index is the degree of satisfaction within a group. The degree of satisfaction is a value that quantifies the degree of emotion of participants that they are satisfied or agreeing with. For example, by using artificial intelligence to learn the facial expressions and degree of satisfaction of participants shown in their video information, it is possible to determine the degree of satisfaction of participants from their video information. The average of the degrees of satisfaction of multiple participants belonging to each group is used as the second state index of the group. Note that if there is a large imbalance in the degrees of satisfaction among participants, it cannot be said that the degree of satisfaction of the group is high. For this reason, the second state index of the group may be a value obtained by adjusting the average of the degrees of satisfaction of multiple participants according to the imbalance in the degrees of satisfaction among participants.

[0062] The change unit 255 generates change information by changing the group information based on the conference status. The change unit 255 outputs to the support unit 257 the change portion that differs between the change information and the group information. The change unit 255 changes the group information using a first status index of the conference information during a first period, and changes the group information using a second status index of the conference information during a second period. The second period is a period after the first period. The second period is a period from a time a predetermined time before the end time of the target conference to the end time. The first period is any period from the start time to the end time of the target conference, excluding the second period. For example, if the predetermined time is 30 minutes, the first period is a period from the start time to 30 minutes before the end time, and the second period is a period from 30 minutes before the end time to the end time.

[0063] The change unit 255 determines whether to change the group information at predetermined time intervals. The change unit 255 compares the conference information of each of the multiple groups with a predetermined threshold at predetermined time intervals, and determines to change the group information when the conference information of any of the multiple groups is equal to or less than the threshold TH. If the timing of the determination is a first period, the change unit 255 determines to change the group information when the first state index of any of the multiple groups is equal to or less than the first threshold TH1. If the timing of the determination is a second period, the change unit 255 determines to change the group information when the second state index of any of the multiple groups is equal to or less than the second threshold TH2.

[0064] As described above, the change unit 255 changes the group information using the first state index and the first threshold TH1, or the second state index and the second threshold TH2. The processing procedure is the same when the change unit 255 changes the group information using the first state index and the first threshold TH1 as when it changes the group information using the second state index and the second threshold TH2. Hereinafter, the first state index and the second state index will be collectively referred to as state indexes, and the first threshold TH1 and the second threshold TH2 will be collectively referred to as thresholds.

[0065] When changing the group information, the change unit 255 determines at least one assistant from among the multiple participants based on the conference status of each of the multiple groups. When changing the group information, the change unit 255 also determines an auxiliary group from among the multiple groups that the assistant belonged to before the change based on the conference status of each of the multiple groups. The change unit 255 generates change information that changes the group information to a group formation in which the assistant belongs to an auxiliary group.

[0066] Specifically, the change unit 255 determines, as an assistant, a facilitator who belongs to a support group with the highest status index among one or more groups to which facilitators of multiple groups belong. A facilitator is a participant who plays a role in controlling the progress of a meeting in a group and is predetermined from among the multiple participants. There may be one or more facilitators. The change unit 255 does not determine, as a support group, a group to which a facilitator who has already been determined to be an assistant belongs, even if the group includes one or more groups to which the facilitator belongs. In other words, the change unit 255 determines, as an assistant, a facilitator who belongs to a support group with the highest status index among one or more groups to which a facilitator who has not been determined to be an assistant belongs. Note that the change unit 255 may determine, as a support group, a facilitator who belongs to a group with the highest status index among one or more groups to which the facilitator belongs, even if that group has already been determined to be an assistant.

[0067] The change unit 255 determines the group with the smallest state index from among one or more groups to which the facilitator of the multiple groups does not belong as the support group. The change unit 255 generates change information that changes the group information to a group formation in which the assistant belongs to the support group. The change unit 255 outputs the group information and the change information to the support unit 257. Note that the change unit 255 may also generate change information that changes one of the multiple participants belonging to the support group to a group formation in which the assistant belongs to the support group. In this case, it is possible to prevent the number of participants belonging to each of the multiple groups from changing.

[0068] The support unit 257 compares the change information with the group information and notifies the participants of the different changed parts. Specifically, the support unit 257 compares the change information with the group information and determines the assistant and the assistant group as the changed parts. The support unit 257 notifies the participants of a message urging the assistant to move to the assistant group.

[0069] For example, the support unit 257 displays a message on the PCs 100-1 to 100-N. Alternatively, the support unit 257 may notify only the assistant. In this case, the support unit 257 displays the message on a device among the PCs 100-1 to 100-N that is operated by the assistant. The assistant who views the message can change the group formation so that the assistant belongs to the auxiliary group. Alternatively, the message may be notified to an administrator who manages the conference. In this case, the support unit 257 displays the message on a device among the PCs 100-1 to 100-N that is operated by the administrator. The administrator who views the message can change the group formation so that the assistant belongs to the auxiliary group. When the group formation is changed, since the assistant is present in the auxiliary group, it is expected that discussions will become more lively in the conference and the conference state will improve. Therefore, by the support unit 257 notifying the message, it is possible to adjust the progress of the conference so that it progresses in a balanced manner among multiple groups.

[0070] When the group formation is changed by the administrator or assistant, the group information determination unit 251 acquires new group information after the group formation is changed by the administrator. When the group information is changed, a pair of group information and state information before and after the change is stored in the HDD 204. Furthermore, state information after the pair of group information and state information before and after the change is stored is stored in the HDD 204. As a result, a pair of group information and state information before the change and a pair of group information and state information after the change are stored in the HDD 204. A threshold for determining whether to change the group information or the timing of assistant movement may be learned by artificial intelligence based on the pair of group information and state information before and after the change. The threshold for determining whether to change the group information is, for example, a first threshold TH1 for the first state index and a second threshold TH2 for the second state index.

[0071] FIG. 6 is a flowchart showing an example of the flow of the group adjustment process. The group adjustment process is executed by CPU 201 included in server 200 as CPU 201 executes a remote conference program stored in ROM 202, HDD 204, or CD-ROM 211. Referring to FIG. 6, CPU 201 included in server 200 determines group information (step S01) and proceeds to step S02. Group information is generated including a participant record that associates, for each participant, participant identification information for identifying that participant with group identification information for identifying the group. The group information is determined by an administrator of one of the participants in the conference and input to remote conference system 1. For example, when the administrator operates PC 100, if the administrator inputs group information including participant records for each of multiple participants into the remote conference system via PC 100-1, the group information received from PC 100 is acquired.

[0072] In step S02, audio information and video information are acquired for each participant, and the process proceeds to step S03. CPU 201 controls communication unit 205 to acquire audio information indicating the audio of the participants received by communication unit 205 from each of PCs 100-1 to 100-N, and video information obtained by capturing images of the faces of the participants. The audio information and video information are stored in HDD 204 in association with the participants.

[0073] In step S03, it is determined whether a predetermined time has elapsed. It is determined whether a predetermined time has elapsed since the group adjustment process was executed or since step S03 was executed in the past. If the predetermined time has elapsed, the process proceeds to step S04, but if not, the process returns to step S01. The process from step S04 onwards is executed at predetermined time intervals.

[0074] In step S04, a conference status acquisition process is executed, and the process proceeds to step S05. The conference status acquisition process, the details of which will be described later, is a process of calculating a status index. The status index includes a first status index and a second status index that has a different degree of evaluation from the first status index. In this embodiment, the first status index is an index that indicates a larger value as the level of activity of the conference increases, and is represented as the speaking rate in the group. The second status index is an index that indicates the degree of satisfaction of each participant in the conference, and is a value that quantifies the degree of satisfaction or agreement. The second status index is determined by having artificial intelligence learn images of the participants' faces and their satisfaction levels, and the satisfaction level is determined from the expressions that can be read from the participants' faces.

[0075] In step S05, it is determined whether the current time is within the first period. If it is within the first period, the process proceeds to step S06; if not, the process proceeds to step S07. In step S06, the state index is set to the first state index, and the process proceeds to step S10. In step S07, it is determined whether the current time is within the second period. If it is within the second period, the process proceeds to step S08; if not, the process returns to step S01. The second period is a period that follows the first period. In step S08, the state index is set to the second state index, and the process proceeds to step S09.

[0076] In step S09, it is determined whether the minimum state index is equal to or less than a threshold value. If a first state index is set in the state information, it is determined whether the minimum first state index is equal to or less than a first threshold value TH1. If a second state index is set in the state information, it is determined whether the minimum second state index is equal to or less than a second threshold value TH2. If the minimum state index is equal to or less than the threshold value, the process proceeds to step S10; if not, the process returns to step S01.

[0077] In step S10, a change information generation process is executed, and the process proceeds to step S11. The change information generation process, which will be described in detail later, is a process for generating change information in which the group information has been changed based on the state information. By executing the change information generation process, at least one pair of an assistant and an assistant group is determined. In step S11, the changed portion is notified, and the process proceeds to step S12. The changed portion is a portion that differs between the change information and the group information. Specifically, a message urging the assistant to move to an assistant group is notified to the participants. For example, the message is displayed on PCs 100-1 to 100-N operated by the participants. The message may also be notified to the assistant or the administrator, rather than to all participants. The assistant or the administrator can set an operation in the remote conference system to change the group to which the assistant belongs to an assistant group. Furthermore, if the timing for moving the assistant is determined by artificial intelligence, the assistant is notified of the timing for moving.

[0078] In step S12, it is determined whether the assistant has moved to a different group. If the assistant has moved to a different group, the process proceeds to step S13; if not, the process returns to step S01. In step S13, the group information is updated, and the process proceeds to step S14. The group information is changed to a group arrangement in which the assistant belongs to the different group. In step S14, the conference status and group information are stored in HDD 204, and the process returns to step S01.

[0079] 7 is a flowchart showing an example of the flow of the conference status acquisition process. The conference status acquisition process is a process executed in step S04 of the group adjustment process. Referring to FIG. 7, a group to be processed is selected (step S21), and the process proceeds to step S22. A group from which the conference status is to be acquired is selected from among the multiple groups defined by the group information.

[0080] In step S22, a first status index is calculated, and the process proceeds to step S23. The first status index is an index indicating the degree of activity of the conference, and is the speech rate in the group. The speech rate is expressed as the rate of speech time per unit time. The speech rate of each of the multiple participants is calculated based on the voice information acquired from each of PCs 100-1 to 100-N. The first status index of each of the multiple groups is the sum of the speech rates of each of the multiple participants belonging to that group.

[0081] In step S23, a second state index is calculated, and the process proceeds to step S24. The second state index is an index indicating the degree of satisfaction of each participant in the meeting, and is the satisfaction level of the group. The satisfaction level is a value obtained by quantifying the degree of satisfaction or agreement of the participants. The satisfaction level of each participant is calculated based on the video information acquired from each of PCs 100-1 to 100-N. For example, by using artificial intelligence to learn, the satisfaction level can be determined from the participant's face. The second state index of each of multiple groups is the average of the satisfaction levels of each of the multiple participants belonging to that group.

[0082] In step S24, it is determined whether or not there is a group that has not been selected for processing. If there is an unselected group, the process returns to step S21; otherwise, the process returns to the group adjustment process. As a result, a first status index and a second status index are calculated as the conference status of each of the multiple groups defined by the group information.

[0083] 8 is a flowchart showing an example of the flow of the change information generation process. The change information generation process is a process executed in step S10 of the group adjustment process. Referring to FIG. 8, the number of facilitators is set as a constant N (step S31), and the process proceeds to step S32. One or more facilitators are determined from the group information among the multiple participants.

[0084] In step S32, variable i is set to "1" and the process proceeds to step S33. Variable i is a value indicating the number of facilitators to be processed, and is incremented in step S35 or step S40, which will be described later. In step S33, the group to be processed is selected from the multiple groups in ascending order of state index.

[0085] In step S33, a group to be processed is selected from the multiple groups in ascending order of state index. In the next step S34, it is determined whether a facilitator is present in the selected group. If a facilitator is present, the process proceeds to step S35; if not, the process proceeds to step S37. In step S35, the variable i is incremented, and the process proceeds to step S36. In step S36, it is determined whether the variable i is greater than a constant N. If the variable i is greater than the constant N, the process returns to the group adjustment process; if not, the process returns to step S33.

[0086] In step S37, the group selected for processing is determined to be an auxiliary group, and processing proceeds to step S38. In step S38, the group to be processed is selected from the multiple groups in descending order of state index, and processing proceeds to step S39. In step S39, it is determined whether a facilitator is present in the group selected in step S38. If a facilitator is present, processing proceeds to step S40; if not, processing returns to step S38.

[0087] In step S40, the variable i is incremented, and the process proceeds to step S41. In step S41, it is determined whether or not a facilitator belonging to the group selected in step S38 has been moved. A facilitator who has already been determined as an assistant is determined to have been moved. If the facilitator has been moved, the process proceeds to step S42; if not, the process proceeds to step S43. In step S42, it is determined whether or not the variable i is greater than a constant N. If the variable i is greater than the constant N, the process returns to the group adjustment process; if not, the process returns to step S38.

[0088] In step S43, the group selected in step S38 is determined to be the support group, and the process proceeds to step S44. In step S44, the facilitators belonging to the support group are determined to be assistants, and the process proceeds to step S45. In step S45, it is determined whether the variable i is greater than a constant N. If the variable i is greater than the constant N, the process returns to the group adjustment process; otherwise, the process returns to step S33.

[0089] <First Modification> In the above-described embodiment, the facilitator belongs to at least one of the multiple groups. The change information generation process in the first modified example is a process that is executed when the facilitator does not belong to multiple groups.

[0090] Fig. 9 is a flowchart showing an example of the flow of the change information generation process in Modification 1. Referring to Fig. 9, the number of groups is set to constant N (step S31A), and the process proceeds to step S32.

[0091] In step S32, variable i is set to "1", and the process proceeds to step S33. Variable i is a value indicating the number of groups to be processed, and is incremented in step S40, which will be described later. In step S33, groups to be processed are selected from the multiple groups in ascending order of state index, and the process proceeds to step S34A. In the first period, groups to be processed are selected in descending order of first state index, and in the second period, groups to be processed are selected in descending order of second state index.

[0092] In step S34A, it is determined whether the condition index is equal to or less than a threshold value. In the first period, the first condition index is compared with a first threshold value TH1, and in the second period, the second condition index is compared with a second threshold value TH2. If the condition index is equal to or less than the threshold value, the process proceeds to step S37; otherwise, the process returns to the group adjustment process. In step S37, the group selected to be processed in step S33 is determined to be an auxiliary group, and the process proceeds to step S38. In step S38, groups to be processed are selected from the multiple groups in descending order of condition index, and the process proceeds to step S40. In the first period, groups to be processed are selected in descending order of first condition index, and in the second period, groups to be processed are selected in descending order of second condition index. In step S40, the variable i is incremented, and the process proceeds to step S43.

[0093] In step S43, the group selected in step S38 is determined to be the support group, and processing proceeds to step S44. In step S44, one of the multiple participants belonging to the support group is determined to be the assistant, and processing proceeds to step S45A. For example, the participant with the highest speaking rate in both the first period and the second period is determined to be the assistant. In step S45A, it is determined whether the variable i is greater than 1 / 2 of the constant N. If the variable i is greater than 1 / 2 of the constant N, processing returns to the group adjustment processing; otherwise, processing returns to step S33.

[0094] <Second Modification> In the above-described embodiment, the conference state includes a first state index and a second state index, where the first state index is calculated from the speech rate of each participant belonging to the group, and the second state index is calculated from the satisfaction level of the participants belonging to the group. However, the conference state is not limited to this.

[0095] The conference status may be measured from conversations. The conference status may be determined from text information obtained by recognizing the voice information of the participants. The conference status may be determined from the connection status of each of the PCs 100-1 to 100-N operated by the participants. Furthermore, the conference information may be determined from the volume of the voices of the participants. Furthermore, the conference status may be determined based on the emotions of the participants, such as joy, anger, sadness, and happiness.

[0096] <Third Modification> In the above-described embodiment, the conference state is determined using different indicators for two periods, the first period and the second period. The number of periods and the number of indicators are not limited to these. The conference state may be determined using more periods and more different conference states.

[0097] As described above, according to an aspect of the present embodiment, remote conference system 1 includes server 200 and PCs 100-1 to 100-N, and multiple participants can participate by operating PCs 100-1 to 100-N. Server 200 determines group information for multiple groups, each consisting of two or more participants, acquires a conference status indicating the conference status of each of the multiple groups, generates change information by modifying the group information based on the conference status of each of the multiple groups, and notifies the changes from the group information. This makes it possible to change participants between groups with good conference status and groups with bad conference status. Furthermore, since the changes from the group information are notified, it is possible to prompt a change to the two or more participants constituting each of the multiple groups so as to improve the conference status of a group with a bad conference status. This makes it possible to improve the conference status of each of the multiple groups, and adjust the progress of the conference to proceed balanced among the multiple groups.

[0098] Furthermore, the server 200 determines at least one assistant from among the participants based on the conference status of each of the groups, and determines an auxiliary group from among the groups that is different from the group to which the assistant belonged before the change based on the conference status of each of the groups. Therefore, the group information is changed so that the assistant belongs to an auxiliary group, making it easy to create change information that changes the group information.

[0099] The conference status also includes a first status index and a second status index, which have larger values ​​as the status of the two or more participants constituting the group improves. The server 200 determines two target groups with different first status indices from among the multiple groups during a first period, selects one participant from the target group with the larger first status index as an assistant, and selects the other target group with the smaller first status index as an assistant group. The server also determines two target groups with different second status indices from among the multiple groups during a second period after the first period, selects one participant from the target group with the larger second status index as an assistant, and selects the other target group with the smaller second status index as an assistant group. This allows a candidate from a group with participants in a good status to be assigned to a group with participants in a less good status. This allows support to be provided to improve the conference status of a group with participants in a less good status. Furthermore, change information is generated based on the first status index during the first period, and change information is generated based on the second status index during a second period after the first period, allowing multiple groups to be compared in different statuses during different periods. This makes it possible to compare multiple groups based on the state of the meeting as it progresses. For example, if the first state index represents the level of activity in the meeting and the second state index represents the level of satisfaction among the participants, support can be provided to encourage active discussion in groups where discussion is not active in the first period, and support can be provided to encourage consensus in groups where the opinions of multiple participants are not in agreement in the later second period.

[0100] Furthermore, the multiple participants include a predetermined facilitator (assistant candidate), and the server 200 determines one target group from one or more groups to which the facilitator belongs, and determines the other target group from one or more groups to which the facilitator does not belong, making it easy to determine the assistant and the assistant group.

[0101] Furthermore, the server 200 determines the group with the highest condition index among one or more groups to which the assistant candidate belongs as one target group, and determines the group with the lowest condition index among one or more groups to which the assistant candidate does not belong as the other target group. For example, the server 200 determines the group with the highest condition index among one or more groups to which the facilitator belongs as the support group to which the assistant belongs, and determines the group with the lowest condition index among one or more groups to which the facilitator does not belong as the assistant group. This makes it easy to determine the assistant and the assistant group.

[0102] Furthermore, the server 200 determines the conference status of each of the multiple groups at predetermined time intervals, and therefore, it is possible to change the group information in accordance with the conference status that changes over time.

[0103] Furthermore, the server 200 generates change information when the conference state of any of the multiple groups becomes a predetermined state. Specifically, the server 200 generates change information when the first state index of any of the multiple groups becomes equal to or less than a first threshold TH1 during a first period, and when the second state index of any of the multiple groups becomes equal to or less than a second threshold TH2 during a second period. This makes it possible to change group information according to the conference state that changes over time.

[0104] Furthermore, the server 200 stores history information including pairs of group information and conference statuses in the HDD 204, and learns the first threshold TH1 and the second threshold TH2 using the history information as training data, thereby enabling the group information to be changed at an appropriate timing.

[0105] Furthermore, the server 200 acquires the conference status based on the voices of each of the participants and / or images of the faces of the participants, thereby making it possible to acquire the conference status appropriately.

[0106] The embodiments disclosed herein should be considered to be illustrative in all respects and not restrictive. The scope of the present invention is defined by the claims, not by the above description, and is intended to include all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0107] 1 Remote conference system, 5 Internet, 100-1 to N PCs, 200 Server, 108 Camera, 109 Speaker, 110 Microphone, 201 CPU, 202 ROM, 203 RAM, 204 HDD, 205 Communication unit, 206 Display unit, 207 Operation unit, 210 External storage device, 211 CD-ROM, 213 Bus, 251 Group information determination unit, 253 Conference status acquisition unit, 255 Change unit, 257 Support unit.

Claims

1. A remote conference system in which multiple participants can participate, a group information determination means for determining group information regarding a plurality of groups each consisting of two or more of the participants; a conference status acquisition means for acquiring a conference status indicating a conference status in each of the plurality of groups; a change unit that generates change information by changing the group information based on the conference status of each of the plurality of groups; a support means for notifying a change from the group information; The change means determines at least one assistant from among the plurality of participants based on the conference state in each of the plurality of groups, and determines an auxiliary group from among the plurality of groups other than the group to which the assistant belonged before the change based on the conference state in each of the plurality of groups.

2. the conference status includes a status index that indicates a larger value as the status of the two or more participants constituting the group is better; 2. The remote conference system of claim 1, wherein the change means determines two target groups from among the plurality of groups having different condition indices, determines one of the participants belonging to one of the target groups having the larger condition index as the assistant, and determines the other target group having the smaller condition index as the assistant group.

3. the condition index includes a first condition index and a second condition index, each of which indicates a different condition; 3. The remote conference system according to claim 2, wherein the modification means generates the modification information based on the first condition index during a first period, and generates the modification information based on the second condition index during a second period after the first period.

4. the plurality of participants includes predetermined auxiliary candidates; 4. The remote conference system according to claim 2, wherein the change means determines one of the target groups from one or more groups to which the assistant candidate belongs, and determines the other of the target groups from one or more groups to which the assistant candidate does not belong.

5. 5. The remote conference system according to claim 4, wherein the change means determines the group having the largest status index among the one or more groups to which the assistant candidate belongs as one of the target groups, and determines the group having the smallest status index among the one or more groups to which the assistant candidate does not belong as the other target group.

6. 6. The remote conference system according to claim 1, wherein the change unit determines the conference state of each of the plurality of groups at predetermined time intervals.

7. The remote conference system of any one of claims 1 to 6, wherein the change means generates the change information in response to the conference state of any of the plurality of groups becoming a predetermined state in which a first state index of any of the plurality of groups is less than or equal to a first threshold value TH1 during a first period, and a second state index of any of the plurality of groups is less than or equal to a second threshold value TH2 during a second period.

8. a history storage means for storing history information including a set of the group information and the conference status; 8. The remote conference system according to claim 7, further comprising: a learning unit that learns the predetermined state using the history information as teaching data.

9. 9. The remote conference system according to claim 1, wherein the conference state acquisition means acquires the conference state based on images of the voices of the plurality of participants and / or faces of the participants.

10. 1. A remote conferencing method executed by a computer for providing a remote conferencing service in which multiple participants can participate, comprising: a group information determination step of determining group information regarding a plurality of groups, each of which is composed of a plurality of two or more of the participants; a conference status acquisition step of acquiring a conference status indicating a conference status in each of the plurality of groups; a modifying step of generating modified information by modifying the group information based on the conference status of each of the plurality of groups; a support step of notifying a change in the group information; The change step includes a step of determining an assistant from among the plurality of participants based on the conference state in each of the plurality of groups, and a step of determining an assistant group from among the plurality of groups other than the group to which the assistant belonged before the change based on the conference state in each of the plurality of groups.

11. A remote conference program executed on a computer that provides a remote conference service in which multiple participants can participate, a group information determination step of determining group information regarding a plurality of groups, each of which is composed of a plurality of two or more of the participants; a conference status acquisition step of acquiring a conference status indicating a conference status in each of the plurality of groups; a modifying step of generating modified information by modifying the group information based on the conference status of each of the plurality of groups; a support step of notifying a change in the group information; The change step includes a step of determining an assistant from among the plurality of participants based on the conference state in each of the plurality of groups, and a step of determining an assistant group from among the plurality of groups other than the group to which the assistant belonged before the change based on the conference state in each of the plurality of groups.

Images