Teleconference system, teleconference method, and recording medium

Abstract

The present invention is for adjusting to promote progress of a conference equally among a plurality of groups. A remote conference system capable of being attended by a plurality of participants has a group information decision section (251) that decides group information related to a plurality of groups each composed of two or more of the plurality of participants, a conference state acquisition section (253) that acquires a conference state indicating a state of a conference in each of the plurality of groups, a change section (255) that changes the group information based on the conference state in each of the plurality of groups, and a support section (257) that notifies of a portion of the group information that has been changed.

Description

Technical Field

[0001] This invention relates to a remote conferencing system, a remote conferencing method, and a remote conferencing program, particularly to a remote conferencing system with group discussion room functionality, a remote conferencing method executed by the remote conferencing system, and a remote conferencing program that enables a computer to execute the remote conferencing method. Background Technology

[0002] In recent years, remote conferencing systems, which allow multiple participants in separate locations to hold meetings via computer, have become increasingly popular. Furthermore, technologies supporting the advancement of remote conferencing have been developed. For example, Japanese Patent Application Publication No. 2007-325063 discloses a remote conferencing system that connects multiple locations via a network. The system is characterized by having a server connected to the network at each location and a display device connected to the server. The server includes a recording unit that records data related to the participants' speeches for each participant, and a first display control unit that displays the data recorded by the recording unit at each location on the display device of the meeting host, a specific participant, or all participants.

[0003] On the other hand, in remote conferencing systems, there are known breakout room functions that divide meeting participants into multiple groups, with each group participating in a separate discussion room. In the remote conferencing system described in Japanese Patent Application Publication No. 2007-325063, while the breakout room function allows for smooth conferencing within each of the multiple groups, there are instances where the status of the meetings differs between groups. Consequently, there are situations where meeting end times vary significantly, and the outcomes of the meetings differ considerably.

[0004] Patent document 1: Japanese Patent Application Publication No. 2007-325063. Summary of the Invention

[0005] One of the objectives of this invention is to provide a remote conferencing system that can be adjusted to advance the progress of a meeting evenly across multiple groups.

[0006] Another objective of this invention is to provide a remote conferencing method that can be adjusted to advance the progress of a meeting evenly across multiple groups.

[0007] Another objective of this invention is to provide a remote conferencing program that can be adjusted to advance the progress of a meeting in a balanced manner across multiple groups.

[0008] According to one aspect of the invention, a remote conferencing system is a remote conferencing system that can be attended by multiple participants, comprising: a group information determination unit that determines group information related to multiple groups consisting of two or more participants; a meeting status acquisition unit that acquires meeting status indicating the meeting status of each of the multiple groups; a change unit that generates change information that changes the group information based on the meeting status of each of the multiple groups; and a support unit that notifies the change portion of the group information.

[0009] Based on this aspect, change information is generated after changing two or more participants who constitute multiple groups, according to the respective meeting status of each group. Therefore, it is possible to change participants between groups with good and poor meeting status. Furthermore, due to the notification and group information change component, it is possible to urge changes to two or more participants who constitute multiple groups, thereby improving the meeting status of groups with poor status. As a result, a remote conferencing system is provided that can improve the meeting status of multiple groups and adjust to promote the progress of meetings evenly across multiple groups.

[0010] Preferably, the change unit determines at least one assistant among multiple participants based on the meeting status of each of the multiple groups, and determines an assistant group among the multiple groups that is different from the group to which the assistant belonged before the change, based on the meeting status of each of the multiple groups.

[0011] Based on this, at least one assistant is selected from multiple participants, and an assistant group is selected from multiple groups that is different from the group the assistant belonged to before the change. Therefore, by changing the group information so that the assistant belongs to an assistant group, generating change information for the changed group information becomes easy.

[0012] Preferred: The meeting status includes a status index. The better the status of two or more participants in a group, the larger the status index value. The change unit determines two object groups with different status indices from multiple groups. A participant from the object group with the larger status index is determined as an assistant, and the other object group with the smaller status index is determined as the assistant group.

[0013] Based on this approach, two groups with different state indices are selected from multiple groups. A participant from the group with the higher state index is designated as an assistant, and the participant from the group with the lower state index is designated as the assistant group. Therefore, it is possible to assign a candidate from a participant's well-being group to a participant's poor-being group. Thus, support can be provided to improve the meeting status of the group whose participants are in a poor state.

[0014] Preferably, the state index includes a first state index and a second state index that represent different states respectively. The change unit generates change information based on the first state index in the first period and generates change information based on the second state index in the second period after the first period.

[0015] Based on this, change information is generated based on a first state index during the first period, and change information is generated based on a second state index during the second period following the first period. Therefore, multiple groups are compared with different states during different periods. Thus, it is possible to compare multiple groups with meeting states corresponding to the progress of the meeting.

[0016] Preferred option: Multiple participants include pre-determined auxiliary candidates. The change unit determines one object group from one or more groups to which the auxiliary candidates belong, and another object group from one or more groups to which the auxiliary candidates do not belong.

[0017] Based on this, the group to which the assistant candidate belongs is determined from one or more groups to which the assistant candidate belongs, and the assistant group is determined from one or more groups to which the assistant candidate does not belong. Therefore, the determination of the assistant and the assistant group becomes easy.

[0018] Preferred option: The change unit determines the group with the largest state index among the more than one groups to which the auxiliary candidate belongs as an object group, and determines the group with the smallest state index among the more than one groups to which the auxiliary candidate does not belong as another object group.

[0019] Based on this, the group with the highest state index among the multiple groups to which the assistant candidate belongs is determined as the assistant group, and the group with the lowest state index among the multiple groups to which the assistant candidate does not belong is determined as the assistant group. Therefore, the determination of the assistant and the assistant group becomes easy.

[0020] Preferred option: The change unit determines the meeting status of each of the multiple groups at each specified time.

[0021] Based on this, since the meeting status of each of the multiple groups is judged according to each specified time, the group information can be changed according to the meeting status that changes as time goes by.

[0022] Preferred option: The change unit generates change information based on the meeting status of any one of the multiple groups becoming a specified status.

[0023] Based on this, change information is generated when the meeting status of any one of the multiple groups becomes a specified state, so group information can be changed according to the meeting status that changes over time.

[0024] Preferably, it also includes a history storage unit for storing historical information of groups containing group information and meeting status, and a learning unit for using historical information as teacher data to learn the prescribed status.

[0025] Based on this, the historical information of groups, including group information and meeting status, is used as teacher data to learn the timing for generating change information, so that group information can be changed at appropriate times.

[0026] Preferably, the meeting status acquisition unit acquires the meeting status based on the individual voices of multiple participants and / or images obtained by capturing the faces of the participants.

[0027] Based on this, the meeting status can be appropriately obtained.

[0028] According to other aspects of the invention, the remote conferencing method is a remote conferencing method performed by a computer that provides remote conferencing services that multiple participants can attend, causing the computer to perform the following steps: a group information determination step, determining group information associated with multiple groups consisting of two or more participants respectively; a meeting status acquisition step, acquiring meeting status indicating the meeting status of each of the multiple groups; a change step, generating change information that changes the group information based on the meeting status of each of the multiple groups; and a support step, notifying the changed group information.

[0029] Based on this, a remote conferencing method can be provided that can be adjusted to promote the progress of the meeting in a balanced manner across multiple groups.

[0030] According to other aspects of the invention, the remote conferencing program is a remote conferencing program executed by a computer that provides remote conferencing services that multiple participants can join, causing the computer to perform the following steps: a group information determination step, determining group information associated with multiple groups consisting of two or more participants respectively; a meeting status acquisition step, acquiring meeting status indicating the meeting status of each of the multiple groups; a change step, generating change information that changes the group information based on the meeting status of each of the multiple groups; and a support step, notifying the changed group information.

[0031] Based on this, it is possible to provide remote conferencing programs that can be adjusted to promote the progress of meetings in a balanced manner across multiple groups. Attached Figure Description

[0032] Figure 1 This is a diagram illustrating an example of the system structure of a remote conferencing system according to one embodiment of the present invention.

[0033] Figure 2 This is a diagram illustrating an example of a remote conferencing system according to one embodiment of the present invention.

[0034] Figure 3 This is a block diagram representing an example of the hardware structure of a server.

[0035] Figure 4 This is a block diagram representing an example of the hardware structure of a PC.

[0036] Figure 5 This diagram illustrates an example of the functions of the CPU in the server described in this embodiment.

[0037] Figure 6 This is a flowchart illustrating an example of the group adjustment process.

[0038] Figure 7 This is a flowchart illustrating an example of the process for obtaining and processing meeting status.

[0039] Figure 8 This is a flowchart illustrating an example of the process for generating and processing change information.

[0040] Figure 9 This is a flowchart illustrating an example of the change information generation and processing flow in the first variation.

[0041] Explanation of reference numerals in the attached figures: 1… Remote conferencing system, 5… Internet, 100-1 to N… PC, 200… Server, 108… Camera, 109… Speaker, 110… Microphone, 201… CPU, 202… ROM, 203… RAM, 204… HDD, 205… Communication unit, 206… Display unit, 207… Operation unit, 210… External storage device, 211… CD-ROM, 213… Bus, 251… Group information decision unit, 253… Conference status acquisition unit, 255… Change unit, 257… Support unit. Detailed Implementation

[0042] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the following description, the same reference numerals are used for the same components. Their names and functions are also the same. Therefore, their detailed descriptions will not be repeated.

[0043] Figure 1 This is a diagram illustrating an example of the system architecture of a remote conferencing system according to one embodiment of the present invention. (Refer to...) Figure 1 The remote conferencing system 1 includes a server 200 and personal computers (hereinafter referred to as "PCs") 100-1, 100-2, 100-3, 100-4 to 100-N. Here, N is a positive integer, specifically 5 or higher. The server 200 and PCs 100-1 to N are connected to the Internet 5 and can communicate with each other.

[0044] PC100-1 to PC100-N are equipped with a camera, a microphone for collecting sound, and a speaker for outputting sound, respectively. PC100-1 to PC100-N are general-purpose computers with the same main hardware structure and functions, so they are collectively referred to as PC100.

[0045] Furthermore, instead of PC100-1 to N, a device equipped with a camera, microphone, speaker, and communication functions could be used, such as a PDA (Personal Digital Assistant) or a smartphone. Additionally, it is not limited to the Internet; any other network could be used as long as the server 200 and PC100-1 to N can communicate with each other. For example, a local area network (LAN) or a wide area network (WAN) could also be used.

[0046] Figure 2 This is a diagram illustrating an example of a remote conferencing system according to one embodiment of the present invention. (Refer to...) Figure 2 In the remote conferencing system 1, participants access the meeting by operating any one of PCs 100-1 to N. In this embodiment, the remote conferencing system 1 will be described as follows: The participants include a meeting administrator and at least one facilitator who assists in the meeting's progress. When the remote conferencing system 1 uses the breakout room function to conduct meetings in multiple groups, the facilitator belongs to any group and assists in the progress of that group's meeting. The administrator, facilitator, and participants are predetermined, and the administrator sets the participants and facilitator in the remote conferencing system 1. Here, the example will be the participant operating PC 100-1 who is both the administrator and the facilitator.

[0047] Each of PCs 100-1 to N has a program installed for participating in the meeting. PCs 100-1 to N communicate with server 200 to hold the meeting. In addition to a dedicated program for communicating with server 200, the programs installed on PCs 100-1 to N can also be general browser programs if server 200 provides web services.

[0048] Server 200 implements a remote conferencing system by executing a remote conferencing program. Server 200 communicates with PCs 100-1 to N, and sends the data received from PCs 100-1 to N to the other PCs 100-1 to N respectively.

[0049] The data transmitted and received between each PC100-1 to N and the server 200 includes audio information representing sound, video information representing images, and application data. The data transmitted and received between each PC100-1 to N and the server 200 can be either compressed or uncompressed.

[0050] Server 200 aggregates the data sent to each PC 100-1 to N. For example, server 200 sends the audio data received from each PC 100-1 to N to all PCs 100-1 to N. In addition, regarding video information, server 200 aggregates the video data received from each PC 100-1 to N and sends it to each PC 100-1 to N separately.

[0051] Server 200 determines and sends the images and application data based on requests from each PC 100-1 to N. Therefore, the images displayed on each PC 100-1 to N may be the same or different. Furthermore, each PC 100-1 to N can also process the image information received from server 200 and display it. In this case, no image information processing is required on server 200, thus reducing the load.

[0052] Server 200 stores group information related to multiple groups A through E and determines the respective meeting status of each group. Group information identifies multiple participants belonging to each of the multiple groups A through E. Server 200 determines the respective meeting status of groups A through E based on audio and video information received from PCs 100-1 through N. Here, we show the case where the meeting status of group A is determined to be 91%, the meeting status of group B is determined to be 78%, the meeting status of group C is determined to be 18%, the meeting status of group D is determined to be 42%, and the meeting status of group E is determined to be 36%.

[0053] Server 200 determines assistants and auxiliary groups based on the meeting states of multiple groups A through E, and sends notifications accordingly. Server 200 obtains the group to which the facilitator belongs (groups D and E) based on the facilitator information indicating which participant is the facilitator. Server 200 designates group D, which has the highest meeting state among groups D and E to which the facilitator belongs, as the support group, and designates the facilitator belonging to the support group as an assistant. Furthermore, Server 200 designates group C, which has the lowest meeting state among groups A through C to which the facilitator does not belong, as the auxiliary group. Then, it notifies the assistant or administrator of a message urging the assistant to move to the auxiliary group.

[0054] Figure 3 This is a block diagram illustrating an example of a server's hardware architecture. (See reference...) Figure 3The server 200 is a computer that performs computational processing and includes a central processing unit (CPU) 201 connected to the bus 213 for controlling the overall operation of the server 200, a ROM (Read Only Memory) 202 for storing programs executed by the CPU 201, a RAM (Random Access Memory) 203 used as the working area of ​​the CPU 201, an HDD 204 for non-volatile data storage, a communication unit 205 for connecting the CPU 201 to the Internet 5, a display unit 206 for displaying images, an operation unit 207 for accepting operation input, and an external storage device 210.

[0055] The communication unit 205 is an interface for connecting the server 200 to the Internet 5. Therefore, the CPU 201 can communicate with the PCs 100-1 to 100N connected to the Internet 5 via the communication unit 205.

[0056] External storage device 210 is equipped with CD-ROM (Compact Disk Read Only Memory) 211. CPU 201 controls external storage device 210 to read data stored in CD-ROM 211.

[0057] In this embodiment, CPU 201 executes a program stored in ROM 202 or HDD 204. Alternatively, CPU 201 can control external storage device 210 to read a program for CPU 201 to execute from CD-ROM 211, store the read program in RAM 203, and execute it.

[0058] Furthermore, CPU 201 downloads a program from a computer connected to the Internet 5 and stores it in HDD 204. Additionally, if a computer connected to the Internet 5 writes a program to HDD 204, the program is stored in HDD 204. CPU 201 can also load the program stored in HDD 204 into RAM 203 and execute it.

[0059] Furthermore, the recording medium for storing programs executed by the CPU 201 is not limited to the CD-ROM 211; it can also be a floppy disk, magnetic tape, optical disc (MO / MD / DVD), IC card, optical card, mask ROM, EPROM (Erasable Programmable Read-Only Memory), or other semiconductor memory media. The programs referred to here include not only programs that can be directly executed by the CPU 201, but also source programs, compressed programs, encrypted programs, etc.

[0060] Figure 4 This is a block diagram illustrating an example of the hardware architecture of a PC. (See reference...) Figure 4 PC100 is a computer that performs arithmetic processing and includes a CPU 101 connected to a bus 112 for controlling the overall operation of PC100, a ROM 102 for storing programs executed by the CPU 101, a RAM 103 used as the working area of ​​the CPU 101, an HDD 104 for non-volatile data storage, a communication unit 105 for connecting the CPU 101 to the Internet 5, a display unit 106 for displaying images, an operation unit 107 for accepting input from users, a camera 108 for capturing images of users, a speaker 109 for outputting sound, a microphone 110 for collecting the voice of the operator, and an external storage device 111.

[0061] External storage device 111 is equipped with CD-ROM 111A. CPU 101 controls external storage device 111 to read data stored in CD-ROM 111A.

[0062] Alternatively, at least two of the camera 108, speaker 109, and microphone 110 can be integrated into a module and connected to the PC 100. The module may include, for example, a headset receiver in which the speaker 109 and microphone 110 are integrated.

[0063] Figure 5 This diagram illustrates an example of the functions of the CPU in the server described in this embodiment. Figure 5 The function shown is implemented by the CPU 201, which executes a remote conferencing program stored in ROM 202, HDD 204, or CD-ROM 211, through the CPU 201 of the server 200. (See reference...) Figure 5 The CPU 201 includes a group information decision unit 251, a meeting status acquisition unit 253, a change unit 255, and a support unit 257.

[0064] The group information determination unit 251 determines group information related to multiple groups. Group information determines multiple groups and the participants belonging to each group. Here, group information includes participant records that establish a relationship between participant identification information used to identify the participant and group identification information used to identify the group for each participant. The participants in the meeting belong to any one of the multiple groups. Multiple groups are determined by any one of the multiple participants' managers. Here, the case where the participant operating PC100-1 is the manager is described as an example. The manager has prior knowledge of multiple participants and inputs participant identification information and group identification information into the remote conferencing system via PC100-1 according to the multiple participants. The group information determination unit 251 acquires the group information input by the manager into PC100-1 and outputs it to the meeting status acquisition unit 253 and the change unit 255. Furthermore, although this embodiment describes the case where the manager inputs group information into PC100-1 as an example, the group information determination unit 251 can also generate group information based on the information of multiple participants. For example, multiple participants can be identified based on meeting announcements and reply emails, and then grouped according to their respective attributes. Attributes include the participant's organization, work experience, and field of expertise.

[0065] The meeting status acquisition unit 253 acquires the meeting status representing the status of each of the multiple groups. The meeting status is determined based on the individual status of each participant belonging to a group. The participant's status is determined based on their voice and / or an image obtained by photographing their face. The meeting status acquisition unit 253 outputs the meeting status of each of the multiple groups to the change unit 255. Meeting information is information where a higher value indicates a better meeting status within a group.

[0066] The meeting status acquisition unit 253 acquires audio information representing the participants' voices and image information obtained from PCs 100-100A to 100N operated by multiple participants. PCs 100-1 to 100N respectively send audio information (converted from sound collected by microphone 110 into digital signals) and image information (converted from images of participants' faces captured by camera 108 into digital signals) to the server 200. The meeting status acquisition unit 253 stores the audio and image information received by the communication unit 205 from PCs 100-1 to 100N in association with the participants in HDD 204. The audio and image information can be stored in association with either PCs 100-1 to 100N or with the participants operating PCs 100-1 to 100N. During the login phase of the remote conferencing system where a participant logs in using any of PCs 100-1 to 100N, the server 200 authenticates and confirms the participant.

[0067] The meeting status is indicated by a first status index and a second status index. The first status index and the second status index are indices with different evaluation criteria. The meeting status acquisition unit 253 determines the first status index and the second status index for each group based on the individual statuses of the multiple participants belonging to that group.

[0068] In this embodiment, the first state index is used to represent the level of activity in the meeting, and the second state index is used to represent the level of agreement among the participants in the meeting. The greater the level of activity in the meeting, the larger the value of the first state index. The greater the level of agreement among the participants in the meeting, the larger the value of the second state index.

[0069] Specifically, let's take the case where the first state index is the speaking rate within a group as an example. The speaking rate is expressed as the proportion of speaking time per unit of time. The speaking time is calculated based on the participants' vocal information. The sum of the speaking rates of all participants belonging to each group is used as the first state index of the group. Furthermore, if one person's speaking rate is higher than that of the other participants, it cannot be said that the group is highly active. Therefore, the first state index of the group can also be the value after adjusting for the deviation of the individual speaking rates of the participants. For example, the first state index of the group can also be the value divided by the difference between the maximum value (or the average of the top few) of the individual speaking rates of the participants and the overall average value.

[0070] Furthermore, let's take the case where the second state index is the approval rating within a group as an example. Approval rating is a numerical value representing the degree of a participant's approval or agreement. For example, artificial intelligence can be used to learn from the participants' facial expressions and approval ratings represented by their video footage to determine their approval ratings. The average approval rating of each participant belonging to a group is then used as the group's second state index. Moreover, if there is a large discrepancy in approval ratings among participants, it cannot be said that the group's approval rating is high. Therefore, the group's second state index can also be an adjusted value, calculated based on the discrepancy in approval ratings among participants, using the average of the individual approval ratings of multiple participants.

[0071] The change unit 255 generates change information based on the meeting status change group information. The change unit 255 outputs the different change portions of the change information and group information to the support unit 257. The change unit 255 uses the first status index change group information of the meeting information in the first period and the second status index change group information of the meeting information in the second period. The second period is the period following the first period. The second period is the period from a predetermined time before the end time of the meeting to the end time. The first period is any period other than the second period, which is the period from the start time to the end time of the meeting. For example, if the predetermined time is thirty minutes, the first period is the period from the start time to thirty minutes before the end time, and the second period is the period from thirty minutes before the end time to the end time.

[0072] The change unit 255 determines whether to change group information at predetermined time intervals. The change unit 255 compares the meeting information of each of multiple groups with a predetermined threshold at predetermined time intervals. If the meeting information of any one of the multiple groups is below the threshold TH, it determines that the group information has changed. If the timing of the determination is a first period, then if the first state index of any one of the multiple groups is below the first threshold TH1, the change unit 255 determines that the group information has changed. If the timing of the determination is a second period, then if the second state index of any one of the multiple groups is below the second threshold TH2, the change unit 255 determines that the group information has changed.

[0073] As described above, the modification unit 255 modifies the group information using either a first state index and a first threshold TH1, or a second state index and a second threshold TH2. The processing order is the same whether the modification unit 255 modifies the group information using the first state index and the first threshold TH1, or using the second state index and the second threshold TH2. Hereinafter, the first state index and the second state index will be collectively referred to as the state index, and the first threshold TH1 and the second threshold TH2 will be collectively referred to as the threshold.

[0074] When group information is changed, the change unit 255 determines at least one assistant from among the participants based on the meeting status of each of the multiple groups. Additionally, when group information is changed, the change unit 255 determines an assistant group from among the multiple groups that is different from the group the assistant belonged to before the change, based on the meeting status of each of the multiple groups. The change unit 255 generates change information that modifies the group information to reflect the assistant's group affiliation.

[0075] Specifically, the modification unit 255 designates the promoter of the support group with the highest status index among the groups to which the promoter belongs (among multiple groups) is a facilitator. The facilitator is a participant responsible for controlling the progress of meetings within the group, and is pre-selected from multiple participants. It is sufficient for one or more facilitators to exist. The modification unit 255 does not designate a group to which a promoter, although belonging to one or more groups, belongs, but has already been designated as a facilitator, as a support group. In other words, the modification unit 255 designates the promoter of the support group with the highest status index among the groups to which the promoter, who has not been designated as a facilitator, belongs, as a facilitator. Furthermore, the modification unit 255 may designate a group as a support group even if the promoter of the group with the highest status index among the groups to which the promoter belongs is already designated as a facilitator.

[0076] The modification unit 255 determines the group with the smallest state index among more than one group to which the promoters of multiple groups do not belong as the auxiliary group. The modification unit 255 generates modification information that makes the group information such that the promoters belong to the auxiliary group. The modification unit 255 outputs the group information and modification information to the support unit 257. Furthermore, the modification unit 255 can also generate modification information by making any one of the multiple participants belonging to the auxiliary group belong to the support group. In this case, the number of participants in each of the multiple groups remains unchanged.

[0077] Support Department 257 compares the change information with the group information and notifies the participants of the different changes. Specifically, Support Department 257 compares the change information with the group information and determines the assistant and the assistant group as the changes. Support Department 257 then notifies the participants of a message urging the assistant to move to the assistant group.

[0078] For example, support unit 257 displays messages on PCs 100-1 to N. Alternatively, support unit 257 can notify only the assistants. In this case, support unit 257 displays messages on the devices operated by the assistants on PCs 100-1 to N. Assistants viewing the messages can change their group assignments based on their assigned assistant group. Alternatively, messages can be notified to the administrators managing the meeting. In this case, support unit 257 displays messages on the devices operated by the administrators on PCs 100-1 to N. Administrators viewing the messages can change their group assignments based on the assistants' assigned assistant groups. When group assignments are changed, since assistants are present in the assistant groups, it is expected that the meeting's progress will improve due to increased discussion during the meeting. Therefore, by notifying the meeting through support unit 257, the progress of the meeting can be balanced across multiple groups.

[0079] When the group composition is changed by the manager or assistant, the group information decision unit 251 obtains the new group information after the manager's change. In the case of a change in group information, the group information and status information before and after the change are stored in HDD 204. Furthermore, the status information of the group after storing the group information and status information before and after the change is also stored in HDD 204. Thus, HDD 204 stores both the group information and status information before the change and the group information and status information after the change. Artificial intelligence can also be used to learn thresholds for determining whether to change group information or assistant movement timing based on the group information and status information before and after the change. The thresholds for determining whether to change group information include, for example, a first threshold TH1 for a first status index and a second threshold TH2 for a second status index.

[0080] Figure 6 This is a flowchart illustrating an example of a group adjustment process. The group adjustment process is performed by the CPU 201, which executes a remote conferencing program stored in ROM 202, HDD 204, or CD-ROM 211, via the CPU 201 of the server 200. (See reference...) Figure 6 The CPU 201 of server 200 determines the group information (step S01) and initiates the process to step S02. Group information is generated, comprising participant records that establish a correlation between participant identification information used to identify each participant and group identification information used to identify the group. The group information is determined by any administrator among the meeting participants and input into the remote conferencing system 1. For example, when an administrator operates PC 100, and the administrator inputs group information containing participant records for each of multiple participants into the remote conferencing system via PC 100-1, the group information received from PC 100 is retrieved.

[0081] In step S02, audio and video information are acquired for each participant, and the process proceeds to step S03. CPU 201 controls communication unit 205, which acquires audio information representing the participant's voice received from PCs 100-1 to N, and video information obtained by capturing the participant's face. The audio and video information are stored in HDD 204 in association with the participant.

[0082] In step S03, it is determined whether a predetermined time has elapsed. This involves determining whether a predetermined time has elapsed since the start of the group adjustment process, or since the execution of step S03 in the past. If the predetermined time has elapsed, the process proceeds to step S04; otherwise, the process returns to step S01. Step S04 and subsequent processes are executed at predetermined time intervals.

[0083] In step S04, the meeting status acquisition process is performed, and the process proceeds to step S05. Although the meeting status acquisition process is described in detail later, it involves calculating a status index. The status index includes a first status index and a second status index, which represents a different level of evaluation than the first status index. In this embodiment, the first status index is an index that indicates a higher level of activity in the meeting, and is the speaking rate within the group. The second status index represents the degree of approval or agreement among the participants, and is a numerical value representing the level of approval or agreement. Regarding the second status index, artificial intelligence learns from images of participants' faces and their approval levels, determining the approval level based on facial expressions read from the participants' faces.

[0084] In step S05, it is determined whether the current time is within the first period. If it is within the first period, the process proceeds to step S06; otherwise, it proceeds to step S07. In step S06, the first state index is set as the state index, and the process proceeds to step S10. In step S07, it is determined whether the current time is within the second period. If it is within the second period, the process proceeds to step S08; otherwise, the process returns to step S01. The second period is the period following the first period. In step S08, the second state index is set as the state index, and the process proceeds to step S09.

[0085] In step S09, it is determined whether the minimum state index is below a threshold. If the first state index is set as state information, it is determined whether the minimum first state index is below the first threshold TH1. If the second state index is set as state information, it is determined whether the minimum second state index is below the second threshold TH2. If the minimum state index is below the threshold, the process proceeds to step S10; otherwise, the process returns to step S01.

[0086] In step S10, a change information generation process is performed, and the process proceeds to step S11. Although the details of the change information generation process are described later, it is a process that generates change information based on the status information, indicating a change in group information. By performing the change information generation process, at least one assistant and one assistant group are determined. In step S11, the change portion is notified, and the process proceeds to step S12. The change portion is the part that differs between the change information and the group information. Specifically, a message urging the assistant to move to the assistant group is sent to the participants. For example, the message is displayed on PCs 100-1 to N where the participants are operating. Alternatively, the message may not be sent to all participants, but only to the assistant or the administrator. The assistant or the administrator can set up an operation in the remote conferencing system to change the assistant's group to the assistant group. In addition, if the assistant's movement timing is determined by artificial intelligence, the movement timing is notified to the assistant.

[0087] In step S12, it is determined whether the assistant has moved to a new group. If the assistant has moved to a new assistant group, the process proceeds to step S13; otherwise, it returns to step S01. In step S13, the group information is updated, and the process proceeds to step S14. The group information is changed to reflect the assistant's group affiliation. In step S14, the meeting status and group information are stored in HDD204, and the process returns to step S01.

[0088] Figure 7 This is a flowchart illustrating an example of the meeting status acquisition process. The meeting status acquisition process is performed in step S04 of the group adjustment process. (See reference...) Figure 7 Select the group to be processed (step S21), and proceed to step S22. Select the group to be processed from the multiple groups determined based on the group information.

[0089] In step S22, a first state index is calculated, and the process proceeds to step S23. The first state index is an index representing the level of activity in the meeting, and is the speaking rate within a group. The speaking rate is expressed as a proportion of speaking time per unit of time. Based on the audio information acquired from PC100-1 to N, the speaking rate of each participant is calculated. The first state index of each group is the sum of the speaking rates of the participants belonging to that group.

[0090] In step S23, the second state index is calculated, and the process proceeds to step S24. The second state index is an index representing the degree of approval of each participant in the meeting; it is the approval level within a group. Approval level is a numerical value representing the degree of approval or agreement felt by a participant. The approval level of each participant is calculated based on image information acquired from PC100-1 to N. For example, approval level can be determined based on the participant's facial features using artificial intelligence learning. The second state index for each of the multiple groups is the average of the approval levels of the multiple participants belonging to that group.

[0091] In step S24, it is determined whether there are any groups that have not been selected as processing objects. If there are unselected groups, the process returns to step S21; otherwise, the process returns to the group adjustment process. Therefore, a first state index and a second state index are calculated as the respective meeting states of the multiple groups determined based on the group information.

[0092] Figure 8 This is a flowchart illustrating an example of the change information generation process. The change information generation process is performed in step S10 of the group adjustment process. (See reference...) Figure 8 Set the number of promoters for the constant N (step S31), and proceed to step S32. Based on the group information, determine one or more participants as promoters.

[0093] In step S32, variable i is set to "1", and the process proceeds to step S33. Variable i represents the number of promoters who become the subjects of processing, and is incremented by 1 in step S35 or step S40 described later. In step S33, groups that become the subjects of processing are selected sequentially, starting from the group with the smaller state index among multiple groups.

[0094] In step S33, groups are selected as the processing targets according to their state indices in ascending order. In the next step S34, it is determined whether a promoter exists in the selected group. If a promoter exists, the process proceeds to step S35; otherwise, it proceeds to step S37. In step S35, variable i is incremented by 1, and the process proceeds to step S36. In step S36, it is determined whether variable i is greater than the constant N. If variable i is greater than the constant N, the process returns to the group adjustment process; otherwise, it returns to step S33.

[0095] In step S37, the group selected as the processing target is designated as an auxiliary group, and the process proceeds to step S38. In step S38, starting from the group with the largest state index among multiple groups, groups are sequentially selected as processing targets, and the process proceeds to step S39. In step S39, it is determined whether there is a promoter among the groups selected in step S38. If there is a promoter, the process proceeds to step S40; otherwise, the process returns to step S38.

[0096] In step S40, variable i is incremented by 1, and the process proceeds to step S41. In step S41, it is determined whether the promoter belonging to the group selected in step S38 has completed its move. Promoters that have been determined to be assistants are deemed to have completed their moves. If the move is completed, the process proceeds to step S42; otherwise, the process proceeds to step S43. In step S42, it is determined whether variable i is greater than the constant N. If variable i is greater than the constant N, the process returns to the group adjustment process; otherwise, the process returns to step S38.

[0097] In step S43, the group selected in step S38 is designated as the support group, and the process proceeds to step S44. In step S44, the promoters belonging to the support group are designated as assistants, and the process proceeds to step S45. In step S45, it is determined whether variable i is greater than the constant N. If variable i is greater than the constant N, the process returns to the group adjustment process; otherwise, the process returns to step S33.

[0098] <First Variation>

[0099] In the above embodiments, the propeller belongs to at least one of multiple groups. The change information generation process in the first variation is performed when the propeller does not belong to multiple groups.

[0100] Figure 9This is a flowchart illustrating an example of the change information generation and processing flow in the first variation. (See also...) Figure 9 Set the number of groups for the constant N (step S31A), and proceed to step S32.

[0101] In step S32, variable i is set to "1", and the process proceeds to step S33. Variable i is a value representing the number of groups that become processing objects, and it is incremented by 1 in step S40 (described later). In step S33, groups that become processing objects are selected sequentially, starting from the group with the smaller state index among the multiple groups, and the process proceeds to step S34A. In the first period, groups that become processing objects are selected sequentially, starting from the group with the smaller first state index; in the second period, groups that become processing objects are selected sequentially, starting from the group with the smaller second state index.

[0102] In step S34A, it is determined whether the state index is below a threshold. During the first period, the first state index is compared with a first threshold TH1; during the second period, the second state index is compared with a second threshold TH2. If the state index is below the threshold, the process proceeds to step S37; otherwise, the process returns to the group adjustment process. In step S37, the group selected as the processing target in step S33 is determined as an auxiliary group, and the process proceeds to step S38. In step S38, starting from the group with the larger state index among multiple groups, groups are sequentially selected as processing targets, and the process proceeds to step S40. During the first period, groups are sequentially selected as processing targets starting from the group with the larger first state index; during the second period, groups are sequentially selected as processing targets starting from the group with the larger second state index. In step S40, variable i is incremented by 1, and the process proceeds to step S43.

[0103] In step S43, the group selected in step S38 is designated as the support group, and the process proceeds to step S44. In step S44, any one of the participants belonging to the support group is designated as an assistant, and the process proceeds to step S45A. For example, regardless of whether it is the first period or the second period, the participant with the highest speaking rate is designated as the assistant. In step S45A, it is determined whether variable i is greater than 1 / 2 of the constant N. If variable i is greater than 1 / 2 of the constant N, the process returns to the group adjustment process; otherwise, the process returns to step S33.

[0104] <Second Variation>

[0105] In the above implementation, the meeting state includes a first state index and a second state index. The first state index is calculated based on the speaking rate of each participant belonging to the group, and the second state index is calculated based on the approval rate of each participant belonging to the group. The meeting state is not limited to this.

[0106] The meeting status can also be determined based on the conversation. It can also be determined based on character information obtained from recognizing participants' voice information. Furthermore, it can be determined based on the connection status of the PCs 100-1 to N operated by the participants. Additionally, it can be determined based on the volume of the participants' voices. Finally, it can be determined based on the participants' emotional state, such as joy, anger, sorrow, or happiness.

[0107] <Third Variation>

[0108] In the above implementation, the meeting status is determined using different indicators during the first and second periods. The number of periods and indicators is not limited to this. The meeting status can also be determined using more periods and more different meeting statuses.

[0109] As explained above, according to one aspect of the invention in this embodiment, the remote conferencing system 1 includes a server 200 and PCs 100-1 to N, and multiple participants can operate PCs 100-1 to N to participate in the meeting. The server 200 determines group information related to multiple groups, each consisting of two or more participants, acquires meeting status indicating the meeting status of each of the multiple groups, generates change information based on the meeting status of each of the multiple groups, and notifies the group information change unit. Therefore, it is possible to change participants between groups with good meeting status and groups with poor meeting status. Furthermore, by notifying the group information change unit, it is possible to urge changes of two or more participants constituting the multiple groups to improve the meeting status of groups with poor meeting status. Therefore, it is possible to ensure that the meeting status of each of the multiple groups is good, and to adjust the progress of the meeting to proceed evenly among the multiple groups.

[0110] Furthermore, server 200 determines at least one assistant among multiple participants based on the meeting status of each of the multiple groups, and determines an assistant group within the multiple groups that is different from the group the assistant belonged to before the change, also based on the meeting status of each of the multiple groups. Therefore, changing the group information to indicate that the assistant belongs to an assistant group makes generating change information for the changed group information easier.

[0111] Furthermore, the meeting status includes a first status index and a second status index, where a higher value indicates a better status for two or more participants forming a group. During the first period, server 200 determines two object groups with different first status indices from multiple groups, designating a participant from the object group with the larger first status index as an assistant and the other object group with the smaller first status index as an assistant group. In the second period following the first, server determines two object groups with different second status indices from multiple groups, designating a participant from the object group with the larger second status index as an assistant and the other object group with the smaller second status index as an assistant group. Therefore, it is possible to assign a candidate from a participant's good status group to a participant's bad status group. Thus, support can be provided in a way that improves the meeting status of participants with bad status groups. Additionally, change information is generated based on the first status index during the first period and based on the second status index during the second period following the first, allowing comparison of multiple groups based on different statuses at different times. Therefore, it is possible to compare multiple groups based on the meeting status corresponding to the progress of the meeting. For example, if the first state index is the activity level of the meeting and the second state index is the participants' approval level, support is provided in the initial first period by actively engaging in discussions in groups where discussions were not active, and in the subsequent second period by bringing consensus to groups where multiple participants in a group disagree.

[0112] In addition, multiple participants include pre-determined promoters (assistant candidates). Server 200 determines one object group from more than one group to which the promoter belongs, and another object group from more than one group to which the promoter does not belong. Therefore, the determination of assistants and auxiliary groups becomes easy.

[0113] Furthermore, server 200 determines the group with the highest status index among the more than one groups to which the support candidate belongs as one object group, and determines the group with the lowest status index among the more than one groups to which the support candidate does not belong as another object group. For example, server 200 determines the group with the highest status index among the more than one groups to which the promoter belongs as the support group to which the supporter belongs, and determines the group with the lowest status index among the more than one groups to which the promoter does not belong as the support group. Therefore, the determination of the supporter and the support group becomes easy.

[0114] In addition, server 200 determines the meeting status of each of the multiple groups at each specified time. Therefore, it is possible to update group information based on the changing meeting status over time.

[0115] Furthermore, server 200 generates change information based on the meeting status of any one of the multiple groups reaching a predetermined state. Specifically, server 200 generates change information when, during a first period, the first state index of any one of the multiple groups falls below a first threshold TH1, and during a second period, the second state index of any one of the multiple groups falls below a second threshold TH2. Therefore, group information can be changed based on the meeting status changing over time.

[0116] Additionally, server 200 stores historical group information, including group information and meeting status, in HDD 204. This historical information is used as teacher data to learn the first threshold TH1 and the second threshold TH2. Therefore, group information can be changed at appropriate time intervals.

[0117] Additionally, server 200 obtains the meeting status based on the individual voices of multiple participants and / or images obtained by capturing the participants' faces. Therefore, it is possible to appropriately obtain the meeting status.

[0118] The embodiments disclosed herein should be considered illustrative of all points and not restrictive embodiments. The scope of the invention is defined not by the foregoing description but by the claims, and includes all modifications within the meaning and scope of the claims.

Claims

1. A remote conferencing system, wherein multiple participants can join, The aforementioned remote conferencing system has the following features: The group information determination unit determines the group information associated with multiple groups consisting of two or more of the aforementioned participants. The meeting status acquisition unit acquires the meeting status representing the status of each of the above groups; The change unit generates change information that modifies the information of the aforementioned groups based on the meeting status of each of the aforementioned groups. as well as The support unit notifies the portion of the information related to the aforementioned groups of changes. The aforementioned change unit determines at least one assistant among the participants based on the meeting status of each of the aforementioned groups, and determines an assistant group among the aforementioned groups that is different from the group to which the assistant belonged before the change, based on the meeting status of each of the aforementioned groups.

2. The remote conferencing system according to claim 1, wherein, The aforementioned meeting status includes a status index; the better the status of two or more participants constituting the aforementioned group, the higher the value of the aforementioned status index. The aforementioned change unit determines two object groups with different state indices from among the multiple aforementioned groups, designates one participant belonging to the object group with the larger state index as the assistant, and designates the other object group with the smaller state index as the assistant group.

3. The remote conferencing system according to claim 2, wherein, The aforementioned state indices include a first state index and a second state index, each representing a different state. The aforementioned change unit generates the aforementioned change information based on the aforementioned first state index during the first period, and generates the aforementioned change information based on the aforementioned second state index during the second period following the aforementioned first period.

4. The remote conferencing system according to claim 2, wherein, Several of the aforementioned participants included pre-selected auxiliary candidates. The aforementioned change unit determines one of the aforementioned target groups from one or more of the aforementioned groups to which the aforementioned auxiliary candidate belongs, and determines another of the aforementioned target groups from one or more of the aforementioned groups to which the aforementioned auxiliary candidate does not belong.

5. The remote conferencing system according to claim 4, wherein, The aforementioned modification unit determines the group with the largest status index among the more than one groups to which the aforementioned auxiliary candidate belongs as one of the aforementioned object groups, and determines the group with the smallest status index among the more than one groups to which the aforementioned auxiliary candidate does not belong as another of the aforementioned object groups.

6. The remote conferencing system according to any one of claims 1 to 5, wherein, The aforementioned change unit determines the status of the aforementioned meetings in each of the aforementioned groups at each specified time.

7. The remote conferencing system according to any one of claims 1 to 5, wherein, The aforementioned change unit generates the aforementioned change information based on the meeting status of any one of the aforementioned groups becoming a specified state.

8. The remote conferencing system according to claim 7, wherein, It also has: Historical record storage unit stores historical record information, including the aforementioned group information and the aforementioned meeting status group; and The learning unit uses the aforementioned historical data as teacher data to learn the aforementioned prescribed states.

9. The remote conferencing system according to any one of claims 1 to 5, wherein, The meeting status acquisition unit acquires the meeting status based on the individual voices of the participants and / or images obtained by capturing their faces.

10. A remote conferencing method, which is a remote conferencing method performed by a computer providing remote conferencing services that can be attended by multiple participants, wherein, Have the above computer perform the following steps: The group information determination step determines the group information associated with multiple groups, each consisting of two or more of the aforementioned participants. The meeting status acquisition step involves acquiring the meeting status representing the status of each of the multiple groups mentioned above. The change process involves generating change information for the aforementioned groups based on the meeting status of each of the aforementioned groups; as well as Support steps, notification of changes to the above group information, In the above change steps, at least one assistant among the participants is determined based on the meeting status of each of the multiple groups, and an assistant group is determined from among the multiple groups that is different from the group to which the assistant belonged before the change, based on the meeting status of each of the multiple groups.

11. The remote conferencing method according to claim 10, wherein, The aforementioned meeting status includes a status index; the better the status of two or more participants constituting the aforementioned group, the higher the value of the aforementioned status index. In the above-mentioned change steps, two object groups with different state indices are selected from the multiple groups mentioned above. One participant from the object group with the larger state index is selected as the assistant, and the other object group with the smaller state index is selected as the assistant group.

12. The remote conferencing method according to claim 11, wherein, The aforementioned state indices include a first state index and a second state index, each representing a different state. In the above-mentioned change steps, the change information is generated based on the first state index during the first period, and the change information is generated based on the second state index during the second period after the first period.

13. The remote conferencing method according to claim 11, wherein, Several of the aforementioned participants included pre-selected auxiliary candidates. In the above-mentioned change steps, one of the above-mentioned target groups is determined from one or more of the above-mentioned groups to which the above-mentioned auxiliary candidate belongs, and another of the above-mentioned target groups is determined from one or more of the above-mentioned groups to which the above-mentioned auxiliary candidate does not belong.

14. The remote conferencing method according to claim 13, wherein, In the above-mentioned change steps, the group with the largest status index among the more than one group to which the auxiliary candidate belongs is determined as one object group, and the group with the smallest status index among the more than one group to which the auxiliary candidate does not belong is determined as another object group.

15. The remote conferencing method according to any one of claims 10 to 14, wherein, In the above change steps, the meeting status of each of the above groups is determined according to each specified time.

16. The remote conferencing method according to any one of claims 10 to 14, wherein, In the above change steps, the above change information is generated based on the meeting status of any one of the above groups becoming the specified status.

17. The remote conferencing method according to claim 16, wherein, It also has: The historical record storage step stores historical record information, which includes the aforementioned group information and the aforementioned meeting status group; and The learning process involves using the aforementioned historical data as teacher data to learn the prescribed states.

18. The remote conferencing method according to any one of claims 10 to 14, wherein, In the above-mentioned meeting status acquisition step, the meeting status is acquired based on the individual voices of the participants and / or images obtained by photographing their faces.

19. A computer-readable recording medium storing a remote conferencing program, wherein the recording medium stores a remote conferencing program executed by a computer providing remote conferencing services that can be attended by multiple participants, and wherein... Have the above computer perform the following steps: The group information determination step determines the group information associated with multiple groups, each consisting of two or more of the aforementioned participants. The meeting status acquisition step involves acquiring the meeting status representing the status of each of the multiple groups mentioned above. The change process involves generating change information for the aforementioned groups based on the meeting status of each of the aforementioned groups; as well as Support steps, notification of changes to the above group information, In the above change steps, at least one assistant among the participants is determined based on the meeting status of each of the multiple groups, and an assistant group is determined from among the multiple groups that is different from the group to which the assistant belonged before the change, based on the meeting status of each of the multiple groups.

20. The computer-readable recording medium storing a remote conferencing program as described in claim 19, wherein, The aforementioned meeting status includes a status index; the better the status of two or more participants constituting the aforementioned group, the higher the value of the aforementioned status index. In the above-mentioned change steps, two object groups with different state indices are selected from the multiple groups mentioned above. One participant from the object group with the larger state index is selected as the assistant, and the other object group with the smaller state index is selected as the assistant group.

21. The computer-readable recording medium storing a remote conferencing program as described in claim 20, wherein, The aforementioned state indices include a first state index and a second state index, each representing a different state. In the above-mentioned change steps, the change information is generated based on the first state index during the first period, and the change information is generated based on the second state index during the second period after the first period.

22. The computer-readable recording medium storing a remote conferencing program as described in claim 20, wherein, Several of the aforementioned participants included pre-selected auxiliary candidates. In the above-mentioned change steps, one of the above-mentioned target groups is determined from one or more of the above-mentioned groups to which the above-mentioned auxiliary candidate belongs, and another of the above-mentioned target groups is determined from one or more of the above-mentioned groups to which the above-mentioned auxiliary candidate does not belong.

23. The computer-readable recording medium storing a remote conferencing program as described in claim 22, wherein, In the above-mentioned change steps, the group with the largest status index among the more than one group to which the auxiliary candidate belongs is determined as one object group, and the group with the smallest status index among the more than one group to which the auxiliary candidate does not belong is determined as another object group.

24. The computer-readable recording medium storing a remote conferencing program according to any one of claims 19 to 23, wherein, In the above change steps, the meeting status of each of the above groups is determined according to each specified time.

25. The computer-readable recording medium storing a remote conferencing program according to any one of claims 19 to 23, wherein, In the above change steps, the above change information is generated based on the meeting status of any one of the above groups becoming the specified status.

26. The computer-readable recording medium storing a remote conferencing program as described in claim 25, wherein, It also causes the aforementioned computer to perform: The historical record storage step stores historical record information, which includes the aforementioned group information and the aforementioned meeting status group; and The learning process involves using the aforementioned historical data as teacher data to learn the prescribed states.

27. The computer-readable recording medium storing a remote conferencing program according to any one of claims 19 to 23, wherein, In the above-mentioned meeting status acquisition step, the meeting status is acquired based on the individual voices of the participants and / or images obtained by photographing their faces.

Images