Conference support system and method

Abstract

To propose a conference support system and method for activating a conference to support to enable holding a conference with higher productivity.SOLUTION: A conference support system is adapted to: acquire information related to action and a state of each participant of a conference; analyze and determine the action and the state of each participant based on the acquired information; determine an individual type of each participant based on analysis and determination results of the action and the state of each participant; determine a group type of a group constituted of the respective participants based on determination results of the individual type of each participant; and based on the determination results of the individual type of each participant and determination results of the group type of the group, display necessary first advice for a necessary participant so as to bring the group close to an ideal group type.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a conference support system and method, and is suitable for application to a conference support system that supports the activation of a conference. [Background technology]

[0002] As an invention for understanding the status of a conversation, for example, Patent Document 1 discloses an invention that processes audio signals to detect each utterance of participants in a conversation, processes image signals to detect each state of the participants in the conversation (for example, the direction they are looking), and visualizes and displays the status of the conversation based on these detection results.This invention has the advantage of being able to determine the status of a conversation and the presence of participants with higher accuracy than existing technologies that determine the status of a conversation and the presence of participants only based on the utterances of the participants.

[0003] Furthermore, Patent Document 2 discloses an invention that determines an intervener in a conversation between multiple speakers based on an overall speech level related to a first speech duration of at least one of the multiple speakers in a conference and an overall speaker balance of the multiple speakers obtained from multiple second speech durations of each speaker, and displays information encouraging the determined intervener to intervene in the conversation. This invention has the advantage of being able to encourage a conference between speakers to move from a low-productivity state to a high-productivity state. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2019 / 139101 [Patent Document 2] Japanese Patent Publication No. 2021-140569 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the invention disclosed in Patent Document 1 has a problem in that it is difficult for participants themselves to recognize what actions they should take to improve the conversation.

[0006] Furthermore, according to the invention disclosed in Patent Document 2, although it is expected that conversation will be made more active by encouraging the relevant person to "intervene in the conversation," there is a problem in that no consideration is given to the atmosphere of the meeting as a whole. In this case, in order to make a meeting more active, it is not enough for all participants to simply speak to the same extent; it is also necessary to create an ideal state for the group made up of participants, such as an atmosphere in which participants feel comfortable speaking, and it is believed that creating such a state will enable more productive meetings.

[0007] The present invention has been made in consideration of the above points, and aims to propose a conference support system and method that can help make conferences more active and productive. [Means for solving the problem]

[0008] In order to solve the above problem, the present invention provides a conference support system for supporting a conference, comprising: a sensor group that acquires information on the behavior and state of each participant in the conference; an analysis and determination unit that analyzes and determines the behavior and state of each participant based on the information acquired by the sensor group; an individual type determination unit that determines the individual type of each participant based on the analysis and determination result of the analysis and determination unit; a group type determination unit that determines the group type of a group made up of each participant based on the determination result of the individual type determination unit; and a display processing unit that displays necessary first advice to the necessary participants so as to bring the group closer to an ideal group type based on the determination results of the individual type determination unit and the group type determination unit. The individual type determination unit determines whether the individual type of each participant is a first individual type that initiates conversations the most, a second individual type that has the loudest speech volume of the other participants after the participant speaks, a third individual type that has a speech time equal to or greater than a predetermined first threshold and less than a predetermined second threshold, or a fourth individual type other than the first to third individual types; and the group type determination unit determines whether the group type of the group is a first group type that includes all participants of the first individual type, the second individual type, and the third individual type, or a second group type other than the first group type, and the ideal group type is the first group type. I did so.

[0009] A conference support method executed by a conference support system that supports a conference, the method comprising: a first step of acquiring information on the behavior and state of each participant of the conference; a second step of analyzing and determining the behavior and state of each participant based on the acquired information; a third step of determining an individual type of each participant based on the results of analyzing and determining the behavior and state of each participant; a fourth step of determining a group type of a group made up of each participant based on the results of determining the individual type of each participant; and a fifth step of displaying necessary first advice to the participants based on the results of determining the individual type of each participant and the results of determining the group type of the group, so as to bring the group closer to an ideal group type. In the third step, the conference support system determines whether the individual type of each of the participants is a first individual type that initiates conversations the most, a second individual type that has the loudest speech volume of the other participants after the participant speaks, a third individual type that has a speech time that is equal to or greater than a predetermined first threshold and less than a predetermined second threshold, or a fourth individual type other than the first to third individual types; and in the fourth step, the conference support system determines whether the group type of the group is a first group type that includes all participants of the first individual type, the second individual type, and the third individual type, or a second group type other than the first group type, and the ideal group type is the first group type. I did so.

[0010] According to the conference support system and method of the present invention, a group can be made closer to an ideal group type. [Effects of the Invention]

[0011] According to the present invention, a conference support system and method can be realized that can help make conferences more active and more productive. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a block diagram showing the overall configuration of a conference support system according to an embodiment of the present invention; [Figure 2] 1 is a block diagram showing a logical configuration of a conference support system according to an embodiment of the present invention. [Figure 3] 10 is a diagram illustrating an example of the configuration of a participant status management table. [Figure 4] 10 is a diagram showing an example of the configuration of a first analysis and judgment result management table. [Figure 5] 10 is a diagram showing an example of the configuration of a second analysis and judgment result management table. [Figure 6] 10 is a diagram showing an example of the configuration of a third analysis and judgment result management table. [Figure 7]10 is a diagram showing an example of the configuration of an evaluation management table. [Figure 8] 10 is a diagram showing an example of the configuration of an evaluation message management table; [Figure 9] 10 is a diagram showing an example of the configuration of an advice management table; [Figure 10] 10 is a diagram illustrating an example of the configuration of a group evaluation message management table. [Figure 11] FIG. 10 is a diagram showing an example of the screen configuration of a real-time analysis and determination result screen. [Figure 12] FIG. 10 is a diagram showing an example of a screen configuration of a review screen. [Figure 13] 10 is a flowchart showing a processing procedure for a personal type determination process. [Figure 14] FIG. 10 is a diagram illustrating a personal type determination process. [Figure 15] 10 is a flowchart showing a processing procedure for group type determination processing. [Figure 16] 10 is a flowchart showing a processing procedure for screen display processing. DETAILED DESCRIPTION OF THE INVENTION

[0013] An embodiment of the present invention will be described in detail below with reference to the drawings.

[0014] (1) Configuration of the conference support system according to this embodiment In Fig. 1, the overall system for supporting a conference according to this embodiment is designated by reference numeral 1. This conference supporting system 1 is configured with a sensor group 2 that acquires information on the behavior and status of each participant in a conference by measurement or the like, and a conference supporting device 4 that determines the behavior and status of each participant based on the information acquired by the sensor group 2 and displays necessary advice for the necessary participants on a display device 3.

[0015] The sensor group 2 is composed of an imaging device 5, a sound collection device 6, and multiple vibration sensors 7. The imaging device 5 is composed of, for example, a 360° camera, and is used to capture the facial expressions of each participant in the conference. However, the imaging device 5 may also be composed of multiple cameras arranged so as to capture the facial expressions of each participant. The imaging device 5 outputs a video signal SG1 obtained by capturing images to the conference support device 4.

[0016] The sound collection device 6 is configured with, for example, a multi-channel microphone array capable of identifying the direction of a sound source, and is used to collect the voices of each participant. However, the sound collection device 6 may also be configured with multiple microphones arranged to collect the voices emitted by each participant. The sound collection device 6 outputs an audio signal SG2 corresponding to the collected voices to the conference support device 4.

[0017] The vibration sensor 7 is configured by, for example, an acceleration sensor, and is placed on the seat of the chair on which each participant sits. The vibration sensor 7 detects vibrations caused by the participant nodding in the chair on which the vibration sensor 7 is placed, and outputs a vibration signal SG3 corresponding to the detected vibrations to the conference support device 4.

[0018] The conference support device 4 is configured from a general-purpose computer device equipped with information processing resources such as a CPU (Central Processing Unit) 10, a memory 11, and a storage device 12, and an input device 13.

[0019] The CPU 10 is a processor that controls the overall operation of the conference support device 4. The memory 11 is composed of, for example, a volatile semiconductor memory, and is used as a work memory for the CPU 10. The storage device 12 is composed of, for example, a non-volatile large-capacity storage device such as a hard disk drive or SSD (Solid State Drive), and is used to store various programs and data that needs to be stored for a long period of time. When the conference support device 4 is started up or when needed, necessary programs are loaded from the storage device 12 into the memory 11, and the CPU 10 executes the programs loaded into the memory 11, thereby performing various processes of the conference support device 4 as a whole, as described below.

[0020] In this embodiment, the memory 11 stores an image processing program 14, a voice processing program 15, a nod detection program 16, an analysis / determination program 17, an individual type determination program 18, a group type determination program 19, and a display processing program 20, which are all loaded from the storage device 12, and the storage device 12 stores an information management database 21 and a message / advice management database 22, which are all described below.

[0021] The input device 13 is configured by, for example, a mouse and a keyboard, and is used by any participant, such as a facilitator, to input various operations to the conference support device 4.

[0022] The display device 3 is composed of, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display, and is used by the meeting support device 4 to display a real-time analysis and judgment result screen 50, which will be described later with reference to FIG. 11, and a review screen 70, which will be described later with reference to FIG. 12.

[0023] Fig. 2 shows the logical configuration of the conference support device 4. As shown in Fig. 2, the conference support device 4 is configured to include an image processing unit 30, a voice processing unit 31, a nod detection unit 32, an analysis and determination unit 33, an individual type determination unit 34, a group type determination unit 35, and a display processing unit 36.

[0024] In the following, it is assumed that the position of each chair installed in the conference room and the unique identifier (hereinafter referred to as chair ID) assigned to each chair have been given in advance to the image processing unit 30, the audio processing unit 31, and the analysis and determination unit 33. In the following, it is assumed that the chair ID of each chair is used as the identifier of the participant sitting in that chair (hereinafter referred to as participant ID).

[0025] 1 loaded into memory 11. Image processing unit 30 determines the facial expression of each participant at a predetermined first period (e.g., every one second) based on video signal SG1 provided from imaging device 5, and when it detects a smile from any participant, it notifies analysis and determination unit 33 of the participant ID of that participant.

[0026] The image processing unit 30 also determines the facial direction of each participant based on the video signal SG1 provided from the imaging device 5 at the first period described above, and if it determines that any participant is looking at another participant, it notifies the analysis and judgment unit 33 of the participant ID of that participant and the participant ID of the other participant that the participant was looking at.

[0027] 1 loaded into the memory 11. The audio processing unit 31 is a functional unit that is realized by the CPU 10 executing the audio processing program 15 of Fig. 1 that is loaded into the memory 11. Based on the audio signal SG2 provided from the sound collection device 6, the audio processing unit 31 identifies the participant who made the utterance when one of the participants spoke and measures the volume of the utterance at that time, at the first cycle described above, and notifies the analysis and determination unit 33 of the identification result (the participant ID of the participant) and the measurement result.

[0028] 1 loaded into memory 11. Nodding detection unit 32 analyzes vibration signals SG3 provided from each vibration sensor 7 at the first period described above, and when detecting that any participant is nodding, notifies analysis and determination unit 33 of the participant ID of that participant.

[0029] The analysis and determination unit 33 is a functional unit that is realized when the CPU 10 executes the analysis and determination program 17 of Fig. 1 that is loaded into the memory 11. The analysis and determination unit 33 sequentially registers and manages each piece of information obtained based on the notifications given by the image processing unit 30, the audio processing unit 31, and the nod detection unit 32 in the first cycle described above, in a participant status management table 40 provided in the information management database 21.

[0030] 3 shows an example of the configuration of such a participant status management table 40. The participant status management table 40 is a table used to store information based on notifications given by the image processing unit 30, the audio processing unit 31, and the nodding detection unit 32 in the first cycle, for each participant, and is generated for each participant.

[0031] 3, the participant status management table 40 includes an elapsed time column 40A, a smile column 40B, a direction column 40C, a speaking column 40D, a volume column 40E, and a nodding column 40F. In the participant status management table 40, one record (row) corresponds to one analysis result notified from the image processing unit 30, the audio processing unit 31, and the nodding detection unit 32 in the first cycle. Hereinafter, the timing at which the image processing unit 30, the audio processing unit 31, and the nodding detection unit 32 perform analysis and judgment, which arrives in the first cycle, will be referred to as the first analysis and judgment timing.

[0032] The elapsed time field 40A stores the time elapsed since the start of the conference at the corresponding first analysis / determination timing. The smile field 40B stores a flag (hereinafter referred to as a smile flag) indicating whether the image processing unit 30 detected a smile of the corresponding participant at the corresponding first analysis / determination timing. In the example of FIG. 3, if the image processing unit 30 detected a smile of the corresponding participant at the first analysis / determination timing, the smile flag is set to "1," and if not, the smile flag is set to "0."

[0033] In addition, the orientation column 40C stores the participant IDs of the corresponding participant and the other participants notified by the image processing unit 30 when the image processing unit 30 determines that the face of the corresponding participant was facing the other participants (i.e., the corresponding participant was looking at the other participants) at the first analysis / determination timing.

[0034] Furthermore, the speech column 40D stores a flag (hereinafter referred to as a speech flag) indicating whether or not the voice processing unit 31 detected the speech of the corresponding participant at the corresponding first analysis / determination timing. In the example of Fig. 3, if the voice processing unit 31 detected the speech of the corresponding participant at the first analysis / determination timing, the speech flag is set to "1", and if not, the speech flag is set to "0".

[0035] Furthermore, the volume column 40E stores the speech volume of the corresponding participant measured by the audio processing unit 31 at the corresponding first analysis and determination timing.

[0036] Furthermore, the nod column 40F stores a flag (hereinafter referred to as a nod flag) indicating whether the nod detection unit 32 detected that the corresponding participant nodded at the corresponding first analysis and judgment timing. In the example of Fig. 3, if the nod detection unit 32 detected that the corresponding participant nodded at that first analysis and judgment timing within the period from the immediately preceding first analysis and judgment timing to the current first analysis and judgment timing, the nod flag is set to "1," and if not detected, the nod flag is set to "0."

[0037] Therefore, in the example of Figure 3, the corresponding participant turns his / her face toward the participant with participant ID "1" without smiling or speaking between "00:00:01" and "00:00:02" after the start of the meeting, and nods at least once during this time.

[0038] In addition, the analysis and judgment unit 33 analyzes and judges the behavior and status of each participant at a second period (for example, one minute period) longer than the first period based on the information stored in each participant's participant status management table 40, and stores and manages the analysis and judgment results in a first analysis and judgment result management table 41, a second analysis and judgment result management table 42, and a third analysis and judgment result management table 43 provided in the information management database.

[0039] 4 shows an example of the configuration of the first analysis and judgment result management table 41. The first analysis and judgment result management table 41 is a table used to store the analysis and judgment results regarding the behavior and status of each participant obtained by the analysis and judgment process executed by the analysis and judgment unit 33 in the second cycle. A first analysis and judgment result management table 41 is created for each participant.

[0040] 4, this first analysis and judgment result management table 41 is configured to include an elapsed time column 41A, a smile rate column 41B, a direction column 41C, a synchronization column 41D, a conversation energy column 41E, a nod count column 41F, and an excitement level column 41G. In the first analysis and judgment result management table 41, one record (row) corresponds to one analysis and judgment result obtained by the analysis and judgment process executed by the analysis and judgment unit 33 in the second cycle described above. Note that, hereinafter, the timing at which the analysis and judgment unit 33 performs the analysis and judgment process in the second cycle (the timing that arrives in the second cycle) is referred to as the second analysis and judgment timing.

[0041] The elapsed time field 41A stores the time that has elapsed since the start of the conference at the corresponding second analysis and judgment timing. The smile rate field 41B stores the smile rate calculated by the analysis and judgment unit 33 at the corresponding second analysis and judgment timing using the following formula:

number

[0042] In addition, the "time within the period" in formula (1) is, for example, 60 seconds if the second cycle is 1 minute. Also, the "smiling time" is the time obtained from the participant status management table 40 (FIG. 3) of the corresponding participant. For example, if the first cycle is 1 second, it can be obtained as the number of smile flags set to "1" in the period from the immediately preceding second analysis / determination timing to the current second analysis / determination timing in the participant status management table 40.

[0043] The direction column 41C stores the participant IDs of other participants who were determined to be facing (looking) at the corresponding participant during the period from the immediately preceding second analysis and judgment timing to the current second analysis and judgment timing. The participant IDs of these other participants can be obtained as the participant IDs stored in the direction column 40C for each participant during the period from the immediately preceding second analysis and judgment timing to the current second analysis and judgment timing in the participant status management table 40 (FIG. 3) for the corresponding participant.

[0044] The synchronization column 41D stores a flag (hereinafter referred to as a synchronization flag) that indicates whether or not a majority of the participants in the conference, including a corresponding participant, were visually watching another participant who was visually watching that participant (hereinafter referred to as a synchronized state). In the example of Figure 4, if a synchronized state occurred within the corresponding period, the synchronization flag is set to "1," and if a synchronized state did not occur, the synchronization flag is set to "0."

[0045] In the conversation heat quantity field 41E, the analysis and judgment unit 33 calculates the heat quantity by the following formula at the corresponding second analysis and judgment timing.

number

[0046] The nod count field 41F stores the total number of times the participant nodded during the period from the immediately preceding second analysis and judgment timing to the current second analysis and judgment timing. This total number is calculated as the total number of nod flags set to "1" among the nod flags stored in each nod field 40F in the participant status management table 40 (FIG. 3) for the corresponding participant during the period from the immediately preceding second analysis and judgment timing to the current second analysis and judgment timing.

[0047] Furthermore, the excitement level column 41G stores the sum of the conversation energy stored in the conversation energy column 41E of the same record and the number of nods stored in the number of nods column 41F of the same record as the excitement level of that participant during the period from the immediately preceding second analysis / judgment timing to the current second analysis / judgment timing.

[0048] On the other hand, the second analysis and determination result management table 42 is a table used to manage the status of each participant, such as the number of times each participant has spoken and the duration of each speech, from the start of the conference to the present, and is updated at a second interval by the analysis and determination unit 33. As shown in Fig. 5, the second analysis and determination result management table is configured with a participant ID column 42A, a smile rate column 42B, a number of speeches column 42C, a speech duration column 42D, a number of openings column 42E, a duration column 42F, a number of triggers column 42G, a total number of nods column 42H, and an excitement level column 42I. The second analysis and determination result management table 42 has records (rows) corresponding to each participant.

[0049] The participant ID column 42A stores the participant ID of the participant corresponding to that record, and the smile rate column 42B stores the smile rate of the corresponding participant for the period from the start of the meeting to the present, calculated by the analysis and judgment unit 33 using the above-mentioned formula (1).

[0050] The number of utterances column 42C stores the total number of utterances made by the corresponding participant from the start of the conference to the present (hereinafter referred to as the number of utterances). The number of utterances made by each participant is obtained by extracting, as the time periods in which the participant spoke, time periods in which the utterance flag set to "1" exists singly or consecutively from among the utterance flags stored in each utterance column 40D of the participant status management table 40 (FIG. 3) corresponding to that participant, and by obtaining the number of extracted time periods.

[0051] Furthermore, the speech time field 42D stores the average time of the speech (speech time) calculated by the analysis and determination unit 33. The speech time is calculated as the average time during the time period when the corresponding participant extracted as described above spoke.

[0052] Furthermore, the number of times a participant has initiated a topic from the start of the conference to the present (hereinafter referred to as the number of times an participant initiated a topic) is stored in the number of times column 42E. "Initiating" here means being the first to speak when a discussion on a certain topic is held. Usually, in a conference, a "discussion" in which one or more participants speak continuously on a certain topic is held for a certain period of time, and then, after a certain period of silence, a "discussion" on the next topic begins. Therefore, the analysis and judgment section 33 refers to the participant status management table 40 of each participant, sequentially detects quiet periods (periods during which no participant is speaking) of a certain duration or more, extracts the utterances of each participant between the quiet periods as a series of utterances that may have been the start of a "discussion" on a single topic, and identifies the participant who spoke first in the utterance series as the participant who initiated the "discussion." The analysis and determination unit 33 also counts the number of utterances for each participant and stores the count results in the utterance count column 42E.

[0053] Furthermore, the duration column 42F stores the total duration of each topic initiated by the corresponding participant. Since the duration of each topic is the duration of a series of utterances corresponding to that topic extracted as described above, the analysis and determination unit 33 extracts, for each participant, a series of utterances corresponding to each topic initiated by that participant, and stores the total duration of these extracted utterances in the duration column 42F corresponding to that participant as the total duration of the topics initiated by that participant.

[0054] Furthermore, the number of times that other participants spoke as a result of an utterance by the corresponding participant (the number of times that the corresponding participant made an utterance that induced other participants to speak, hereinafter referred to as the number of times of utterance) is stored in the number of times of utterance. To count the number of times of utterance, the analysis and determination unit 33 calculates the total volume of the speech of the other participants over a certain period of time each time a participant speaks, and when this total value reaches or exceeds a preset threshold, it determines that the participant has induced other participants to speak and counts up the number of times.

[0055] Furthermore, the total nod count field 42H stores the number of times the corresponding participant has nodded from the start of the conference to the present (total nod count). This total nod count is calculated as the sum of the nod counts stored in each nod count field 41F of the first analysis and judgment result management table 41 (FIG. 4) corresponding to that participant.

[0056] Furthermore, the excitement level column 42I stores the excitement level of the corresponding participant from the start of the conference to the present. This excitement level is calculated as the sum of the excitement levels stored in the respective excitement level columns 41G of the first analysis and judgment result management table 41 corresponding to that participant.

[0057] On the other hand, the third analysis and judgment result management table 43 is a table for storing the nodding rate and conversational energy for each second analysis and judgment timing for the entire group made up of all participants attending the current conference, calculated by the analysis and judgment unit 33 at each of the second analysis and judgment timings described above. As shown in FIG. 6, the third analysis and judgment result management table 43 is configured with an elapsed time column 43A, a nodding rate column 43B, and a conversational energy column 43C. In the third analysis and judgment result management table 43, one record (row) corresponds to the analysis and judgment result calculated at the corresponding second analysis and judgment timing.

[0058] The elapsed time field 43A stores the elapsed time from the start of the meeting at the corresponding second analysis / determination timing, and the nod rate field 43B stores the nod rate calculated by the following formula:

number

[0059] The conversation energy field 43C stores the conversation energy of the entire group. This conversation energy is calculated as the sum of the speech volumes of each participant measured in the first cycle from the immediately preceding second analysis / judgment timing to the current second analysis / judgment timing.

[0060] Therefore, in the example of FIG. 6, for example, the nodding rate of all participants at "00:01:00" was "10"%, and the conversational energy of all participants was "15".

[0061] On the other hand, when an instruction to display the review screen 70 described later in FIG. 12 is input via the input device 13 (FIG. 1) (more precisely, when the review button 54 on the real-time analysis and judgment result screen 50 described later in FIG. 11 is clicked), the analysis and judgment unit 33 evaluates the state of the conference from the start of the conference to that point in time (hereinafter referred to as the point in time when the evaluation result display screen is instructed to be displayed) based on the first to third analysis and judgment result management tables 41 to 43 described above, and generates an evaluation management table 44 based on the evaluation results.

[0062] As shown in FIG. 7, the evaluation management table 44 includes a synchronization frequency column 44A, a synchronization rate column 44B, a smile rate column 44C, a topic count column 44D, a nod rate column 44E, and an excitement level column 44F.

[0063] The synchronization count column 44A stores the number of times a synchronization state occurred within the time from the start of the meeting to the time when the evaluation result display screen display instruction is issued (hereinafter referred to as the meeting time), which can be recognized by referring to the synchronization column 41D of each participant's first analysis / judgment result management table 41 (Figure 4).

[0064] Also, in the synchronization rate column 44B,

number

[0065] Furthermore, the smile rate column 44C stores a smile rate that indicates the degree of relaxation of all participants during the conference. This smile rate is calculated as the average value of the smile rates stored in the smile rate column 42B of each participant in the second analysis and judgment result management table 42 (FIG. 5).

[0066] Furthermore, the topic count column 44D stores the total number of topics (hereinafter referred to as the topic count) from the start of the conference to the present. This topic count is calculated by adding up the number of openings by each participant stored in each opening count column 42E of the second analysis and judgment result management table 42. Furthermore, the nod rate column 44E stores the nod rate calculated using the "period" in the above formula (3) as the conference time.

[0067] Furthermore, in the excitement column on 44F,

number

[0068] After generating the evaluation management table 44 as described above, the analysis and determination unit 33 notifies the personal type determination unit 34 of the generation of the evaluation management table 44.

[0069] The personal type determination unit 34 is a functional unit that is realized by the CPU 10 (FIG. 1) executing the personal type determination program 18 of FIG. 1 that is loaded into the memory 11 (FIG. 1). When the personal type determination unit 34 receives such a notification from the analysis and determination unit 33, it refers to the second analysis and determination result management table 42 (FIG. 5) and determines the personal type of each participant. but It is determined whether the type is an "initiator type," "mood maker type," "smiley type," "speaker type," "imaginator type," or "retweeter type."

[0070] Here, the "initiator type" is the type of participant who breaks silence in a meeting, and corresponds to the participant who speaks the most. The "mood maker type" is the type of participant who livens up the meeting after he or she speaks, and corresponds to the participant who speaks the most (total value of speech volume) after he or she speaks. The "smiley type" is the type of participant who spends a lot of time participating in a meeting with a smile, and corresponds to a participant other than the "initiator type" or "mood maker" who, for example, smiles for more than half of the meeting time.

[0071] On the other hand, the "speaker type" is a type of participant who has been speaking for a long time throughout the entire conference, and corresponds to a participant whose speaking time is longer than a predetermined threshold value set in advance for the speaking time (hereinafter referred to as the "speech time threshold"). The "imaginator type" is a type of participant who has not spoken much, and corresponds to a participant whose speaking time is less than the speaking time threshold and whose number of times of speaking is less than a predetermined threshold value set in advance for the number of times of speaking (hereinafter referred to as the "speech count threshold").

[0072] Furthermore, the "retweeter type" is a type of participant who often speaks after other participants have spoken and is good at piggybacking on the opinions of other participants, and is a participant whose number of speeches is equal to or greater than the speech count threshold and whose speech time is less than the speech time threshold.

[0073] A specific method for determining the individual type of each participant by the individual type determination unit 34 will be described later. After determining the individual type of each participant, the individual type determination unit 34 notifies the group type determination unit 35 and the display processing unit 36 ​​of the determination results.

[0074] 1 loaded into memory 11. When the group type determination unit 35 is notified of the individual type of each participant by the individual type determination unit 34, the group type determination unit 35 determines which type (hereinafter referred to as the group type) the group made up of these participants is, from among "High Creation Group," "High Synchro Group," "High Excite Group," "High Smiley Group," "High Thinking Group," and "High Speaker Group," based on the notified individual type of each participant and the evaluation management table 44.

[0075] Here, a "High Creativity Group" is a group in which participants are expected to be able to smoothly conduct a meeting, and corresponds to a group that includes participants of the "Retweeter Type," "Initiator Type," and "Mood Maker Type."

[0076] A "High Synchronization Group" is a group where multiple participants (for example, more than half of the participants) frequently behave in the same way (here, a synchronized state). Furthermore, a "High Excite Group" is a group where lively discussions take place, and a "High Smiley Group" is a group where a high percentage of participants smile during the meeting.

[0077] A "High Thinking Group" is a group with a large number of nervous participants (who speak little or smile little), and corresponds to a group in which more than half of the participants are "Imaginator Type" or less than half of the participants are "Smiley Type." A "High Speaker Group" is a group in which a specific participant continues to speak for a long time, and corresponds to a group with a large number of "Speaker Type" participants.

[0078] A specific method for determining the group type of a group by the group type determination unit 35 will be described later. After determining the group type of the group, the group type determination unit 35 notifies the display processing unit 36 ​​of the determination result.

[0079] The display processing unit 36 ​​is a functional unit that is realized when the CPU 10 executes the display processing program 20 of Fig. 1 that is loaded into the memory 11. When a predetermined input operation is performed via the input device 13 (Fig. 1) by a participant, such as a conference facilitator, before or after the start of the conference, the display processing unit 36 ​​displays on the display device 3 (Fig. 2) a real-time analysis and judgment result screen 50 (described later with reference to Fig. 11) that displays analysis and judgment results of the conference status from the start of the conference to the present, updated in real time, based on the participant status management table 40 stored in the information management database 21 (Fig. 2) and the information stored in the first to third analysis and judgment result management tables 41 to 43.

[0080] Meanwhile, the display processing unit 36 ​​manages an evaluation message management table 45, an advice management table 46, and a group evaluation message management table, which are pre-stored in the message / advice management database 22. The evaluation message management table 45 is a table storing evaluation messages to be presented to participants of each individual type, as shown in Fig. 8, and the advice management table 46 is a table storing advice to be presented to participants of each individual type for each group type, as shown in Fig. 9. Furthermore, the group evaluation message management table 47 is a table storing messages to be presented to groups of each group type, as shown in Fig. 10.

[0081] When the display processing unit 36 ​​is notified of the individual types of each participant in the group from the individual type determination unit 34 and the group type of the group from the group type determination unit 35, the display processing unit 36 ​​acquires necessary information from the evaluation management table 44 stored in the information management database 21 and the evaluation message management table 45, advice management table 46, and group evaluation message management table 47 stored in the message / advice management database 22. The display processing unit 36 ​​then generates a review screen 70, which will be described later with reference to FIG. 12, based on the acquired information, and displays the generated review screen 70 on the display device 3.

[0082] (2) Screen layout of various screens displayed on the display device 11 shows an example of the screen configuration of a real-time analysis and determination result screen 50 that is displayed on the display device 3 by the display processing unit 36 ​​of the conference support device 4 during the conference. This real-time analysis and determination result screen 50 is a display screen that displays in real time the analysis and determination results for the state of the conference obtained based on various information collected by the sensor group 2 (FIG. 1) during the conference.

[0083] This real-time analysis and judgment result screen 50 is composed of a group status display area 51 on the left side of the screen, a personal evaluation display area 52 on the right side of the screen, a pause button 53 and a review button 54 on the upper side of the personal evaluation display area 52, and first and second pin buttons 55A and 55B on the lower side of the personal evaluation display area 52.

[0084] The group status display area 51 is made up of an "activity synchronization" area 51A, a "relaxation" area 51B, a "conversation tendency" area 51C, a "nodding" area 51D, and an "excitement" area 51E. A common time axis (not shown) is defined horizontally in the "activity synchronization" area 51A, the "relaxation" area 51B, the "conversation tendency" area 51C, the "nodding" area 51D, and the "excitement" area 51E, and the latest information in the first or second cycle is displayed at the left end of each of the areas 51A to 51E, and the displayed information then flows to the right as time passes.

[0085] The "activity synchronization" area 51A has horizontal lines corresponding to each participant, and within these lines, first marks 60 (in FIG. 11, the first marks 60 are displayed as "●" due to space limitations) representing the facial orientation of the corresponding participant at that time are displayed sequentially at the right ends of these lines in a second cycle. Also, in the "activity synchronization" area 51A, if the above-mentioned synchronization state occurs between the immediately preceding second analysis / determination timing and the current second analysis / determination timing, a predetermined second mark 61 indicating that synchronization state has occurred is displayed in place of the first mark 60 within the line corresponding to each participant with whom the synchronization state has occurred.

[0086] In the "Relax" area 51B, a line graph 62 showing the change in the smile rate for each participant is displayed while being updated in a second cycle. At this time, the line graph 62 for each participant is colored in a different color.

[0087] Furthermore, in the "conversation tendency" area 51C, a horizontal line corresponding to each participant is provided, and when a participant speaks, a track-shaped third mark 63 having a length corresponding to the speaking time of the utterance is displayed within the corresponding line while being updated at a first cycle. At this time, each third mark 63 is displayed in a different color for each participant.

[0088] Furthermore, the "nodding" area 51D has horizontal lines corresponding to each participant, and within these lines, a fourth mark 64 representing the timing at which the corresponding participant nods is displayed sequentially at the right end of the corresponding line in a first cycle. At this time, the fourth mark 64 is displayed in a different color for each participant.

[0089] Furthermore, in the "Excitement" area 51E, a first curve 65 representing the "nodding rate" of the entire group as described above with reference to Figure 7 and a second curve 66 representing the "energy of conversation" of the entire group are displayed while being updated at a second cycle.

[0090] On the other hand, the individual evaluation display area 52 displays a list of the number of times each participant started a conversation from the start of the meeting to the present ("Number of times a conversation started"), the duration of the conversation ("Duration of the conversation"), and the level of excitement ("Level of excitement"), as described above in the second analysis and judgment result management table 42 of Figure 5.

[0091] On the real-time analysis and judgment result screen 50, by clicking the first pin button 55A or the second pin button 55B at a desired timing, a fifth mark 67A, 67B (a heart mark for the first pin button, and an inverted triangle mark for the second pin button) corresponding to the first or second pin button 55A, 55B can be displayed at the upper left edge of the “activity synchronization” area 51A at that timing. The fifth mark 67A, 67B is also displayed in synchronization with the information displayed in the “activity synchronization” area 51A, the “relaxation” area 51B, the “conversation tendency” area 51C, the “nodding” area 51D, and the “excitement” area 51E, flowing from the left edge to the right edge of the areas 51A-51E over time.

[0092] Furthermore, on the real-time analysis and judgment result screen 50, by clicking the pause button 53, the updating of the display contents of the group status display area 51 and the individual evaluation display area 52 (more precisely, the updating of the participant status management table 40 (Figure 3) and the first to third analysis and judgment result management tables 41 to 43 by the analysis and judgment unit 33) can be paused, and by clicking the review button 54, the screen displayed on the display device 3 can be switched from the real-time analysis and judgment result screen 50 to the review screen 70 shown in Figure 12.

[0093] This review screen 70 is a screen for displaying the status of the entire group from the start of the meeting to the present, the current evaluation of each participant, and advice for necessary participants on their attitude toward participating in future meetings.It is composed of a group status display area 71 on the left side of the screen, a group type display area 72 above the group status display area 71, an individual evaluation display area 73 on the right side of the screen, and a resume button 74 and an end button 75 above the individual evaluation display area 73.

[0094] On the left side of the group status display area 71 are provided an "Activity Synchronization" area 71A, a "Relax" area 71B, a "Conversation Tendency" area 71C, a "Nodding" area 71D, and an "Excitement" area 71E, and in these "Activity Synchronization" area 71A, "Relax" area 71B, "Conversation Tendency" area 71C, "Nodding" area 71D, and "Excitement" area 71E, the same information that was displayed in the "Activity Synchronization" area 71A, "Relax" area 71B, "Conversation Tendency" area 71C, "Nodding" area 71D, and "Excitement" area 71E, respectively, immediately before the review button 54 on the real-time analysis / judgment result screen 50 was clicked is displayed with updates stopped.

[0095] Also, on the right side of the group status display area 71, the number of synchronizations, smile rate, number of topics, nodding rate, and excitement level of the entire group from the start of the meeting to the present (more precisely, from the start of the meeting until the review button 54 on the real-time analysis and judgment result screen 50 is clicked) are displayed. The number of synchronizations, smile rate, number of topics, nodding rate, and excitement level here are all read and displayed from the evaluation management table 44 described above with reference to FIG. 7.

[0096] The group type display area 72 also displays the group type of the group determined by the group type determination unit 35 (Fig. 2), an evaluation of the group's progress in the meeting up to now, and advice for future meeting progress. For example, Fig. 12 shows an example in which the group type of the group is determined to be a "High Creativity Group," and the message "Members were actively discussing each other" is displayed as the evaluation of the group's progress in the meeting up to now, and the advice for future meeting progress is "Let's continue the active discussion together."

[0097] On the other hand, the personal evaluation display area 73 is provided with an evaluation / advice display area 73A corresponding to each participant, and in these evaluation / advice display areas 73A, the personal type determined for that participant up to now, an evaluation message regarding the participant's participation attitude in the meeting so far, and advice regarding the participant's participation attitude in future meetings are displayed.

[0098] At this time, the above-mentioned advice displayed for each participant in each evaluation / advice display area 73A is advice that will bring the group type of the group closer to the "High Creation Type," with the "High Creation Type" being the ideal group type state.

[0099] For example, if the group type is a "High Synchro Type" or a "High Excite Group," there are no "Retweeter Type" participants in the group, and so advice to create "Retweeter Type" participants is displayed to participants who need it. Specifically, a participant determined to be a "Speaker Type" is shown advice to respond to the opinions of other participants, such as "Listen to and respond to the opinions of others." Furthermore, a participant determined to be an "Imaginator Type" is shown advice encouraging them to speak, such as "Speak what you're thinking," if they speak less frequently and for less time, and advice to tone down their speech, such as "Listen to and speak other people's opinions," if they speak more frequently and for more time.

[0100] Also, if the group type of the group is "High Smiley Type," there will be a large number of participants in the group who are determined to be "Smiley Type," so in order to create participants who are "Retweeter Type," advice on how to respond to the opinions of other participants, such as "Listen to and respond to the opinions of others," will be displayed to each participant determined to be "Smiley Type."

[0101] Furthermore, if the group type is "High Thinking Type," there are many participants in the group who are determined to be "Imaginator Type," so in order to create "Retweeter Type" participants, each participant determined to be an "Imaginator Type" will be shown advice encouraging them to speak, such as "Say what you're thinking," if they speak less frequently or for less time, and advice to moderate their speech, such as "Listen to other people's opinions and speak up," if they speak more frequently or for more time.

[0102] Furthermore, if the group type of the group is "High Speaker Type," and there are participants in the group who are determined to be "Speaker Type," advice on how to respond to the opinions of other participants, such as "Listen to and respond to the opinions of others," is displayed to each participant determined to be "Speaker Type" in order to create participants who are "Retweeter Type."

[0103] If the group type of the group is "High Creativity Type," the group is in an ideal state, so no advice is displayed to each participant, and only an evaluation message is displayed.

[0104] On this review screen 70, by clicking the resume button 74, the display screen of the display device 3 can be returned to the real-time analysis and determination result screen 50 described above with reference to Fig. 11. In this case, updating of the display contents on the real-time analysis and determination result screen 50 is resumed.

[0105] Therefore, when the participant in charge of facilitating the meeting takes a break, he or she can click the review button 54 on the real-time analysis and judgment result screen 50 to have the display device 3 display the review screen 70, which shows evaluations and advice on the status of the entire group and each participant during the meeting from the start of the meeting to that point, and when the break is over, he or she can click the resume button 74 on the review screen 70 to return the display screen of the display device 3 to the real-time analysis and judgment result screen 50, after which the meeting support device 4 can resume analyzing the status of the meeting.

[0106] The review screen 70 can be closed when the meeting is over by clicking the end button 75. By clicking the file output button 76 before closing the review screen 70, the screen data file of the review screen 70 at that time can be output to a desired output destination.

[0107] (3) Various processes executed by the conference support device Next, we will explain the flow of various processes executed by the individual type determination unit 34, group type determination unit 35, and display processing unit 36 ​​of the meeting support device 4 to display the real-time analysis / determination result screen 50 and review screen 70 described above.

[0108] In the following, the entities that execute various processes will be described as the individual type determination unit 34, the group type determination unit 35, or the display processing unit 36, but in reality, it goes without saying that the CPU 10 (Figure 1) of the conference support device executes the processes based on the corresponding programs (individual type determination program 18, group type determination program 19, or display processing program 20) described above with reference to Figure 1.

[0109] (3-1) Personal type determination process 13 shows the processing steps of the individual type determination process executed by the individual type determination unit 34 when the review button 54 on the real-time analysis and determination result screen 50 is clicked as described above, causing the analysis and determination unit 33 to generate the evaluation management table 44, and then a notification that the evaluation management table 44 has been reproduced is sent from the analysis and determination unit 33 to the individual type determination unit 34. The individual type determination unit 34 determines the individual type of each participant according to the processing steps shown in FIG.

[0110] In practice, when the individual type determination unit 34 receives such notification from the analysis and determination unit 33, it starts the individual type determination process shown in Figure 13, and first determines the person who has initiated the most events as the "initiator type" based on the number of events for each participant stored in the second analysis and determination result management table 42 (S1).

[0111] Next, the individual type determination unit 34 determines the participant who has been induced the most times from among the remaining participants as the "mood maker type" based on the number of times each participant has been induced, which is stored in the second analysis and determination result management table 42 (S2).

[0112] Next, the individual type determination unit 34 determines, based on the smile rate of each participant stored in the second analysis and determination result management table 42, all of the remaining participants whose smile rate from the start of the meeting to the present is 50% or more as the "smiley type" (S3).

[0113] After this, the individual type determination unit 34 assigns the remaining participants to one of the first to fourth quadrants as shown in Figure 14 based on the number of times they have spoken and the duration of their speeches, which are stored in the second analysis and determination result management table 42 (S4).

[0114] Specifically, the individual type determination unit 34 assigns participants whose previous number of utterances is equal to or greater than the predetermined utterance count threshold and whose previous utterance duration is equal to or greater than the predetermined utterance duration threshold to quadrant 1. The individual type determination unit 34 also assigns participants whose previous number of utterances is less than the utterance count threshold but whose previous utterance duration is equal to or greater than the utterance duration threshold to quadrant 2.

[0115] Furthermore, the individual type determination unit 34 assigns participants whose previous speech time is less than the speech time threshold and whose previous number of speeches is also less than the speech count threshold to the third quadrant, and assigns participants whose previous speech time is less than the speech time threshold but whose number of speeches is equal to or greater than the speech count threshold to the fourth quadrant.

[0116] The individual type determination unit 34 then determines the individual types of the participants assigned to the first and second quadrants as "speaker type," determines the individual types of the participants assigned to the third quadrant as "imaginator type," and further determines the individual types of the participants assigned to the fourth quadrant as "retweeter type" (S5).

[0117] The individual type determination unit 34 then notifies the group type determination unit 35 and the display processing unit 36 ​​of the individual types of each participant determined in this manner (S6), and then ends this individual type determination process.

[0118] (3-2) Group type determination process 15 shows the procedure for group type determination processing executed by the group type determination unit 35, which has received notification of the individual types of each participant from the individual type determination unit 34. The group type determination unit 35 determines the group type of the group according to the procedure shown in FIG.

[0119] In practice, when the group type determination unit 35 is notified of the individual types of each participant by the individual type determination unit 34, it starts the group type determination process shown in Figure 15, and first determines whether the group contains participants of the ``retweeter type,'' ``initiator type,'' and ``mood maker type'' (S10).

[0120] If the group type determination unit 35 obtains a positive result in this judgment, it determines that the group type of the group is a "High Creation Group" (S11), notifies the display processing unit of this determination result (S21), and then terminates the group type determination process.

[0121] In contrast, if the group type determination unit 35 obtains a negative result in the judgment of step S10, it obtains the "synchronization rate" of the group from the evaluation management table 44 and determines whether the obtained "synchronization rate" is greater than a threshold value that is pre-set for the "synchronization rate" in the range of 0 to 100 (S12).

[0122] If the group type determination unit 35 obtains a positive result in this judgment, it determines that the group type of the group is a "High sync group" (S13), notifies the display processing unit 36 ​​of this judgment result (S21), and then terminates the group type determination process.

[0123] In contrast, if the group type determination unit 35 obtains a negative result in the judgment of step S12, it obtains the ``excitement level'' of the group from the evaluation management table 44 and determines whether the obtained ``excitement level'' is greater than a threshold value that is pre-set in the range of 0 to 100 for ``excitement level'' (S14).

[0124] If the group type determination unit 35 obtains a positive result in this judgment, it determines that the group type of the group is a "High Excite Group" (S15), and after notifying the display processing unit 36 ​​of this judgment result (S21), ends the group type determination process.

[0125] On the other hand, if the group type determination unit 35 obtains a negative result in the determination of step S14, it determines whether the ratio of the number of participants determined to be of the "smiley type" to the total number of participants is equal to or greater than a predetermined threshold value (S16).

[0126] If the group type determination unit 35 obtains a positive result in this judgment, it determines that the group type of the group is a "High Smiley Group" (S17), and after notifying the display processing unit 36 ​​of this judgment result (S21), ends the group type determination process.

[0127] On the other hand, if the group type determination unit 35 obtains a negative result in the determination of step S16, it determines whether the ratio of the number of participants determined to be of the "Imaginator type" to the total number of participants is equal to or greater than a predetermined threshold value (S18).

[0128] If the group type determination unit 35 obtains a positive result in this judgment, it determines that the group type of the group is a "High thinking group" (S19), and after notifying the display processing unit 36 ​​of this judgment result (S21), ends the group type determination process.

[0129] Furthermore, if the group type determination unit 35 obtains a positive result in the determination of step S18, it determines that the group type of the group is a "High speaker group" (S20), and after notifying the display processing unit 36 ​​of this determination result (S21), terminates the group type determination process.

[0130] (3-3) Screen display processing 16 shows the procedure for screen display processing executed by the display processing unit 36. The display processing unit 36 ​​displays the real-time analysis and determination result screen 50 (FIG. 11) and the review screen 70 (FIG. 12) on the display device 3 in accordance with the procedure shown in FIG.

[0131] Note that Figure 16 only describes the processing steps for a series of processes to display the real-time analysis / determination result screen 50 and the review screen 70, and omits processing that occurs when, for example, the pause button 53 (Figure 11) on the real-time analysis / determination result screen 50 is clicked or the file output button 76 (Figure 12) on the review screen 70 is clicked.

[0132] When a predetermined operation is performed on the input device 13 (FIG. 1) and the image processing unit 30, audio processing unit 31, nod detection unit 32, and analysis / judgment unit 33 (all of which are shown in FIG. 2) start the judgment processing in the first or second cycle described above, the display processing unit 36 ​​starts the screen display processing shown in FIG. 16, and first generates a real-time analysis / judgment result screen 50 based on the participant status management table 40 (FIG. 3) for each participant stored in the information management database 21 (FIG. 2) and the information registered in the first to third analysis / judgment result management tables 41 to 43, and displays the generated real-time analysis / judgment result screen 50 on the display device 3 (S30).

[0133] For example, based on the information stored in the orientation column 41C and synchronization column 41D of the first analysis and judgment result management table 41 (Figure 4) for each participant, the display processing unit 36 ​​displays a first mark 60 (Figure 11) representing the facial orientation of each participant and a second mark 61 (Figure 11) representing the synchronization state in the ``Activity Synchronization'' area 51A of the group status display area 51 on the real-time analysis and judgment result screen 50, each at a second period.

[0134] In addition, based on the information stored in the smile rate column 42B of the second analysis / judgment result management table 42 (Figure 5), the display processing unit 36 ​​displays a line graph 62 (Figure 11) for each participant in the ``Relax'' area 51B of the group status display area 51 on the real-time analysis / judgment result screen 50, updating it at a second cycle.

[0135] Furthermore, based on the value of the speech flag stored in the speech column 40D of the participant status management table 40 (Figure 3) for each participant, the display processing unit 36 ​​displays a third mark 63 (Figure 11) indicating that the participant is in a speaking state in the ``Conversation Tendency'' area 51C of the group status display area 51 on the real-time analysis / judgment result screen 50, updating it at a first period.

[0136] Furthermore, based on the nod flag stored in the nod column 40F of the participant status management table 40 for each participant, the display processing unit 36 ​​displays a fourth mark 64 (Figure 11) indicating that the corresponding participant has nodded in the ``nod'' area 51D of the group status display area 51 on the real-time analysis / judgment result screen 50 at a first period.

[0137] Furthermore, based on the nodding rate and conversation energy stored respectively in the nodding rate column 43B and conversation energy column 43C of the third analysis / judgment result management table 43, the display processing unit 36 ​​displays a first curve 65 (Figure 11) representing the nodding rate of the entire group and a second curve 66 representing the excitement level of the entire group in the ``Excitement'' column 51E of the group status display area 51 on the real-time analysis / judgment result screen 50, updating them at a second cycle.

[0138] Furthermore, the display processing unit 36 ​​updates the number of pilots, duration and excitement level stored in the pilots count column 42E, duration column 42F and excitement level column 42I of the second analysis and judgment result management table 42 at a second period and displays these values ​​in the personal evaluation display area 52 of the real-time analysis and judgment result screen 50.

[0139] Next, the display processing unit 36 ​​determines whether the review button 54 (FIG. 11) on the real-time analysis and determination result screen 50 has been clicked (S31). If the display processing unit 36 ​​obtains a negative result in this determination, it returns to step S30, and thereafter repeats the loop of step S30-step S31-step S30.

[0140] When the display processing unit 36 ​​eventually obtains a positive result in step S31 by clicking the review button 54 on the real-time analysis and judgment result screen 50, it notifies the analysis and judgment unit 33 (Figure 2) that the review button 54 has been clicked (S32).

[0141] As a result, the analysis and determination unit 33 generates an evaluation management table 44 (Fig. 7) of the content up to that point, and notifies the individual type determination unit 34 (Fig. 2) to that effect, which then executes the individual type determination process described above with reference to Fig. 13. Furthermore, when the individual type determination process by the individual type determination unit 34 is completed, this is notified to the group type determination unit 35, which then executes the group type determination process described above with reference to Fig. 15.

[0142] Meanwhile, after executing step S32, the display processing unit 36 ​​waits for notification of the group type of the group at that time from the group type determination unit 35 (S33). Then, when the display processing unit 36 ​​is eventually notified of the group type of the group from the group type determination unit 35, it collects information necessary to display the review screen 70 (S34).

[0143] Specifically, the display processing unit 36 ​​acquires the number of synchronizations, smile rate, number of topics, nodding rate, and excitement level of the entire group from the start of the meeting to that time from the evaluation management table 44 of the information management database 21.

[0144] The display processing unit 36 ​​also obtains the evaluation messages for each participant's individual type notified by the individual type determination unit 34 from the evaluation message management table 45 (Figure 8) of the message / advice management database 22 (Figure 2), and obtains the evaluation messages for the group type of the group notified by the group type determination unit 35 from the group evaluation message management table 47 (Figure 10) of the message / advice management database 22.

[0145] Furthermore, the display processing unit 36 ​​obtains advice for each participant from the advice management table 46 (Figure 9) based on the individual type of each participant notified by the individual type determination unit 34 and the group type of the group notified by the group type determination unit 35.

[0146] The display processing unit 36 ​​then generates the review screen 70 described above with reference to FIG. 12 based on the information collected as described above and the various pieces of information data that had been displayed up until then in the group status display area 51 of the real-time analysis and judgment result screen 50, and displays the generated review screen 70 on the display device 3 in place of the real-time analysis and judgment result screen 50 that had been displayed up until then (S35).

[0147] Thereafter, the display processing unit 36 ​​waits for the resume button 74 (FIG. 12) or the end button 75 (FIG. 12) on the review screen 70 to be clicked (S36 and S37). When the resume button 74 is clicked, the display processing unit 36 ​​returns to step S30 and switches the screen displayed on the display device 3 to the real-time analysis and determination result screen 50, and thereafter repeats the processes of steps S30 to S36 until a positive result is obtained in step S37. When the end button 75 is eventually clicked, the display processing unit 36 ​​closes the review screen 70 and ends this screen processing.

[0148] (4) Effects of this embodiment As described above, the conference support system 1 of this embodiment determines the individual type of each participant based on various information acquired by the sensor group 2, determines the group type of the group consisting of these participants based on the determination result, and displays necessary advice to necessary participants based on the determination result so as to bring the group closer to the ideal group type, a "High Creation Group."

[0149] Therefore, according to the conference support system 1, the group type of the group can be made closer to the ideal group type, thereby making the conference more lively and more productive.

[0150] (5) Other embodiments In the above embodiment, the case where the nodding of participants is detected using a vibration sensor is described, but the present invention is not limited to this, and the nodding of participants may also be detected using captured images of each participant based on video signals output from an imaging device.

[0151] In the above-described embodiment, the following functions are provided in a single conference support device: an analysis and judgment unit (image processing unit 30, audio processing unit 31, nod detection unit 32, and analysis and judgment unit 33) that analyzes information acquired by the sensor group 2 and judges the behavior and state of each participant; an individual type judgment unit 34 that judges the individual type of each participant based on the analysis and judgment results of the analysis and judgment unit; a group type judgment unit 35 that judges the group type of a group made up of each participant based on the analysis and judgment results of the individual type judgment unit 34; and a display processing unit 36 ​​that displays necessary advice to necessary participants so as to bring the group closer to the ideal group type based on the analysis and judgment results of the individual type judgment unit 34 and the group type judgment unit 35. However, the present invention is not limited to this, and these functions may be distributed and provided in multiple computer devices that make up a distributed computing system.

[0152] Furthermore, in the above embodiment, the individual type determination unit 34 determines whether the individual type of a participant is one of six types, namely, "initiator type," "mood maker type," "smiley type," "speaker type," "imaginator type," and "retweeter type." However, the present invention is not limited to this, and the individual type determination unit 34 may determine whether the individual type is one of four types: "initiator type," "mood maker type," and "retweeter type," which are at least necessary for determining the group type, and other individual types. Furthermore, there may be more than one "other individual type."

[0153] Similarly, in the above embodiment, the group type determination unit 35 determines whether the group type of a group is one of six types, "High Creation Group," "High Synchro Group," "High Excite Group," "High Smiley Group," "High Thinking Group," and "High Speaker Group," but the present invention is not limited to this, and the determination may be one of two types, "High Creation Group" and other group types. Furthermore, there may be more than one "other group type."

[0154] Furthermore, in the above embodiment, the advice for each participant is displayed together on the display device 3, but the present invention is not limited to this. For example, if there is a monitor for each participant, advice and evaluation messages for the participants corresponding to these monitors may be displayed, and the evaluations and advice for each participant may not be displayed on the review screen 70.

[0155] Furthermore, in the above embodiment, the case where all participants gather in one place to hold the conference has been described, but the present invention is not limited to this, and can also be applied to a case where some or all of the participants participate in the conference remotely using their own computer devices. In this case, advice and evaluation messages for each participant may be displayed on the computer device of each participant. [Industrial Applicability]

[0156] The present invention relates to a conference support system and method, and can be widely applied to conference support systems of various configurations that support the activation of conferences. [Explanation of symbols]

[0157] 1...Conference support system, 2...Sensor group, 3...Display device, 4...Conference support device, 5...Imaging device, 6...Sound collection device, 7...Vibration sensor, 10...CPU, 13...Input device, 30...Image processing unit, 31...Audio processing unit, 32...Nodding detection unit, 33...Analysis and judgment unit, 34...Individual type judgment unit, 35...Group type judgment unit, 36...Display processing unit, 40...Participant status management table, 41...First analysis and judgment result management table, 42...Second analysis and judgment result management table, 43...Third analysis and judgment result management table, 44...Evaluation management table, 45...Evaluation message management table, 46...Advice management table, 47...Group evaluation message management table, 50...Real-time analysis and judgment result screen, 70...Review screen, 73...Individual evaluation display area, 73A...Evaluation and advice display area.

Claims

1. A conference support system for supporting a conference, a group of sensors that acquire information about the behavior and status of each participant in the conference; an analysis and determination unit that analyzes and determines the behavior and state of each participant based on the information acquired by the sensor group; an individual type determination unit that determines the individual type of each participant based on the analysis and determination results of the analysis and determination unit; a group type determination unit that determines a group type of a group formed by each of the participants based on a determination result of the individual type determination unit; a display processing unit that displays necessary first advice to the participants based on the determination results of the individual type determination unit and the group type determination unit so as to bring the group closer to an ideal group type; Equipped with The personal type determination unit determine whether each participant has an individual type of a first individual type that initiates conversations the most, a second individual type that has the loudest speech volume of other participants after the participant speaks, a third individual type that has a speech time equal to or greater than a predetermined first threshold and less than a predetermined second threshold, or a fourth individual type other than the first to third individual types; The group type determination unit determining whether the group type of the group is a first group type in which participants of the first individual type, participants of the second individual type, and participants of the third individual type are all present, or a second group type other than the first group type; The ideal group type is the first group type. A conference support system characterized by:

2. The display processing unit Displaying second advice for the entire group in addition to the first advice required for the necessary participants 2. The conference support system according to claim 1.

3. A conference support method executed by a conference support system that supports a conference, a first step of obtaining information about the activity and status of each participant of the conference; a second step of analyzing and determining the behavior and state of each participant based on the acquired information; a third step of determining a personal type of each participant based on the analysis and determination results of the behavior and state of each participant; a fourth step of determining a group type of a group constituted by each of the participants based on the determination result of the individual type of each of the participants; a fifth step of displaying necessary first advice to the participants based on the results of determining the individual type of each participant and the results of determining the group type of the group, so as to bring the group closer to an ideal group type; Equipped with In the third step, the conference support system determine whether each participant has an individual type of a first individual type that initiates conversations the most, a second individual type that has the loudest speech volume of other participants after the participant speaks, a third individual type that has a speech time equal to or greater than a predetermined first threshold and less than a predetermined second threshold, or a fourth individual type other than the first to third individual types; In the fourth step, the conference support system determining whether the group type of the group is a first group type in which participants of the first individual type, participants of the second individual type, and participants of the third individual type are all present, or a second group type other than the first group type; The ideal group type is the first group type. A conference support method comprising:

4. In the fifth step, the conference support system Displaying second advice for the entire group in addition to the first advice required for the necessary participants 4. The conference support method according to claim 3.

Images