Video meeting evaluation system and video meeting evaluation server

Abstract

To evaluate a video meeting itself by evaluating a video image acquired in the video meeting and to make it possible to make use of analysis results thereof.SOLUTION: A video meeting evaluation system comprises: a video meeting service terminal for providing a video meeting to at least a first user terminal and a second user terminal, and also acquiring a video image acquired by using at least a first camera unit and a second camera unit; and an evaluation terminal for making an evaluation pertaining to a video meeting. The video meeting service terminal shares a video image simultaneously or substantially simultaneously at least with the evaluation terminal, in response to an operation of at least the first user terminal. The evaluation terminal identifies, for respective predetermined frame units, at least a face image included in the shared video image, and also calculates an evaluation value pertaining to the face image.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a video meeting evaluation system and a video meeting evaluation server.

Background Art

[0002] Conventionally, a system for teaching some knowledge or giving explanations online has been known (see, for example, Patent Document 1).

[0003] Also, as a method for measuring the effect of a video meeting conducted online, for example, a method of conducting a questionnaire or the like after the video meeting has been proposed (see, for example, Patent Document 2).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] The above-described method of measuring the effect by questionnaire tends to be subjective and is insufficient as a method for objectively measuring the effect regarding the content of the video meeting.

[0006] Also, a method of monitoring a video meeting by a third party and obtaining an objective evaluation by the third party can be considered, but it is too time-consuming and laborious to be practical.

[0007] Therefore, an object of the present invention is to objectively evaluate the content of a video meeting in particular.

Means for Solving the Problems

[0008] According to the present invention, A video conferencing service terminal that provides video conferencing to at least the first user terminal and the second user terminal and acquires moving images acquired by at least the first camera unit or the second camera unit, An evaluation terminal that evaluates the video conferencing, A video conferencing evaluation system including: The video conferencing service terminal shares the moving image with at least the evaluation terminal simultaneously or substantially simultaneously in response to an operation of at least the first user terminal, The evaluation terminal identifies at least face images included in the shared moving image for each predetermined frame unit and calculates an evaluation value related to the face image, A video conferencing evaluation system is obtained.

Effect of the Invention

[0009] According to the present disclosure, by evaluating a moving image of video conferencing, it is possible to objectively evaluate particularly the content.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Mode for Carrying Out the Invention

[0011] The content of the embodiment of the present disclosure will be listed and described. The present disclosure has the following configuration. [Item 1] A video conferencing service terminal that provides video conferencing to at least the first user terminal and the second user terminal and acquires moving images acquired by at least the first camera unit or the second camera unit, An evaluation terminal that evaluates the video conferencing, A video conferencing evaluation system including: The video conferencing service terminal shares the moving image with at least the evaluation terminal simultaneously or substantially simultaneously in response to an operation of at least the first user terminal, The evaluation terminal identifies at least face images included in the shared moving image for each predetermined frame unit and calculates an evaluation value regarding the face images, A video conferencing evaluation system is obtained. [Item 2] The video conferencing evaluation system according to Item 1, The evaluation terminal provides graph information based on the time series of the evaluation values. Video conferencing evaluation system. [Item 3] The video conferencing evaluation system according to Item 1 or Item 2, The evaluation terminal calculates a plurality of evaluation values obtained by evaluating the face image from a plurality of different viewpoints. Video conferencing evaluation system. [Item 4] The video conferencing evaluation system according to any one of Items 1 to 3, The evaluation terminal calculates the evaluation value together with the voice included in the moving image. Video conferencing evaluation system. [Item 5] The video conferencing evaluation system according to any one of Items 1 to 4, The evaluation terminal calculates the evaluation value together with objects other than the face image included in the moving image. Video conferencing evaluation system. [Item 6] A video conferencing evaluation server communicably connected to a video conferencing service terminal that provides video conferencing to at least the first user terminal and the second user terminal and stores moving images acquired by at least a first camera unit of the first user terminal or a second camera unit of the second user terminal, comprising: means for acquiring the moving image; means for identifying at least face images included in the moving image for each predetermined frame unit; means for calculating an evaluation value regarding the face image; A video conferencing evaluation server comprising:

[0012] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted.

[0013] <Basic Function>

[0014] The emotion analysis system of the present embodiment is a system for analyzing specific emotions (feelings that occur in response to the actions of oneself or others, such as pleasure, discomfort, or the degree thereof) that are different from others for an analysis target person among a plurality of people in an environment where video conferencing (hereinafter referred to as an online session including one-way and two-way) is performed by a plurality of people.

[0015] The online session is, for example, an online meeting, an online class, an online chat, etc., in which terminals installed at a plurality of locations are connected to a server via a communication network such as the Internet, and moving images can be exchanged between the plurality of terminals through the server.

[0016] The moving images handled in the online session include the face images and voices of the users who use the terminals. Also, the moving images include images such as materials shared and viewed by multiple users. It is possible to switch between the face image and the material image on the screen of each terminal and display only one of them, or to divide the display area and display the face image and the material image simultaneously. Also, it is possible to display the image of one person among multiple people in full screen, or to divide and display the images of some or all of the users in small screens.

[0017] Among the multiple users who participate in the online session using the terminals, it is possible to specify one or more of them as the analysis target. For example, the leader, facilitator, or administrator of the online session (hereinafter collectively referred to as the organizer) specifies one of the users as the analysis target. The organizer of the online session is, for example, a lecturer in an online class, a chairperson or facilitator in an online meeting, a coach in a session for coaching purposes, etc. The organizer of the online session is usually one of the multiple users who participate in the online session, but may also be someone who does not participate in the online session. Note that it is also possible to analyze all participants without specifying the analysis target.

[0018] Also, the leader, facilitator, or administrator of the online session (hereinafter collectively referred to as the organizer) can also specify one of the users as the analysis target. The organizer of the online session is, for example, a lecturer in an online class, a chairperson or facilitator in an online meeting, a coach in a session for coaching purposes, etc. The organizer of the online session is usually one of the multiple users who participate in the online session, but may also be someone who does not participate in the online session.

[0019] As shown in FIG. 1, the video conferencing evaluation system according to the present embodiment includes user terminals 10 and 20 having at least an input unit such as a camera unit and a microphone unit, a display unit such as a display, and an output unit such as a speaker, a video conferencing service terminal 30 that provides two-way video conferencing to the user terminals 10 and 20, and an evaluation terminal 40 that evaluates video conferencing.

[0020] <Hardware Configuration Example> FIG. 2 is a diagram showing a hardware configuration example of a computer that realizes each of the terminals 10 to 40 according to the present embodiment. The computer includes at least a control unit 11, a memory 12, a storage 13, a communication unit 14, and an input / output unit 15. These are electrically connected to each other through a bus 16.

[0021] The control unit 11 is an arithmetic device that controls the operation of the entire terminal, controls the transmission and reception of data between each element, and performs information processing necessary for the execution and authentication processing of applications. For example, the control unit 11 is a processor such as a CPU, and executes programs and the like stored in the storage 13 and developed in the memory 12 to perform each information processing.

[0022] The memory 12 includes a main memory composed of a volatile storage device such as a DRAM, and an auxiliary memory composed of a non-volatile storage device such as a flash memory or an HDD. The memory 12 is used as a work area of the control unit 11, and stores a BIOS executed when the information sharing support device 10 is started, and various setting information and the like.

[0023] The storage 13 stores various programs such as application programs. A database storing data used for each process may be constructed in the storage 13. In particular, the video conferencing service terminal 30 may record moving images in an online session and store them in the storage 13. Further, the evaluation terminal 40 may acquire moving images and store them together with the analysis results (evaluation results) in the storage 13 managed by the evaluation terminal 40.

[0024] The communication unit 14 connects the information sharing support device 10 to a network. The communication unit 14 communicates with external devices directly or via a network access point, for example, by means of a wired LAN, a wireless LAN, Wi-Fi (registered trademark), infrared communication, Bluetooth (registered trademark), short-range or contactless communication, etc.

[0025] The input / output unit 15 is, for example, an information input device such as a keyboard, a mouse, a touch panel, and an output device such as a display.

[0026] The bus 16 is commonly connected to the above-mentioned respective elements and transmits, for example, address signals, data signals, and various control signals.

[0027] In particular, the evaluation terminal according to the present embodiment acquires a moving image from a video conferencing service terminal, identifies at least a face image included in the moving image for each predetermined frame unit, and calculates an evaluation value regarding the face image (details will be described later).

[0028] <Method for acquiring video> As shown in FIG. 3, the video conferencing service (hereinafter sometimes simply referred to as "this service") provided by the video conferencing service terminal enables two-way communication by image and voice with the user terminals 10 and 20. This service can display a moving image acquired by the camera unit of the other user terminal on the display of the user terminal, and output the voice acquired by the microphone unit of the other user terminal from the speaker.

[0029] In addition, this service is configured to be able to record (by means of recording) moving images and sounds (collectively referred to as "moving images, etc.") by either or both user terminals. The recorded information Vs1, Vs2 (hereinafter referred to as "recorded information") is temporarily cached in the user terminal that started the recording, and will be recorded either only on the video meeting service terminal side, or only locally on either user terminal, or on both. The user can view the recorded information by himself / herself, share it with others, etc. within the scope of using this service.

[0030] The evaluation terminal 40 acquires the recorded information and performs analysis and evaluation as described later. As a method for acquiring the recorded information, for example, a direct download request may be made, or access may be made via a predetermined URL.

[0031] In particular, the video meeting service terminal 30 according to this embodiment shares (streams) the moving image to at least the evaluation terminal 40 simultaneously or substantially simultaneously in response to at least the operation of the first user terminal 10.

[0032] The evaluation terminal 40 evaluates the video acquired as described above through the following analysis.

[0033] <Example 1> Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 4 is a block diagram showing a configuration example according to this embodiment. As shown in FIG. 4, the video meeting evaluation system of this embodiment includes, as a functional configuration, a moving image acquisition unit 11, a biological reaction analysis unit 12, a specific determination unit 13, a related event identification unit 14, a clustering unit 15, and an analysis result notification unit 16.

[0034] Each of the above functional blocks 11 to 16 can be configured by, for example, any of the hardware, DSP (Digital Signal Processor), and software provided in the evaluation terminal 40. For example, when configured by software, each of the above functional blocks 11 to 16 is actually configured with a computer's CPU, RAM, ROM, etc., and is realized by the operation of a program stored in a recording medium such as RAM, ROM, hard disk, or semiconductor memory.

[0035] The moving image acquisition unit 11 acquires, from each terminal, a moving image obtained by photographing a plurality of people (a plurality of users) with a camera provided in each terminal during an online session. Whether the moving image acquired from each terminal is set to be displayed on the screen of each terminal or not does not matter. That is, the moving image acquisition unit 11 acquires moving images from each terminal, including the moving images being displayed and not being displayed on each terminal.

[0036] The biological reaction analysis unit 12 analyzes changes in the biological reactions of each of the plurality of people based on the moving image (regardless of whether it is being displayed on the screen) acquired by the moving image acquisition unit 11. In the present embodiment, the biological reaction analysis unit 12 separates the moving image acquired by the moving image acquisition unit 11 into a set of images (a collection of frame images) and audio, and analyzes changes in the biological reactions from each of them.

[0037] For example, the biological reaction analysis unit 12 analyzes changes in the biological reaction related to at least one of the expression, eye line, pulse, and facial movement by analyzing the user's face image using the frame images separated from the moving image acquired by the moving image acquisition unit 11. In addition, the biological reaction analysis unit 12 analyzes changes in the biological reaction related to at least one of the user's speech content and voice quality by analyzing the audio separated from the moving image acquired by the moving image acquisition unit 11.

[0038] When a person's emotions change, they manifest as changes in physiological reactions such as facial expressions, eye gaze, pulse, facial movements, speech content, voice quality, etc. In this embodiment, by analyzing the changes in the user's physiological reactions, the changes in the user's emotions are analyzed. The emotion analyzed in this embodiment is, as an example, the degree of pleasure / displeasure. In this embodiment, the physiological reaction analysis unit 12 calculates a physiological reaction index value that reflects the content of the change in the physiological reaction by quantifying the change in the physiological reaction according to a predetermined standard.

[0039] The analysis of the change in facial expression is performed, for example, as follows. That is, for each frame image, the face region is identified from the frame image, and the facial expressions of the identified face are classified into a plurality according to a pre-trained image analysis model. Then, based on the classification results, it is analyzed whether a positive facial expression change has occurred between consecutive frame images, whether a negative facial expression change has occurred, and the magnitude of the facial expression change that has occurred, and a facial expression change index value corresponding to the analysis result is output.

[0040] The analysis of the change in eye gaze is performed, for example, as follows. That is, for each frame image, the eye region is identified from the frame image, and by analyzing the directions of both eyes, it is analyzed where the user is looking. For example, it is analyzed whether the user is looking at the face of the speaker being displayed, the shared material being displayed, outside the screen, etc. Also, it may be analyzed whether the movement of the eye gaze is large or small, and whether the frequency of the movement is high or low. The change in eye gaze is also related to the user's concentration. The physiological reaction analysis unit 12 outputs an eye gaze change index value according to the analysis result of the change in eye gaze.

[0041] The analysis of the change in pulse is performed as follows, for example. That is, for each frame image, the face region is specified from within the frame image. Then, using a learned image analysis model that captures the numerical value of the face color information (G of RGB), the change in the G color on the face surface is analyzed. By arranging the results along the time axis, a waveform representing the change in color information is formed, and the pulse is specified from this waveform. When a person is nervous, the pulse becomes faster, and when the person is calm, the pulse becomes slower. The biological reaction analysis unit 12 outputs a pulse change index value according to the analysis result of the change in pulse.

[0042] The analysis of the change in the movement of the face is performed as follows, for example. That is, for each frame image, the face region is specified from within the frame image, and by analyzing the orientation of the face, it is analyzed where the user is looking. For example, it is analyzed whether the user is looking at the face of the speaker being displayed, the shared material being displayed, outside the screen, etc. Also, it may be analyzed whether the movement of the face is large or small, and whether the frequency of movement is high or low. The movement of the face and the movement of the line of sight may be analyzed together. For example, it may be analyzed whether the user is looking straight at the face of the speaker being displayed, looking up or down, or looking obliquely. The biological reaction analysis unit 12 outputs a face orientation change index value according to the analysis result of the change in the orientation of the face.

[0043] The analysis of the speech content is performed as follows, for example. That is, the biological reaction analysis unit 12 performs known speech recognition processing on the speech for a specified time (for example, a time of about 30 to 150 seconds) to convert the speech into a character string, and by performing morphological analysis on the character string, words that are unnecessary for representing conversation such as particles and articles are removed. Then, the remaining words are vectorized, and it is analyzed whether a positive emotional change is occurring, whether a negative emotional change is occurring, and the magnitude of the emotional change that is occurring, and a speech content index value according to the analysis result is output.

[0044] The analysis of voice quality is performed as follows, for example. That is, the biological reaction analysis unit 12 identifies the acoustic characteristics of the voice by performing known voice analysis processing on the voice for a specified time (for example, a time of about 30 to 150 seconds). Then, based on the acoustic characteristics, it analyzes whether a positive voice quality change has occurred, whether a negative voice quality change has occurred, and the magnitude of the voice quality change, and outputs a voice quality change index value corresponding to the analysis result.

[0045] The biological reaction analysis unit 12 calculates a biological reaction index value using at least one of the facial expression change index value, eye line change index value, pulse rate change index value, face orientation change index value, speech content index value, and voice quality change index value calculated as described above. For example, the biological reaction index value is calculated by performing weighted calculation on the facial expression change index value, eye line change index value, pulse rate change index value, face orientation change index value, speech content index value, and voice quality change index value.

[0046] The specific determination unit 13 determines whether the change in the biological reaction analyzed for the analysis target person is specific compared to the change in the biological reaction analyzed for other persons other than the analysis target person. In the present embodiment, the specific determination unit 13 determines whether the change in the biological reaction analyzed for the analysis target person is specific compared to others based on the biological reaction index values calculated for each of a plurality of users by the biological reaction analysis unit 12.

[0047] For example, the specific determination unit 13 calculates the variance of the biological reaction index values calculated for each of a plurality of persons by the biological reaction analysis unit 12, and determines whether the change in the biological reaction analyzed for the analysis target person is specific compared to others by comparing the biological reaction index value calculated for the analysis target person with the variance.

[0048] As cases where the change in the biological reaction analyzed for the subject to be analyzed is specific compared to others, the following three patterns can be considered. The first is the case where no particularly large change in the biological reaction occurs for others, but a relatively large change in the biological reaction occurs for the subject to be analyzed. The second is the case where no particularly large change in the biological reaction occurs for the subject to be analyzed, but a relatively large change in the biological reaction occurs for others. The third is the case where a relatively large change in the biological reaction occurs for both the subject to be analyzed and others, but the content of the change is different between the subject to be analyzed and others.

[0049] The related event specifying unit 14 specifies an event occurring with respect to at least one of the subject to be analyzed, others, and the environment when a change in the biological reaction determined to be specific by the specificity determination unit 13 occurs. For example, the related event specifying unit 14 specifies the actions and words of the subject to be analyzed himself / herself from a moving image when a specific change in the biological reaction occurs for the subject to be analyzed. In addition, the related event specifying unit 14 specifies the actions and words of others from a moving image when a specific change in the biological reaction occurs for the subject to be analyzed. In addition, the related event specifying unit 14 specifies the environment from a moving image when a specific change in the biological reaction occurs for the subject to be analyzed. The environment is, for example, shared materials being displayed on the screen, things shown in the background of the subject to be analyzed, and the like.

[0050] The clustering unit 15 analyzes the degree of correlation between a change in the biological reaction (for example, one or a combination of one or more of eye line, pulse, facial movement, speech content, voice quality) determined to be specific by the specificity determination unit 13 and an event (an event specified by the related event specifying unit 14) that occurs when the specific change in the biological reaction occurs, and when it is determined that the correlation is equal to or higher than a certain level, clusters the subject to be analyzed or the event based on the analysis result of the correlation.

[0051] For example, when a specific change in a biological reaction corresponds to a negative emotional change and the event occurring when such a specific change in the biological reaction occurs is also a negative event, a correlation equal to or higher than a certain level is detected. The clustering unit 15 clusters the subject to be analyzed or the event into any one of a plurality of pre-segmented classifications according to the content of the event, the degree of negativity, the magnitude of the correlation, and the like.

[0052] Similarly, when a specific change in a biological reaction corresponds to a positive emotional change and the event occurring when such a specific change in the biological reaction occurs is also a positive event, a correlation equal to or higher than a certain level is detected. The clustering unit 15 clusters the subject to be analyzed or the event into any one of a plurality of pre-segmented classifications according to the content of the event, the degree of positivity, the magnitude of the correlation, and the like.

[0053] The analysis result notification unit 16 notifies at least one of the change in the biological reaction determined to be specific by the specificity determination unit 13, the event specified by the related event identification unit 14, and the classification clustered by the clustering unit 15 to the designated person of the subject to be analyzed (the subject to be analyzed or the organizer of the online session).

[0054] For example, when a specific change in the biological reaction different from others occurs in the subject to be analyzed (any of the three patterns described above; the same applies hereinafter), the analysis result notification unit 16 notifies the subject to be analyzed of his or her own speech and actions as the event occurring at that time. Thereby, the subject to be analyzed can grasp that he or she has different emotions from others when performing a certain speech or action. At this time, the specific change in the biological reaction identified for the subject to be analyzed may also be notified to the subject to be analyzed. Further, the change in the biological reaction of the other person to be compared may be further notified to the subject to be analyzed.

[0055] For example, when there is a difference between the emotion received by others from the words and deeds of the subject of analysis performed without particular awareness with their usual emotions, or the words and deeds of the subject of analysis performed with a particular emotion with awareness, and the emotion that the subject of analysis themselves had at the time of the words and deeds, the subject of analysis' own words and deeds at that time are notified to the subject of analysis. As a result, it is also possible to discover words and deeds that are well-received by others or words and deeds that are not well-received by others against one's own consciousness.

[0056] In addition, when an event that occurred when a change in a specific biological reaction different from others occurred for the subject of analysis, the analysis result notification unit 16 notifies the organizer of the online session together with the change in the specific biological reaction. As a result, the organizer of the online session can know what events are affecting what emotional changes as phenomena specific to the designated subject of analysis. And it becomes possible to take appropriate measures for the subject of analysis according to the grasped content.

[0057] In addition, the analysis result notification unit 16 notifies the organizer of the online session of an event that occurred when a change in a specific biological reaction different from others occurred for the subject of analysis or the clustering result of the subject of analysis. As a result, the organizer of the online session can grasp the tendency of the behavior specific to the designated subject of analysis or predict future possible behaviors and states depending on the classification in which the designated subject of analysis was clustered. And it becomes possible to take appropriate measures for the subject of analysis accordingly.

[0058] In the above embodiment, the biological reaction index value is calculated by quantifying the change in the biological reaction according to a predetermined standard, and based on the biological reaction index values calculated for each of a plurality of people, an example of determining whether the change in the biological reaction analyzed for the subject of analysis is specific compared to others has been described, but it is not limited to this example. For example, the following may be done.

[0059] That is, the biological reaction analysis unit 12 analyzes the movement of the line of sight for each of a plurality of persons and generates a heat map indicating the direction of the line of sight. The abnormality determination unit 13 determines whether the change in the biological reaction analyzed for the person to be analyzed is abnormal compared to the change in the biological reaction analyzed for others by comparing the heat map generated for the person to be analyzed by the biological reaction analysis unit 12 with the heat map generated for others.

[0060] <Example 2> Hereinafter, an explanation will be given based on Example 2 of the present invention. FIG. 5 is a block diagram showing a configuration example according to the present embodiment. As shown in FIG. 1, the video meeting evaluation system of the present embodiment includes, as functional components, a moving image acquisition unit 11, a biological reaction analysis unit 12, and a reaction information presentation unit 13a.

[0061] The reaction information presentation unit 13a presents information indicating changes in the biological reaction analyzed by the biological reaction analysis unit 12a including participants not displayed on the screen. For example, the reaction information presentation unit 13a presents information indicating changes in the biological reaction to the leader, facilitator, or administrator of the online session (hereinafter collectively referred to as the host). The host of the online session is, for example, a lecturer of an online class, a chairperson or facilitator of an online meeting, a coach of a session for coaching, etc. The host of the online session is usually one of the plurality of users participating in the online session, but may also be another person who does not participate in the online session.

[0062] By doing so, the host of the online session can also grasp the state of participants not displayed on the screen in an environment where the online session is held by a plurality of people.

[0063] The preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings. However, the technical scope of the present disclosure is not limited to such examples. It is obvious that those with ordinary knowledge in the technical field of the present disclosure can conceive of various modification examples or correction examples within the scope of the technical idea described in the claims, and it is naturally understood that these also belong to the technical scope of the present disclosure.

[0064] The devices described in this specification may be realized as a single device, or may be realized by a plurality of devices (such as a cloud server) partially or entirely connected by a network. For example, the control unit 11 and the storage 13 of the information sharing support device 10 may be realized by different servers connected to each other by a network.

[0065] A series of processes by the devices described in this specification may be realized using any of software, hardware, and combinations of software and hardware. It is possible to create a computer program for realizing each function of the information sharing support device 10 according to this embodiment and install it on a PC or the like. In addition, it is also possible to provide a computer-readable recording medium storing such a computer program. The recording medium is, for example, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, etc. Also, the above computer program may be distributed via a network, for example, without using a recording medium.

[0066] Also, the processes described in this specification using flowcharts do not necessarily have to be executed in the order shown in the figures. Some process steps may be executed in parallel. Also, additional process steps may be adopted, and some process steps may be omitted.

[0067] Also, the effects described in this specification are merely illustrative or exemplary and not limiting. That is, the technology according to the present disclosure may exhibit other effects obvious to those skilled in the art from the description of this specification, together with or instead of the above effects.

[0068] The following configurations may also be included in the present invention. <Configuration 1> A video meeting evaluation system that analyzes specific emotions different from others for an analysis target among a plurality of people in an environment where an online session is conducted by a plurality of people. <Configuration 2> A moving image acquisition unit that acquires a moving image obtained by photographing the plurality of people during the online session, A biological reaction analysis unit that analyzes changes in biological reactions for each of the plurality of people based on the moving image acquired by the moving image acquisition unit, A video meeting evaluation system comprising: a specific determination unit that determines whether or not changes in the biological reaction analyzed for the analysis target are specific compared to changes in the biological reaction analyzed for others other than the analysis target. characterized by the above. <Configuration 3> The biological reaction analysis unit according to Configuration 2, wherein the biological reaction analysis unit analyzes changes in a biological reaction related to at least one of an expression, a gaze, a pulse, and a facial movement by analyzing a face image in the moving image acquired by the moving image acquisition unit. <Configuration 4> The biological reaction analysis unit according to Configuration 2 or 3, wherein the biological reaction analysis unit analyzes changes in a biological reaction related to at least one of a speech content and a voice quality by analyzing a voice in the moving image acquired by the moving image acquisition unit. <Configuration 5> The biological reaction analysis unit calculates a biological reaction index value by quantifying changes in the biological reaction according to a predetermined standard, The specific determination unit determines whether or not changes in the biological reaction analyzed for the analysis target are specific compared to changes in the biological reaction analyzed for others other than the analysis target based on the biological reaction index values calculated for each of the plurality of people by the biological reaction analysis unit. The video meeting evaluation system according to any one of Configurations 2 to 4, characterized in that... <Configuration 6> The specific determination unit calculates the variance of the biological reaction index values calculated for each of the plurality of persons by the biological reaction analysis unit, and based on the comparison between the biological reaction index value calculated for the analysis target person and the variance, determines whether the change in the biological reaction analyzed for the analysis target person is specific compared to the change in the biological reaction analyzed for the other persons. The video meeting evaluation system according to Configuration 5, characterized in that... <Configuration 7> The biological reaction analysis unit analyzes the movement of the line of sight for each of the plurality of persons and generates a heat map indicating the direction of the line of sight. The specific determination unit determines whether the change in the biological reaction analyzed for the analysis target person is specific compared to the change in the biological reaction analyzed for the other persons by comparing the heat map generated for the analysis target person by the biological reaction analysis unit with the heat map generated for the other persons. The video meeting evaluation system according to Configuration 3, characterized in that... <Configuration 8> The video meeting evaluation system according to any one of Configurations 2 to 7, further comprising a related event identification unit that identifies an event occurring with respect to at least one of the analysis target person, the other persons, and the environment when a change in the biological reaction determined to be specific by the specific determination unit occurs. <Configuration 9> The video meeting evaluation system according to Configuration 8, further comprising a clustering unit that analyzes the degree of correlation between the change in the biological reaction determined to be specific by the specific determination unit and the event occurring when the specific change in the biological reaction occurs, and when it is determined that the correlation is at a certain level or higher, clusters the analysis target person or the event based on the analysis result of the correlation. <Configuration 10> An analysis result notification unit that notifies at least one of the change in the biological reaction determined to be specific by the specific determination unit and the event specified by the related event specification unit to the analysis target person or the organizer of the online session. The video meeting evaluation system according to Configuration 8, characterized in that it further comprises. <Configuration 11> An analysis result notification unit that notifies at least one of the change in the biological reaction determined to be specific by the specific determination unit, the event specified by the related event specification unit, and the classification clustered by the clustering unit to the analysis target person or the organizer of the online session. The video meeting evaluation system according to claim 9, characterized in that it further comprises. <Configuration 12> A reaction analysis system that analyzes the reactions of participants based on moving images obtained by photographing the participants during an online session in an environment where an online session is held by multiple participants, regardless of whether the participants are displayed on the screen during the online session, and presents the analysis results. <Configuration 13> A moving image acquisition unit that acquires moving images obtained by photographing the participants during the online session; A biological reaction analysis unit that analyzes changes in biological reactions of the participants based on the moving images acquired by the moving image acquisition unit; And a reaction information presentation unit that presents information indicating the change in the biological reaction analyzed by the biological reaction analysis unit including participants not displayed on the screen. The reaction analysis system according to item 12, characterized in that it is provided with. <Configuration 14> The biological reaction analysis unit analyzes changes in biological reactions related to at least one of facial expressions, eye lines, pulse, and facial movements by analyzing the facial images in the moving images acquired by the moving image acquisition unit. The reaction analysis system according to item 13, characterized in that it is provided with. <Configuration 15> The biological reaction analysis unit analyzes changes in biological reactions related to at least one of the speech content and voice quality by analyzing the voice in the moving image acquired by the moving image acquisition unit. The reaction analysis system according to item 13 or item 14. <Configuration 16> The biological reaction analysis unit analyzes where a participant not displayed on the screen is looking at the shared materials displayed on the screen. The reaction analysis system according to item 13. <Configuration 17> The biological reaction analysis unit analyzes at what timing during the online session a participant not displayed on the screen spoke. The reaction analysis system according to item 13. <Configuration 18> The reaction information presentation unit presents information indicating changes in the biological reaction to the organizer of the online session. The reaction analysis system according to any one of items 13 to 17.

Explanation of symbols

[0069] 10, 20 User terminals 30 Video conferencing service terminals 40 Evaluation terminals

Claims

1. a video meeting service terminal that provides a video meeting to at least the first user terminal and the second user terminal and acquires a video image acquired by at least the first camera unit or the second camera unit; an evaluation terminal for evaluating the video conference; A video meeting evaluation system, comprising: the video meeting service terminal simultaneously or substantially simultaneously shares the video with at least the evaluation terminal in response to an operation of at least the first user terminal; the evaluation terminal identifies at least a facial image included in the shared video for each predetermined frame, and calculates an evaluation value for the facial image; Video meeting evaluation system.

2. 2. The video meeting evaluation system of claim 1, The evaluation terminal provides graph information of the evaluation value in time series. Video meeting evaluation system.

3. 3. The video meeting evaluation system according to claim 1, further comprising: the evaluation terminal calculates a plurality of evaluation values ​​by evaluating the face image from a plurality of different viewpoints; Video meeting evaluation system.

4. A video meeting evaluation system according to any one of claims 1 to 3, The evaluation terminal calculates the evaluation value together with the audio included in the moving image. Video meeting evaluation system.

5. A video meeting evaluation system according to any one of claims 1 to 4, the evaluation terminal calculates the evaluation value together with an object other than the face image included in the moving image; Video meeting evaluation system.

Images