Conference system, reaction mark display method, and program

Abstract

To provide a technique for grasping the reactions of a plurality of participants.SOLUTION: The present invention is a conference system in which a plurality of terminal devices and a conference server can communicate via a network. The conference system includes a reaction type determination unit that determines a reaction type shown by each participant based on moving image data or audio data of conference participants acquired by each of the plurality of terminal devices, and a reaction mark display unit that displays reaction marks corresponding to the number of people determined to be each reaction type for one or more reaction types.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a conference system, a reaction mark display method, and a program. [Background technology]

[0002] Online conferences are known in which multiple terminal devices are connected to a conference server via a network, and each user of the terminal device logs in to the same conference to participate in the conference. In online conferences, participants can turn off the camera of their terminal device at will, but it is difficult for participants at other locations to understand the reaction of participants who have turned the camera off. For this reason, presenters sometimes cannot understand whether their explanation has been conveyed well to participants or how the participants are feeling.

[0003] A technology for knowing the emotions of participants in an online conference has been devised (see, for example, Patent Document 1). Patent Document 1 discloses a technology in which emotional information of a first person is acquired by a sensory information acquisition unit of a terminal device, the emotional information is transmitted to a terminal device of a second person via a network, and the transmitted emotional information is notified to the second person by the terminal device of the second person. Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the conventional technology, when multiple participants are participating in an online conference, a participant cannot grasp the reactions of the multiple participants. For example, there may be multiple participants from the presenter's perspective, and in such a case, it is difficult for the presenter to understand the reactions of the multiple participants as a whole.

[0005] In view of the above-mentioned problems, the present invention provides a technique that makes it possible to grasp the reactions of multiple participants. [Means for solving the problem]

[0006] In view of the above problems, the present invention provides a conference system in which multiple terminal devices and a conference server can communicate via a network, comprising a reaction type determination unit that determines the reaction type shown by each participant based on video data or audio data of the conference participants acquired by each of the multiple terminal devices, and a reaction mark display unit that displays reaction marks corresponding to the number of people determined to be each reaction type for one or more reaction types. Effect of the Invention

[0007] The present invention can provide a technique for grasping the reactions of multiple participants. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram for explaining an outline of a process in which a conference server provides reactions of multiple participants to each participant. [Diagram 2] FIG. 1 illustrates an example of a system configuration of a conference system. [Diagram 3] FIG. 2 is a diagram illustrating an example of a hardware configuration of a conference server and a terminal device. [Figure 4] FIG. 2 is an example of a functional block diagram illustrating functions of a conference server and terminal devices in the conference system, with the functions divided into blocks. [Diagram 5] FIG. 13 is a flowchart illustrating an example of a process in which a participant selects a reaction visualization mode and an icon display format. [Figure 6] FIG. 13 is a diagram showing an example of a functional block diagram operating in a reaction statistics view. [Figure 7] 11 is a sequence diagram illustrating an example of a process in which a terminal device displays icons according to reactions of a plurality of participants in a reaction statistics view. FIG. [Figure 8] FIG. 11 is a diagram showing an example of an icon displayed by the terminal device. [Figure 9] FIG. 13 is a diagram showing an example of an icon of a reaction type with the largest number of participants or a reaction type with the smallest number of participants, displayed by the terminal device. [Figure 10] FIG. 13 is a diagram showing an example of a functional block diagram operating in a reaction statistics view. [Figure 11] 13 is a sequence diagram illustrating an example of a process in which a terminal device displays icons according to reaction types of a plurality of participants in a reaction statistics view. FIG. [Figure 12] FIG. 13 is a diagram showing an example of a functional block diagram operating in a reaction extraction view. [Figure 13] 11 is an example of a sequence diagram illustrating a process in which a terminal device displays icons according to reactions of a plurality of participants in a reaction extraction view. FIG. [Figure 14] FIG. 11 is a diagram showing an example of an icon displayed by the terminal device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, a conference system and a reaction mark display method performed in the conference system will be described as an example of an embodiment of the present invention.

[0010] [First embodiment] <Outline of reaction display method> FIG. 1 is a diagram for explaining an outline of a process in which a conference server 1 provides reactions of multiple participants to each participant in this embodiment. (1) The terminal devices 4 a to 4 n acquire video data of the participants captured by their cameras and transmit the data to the conference server 1 . (2) The conference server 1 receives the video data and estimates the reaction type (e.g., positive / negative) of each participant from the video data. The conference server 1 also tallies the reaction type for each participant. (3) The conference server 1 transmits icon data representing the number of people for each reaction type to each terminal device (only terminal device 4a in the figure). (4) The terminal device 4a receives the icon data and displays the icons 201 and 202 based on the icon data. In Fig. 1, the number of participants with positive reactions is 20 and the number of participants with negative reactions is displayed by the size and number of the icons 201 and 202. Therefore, a participant (which may be a presenter or a general participant) can grasp the overall reactions of the other participants.

[0011] <Terminology> Moving image data refers to images in which image frames are updated one after another and appear to move due to residual images. Moving image data is often called video. In this embodiment, the reaction type is determined during an online conference, so moving image data is sent from the terminal device to the conference server 1, but the reaction type may be determined from a single image frame.

[0012] A reaction is a reaction made by a participant in an online conference to a presentation by another participant. A reaction may include a participant's emotions, mood, feelings, thoughts, and intentions.

[0013] The reaction type refers to the type into which the reaction shown by the participant participating in the online conference is most closely classified. Reaction types include, for example, positive and negative. Positive may be referred to as affirmative, supportive, proactive, optimistic, etc., and negative may be referred to as negative, opposed, or passive. In addition, reaction types may include neutral. Furthermore, the reaction types may be divided into N stages (for example, 5 stages) from negative to positive, and the stage to which the participant's reaction corresponds may be used as the reaction type.

[0014] The reaction mark is a mark displayed to represent a reaction type. The reaction mark refers to an icon, a symbol, a sign, a mark, a design, etc. In this embodiment, the reaction mark is described using the term icon.

[0015] <System configuration> The system configuration of the conference system 300 will be described with reference to FIG. 2. FIG. 2 is a diagram showing an example of the system configuration of the conference system 300. In the conference system 300, Wi-Fi routers 3 and 10 in a conference room 17 and a home 18 are communicatively connected to the conference server 1 via a network N. Terminal devices 4a to 4n and 5 can be communicatively connected to the Wi-Fi routers 3 and 10. Any of the terminal devices 4a to 4n is referred to as a "terminal device 4." The network N is, for example, the Internet, but may also be an on-premise network such as an in-house network. The number of terminal devices 4a to 4n and 5 is just an example, and may be any number greater than or equal to two.

[0016] The conference server 1 is an information processing system having one or more information processing devices. The conference server 1 manages conference information, etc., and manages the start and end of the conference, participant login, etc. The services provided by the conference app executed on the conference server 1 are called conference services.

[0017] A Wi-Fi router 3 is installed in the conference room 17, and each of the participants A1 to An brings a terminal device 4a to 4n and enters the conference room 17. The terminal devices 4a to 4n may be installed in the conference room 17. The terminal devices 4a to 4n communicate with the Wi-Fi router 3 via wireless LAN. Each of the terminal devices 4a to 4n is running a conference app. The conference app may be a native app or a Web app.

[0018] On the other hand, participant B joins the conference from his home 18. A Wi-Fi router 10 is installed in his home 18. Participant B uses a terminal device 5. The Wi-Fi router 10 in participant B's home 18 is communicatively connected to network N. Participant B's terminal device 5 communicates with the Wi-Fi router 10 via wireless LAN. Participant B's terminal device 5 runs a conference app.

[0019] The terminal devices 4a to 4n and 5 are examples of devices operated by the participants of the conference. The terminal devices 4a to 4n and 5 may be information processing devices that have a communication function and run a conference app, such as a PC (Personal Computer), a tablet terminal, a smartphone, or a PDA (Personal Digital Assistant).

[0020] In the configuration shown in FIG. 2, the terminal devices 4 and 5 connect to the same URL for the conference. The URL and password are distributed in advance to the participants by e-mail or the like. The video data and audio data acquired by the terminal device 4a (an example of a first terminal device) are sent to the conference server 1, which then transmits the data to the terminal devices 4b to 4n and 5 (an example of a second terminal device). The terminal devices 4b to 4n and 5 output the received video data and audio data. Similarly, the video data and audio data acquired by the terminal device 4b are sent to the conference server 1, which then transmits the data to the terminal devices 4a, 4c to 4n and 5. The terminal devices 4a, 4c to 4n and 5 output the received video data and audio data. The online conference is carried out by all the terminal devices 4 and 5 repeating this process.

[0021] 2, participants A1 to An do not have to join the conference from the same conference room 17, and may join the conference from different conference rooms. Any participant may join the conference from their own desk, or may join the conference from outside. The terminal devices 4 and 5 may be connected to the network N by wire.

[0022] <Hardware configuration example> <<Conference server and terminal device>> Fig. 3 is a diagram showing an example of a hardware configuration of the conference server 1 and the terminal devices 4 and 5. As shown in Fig. 3, the conference server 1 and the terminal devices 4 and 5 are constructed by a computer 500, and include a CPU 501, a ROM 502, a RAM 503, an HD (Hard Disk) 504, an HDD (Hard Disk Drive) controller 505, a display 506, an external device connection I / F (Interface) 508, a network I / F 509, a bus line 510, a keyboard 511, a pointing device 512, an optical drive 514, a media I / F 516, a camera 517, and a speaker 518.

[0023] Among these, the CPU 501 controls the operation of the entire computer 500. The ROM 502 stores programs such as an IPL used to drive the CPU 501. The RAM 503 is used as a work area for the CPU 501. The HD 504 stores various data such as programs. The HDD controller 505 controls the reading or writing of various data from the HD 504 under the control of the CPU 501. The display 506 displays various information such as a cursor, a menu, a window, a character, or an image. The external device connection I / F 508 is an interface for connecting various external devices. In this case, the external device is, for example, a USB (Universal Serial Bus) memory or a printer. The network I / F 509 is an interface for data communication using a network. The bus line 510 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 501 shown in FIG. 3.

[0024] The keyboard 511 is a type of input means having a plurality of keys used for inputting characters, numbers, various instructions, etc. The pointing device 512 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The optical drive 514 controls reading and writing of various data from an optical storage medium 513, which is an example of a removable storage medium. The optical storage medium 513 may be a CD, a DVD, Blu-ray (registered trademark), etc. The media I / F 516 controls reading and writing (storing) of data from a storage medium 515 such as a flash memory. The camera 517 captures the surroundings of the terminal device, such as facial images of participants, and generates image data. The speaker 518 outputs sound.

[0025] <About the function> Next, the functional configuration of each function of conference system 300 will be described with reference to Fig. 4. Fig. 4 is an example of a functional block diagram explaining the functions of conference server 1 and terminal devices 4 and 5 in conference system 300 by dividing them into blocks.

[0026] <<Terminal Device>> The terminal devices 4 and 5 each have a communication unit 31, a reaction type determination unit 32, a display mode selection unit 33, a reaction mark data reception unit 34, a moving image data acquisition unit 35, a display reaction type selection unit 36, and a reaction mark display unit 37. Each of these units in the terminal devices 4 and 5 is a function or a means for performing a function that is realized when any of the components shown in FIG.

[0027] The communication unit 31 connects to the conference service of the conference server 1 and receives voice data and video data of each participant. The communication unit 31 also transmits to the conference server 1 voice data and video data of the participant who is operating the communication unit 31.

[0028] The reaction type determination unit 32 analyzes the video data acquired by the video data acquisition unit 35 and estimates the reactions of the participants. The reaction type determination unit 32 transmits the analysis result to the conference server 1 via the communication unit 31. The analysis of the video data may be performed by the conference server 1.

[0029] The display mode selection unit 33 displays an ON / OFF button for the reaction visualization mode and accepts ON / OFF selection from the participants. The reaction visualization mode is a mode in which the reactions of multiple participants are visualized with icons. When the reaction visualization mode ON / OFF button is ON, the display mode selection unit 33 further displays selection buttons for the reaction statistics view and the reaction extraction view and accepts selection of the reaction statistics view or the reaction extraction view. The reaction statistics view and the reaction extraction view will be described later.

[0030] The reaction mark data receiving unit 34 receives icon data (an example of a mark indicating a reaction type) generated by the conference server 1.

[0031] Video data acquisition unit 35 acquires video data of participants from camera 317. Video data acquisition unit 35 always acquires video data, regardless of the "camera video display on / off" setting that determines whether or not to transmit video data from camera 317 during a conference to other locations.

[0032] The reaction mark display unit 37 is a means for displaying reaction marks according to the number of participants determined to be each reaction type for each reaction type. Specifically, the reaction mark display unit 37 displays an icon based on the icon data received by the reaction mark data receiving unit 34.

[0033] The display reaction type selection unit 36 ​​receives and stores the reaction types to be displayed by the terminal devices 4 and 5 from the participants. When icon data for each reaction type is transmitted from the conference server 1 to the terminal devices 4 and 5, the terminal devices 4 and 5 display only the icons for the reaction types received by the display reaction type selection unit 36.

[0034] <<Conference Server 1>> The conference server 1 has a communication unit 11, a video data receiving unit 12, a reaction type determining unit 13, a reaction mark generating unit 14, an analysis data receiving unit 15, and an analysis data collecting unit 16. Each of these units of the conference server 1 is a function or a means for performing a function that is realized when any of the components shown in Fig. 3 operates in response to an instruction from the CPU 501 in accordance with a program loaded from the HD 504 to the RAM 503.

[0035] The communication unit 11 is a means for transmitting reaction marks according to the number of people determined to be each reaction type to the terminal devices 4 and 5. Specifically, the communication unit 11 receives video data and audio data transmitted from the terminal devices 4 and 5, and transmits them to the terminal devices 4 and 5 participating in the same conference, excluding the terminal device 4 and 5 that transmitted the data.

[0036] The video data receiving unit 12 receives the video data transmitted from the terminal devices 4 and 5 via the communication unit 11 .

[0037] The reaction type determination unit 13 is a means for determining the type of reaction shown by each participant based on the video data of the participants of the conference received by the video data receiving unit 12. Specifically, the reaction type determination unit 13 inputs the video data into a model constructed by machine learning or the like to determine the type of reaction of the participants. Details will be described later. The reaction type may be determined by the terminal devices 4 and 5.

[0038] When the terminal devices 4 and 5 perform reaction analysis, the analysis data receiving unit 15 receives the reaction analysis results transmitted from the terminal devices 4 and 5 via the communication unit 11.

[0039] The analysis data compilation unit 16 compiles the number of participants exhibiting each reaction type based on the reaction type estimated for each participant.

[0040] The reaction mark generating unit 14 is a means for generating reaction marks according to the reaction types of each participant. Specifically, the reaction mark generating unit 14 generates icon data according to the number of participants for each reaction type, and transmits the icon data to the terminal devices 4 and 5 via the communication unit 11. The conference server 1 may transmit the number of participants for each reaction type to the terminal devices 4 and 5, and the terminal devices 4 and 5 may generate reaction marks.

[0041] <Select reaction statistics view or reaction extraction view> Next, the reaction statistics view and the reaction extraction view will be explained, and a flow in which a participant selects the reaction statistics view and the reaction extraction view will be explained.

[0042] 5 is a flow chart for explaining the process in which a participant selects a reaction visualization mode and an icon display format. The process in FIG.

[0043] The display mode selection unit 33 receives a reaction visualization mode setting from a participant (S101).

[0044] The display mode selection unit 33 determines whether or not the reaction visualization mode is ON (S102). If the reaction visualization mode is not ON, the reaction mark display unit 37 displays a normal icon (S103). A normal icon is an icon in which the reaction of each participant is displayed together with characters that represent the participant, such as initials. Alternatively, the normal icon may represent only the participant without showing a reaction.

[0045] When the reaction visualization mode is ON, the display mode selection unit 33 displays a reaction statistics view and a reaction extraction view in a selectable manner (S104).

[0046] When a participant selects the reaction statistics view, the reaction mark display unit 37 displays an icon generated based on the reaction statistics view (S105). The reaction statistics view is a display format in which the number of people for each reaction type is tallied and the number of people is represented by the size of the icon. A notice that the reaction statistics view has been selected is sent to the conference server. When the conference server 1 receives a notice that the reaction statistics view has been selected, the reaction mark generation unit 14 generates a larger icon the greater the number of people tallied for each reaction type. When the conference server 1 receives a notice that the reaction extraction view has been selected, the reaction mark generation unit 14 generates a reaction mark for each participant according to the reaction type of each participant.

[0047] When a participant selects the reaction extraction view, the reaction mark display unit 37 displays an icon generated based on the reaction extraction view (S106). The reaction extraction view is a display format in which a participant selects a reaction type that he or she wants to display, and only the icons of participants that are determined to have that reaction type are displayed. The fact that the reaction extraction view has been selected is transmitted to the conference server.

[0048] A method of displaying icons according to the reaction statistics view and the reaction extraction view will be described in detail below.

[0049] <Meeting server 1 analyzes reactions and creates icon data in the reaction statistics view> First, the icons displayed by the terminal devices 4 and 5 when the reaction statistics view is selected will be described. The reaction statistics view is a display format in which the number of people is tallied for each reaction type and the number of people is represented by the size of the icon. The size of the icon changes every time the number of people for each reaction type changes.

[0050] 6 is a diagram showing a functional block diagram that operates in the reaction statistics view. The process performed by each function in the reaction statistics view will be described. Note that the communication unit 11 is omitted because it is always passed through in the communication between the conference server 1 and the terminal devices 4 and 5.

[0051] (1) The display mode selection unit 33 of the terminal device 4, 5 accepts the ON of the reaction visualization mode by the participant. The display mode selection unit 33 also accepts the selection of the reaction statistics view. The information that the reaction statistics view has been selected is transmitted to the conference server 1.

[0052] (2) The video data acquisition unit 35 of the terminal devices 4 and 5 acquires the video data of the participants and transmits it to the conference server 1 via the communication unit 31 .

[0053] (3) The video data receiving unit 12 of the conference server 1 receives the video data via the communication unit 11 and transmits it to the reaction type determining unit 13 .

[0054] (4) The reaction type determination unit 13 of the conference server 1 estimates the reaction type and transmits the estimated reaction type (positive / negative) as the analysis result to the analysis data compilation unit 16. The reaction type estimation method will be described below.

[0055] The reaction type determination unit 13 analyzes changes in biological reactions of each of the multiple participants based on the video data. The reaction type determination unit 13 separates the video data into a set of images (a collection of frame images) and audio, and analyzes changes in biological reactions from each.

[0056] For example, the reaction type determination unit 13 analyzes the changes in the biological reaction related to at least one of facial expression, gaze, pulse rate, and facial movement by analyzing the facial images of the participants using frame images separated from the video data. Also, the reaction type determination unit 13 analyzes the changes in the biological reaction related to at least one of the participant's remarks and voice quality by analyzing the voices separated from the video data.

[0057] When a person's emotions change, this is reflected in changes in biological reactions such as facial expressions, gaze, pulse rate, facial movements, speech, and voice quality. The reaction type determination unit 13 analyzes changes in the emotions of the participants by analyzing changes in the biological reactions of the participants. In this embodiment, the emotion analyzed is, for example, the degree of comfort / discomfort. In this embodiment, the reaction type determination unit 13 quantifies the changes in the biological reactions according to a predetermined standard, thereby calculating a biological reaction index value that reflects the details of the changes in the biological reactions.

[0058] The analysis of the change in facial expression is performed, for example, as follows. That is, the reaction type determination unit 13 identifies a facial region from each frame image, and classifies the identified facial expressions into a plurality of types according to an image analysis model that has been machine-learned in advance. Then, based on the classification result, it analyzes whether a positive or negative facial expression change has occurred between consecutive frame images, and the extent of the facial expression change, and outputs a facial expression change index value according to the analysis result.

[0059] The analysis of the change in the line of sight is performed, for example, as follows. That is, the reaction type determination unit 13 identifies the eye area from within each frame image and analyzes the direction of both eyes to analyze where the participant is looking. For example, it analyzes whether the participant is looking at the face of the speaker being displayed, the shared material being displayed, or looking outside the screen. It may also be possible to analyze whether the movement of the line of sight is large or small, and whether the movement is frequent or infrequent. The change in the line of sight is also related to the concentration level of the participant. The reaction type determination unit 13 outputs a line of sight change index value according to the analysis result of the change in the line of sight.

[0060] The analysis of changes in pulse rate is performed, for example, as follows. That is, for each frame image, the reaction type determination unit 13 identifies the face area from within the frame image. Then, the reaction type determination unit 13 analyzes changes in G color on the face surface using a trained image analysis model that captures the numerical value of face color information (G of RGB). The results are arranged along the time axis to form a waveform that represents changes in color information, and the pulse rate is identified from this waveform. When a person is nervous, the pulse rate increases, and when the person feels calm, the pulse rate decreases. The reaction type determination unit 13 outputs a pulse rate change index value according to the analysis result of the changes in pulse rate.

[0061] The analysis of the change in facial movement is performed, for example, as follows. That is, the reaction type determination unit 13 identifies the face area from within each frame image and analyzes the face direction to analyze where the participant is looking. The reaction type determination unit 13 analyzes, for example, whether the participant is looking at the face of the speaker being displayed, the shared material being displayed, or looking outside the screen. The reaction type determination unit 13 may also analyze whether the facial movement is large or small, whether the frequency of the movement is high or low, and the like. The facial movement and the eye movement may be analyzed together. The reaction type determination unit 13 may also analyze, for example, whether the participant is looking straight at the face of the speaker being displayed, looking up or down, or looking at an angle, and the like. The reaction type determination unit 13 outputs a facial direction change index value according to the analysis result of the change in the facial direction.

[0062] The analysis of the utterance contents is performed, for example, as follows. That is, the reaction type determination unit 13 converts the voice into a character string by performing a known voice recognition process on the voice of a specified time (for example, about 30 to 150 seconds), and removes words unnecessary for expressing the conversation, such as particles and articles, by performing a morphological analysis of the character string. Then, it vectorizes the remaining words, analyzes whether a positive or negative emotional change has occurred, and the extent of the emotional change, and outputs a utterance content index value according to the analysis result. Therefore, the reaction type may be determined from the voice data.

[0063] The voice quality analysis is performed, for example, as follows. That is, the reaction type determination unit 13 identifies the acoustic features of the voice by performing a known voice analysis process on the voice for a specified time (for example, about 30 to 150 seconds). Then, based on the acoustic features, it analyzes whether a positive or negative voice quality change has occurred and how much of a voice quality change has occurred, and outputs a voice quality change index value according to the analysis result.

[0064] The reaction type determination unit 13 calculates a biological reaction index value using at least one of the facial expression change index value, eye direction change index value, pulse rate change index value, face direction change index value, statement content index value, and voice quality change index value calculated as described above. The reaction type determination unit 13 determines the reaction type (positive / negative) by, for example, weighting the facial expression change index value, eye direction change index value, pulse rate change index value, face direction change index value, statement content index value, and voice quality change index value.

[0065] (5) Next, the analysis data aggregation unit 16 of the conference server 1 aggregates the number of people for each reaction type and transmits the number of people for each reaction type to the reaction mark generation unit 14. The aggregation is performed for all participants. In other words, the icon displayed by the terminal device 4a includes the aggregation result of the reaction type of participant A1. However, the terminal device 4a can also display the individual reaction type of participant A1.

[0066] (6) The reaction mark generating unit 14 of the conference server 1 generates icon data for each reaction type depending on the number of people for each reaction type, and transmits the icon data to the terminal devices 4 and 5 via the communication unit 11.

[0067] (7) The reaction mark data receiving unit 34 of the terminal devices 4 and 5 receives the generated icon data via the communication unit 31 and transmits it to the reaction mark display unit 37. The reaction mark display unit 37 of the terminal devices 4 and 5 displays an icon based on the icon data.

[0068] In FIG. 6, the conference server 1 generates the icon data, but the conference server 1 may transmit the number of people for each reaction type to the terminal devices 4 and 5, and the terminal devices 4 and 5 may generate the icon data.

[0069] Fig. 7 is a sequence diagram for explaining the process in which the terminal devices 4 and 5 display icons corresponding to the reactions of multiple participants in the reaction statistics view. Note that, although the number of the terminal devices 4 and 5 is three in Fig. 7, the number of the terminal devices 4 and 5 may be more than one. Also, in Fig. 7, an example in which a participant operates the terminal device 4a is explained, but the process in the other terminal devices 4 and 5 is similar.

[0070] S1: A participant turns on the reaction visualization mode and selects the reaction statistics view. The display mode selection unit 33 of the terminal device 4a accepts the participant's turning on of the reaction visualization mode. The display mode selection unit 33 also accepts the selection of the reaction statistics view.

[0071] S2: The video data acquisition unit 35 of the terminal device 4a acquires video data of the participants.

[0072] S3: The communication unit 31 transmits the moving image data of the participants to the conference server 1. The moving image data acquisition units 35 of the terminal devices 4b and 5 also acquire the moving image data of the participants and transmit it to the conference server 1 via the communication units 31.

[0073] S4: The video data receiving unit 12 of the conference server 1 receives the video data via the communication unit 11 and transmits it to the reaction type determining unit 13. The reaction type determining unit 13 of the conference server 1 estimates the reaction type and transmits the estimated reaction type (positive / negative) to the analysis data aggregation unit 16 as the analysis result.

[0074] S5: The analysis data collection unit 16 of the conference server 1 collects the number of people for each reaction type, and transmits the number of people for each reaction type to the reaction mark generation unit 14.

[0075] S6: The reaction mark generation unit 14 of the conference server 1 is notified that it is a reaction statistics view, and generates icon data for each reaction type according to the number of people of each reaction type. The reaction mark generation unit 14 creates larger icon data for each reaction type as the number of people of each reaction type increases.

[0076] S7 to S9: The communication unit 11 transmits icon data for each reaction type to the terminal devices 4a, 4b, and 5.

[0077] S10 to S12: The reaction mark data receiving unit 34 of the terminal devices 4a, 4b, and 5 receives the generated icon data via the communication unit 31 and transmits it to the reaction mark display unit 37. The reaction mark display unit 37 of the terminal devices 4a, 4b, and 5 displays an icon for each reaction type.

[0078] The terminal devices 4a, 4b, and 5 and the conference server 1 repeat the above process at regular time intervals. Thus, the terminal devices 4a, 4b, and 5 can each display an icon showing the most recent reaction type of each participant. Alternatively, the conference server 1 may monitor whether the reaction type of each participant has changed, and generate icon data for each reaction type according to the number of participants of each reaction type only when the reaction type of one or more participants has changed. In this way, the conference server 1 can reduce the communication load.

[0079] Fig. 8 shows an example of icons 201, 202 displayed by the terminal devices 4a, 4b, 5. Two icons 201, 202 and reaction type text information 204, 205 are displayed in Fig. 8. One is icon 201 indicating the number of people with a positive reaction, and the other is icon 202 indicating the number of people with a negative reaction. The reaction mark display unit 37 displays a larger reaction mark for each reaction type as the number of participants determined to be of each reaction type increases. The icons 201, 202 are displayed in a size proportional to the number of people determined to be of each reaction type, for example.

[0080] An example of a method for determining the size will be described. First, the reaction mark generation unit 14 assigns a predetermined diameter R to the largest number of people. The diameter R may be set to an appropriate size in the display area of ​​the icon. Next, the reaction mark generation unit 14 calculates "number of positive people ÷ number of negative people = K". The reaction mark generation unit 14 sets the diameter of the icon to R / K for reaction types that are not the largest number of people. This allows participants to easily compare the number of people with positive reactions and the number of people with negative reactions.

[0081] The diameter of the icons 201, 202 does not have to be proportional to the number of people determined to be of each reaction type. The reaction mark generation unit 14 predetermines the diameter of the icons into, for example, about five stages, and determines the diameter of the icon according to the ratio of the number of people determined to be of each reaction type to the total number of participants. The reaction mark generation unit 14 assigns the following diameters: maximum for 100-80%, large for 80-60%, medium for 60-40%, small for 40-20%, and minimum for 20-0%. In this way, the reaction mark generation unit 14 can gradually change the ratio of the diameters of the icons 201, 202, and the terminal devices 4, 5 can display the same diameter when the number of people is about the same.

[0082] The reaction type text information 204, 205 also displays the number of participants who have been determined to have each reaction type, so that each participant can know the specific number of participants.

[0083] 8, the reaction types are classified into two, but the reaction types may be classified into three or more. The reaction types are, for example, positive, negative, neutral, and the like.

[0084] Moreover, the icons 201 and 202 do not have to be circular, and may be square, triangle, diamond, or other shapes. The icons 201 and 202 may be displayed in different colors depending on the reaction type. The icons 201 and 202 may be displayed in different shapes depending on the number of people of the reaction type (for example, a triangle when the number of people is small and a circle when the number of people is large).

[0085] 8 also displays an icon 203 indicating the participant's own reaction type. The reaction mark display unit 37 displays a reaction mark based on the reaction type determined for the participant operating the terminal device 4, 5. The icons indicating the reactions of the participants operating the terminal devices 4, 5 may be sent by the conference server 1 to the terminal devices 4, 5, or may be analyzed and displayed by the terminal devices 4, 5. The participant can simultaneously check his / her own reaction type using the icon 203 and compare it with the reaction types of the other participants. The terminal devices 4a, 4b, 5 may further display the reaction type for each participant.

[0086] <Display only icons of the reaction type with the most people> The conference server 1 may transmit to the terminal devices 4 and 5 only icon data that represents the most common reaction in the reaction statistics view.

[0087] In this case, in the process of "6." in Fig. 6 or in the process of step S6 in Fig. 7, the reaction mark generator 14 of the conference server 1 generates icon data only for the reaction type that has the largest number of participants among the reaction types determined to be each reaction type. Other processes may be similar. Alternatively, the terminal devices 4a, 4b, and 5 may display an icon only for the reaction type that has the largest number of participants among the reaction types determined to be each reaction type.

[0088] FIG. 9(a) shows an example of the display of an icon 210 for the reaction type with the largest number of people. One icon 210 is displayed in FIG. 9(a). In FIG. 8, the number of people with positive reactions is greater than the number of people with negative reactions, so the reaction type with the largest number of people is positive. For this reason, in FIG. 9(a), only the icon 210 indicating the number of people with positive reactions is displayed. The reaction mark display unit 37 displays only the reaction mark for the reaction type with the largest number of people. For example, the reaction mark generation unit 14 assigns a predetermined diameter R to the largest number of people.

[0089] Alternatively, the icon 210 may be displayed in a size proportional to the number of participants. For example, the reaction mark generation unit 14 calculates a ratio P1 of the number of people of the most numerous reaction type to all participants. The diameter when the ratio is 100% is R, and the reaction mark generation unit 14 determines the diameter of the icon 210 to be R×P1. In this way, the reaction mark of the reaction type with the most numerous participants is displayed in a size according to the number of participants. Each participant can know how many people have the most numerous reaction from the size of the icon 210.

[0090] Also, the reaction mark display unit 37 may display only the icon of the reaction type with the fewest number of participants. FIG. 9(b) shows an example of displaying the icon 211 of the reaction type with the fewest number of participants. One icon 211 is displayed in FIG. 9(b). In FIG. 8, the number of people with negative reactions is smaller than the number of people with positive reactions, so the reaction type with the fewest number of participants is negative. For this reason, in FIG. 9(b), only the icon 211 indicating the number of people with negative reactions is displayed. That is, the reaction mark display unit 37 displays only the reaction mark of the reaction type with the fewest number of participants. In this case, each participant can grasp how many minority opinions there are from the size of the icon 211. In this case, the icon may be displayed in a size proportional to the number of participants. As with the icon 210 in FIG. 9(a), the reaction mark generation unit 14 calculates the ratio P2 of the number of people of the reaction type with the fewest number of participants to the total number of participants. The diameter when the ratio is 100% is R, and the reaction mark generation unit 14 determines the diameter of the icon 211 to be R×P2. Therefore, the reaction mark of the reaction type with the fewest number of participants is displayed in a size according to the number of participants.

[0091] Also, in the case of FIG. 9, the reaction mark generating unit 14 may predetermine the diameter of the icon in, for example, five stages, and determine the diameter of the icon according to the ratio of the number of people judged to have each reaction type to the total number of participants.

[0092] In addition, instead of the conference server 1 transmitting only icon data representing the reaction type with the largest or smallest number of participants to the terminal devices 4 and 5, the conference server 1 may transmit the number of people for each reaction type to the terminal devices 4 and 5, and the terminal devices 4 and 5 may generate icon data for only the reaction type with the largest or smallest number of participants.

[0093] <Major Effects> In the conference system of the present embodiment, an icon is displayed according to the number of people judged to be involved in each reaction type, so that each participant can grasp the overall reactions of the other participants. Each participant can statistically grasp the reactions of the other participants.

[0094] [Second embodiment] In this embodiment, a conference system 300 will be described in which the terminal devices 4 and 5 analyze reactions of participants operating the terminal devices 4 and 5 and display icons in a reaction statistics view.

[0095] In this embodiment, it is assumed that the hardware configuration diagram of FIG. 3 and the functional block diagram of FIG. 4 described in the above embodiment can be used.

[0096] 10 is a diagram showing a functional block diagram that operates in the reaction statistics view. The process performed by each function in the reaction statistics view will be described. Note that the communication unit 11 is omitted because it is always passed through in the communication between the conference server 1 and the terminal devices 4 and 5.

[0097] (1) The display mode selection unit 33 of the terminal device 4, 5 accepts the ON of the reaction visualization mode by the participant. The display mode selection unit 33 also accepts the selection of the reaction statistics view. The information that the reaction statistics view has been selected is transmitted to the conference server 1. (2) The video data acquisition unit 35 of the terminal device 4 , 5 acquires video data of the participants and transmits it to the reaction type determination unit 32 . (3) The reaction type determination unit 32 of the terminal devices 4 and 5 estimates the reaction type. The reaction type estimation method may be the same as that in the first embodiment. The reaction type determination unit 32 transmits the estimated reaction type (positive / negative) as an analysis result to the conference server 1 via the communication unit 31. (4) Next, the analysis data receiving unit 15 of the conference server 1 receives the analysis results via the communication unit 11 and transmits them to the analysis data compilation unit 16 . (5) The analysis data compilation unit 16 compiles the number of people for each reaction type and transmits the number of people for each reaction type to the reaction mark generation unit 14. (6) The reaction mark generating unit 14 of the conference server 1 generates icon data according to the number of people who have made each reaction, and transmits the icon data to the terminal devices 4 and 5 via the communication unit 11. (7) The reaction mark data receiving unit 34 of the terminal devices 4 and 5 receives the generated icon data via the communication unit 31 and transmits it to the reaction mark display unit 37. The reaction mark display unit 37 of the terminal devices 4 and 5 displays an icon based on the icon data.

[0098] In FIG. 10, the conference server 1 generates the icon data, but the conference server 1 may transmit the number of people for each reaction type to the terminal devices 4 and 5, and the terminal devices 4 and 5 may generate the icon data.

[0099] 11 is a sequence diagram for explaining a process in which the terminal devices 4 and 5 display icons according to reaction types of multiple participants in the reaction statistics view. Note that in FIG. 11, differences from FIG. 7 may be mainly explained.

[0100] S11: A participant turns on the reaction visualization mode and selects the reaction statistics view. The display mode selection unit 33 of the terminal device 4a accepts the participant's turning on of the reaction visualization mode. The display mode selection unit 33 also accepts the selection of the reaction statistics view.

[0101] S12: The video data acquisition unit 35 of the terminal device 4a acquires video data of the participants.

[0102] S13: The reaction type determination unit 32 of the terminal device 4a estimates the reaction, and creates an analysis result from the estimated reaction type (positive / negative).

[0103] S14: The communication unit 31 of the terminal device 4a transmits the analysis result to the conference server 1. The communication units 31 of the terminal devices 4b and 5 also transmit the analysis result to the conference server 1.

[0104] S15: The analytical data receiving unit 15 of the conference server 1 receives the analysis results via the communication unit 11, and the analytical data aggregation unit 16 of the conference server 1 aggregates the number of people for each reaction type and transmits the number of people for each reaction type to the reaction mark generation unit 14.

[0105] S16: The reaction mark generating unit 14 of the conference server 1 has been notified that this is a reaction statistics view, and so generates icon data according to the number of people for each reaction type.

[0106] S17 to S19: The communication unit 11 of the conference server 1 transmits the icon data to the terminal devices 4a, 4b, and 5.

[0107] S20 to S22: The reaction mark data receiving unit 34 of the terminal devices 4a, 4b, and 5 receives the generated icon data via the communication unit 31, and transmits it to the reaction mark display unit 37. The reaction mark display unit 37 of the terminal devices 4a, 4b, and 5 displays an icon based on the icon data.

[0108] In this way, the analysis of reaction types may be performed not only by the conference server 1 but also by each of the terminal devices 4 and 5. The icons displayed by the reaction mark display unit 37 may be the same as those in FIG.

[0109] <Only show the most popular reaction icons> In this embodiment as well, the conference server 1 may transmit to the terminal devices 4 and 5 only icon data representing the reaction type with the largest number of participants in the reaction statistics view.

[0110] In this case, in the process of "6." in Fig. 10 or the process of step S16 in Fig. 7, the reaction mark generation unit 14 of the conference server 1 generates icon data of only the reaction type with the largest number of participants among the numbers of participants showing each reaction. Other processes may be similar. Also, a display example of icon 210 of only the reaction type with the largest number of participants may be similar to Fig. 9(a), and a display example of icon 211 of only the reaction type with the smallest number of participants may be similar to Fig. 9(a).

[0111] <Major Effects> In addition to the effects of the first embodiment, the conference system of the present embodiment can reduce the load on the conference server 1 because the terminal devices 4 and 5 analyze the reaction types.

[0112] [Third embodiment] In this embodiment, when the reaction extraction view is selected, the icons displayed by the terminal devices 4 and 5 will be described. The reaction extraction view is a display mode in which a participant selects a reaction type that he or she wants to display, and only the icons of participants determined to have that reaction type are displayed.

[0113] 12 is a diagram showing a functional block diagram that operates in the reaction extraction view. The process performed by each function in the reaction extraction view will be described. Note that the communication unit 11 is omitted because it is always passed through in the communication between the conference server 1 and the terminal devices 4 and 5.

[0114] (1) The display mode selection unit 33 of the terminal devices 4 and 5 accepts ON of the reaction visualization mode by a participant. The display mode selection unit 33 also accepts the selection of a reaction extraction view. The selection of the reaction extraction view is transmitted to the conference server 1. The display reaction type selection unit 36 ​​also accepts the selection of a reaction type that the participant wants to display. The display reaction type selection unit 36 ​​holds the selected reaction type. (2) The video data acquisition unit 35 of the terminal devices 4 and 5 acquires the video data of the participants and transmits it to the conference server 1 via the communication unit 31 . (3) The video data receiving unit 12 of the conference server 1 receives the video data via the communication unit 11 and transmits it to the reaction type determining unit 13 . (4) The reaction type determination unit 13 of the conference server 1 estimates the reaction, and transmits the estimated reaction type (positive / negative) to the reaction mark generation unit 14 as an analysis result. (5) Next, the reaction mark generation unit 14 of the conference server 1 generates icon data for each participant based on the analysis result, and transmits the icon data to the terminal devices 4 and 5 via the communication unit 11. The icon data for each participant is icon data in which a color and a mark are set for each participant according to the reaction type. (6) The reaction mark data receiving unit 34 of the terminal device 4, 5 receives the generated icon data via the communication unit 31 and transmits it to the reaction mark display unit 37. (7) The reaction mark display unit 37 of the terminal device 4, 5 displays the icon of the reaction type stored in "1." based on the icon data. In other words, the reaction mark display unit 37 displays only the reaction marks of the participants whose reaction types are determined to be those specified by the participants.

[0115] 12, the conference server 1 analyzes the reaction types, but the reaction types may be analyzed by the terminal devices 4 and 5. Also, the conference server 1 may transmit the reaction types of each participant to the terminal devices 4 and 5, and the terminal devices 4 and 5 may generate icon data.

[0116] 13 is a sequence diagram for explaining a process in which the terminal devices 4 and 5 display icons corresponding to the reactions of a plurality of participants in a reaction extraction view. Note that the explanation of FIG. 13 will mainly focus on the differences from FIG.

[0117] S31: A participant turns on the reaction visualization mode and selects a reaction extraction view. The display mode selection unit 33 of the terminal device 4a accepts the participant's turning on of the reaction visualization mode. The display mode selection unit 33 also accepts the selection of the reaction extraction view.

[0118] S32, S33: The participant selects the reaction type he / she wishes to display. The display reaction type selection unit 36 ​​displays the reaction types and accepts the reaction type selected by the participant.

[0119] S34: The display reaction type selection unit 36 ​​holds the selected reaction type.

[0120] S35: The moving image data acquisition unit 35 of the terminal device 4a acquires moving image data of the participants.

[0121] S36: The communication unit 31 of the terminal device 4a transmits the moving image data to the conference server 1. The moving image data acquisition units 35 of the terminal devices 4b and 5 also acquire moving image data of the participants and transmit them to the conference server 1 via the communication units 31.

[0122] S37: The video data receiving unit 12 of the conference server 1 receives the video data via the communication unit 11 and transmits it to the reaction type determining unit 13. The reaction type determining unit 13 of the conference server 1 estimates the reaction type and transmits the estimated reaction type (positive / negative) to the reaction mark generating unit 14 as an analysis result.

[0123] S38: The reaction mark generating unit 14 of the conference server 1 is notified that it is a reaction extraction view, and generates icon data according to the reaction type for each participant. In the reaction extraction view, the icon data according to the reaction type means that the color differs depending on the reaction type, for example.

[0124] S39 to S41: The communication unit 11 of the conference server 1 transmits icon data to the terminal devices 4a, 4b, and 5.

[0125] S42 to S44: The reaction mark data receiving unit 34 of the terminal devices 4a, 4b, and 5 receives the generated icon data via the communication unit 31 and transmits it to the reaction mark display unit 37. The reaction mark display unit 37 of the terminal devices 4a, 4b, and 5 displays an icon based on the icon data of the selected reaction type.

[0126] Fig. 14 shows an example of icons displayed by the terminal device 4a. In Fig. 14, it is assumed that the reaction type selected by participant A is "positive" as an example. Therefore, icons 221 to 224 are displayed for participants who had positive reactions. Icon 221 represents the reaction of AA, icon 222 represents the reaction of BB, icon 223 represents the reaction of GG, and icon 224 represents the reaction of JJ. In other words, it is clear that AA, BB, GG, and JJ had positive reactions. Therefore, participant A can know the individual participants who had positive reactions.

[0127] In the state of Fig. 14, participant A may change the reaction type for displaying participant icons. The display reaction type selection unit 36 ​​accepts the changed reaction type, and the reaction mark display unit 37 can display icons for participants showing reactions of the selected reaction type. Since the terminal device 4a has already received icon data for all reaction types, the reaction mark display unit 37 can simply switch the reaction type for displaying icons.

[0128] 14, the size of the icons 221-224 may depend on the number of people for each reaction type. That is, the reaction mark generating unit 14 displays larger icons 221-224 as the number of people determined to be of each reaction type increases. For example, the icons 221-224 for positive reaction types are larger than the negative icons that are displayed in turn.

[0129] <Combination with the First and Second Embodiments> The terminal devices 4 and 5 may display the icons tallied in the reaction statistics view described in the first embodiment together with or in switchover with the icons 221 to 224 in Fig. 14. Also, in this embodiment, the analysis of reaction types may be performed by either the terminal devices 4 and 5 or the conference server 1.

[0130] <Major Effects> In the conference system of this embodiment, the terminal devices 4 and 5 display the reaction type of each participant, so that each participant can individually check the reaction types of the other participants.

[0131] <Other application examples> The above describes the best mode for carrying out the present invention using examples, but the present invention is not limited to these examples in any way, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.

[0132] For example, in the reaction statistics view mode, the analysis data compilation unit 16 may compile reaction types excluding the participants themselves. For example, when the number of participants is small, each participant may want to know which reaction type has the most people, excluding the participant himself / herself. Each participant may be able to individually set whether or not to include the participant in the compilation.

[0133] Furthermore, when multiple participants are captured on one terminal device 4, 5, the reaction type determination unit 13 may recognize people individually from the video data and determine the reaction type for each person.

[0134] In addition, in this embodiment, the organizer or the like can grasp the reactions of the participants when the participants turn off the cameras of the terminal devices 4 and 5. However, the cameras of the terminal devices 4 and 5 may remain on. When there are a large number of participants, it is difficult for the organizer to determine the reactions of all the participants from the video data, but in this embodiment, the overall reactions of the participants can be easily seen.

[0135] The conference server 1 may also determine the type of reaction of each participant at regular intervals and count the types of reactions shown by the participants during the conference. In other words, the conference server 1 can transmit to the terminal devices 4 and 5 how many times the participants A1 to An and B showed positive reactions and how many times they showed negative reactions.

[0136] In addition, the configuration example in Fig. 4 and the like is divided according to main functions in order to facilitate understanding of the processing by the conference system 300. The present invention is not limited by the manner in which the processing units are divided or the names of the processing units. The processing by the conference system 300 can be divided into more processing units according to the processing contents. Also, it can be divided so that one processing unit includes more processes.

[0137] Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, and a device such as an ASIC (Application Specific Integrated Circuit), a DSP (digital signal processor), an FPGA (field programmable gate array), or a conventional circuit module designed to execute each function described above.

[0138] Embodiments of the present invention provide significant improvements in computer power and functionality. These improvements allow users to utilize computers that provide more efficient and robust interaction with tables, which are ways of storing and presenting information in information processing devices. Furthermore, embodiments of the present invention provide a better user experience through the use of more efficient, powerful and robust user interfaces. Such user interfaces provide better interaction between humans and machines.

[0139] <Additional Notes> [Claim 1] A conference system in which a plurality of terminal devices and a conference server can communicate with each other via a network, a reaction type determination unit that determines a reaction type shown by each participant based on video data or audio data of the participants of the conference that are acquired by each of the plurality of terminal devices; a reaction mark display unit that displays reaction marks for one or more reaction types according to the number of people determined to be in each reaction type; A conference system having the above configuration. [Claim 2] 2. The conference system according to claim 1, wherein the reaction mark display unit displays reaction marks for each reaction type according to the number of participants determined to be each reaction type. [Claim 3] 3. The conference system according to claim 1, wherein the reaction mark display unit displays a larger reaction mark for each reaction type as the number of participants determined to be the reaction type increases. [Claim 4] 2. The conference system according to claim 1, wherein the reaction mark display unit displays only the reaction mark of the reaction type determined to be the reaction type that includes the largest number of participants. [Claim 5] 2. The conference system according to claim 1, wherein the reaction mark display unit displays only the reaction mark of the reaction type determined to be the reaction type associated with the fewest number of participants. [Claim 6] 5. The conference system according to claim 4, wherein the reaction mark of the reaction type with the largest number of participants is displayed in a size according to the number of participants. [Claim 7] 6. The conference system according to claim 5, wherein the reaction mark of the reaction type with the smallest number of participants is displayed in a size according to the number of participants. [Claim 8] The conference server includes a reaction mark generating unit that generates a reaction mark for each participant according to a reaction type of the participant; a communication unit that transmits the reaction marks of each participant generated by the reaction mark generation unit to the plurality of terminal devices, The plurality of terminal devices include 8. The conference system according to claim 1, wherein only the reaction marks of participants that are determined to be of the reaction types designated by participants operating the respective terminal devices are displayed. [Claim 9] 9. The conference system according to claim 1, wherein the reaction mark display unit displays the reaction mark based on a reaction type determined for a participant operating a terminal device. [Explanation of symbols]

[0140] 1. Conference Server 4,5 Terminal Equipment 201~203, 210, 211 Icon [Prior art documents] [Patent documents]

[0141] [Patent Document 1] JP 2022-129628 A

Claims

1. A conference system in which a plurality of terminal devices and a conference server can communicate with each other via a network, a reaction type determination unit that determines a reaction type shown by each participant based on video data or audio data of the participants of the conference that are acquired by each of the plurality of terminal devices; a reaction mark display unit that displays reaction marks for one or more reaction types according to the number of people determined to be in each reaction type; A conference system having the above configuration.

2. The conference system according to claim 1 , wherein the reaction mark display unit displays reaction marks for each reaction type according to the number of participants determined to be each reaction type.

3. 3. The conference system according to claim 1, wherein the reaction mark display unit displays a larger reaction mark for each reaction type as the number of participants determined to be the reaction type increases.

4. The conference system according to claim 1 , wherein the reaction mark display unit displays only the reaction mark of the reaction type determined to be the reaction type that includes the largest number of participants.

5. The conference system according to claim 1 , wherein the reaction mark display unit displays only the reaction mark of the reaction type determined to be the reaction type with the smallest number of participants.

6. The conference system according to claim 4 , wherein the reaction mark of the reaction type having the largest number of participants is displayed in a size according to the number of participants.

7. The conference system according to claim 5 , wherein the reaction mark of the reaction type having the smallest number of participants is displayed in a size according to the number of participants.

8. The conference server includes a reaction mark generating unit that generates a reaction mark for each participant according to a reaction type of the participant; a communication unit that transmits the reaction marks of each participant generated by the reaction mark generation unit to the plurality of terminal devices, The plurality of terminal devices The conference system according to claim 1 , wherein only the reaction marks of participants whose reaction types are determined to be those designated by participants who operate the terminal devices, respectively, are displayed.

9. The conference system according to claim 1 , wherein the reaction mark display unit displays the reaction mark based on a reaction type determined for a participant operating a terminal device.

10. The plurality of terminal devices include a display mode selection unit that accepts selection of a reaction statistics view or a reaction extraction view; When receiving a notification that the reaction statistics view has been selected, the reaction mark generation unit generates a larger reaction mark as the number of people counted for each reaction type increases, When receiving a notification that the reaction extraction view has been selected, the reaction mark generating unit generates the reaction mark for each participant according to the reaction type of each participant; The conference system according to claim 8 , wherein the plurality of terminal devices display only the reaction marks of participants whose reaction types are determined to be those designated by participants operating the plurality of terminal devices.

11. A reaction mark display method performed by a conference system in which a plurality of terminal devices and a conference server can communicate with each other via a network, comprising the steps of: A process of determining a reaction type shown by each participant based on video data or audio data of the participants of the conference acquired by each of the plurality of terminal devices; A process of displaying reaction marks for one or more reaction types according to the number of people determined to be involved in each reaction type; A reaction mark display method having the above structure.

12. A conference server capable of communicating with a plurality of terminal devices via a network, a reaction type determination unit that determines a reaction type shown by each participant based on video data or audio data of the participants of the conference that are acquired by each of the plurality of terminal devices; a communication unit that transmits a reaction mark according to the number of people determined to be the reaction type to one or more terminal devices; The terminal device displays, for one or more reaction types, reaction marks corresponding to the number of people determined to be associated with each reaction type.

Images