Conference control device, conference control method, and computer program

Abstract

To make it possible to easily confirm the status of other participants in a conference held via a network.SOLUTION: A conference control device according to one aspect of the present invention includes, a display information generation unit that performs image processing so as to approach a predetermined standard for an image of a user participating in a conference held over a network, generates display information including the user's image that has undergone image processing, and transmits the display information to a user terminal used by another user, and an audio control unit that controls audio in the conference.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a technology for assisting online conversations.

Background Art

[0002] Conventionally, systems for holding meetings via a network have been proposed. Recently, in particular, a system in which each participant uses a camera to share an image including their own face with others while holding a meeting has also become widespread (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, due to various factors such as the meeting situation and the usage environment of each participant's camera, it may be difficult to confirm the situation of other participants. In view of the above circumstances, an object of the present invention is to provide a technology that enables easier confirmation of the situation of other participants in a meeting conducted via a network.

Means for Solving the Problems

[0005] One aspect of the present invention is a conference control apparatus including: a display information generation unit that performs image processing on an image of a user participating in a conference conducted via a network so as to approach a predetermined standard, generates display information including the image of the user on which the image processing has been performed, and transmits the display information to a user terminal used by another user; and an audio control unit that controls audio in the conference.

[0006] One aspect of the present invention is the above-described conference control apparatus, wherein the predetermined criterion is a criterion related to the size of the image of the user's face.

[0007] One aspect of the present invention is the above-described conference control apparatus, wherein the predetermined criterion is a criterion related to the white balance or contrast of the image of the user's face.

[0008] One aspect of the present invention is the above-described conference control apparatus, further comprising an estimation unit that estimates the emotional state of the user based on the image of the user, and the display information generation unit performs image processing on the image according to the emotional state of the estimation result in the estimation unit.

[0009] One aspect of the present invention is a conference control method including: an image processing step of performing image processing on an image of a user participating in a conference conducted via a network so as to approach a predetermined criterion, and generating display information including the image of the user on which the image processing has been performed; and a voice control step of controlling the voice in the conference, and transmitting the display information to a user terminal used by another user.

[0010] One aspect of the present invention is a computer program for causing a computer to function as the above-described conference control apparatus.

Effects of the Invention

[0011] According to the present invention, in a conference conducted via a network, it becomes possible to more easily confirm the situations of other participants.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Embodiments for Carrying Out the Invention

[0013] Hereinafter, specific configuration examples of the present invention will be described with reference to the drawings. In the following description, a concept showing a virtual connection for two or more users to speak to others is called a conference room. Therefore, the conference room in the following description does not necessarily have to be named "conference room". For example, even if it is simply called a conversation or a session, as long as it is a virtual place where two or more users can speak to others, it corresponds to the conference room in the following description. For example, a virtual connection where a specific user (for example, a lecturer) speaks unidirectionally to a plurality of others (including a large number such as 50 or 100) during a seminar or presentation is also included in the conference room in the following description.

[0014] FIG. 1 is a schematic block diagram showing the system configuration of the conference system 100 of the present invention. The conference system 100 is a system for users operating user terminals 10 to hold a conference via a network 40. The conference system 100 includes a plurality of user terminals 10 and a conference control device 20. The plurality of user terminals 10 and the conference control device 20 are communicably connected via a network 30. The network 30 may be a network using wireless communication or a network using wired communication. The network 30 may be configured by combining a plurality of networks.

[0015] FIG. 2 is a schematic block diagram showing a specific example of the functional configuration of the user terminal 10. The user terminal 10 is configured using an information device such as a smartphone, a tablet, a personal computer, a portable game machine, a stationary game machine, or a dedicated device. The user terminal 10 includes a communication unit 11, an operation unit 12, a display unit 13, an audio input unit 14, an audio output unit 15, a storage unit 16, and a control unit 17.

[0016] The communication unit 11 is a communication device. The communication unit 11 may be configured as, for example, a network interface. The communication unit 11 performs data communication with other devices via the network 30 in accordance with the control of the control unit 17. The communication unit 11 may be a device that performs wireless communication or a device that performs wired communication.

[0017] The operation unit 12 is configured using existing input devices such as a keyboard, a pointing device (mouse, tablet, etc.), buttons, and a touch panel. The operation unit 12 is operated by the user when inputting the user's instructions into the user terminal 10. The operation unit 12 may be an interface for connecting the input device to the user terminal 10. In this case, the operation unit 12 inputs an input signal generated in response to the user's input in the input device into the user terminal 10. The operation unit 12 may be configured using a microphone and a voice recognition device. In this case, the operation unit 12 performs voice recognition on the words spoken by the user and inputs the string information of the recognition result into the user terminal 10. In this case, the operation unit 12 may be configured integrally with the voice input unit 14. The operation unit 12 may be configured in any way as long as it can input the user's instructions into the user terminal 10.

[0018] The display unit 13 is an image display device such as a liquid crystal display or an organic EL (Electro Luminescence) display. The display unit 13 displays the image data used when holding a meeting. The display unit 13 may be an interface for connecting the image display device to the user terminal 10. In this case, the display unit 13 generates a video signal for displaying the image data and outputs the video signal to the image display device connected to itself.

[0019] The voice input unit 14 is configured using a microphone. The voice input unit 14 may be configured as the microphone itself or as an interface for connecting the microphone as an external device to the user terminal 10. The microphone acquires the speech voice of the user holding the meeting. The voice input unit 14 outputs the voice data acquired by the microphone to the control unit 17.

[0020] The voice output unit 15 is configured using a voice output device such as a speaker, headphones, or earphones. The voice output unit 15 may be configured as the voice output device itself, or may be configured as an interface for connecting a voice output device to the user terminal 10 as an external device. It is desirable for the voice output device to output voice so that the user conducting the meeting can hear the voice. The voice output unit 15 outputs voice according to the voice signal output by the control unit 17.

[0021] The storage unit 16 is configured using a storage device such as a magnetic hard disk device or a semiconductor storage device. The storage unit 16 stores data used by the control unit 17. The storage unit 16 may function, for example, as a user information storage unit 161 and a speech information storage unit 162.

[0022] The user information storage unit 161 stores information about the user operating the user terminal 10 (hereinafter referred to as "user information"). The user information may include, for example, the user's identification information and attribute information. The attribute information may include information related to, for example, the user's age and gender.

[0023] The control unit 17 is composed of a processor such as a CPU (Central Processing Unit) and a memory. The control unit 17 functions as a display control unit 171, a conference control unit 172, and an audio control unit 173 when the processor executes a program. Note that all or part of each function of the control unit 17 may be realized using hardware such as an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The above program may be recorded on a computer-readable recording medium. A computer-readable recording medium is, for example, a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, a semiconductor storage device (e.g., SSD: Solid State Drive), or a storage device such as a hard disk or a semiconductor storage device built into a computer system. The above program may be transmitted via a telecommunication line.

[0024] The display control unit 171 receives display information from the conference control device 20 via the communication unit 11. The display control unit 171 generates an image signal based on the acquired display information and causes the display unit 13 to display it. The display information may be, for example, image data indicating the image itself to be displayed. In this case, the entity that generates the image data (image data generation unit) is the conference control device 20. The display information may be, for example, data indicating information necessary for generating the image to be displayed (e.g., information about the participating users). In this case, the display control unit 171 generates image data for display on the display unit 13 based on the display data. In this case, the entity that generates the image data (image data generation unit) is the display control unit 171.

[0025] The conference control unit 172 controls the conference virtually provided in the conference control device 20. For example, when the user operates the operation unit 12 to instruct logging in to the conference service provided by the conference control device 20, the conference control unit 172 performs the process for logging in. For example, when the user operates the operation unit 12 to instruct installing a new conference room, the conference control unit 172 performs the process for installing a new conference room. For example, when the user operates the operation unit 12 to instruct entering a conference room, the conference control unit 172 performs the process for entering the instructed conference room. The entry into the conference room may be performed in any form. For example, in a screen where one or more characters, buttons, or icons indicating the conference room are displayed, the entry into the conference room may be performed by operating any of the characters, buttons, or icons. The entry into the conference room may be performed by accessing the address assigned to each conference room (for example, a specific identification number, a ULR (Uniform Resource Locator), etc.).

[0026] The voice control unit 173 controls the voice communication with the users of other user terminals 10. When entering a conference room, voice transmission and reception are performed with other users who have entered the conference room. The voice control unit 173 transmits, for example, the voice data input from the voice input unit 14 to the conference control device 20 via the communication unit 11. When the voice control unit 173 receives voice data from the conference control device 20, it outputs the received voice data from the voice output unit 15.

[0027] Figure 3 is a schematic block diagram showing a specific example of the functional configuration of the conference control device 20. The conference control device 20 is configured using an information processing device such as a personal computer or a server device. The conference control device 20 includes a communication unit 21, a storage unit 22, and a control unit 23.

[0028] The communication unit 21 is a communication device. The communication unit 21 may be configured as, for example, a network interface. The communication unit 21 performs data communication with other devices via the network 30 in accordance with the control of the control unit 23. The communication unit 21 may be a device that performs wireless communication or a device that performs wired communication.

[0029] The storage unit 22 is configured using a storage device such as a magnetic hard disk device or a semiconductor storage device. The storage unit 22 stores data used by the control unit 23. The storage unit 22 may function as, for example, a user information storage unit 221, a conference room information storage unit 222, and an emotional state information storage unit 223. The user information storage unit 221 stores information (user information) about a plurality of users who operate the user terminal 10.

[0030] The conference room information storage unit 222 stores information about the conference room (hereinafter referred to as "conference room information"). A conference room is a virtual room installed for users to hold meetings in the conference system 100. The conference room information may include, for example, the ID of the conference room, information indicating the name set for the conference room, information indicating the reserved date and time when the conference room is installed, and information related to the attributes of the conference room. The information related to the attributes of the conference room may include, for example, the number of people who can enter the conference room and information indicating the users who can enter the conference room.

[0031] The emotional state information storage unit 223 stores information (hereinafter referred to as "emotional state information") indicating the emotional state of each user (each participant) estimated by the estimation unit 235 of the control unit 23. For example, the emotional state information storage unit 223 may store the emotional state information for each conference. In this case, the emotional state information may represent the estimation results of the emotional state of each participant who participated in the conference at predetermined timings (for example, every 1 second, every 5 seconds, every 1 minute, etc.).

[0032] The control unit 23 is composed of a processor such as a CPU and a memory. By the processor executing a program, the control unit 23 functions as a user control unit 231, a conference room control unit 232, a display information generation unit 233, an audio control unit 234, an estimation unit 235, and an evaluation unit 236. Note that all or part of each function of the control unit 23 may be realized using hardware such as an ASIC, a PLD, or an FPGA. The above program may be recorded on a computer-readable recording medium. A computer-readable recording medium is, for example, a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, a semiconductor storage device (e.g., SSD), or a storage device such as a hard disk or a semiconductor storage device built into a computer system. The above program may be transmitted via a telecommunication line.

[0033] The user control unit 231 performs control processing related to the user. For example, the user control unit 231 may perform processing for login (e.g., authentication processing) for the user terminal 10 that accesses the conference control device 20. The user control unit 231 may register the user information received from the user terminal 10 in the user information storage unit 221.

[0034] The conference room control unit 232 performs control processing related to the conference room. For example, when the conference room control unit 232 receives an instruction from the user terminal 10 to newly install a conference room, it may generate conference room information based on the received information and register it in the conference room information storage unit 222. Also, when it is time to install the conference room, the conference room control unit 232 virtually installs the conference room. The timing to install the conference room is, for example, at that time when an instruction is received from the user terminal 10 to newly install a conference room immediately, or when the date and time have arrived if a reservation for installing the conference room has been registered in advance. When a predetermined operation for the user to participate in the conference room is performed and a predetermined condition is satisfied, the conference room control unit 232 performs processing to allow the user to participate in the conference room. For example, the conference room control unit 232 registers that a new user has participated in the conference room by updating the conference room information storage unit 222.

[0035] The display information generation unit 233 generates information (display information) required for generating an image to be displayed on the user terminal 10. The display information may include, for example, information about the currently installed conference rooms and information about the users of each user terminal 10 that has entered each conference room. The display information may also include information indicating the result estimated by the estimation unit 235 about the emotional state of the users of each user terminal 10.

[0036] The display information may include data of the face images of each user. The display information generation unit 233 determines whether each face image of each user meets a predetermined appropriate criterion regarding the quality of the image. The predetermined criterion regarding the quality of the image may be, for example, a criterion regarding the size of the face portion (for example, a criterion indicating that the size of the face portion is within a predetermined appropriate range). The predetermined criterion regarding the quality of the image may be, for example, a criterion regarding the appropriate white balance (brightness) of the face portion (for example, a criterion indicating the brightness or edge strength of the face portion). The predetermined criterion regarding the quality of the image may be, for example, a criterion regarding the appropriate contrast of the face portion (for example, a criterion indicating the light and dark difference of the face portion). For an image for which the predetermined criterion regarding the quality of the image is not satisfied, the display information generation unit 233 executes predetermined image processing so that the image quality approaches an appropriate criterion. Then, the display information generation unit 233 generates display information using the image on which the image processing has been performed. The display information generation unit 233 transmits the generated display information to the user terminal 10.

[0037] The voice control unit 234 receives voice data from the user terminal 10. The voice control unit 234 generates voice data to be output to each user terminal 10 (hereinafter referred to as "conference voice data"), and transmits the conference voice data to each user terminal 10. The voice control unit 234 may transmit, for example, the conference voice data in the conference room where the user has entered to each user terminal 10.

[0038] The estimation unit 235 estimates the emotional state for each user participating in each conference room. For example, based on the face image of each user, the estimation unit 235 estimates what kind of emotional state it is. The estimated emotional state may be, for example, any one of joy, anger, sorrow, and pleasure, or any one of happiness, anger, surprise, sadness, and normal. The estimated emotional state may be, for example, the degree of interest in the content of the voice flowing in the conference room. The emotional state may be estimated by image recognition based on a predetermined feature amount indicating the expression in the face image. The estimation unit 235 records the estimation result in the emotional state information storage unit 223 along the time series. Such image recognition may be executed using a learned model obtained, for example, by performing machine learning using a teacher image or the like in advance.

[0039] The emotional state may be estimated based on, for example, the size of the face image. When estimated based on the size of the face image, for example, since it is estimated that the closer the face image is to the screen, the larger the face image is, it may be estimated that the user has more interest. When estimated based on the size of the face image, for example, since it is estimated that the farther the face image is from the screen, the smaller the face image is, it may be estimated that the user has less interest.

[0040] The emotional state may be estimated based on the movement of each user. For example, using a learned model obtained by performing machine learning using a teacher image or the like in advance, the nodding motion of each user is detected, and the emotional state may be estimated based on the number of nods per unit time, the size, etc. of the nods. For example, it may be estimated that the more nods per unit time, the more interest the user has. For example, it may be estimated that the larger the size of the nod, the more interest the user has. It may be estimated based on the number of nods and the size per unit time.

[0041] Although several specific examples of the emotional state estimation process have been described, the emotional state may be estimated by a process other than those described above.

[0042] The evaluation unit 236 evaluates the content of the meeting. For example, the evaluation unit 236 may evaluate the content of the meeting based on the emotional state information stored in the emotional state information storage unit 223. For example, the evaluation unit 236 may perform an evaluation indicating that the meeting was good to the extent that the statistical value of the values indicating the degree of interest of each user shows a high value as a value indicating that the interest is shown. The evaluation unit 236 may perform an evaluation indicating that the meeting was good to the extent that there are more people showing joy or surprise and fewer people showing anger or sadness.

[0043] The evaluation unit 236 may evaluate the attendance attitude of each individual participant based on the emotional state information stored in the emotional state information storage unit 223. For example, an evaluation indicating good attendance attitude may be performed to the extent that the time when the emotional information is acquired is relatively long. The fact that the emotional information is acquired indicates that the face was positioned in front of the monitor facing the monitor, and it can be estimated that the person was properly present at the meeting.

[0044] FIG. 4 is a diagram showing a specific example of an image displayed on a user terminal in the prior art. In each user area 91, an image of the user who has entered the conference room is displayed. Each user image displayed in the user area 91 is a moving image taken by a camera connected to the user terminal of each user. The face image of the user in the upper left is displayed with an appropriate size and an appropriate white balance.

[0045] The face image of the user in the upper right is too large. Therefore, a part of the face image protrudes from the user area 91. In addition, the face image of the user in the upper right is taken in an inappropriate white balance state due to the fact that the photographed environment is too bright or the camera parameter settings are inappropriate. Therefore, the face image has become too white (in a so-called 'white out' state). As a result, the face image of the user in the upper right is in a difficult-to-see state.

[0046] The face image of the user in the lower left corner is slightly smaller in size. Also, the face image of the user in the lower left corner was taken in an inappropriate white balance condition due to reasons such as the shooting environment being too dark or the camera parameter settings being inappropriate. Therefore, the face image has become too dark (in a so-called "blackout" state). As a result, the face image of the user in the lower left corner is in a difficult-to-see state.

[0047] The face image of the user in the lower right corner is considerably smaller in size. As a result, the face image of the user in the lower right corner is in a difficult-to-see state.

[0048] FIG. 5 is a diagram showing a specific example of an image displayed on the display unit 13 of the user terminal 10. FIG. 5 shows a specific example of an image displayed on the display unit 13 of the user terminal 10 of the present invention in the same situation as FIG. 4. In FIG. 5, an image of the interior of the conference room is displayed on the display unit 13. The image of the interior of the conference room is an image displayed while the user is entering the conference room. In the image of the interior of the conference room, images of some or all of the users entering the conference room are displayed. The displayed screen is formed by one or a plurality of user areas 51. In each user area 51, an image of the user entering is displayed. Each user's image displayed in the user area 51 may be a moving image taken by a camera connected to the user terminal 10 of each user, or may be a still image (for example, an icon image). The image displayed in each user area 51 is generated by the display information generation unit 233.

[0049] The face image of the user in the upper left corner is an image taken with appropriate size, appropriate white balance, and contrast. That is, in the image data of the face image of the user in the upper left corner, the size of the user's face image is within a predetermined range. Also, in the image data of the face image of the user in the upper left corner, the image quality of the user's face image satisfies predetermined conditions regarding white balance and contrast. Therefore, image processing may not be executed for the face image of the user in the upper left corner.

[0050] When the face image of the upper-right user was captured and received by the conference control device 20, the size of the face was too large. Therefore, the display information generation unit 233 reduces the face image of the upper-right user so that the size of the face becomes a predetermined size. Also, the face image of the upper-right user had white spots. Therefore, the display information generation unit 233 executes image processing such as reducing the brightness of the face image of the upper-right user to reduce the white spots. The data of the face image after such image processing is transmitted to each user terminal 10 as the data of the face image of the upper-right user.

[0051] When the face image of the lower-left user was captured and received by the conference control device 20, the size was slightly smaller. Therefore, the display information generation unit 233 enlarges the face image of the lower-left user so that the size of the face becomes a predetermined size. Also, the face image of the lower-left user had black crush. Therefore, the display information generation unit 233 executes image processing such as increasing the brightness of the face image of the lower-left user to reduce the black crush. The data of the face image after such image processing is transmitted to each user terminal 10 as the data of the face image of the lower-left user.

[0052] When the face image of the lower-right user was captured and received by the conference control device 20, the size was considerably smaller. Therefore, the display information generation unit 233 enlarges the face image of the lower-right user so that the size of the face becomes a predetermined size. The data of the face image after such image processing is transmitted to each user terminal 10 as the data of the face image of the lower-right user.

[0053] In this way, the display information generation unit 233 performs image processing on the images of each user so that the size of the face displayed in each user area 51 becomes a predetermined size (substantially the same size). Also, the display information generation unit 233 performs image processing on the images of each user so that the brightness of the face displayed in each user area 51 becomes a predetermined brightness (for example, the average value of the brightness within the face area is substantially the same).

[0054] FIG. 6 is a diagram showing a specific example of an image displayed on the display unit 13 of the user terminal 10. In the image shown in FIG. 6, image processing based on the result estimated by the estimation unit 235 is performed on the face images of the respective user areas 51 shown in FIG. 5. Specifically, image processing according to the emotional state of each user estimated by the estimation unit 235 is performed. For example, a broken-line circle surrounding the face image is superimposed on the face image of the user estimated to be happy (the upper-left face image). For example, a dashed-line circle surrounding the face image is superimposed on the face image of the user estimated to be angry (the upper-right face image). For example, a dotted-line circle surrounding the face image is superimposed on the face image of the user estimated to be sad (the lower-left face image). For example, a two-dashed-line circle surrounding the face image is superimposed on the face image of the user estimated to be surprised (the lower-right face image). Note that the display mode of each of the above-described circles is merely a specific example. Thus, the line types of the respective circles may be different, or the line thicknesses and colors may be different. Further, the geometric figure surrounding each face image does not need to be limited to a circular shape. For example, it may be a rectangle, an ellipse, or a polygon, or they may be different shapes. Further, the image superimposed on each face image does not need to be a geometric figure such as a circle as described above, and pictograms or characters indicating respective emotional states may be superimposed.

[0055] In addition, in the image processing according to the emotional state, image processing for changing the background color may be performed, or image processing for changing the overall color or pattern of the user area 51 may be performed. At this time, the image processing may be performed so that the face image of the user is not displayed in a visible state. For example, the user area 51 may be filled with a color or pattern corresponding to each emotional state. FIG. 7 is a diagram showing a specific example of an image displayed on the display unit 13 of the user terminal 10. In FIG. 7, the user areas 51 of 48 users are displayed. Each user area 51 is classified into one of the emotions of happiness, anger, surprise, sadness, and normal based on the user's image, and is filled with a pattern corresponding to the classified emotional state.

[0056] When using a normal display device, even if as many as 48 face images are displayed, the size of each user area 51 will be reduced, making it difficult to read the expressions of each user. In such a situation, when each user area 51 is shown in colors and patterns according to the emotional state in this way, users can more easily recognize the emotional states of each user. Also, compared with the case where face images are communicated, communicating an image filled with colors and patterns can reduce the amount of data required for communication. Therefore, it becomes possible to more stably realize a meeting via a network. Also, by reducing the amount of data, even when a device with low drawing ability such as an inexpensive smartphone is used as the user terminal 10, it is possible to reduce the occurrence of problems such as processing glitches.

[0057] FIG. 8 is a diagram showing a specific example of an image displayed on the display unit 13 of the user terminal 10. In FIG. 8, the number of emotional states of users participating in the same meeting room is displayed as a bar graph. The display information generation unit 233 may generate display information so as to display information indicating the number of users classified into each emotional state as shown in FIG. 8. Such display information may be generated so as to be displayed together with the images shown in FIGS. 5 to 7.

[0058] FIG. 9 is a diagram showing a specific example of an image displayed on the display unit 13 of the user terminal 10. In FIG. 9, the number of emotional states of users participating in the same meeting room is displayed as a line graph showing a time-series change. The display information generation unit 233 may generate display information so as to display information indicating the time-series change in the number of users classified into each emotional state as shown in FIG. 9. Such display information may be generated so as to be displayed together with the images shown in FIGS. 5 to 7. In the display information of FIG. 9, information indicating the progress of the meeting (for example, greeting, Part 1, Part 2, and Q&A) may be shown on the horizontal axis indicating the change in time. In the display information of FIG. 9, information indicating a predetermined event (for example, new product release) may be shown at the position corresponding to the time when the event occurred on the horizontal axis indicating the change in time.

[0059] Figure 10 is a sequence chart showing a specific example of the processing flow of the conference system 100. First, the user terminal 10 captures an image of the user (attendee) at a predetermined timing and transmits it to the conference control device 20 (step S101).

[0060] When the conference control device 20 receives the images of each user, it executes image correction processing (step S102). In the image correction processing, the conference control device 20 executes image processing such as making the face sizes substantially the same and making the face brightness substantially the same. The conference control device 20 executes processing for estimating the emotional state based on the user's image (step S103). The conference control device 20 generates display information according to the estimated emotional state (step S104). Then, the conference control device 20 transmits the generated display information to the user terminal 10 (step S105). The user terminal 10 displays an image or characters on the display unit 13 based on the received display information (step S106).

[0061] According to the conference system 100 configured as described above, in a conference conducted via a network, it is possible to more easily confirm the status of other participants. More specifically, it is as follows. In the conference system 100, image processing is performed so that the face images of each user approach an appropriate standard. For example, image processing is performed so that the face size, white balance, and contrast approach appropriate standards. Therefore, it is possible to more easily confirm the facial expressions and states of each participant. Also, in the conference system 100, the emotional state of each user is estimated, and image processing according to the estimation result is performed. By such processing as well, it is possible to more easily confirm the status of each participant.

[0062] As described above, the embodiments of the present invention have been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and designs and the like within the scope not departing from the gist of the present invention are also included.

Description of Reference Numerals

[0063] 100… Conference system, 10… User terminal, 20… Conference control device, 11… Communication unit, 12… Operation unit, 13… Display unit, 14… Voice input unit, 15… Voice output unit, 16… Memory unit, 161… User information memory unit, 17… Control unit, 171… Display control unit, 172… Conference control unit, 173… Voice control unit, 21… Communication unit, 22… Memory unit, 221… User information memory unit, 222… Conference room information memory unit, 223… Emotional state information memory unit, 23… Control unit, 231… User control unit, 232… Conference room control unit, 233… Display information generation unit, 234… Voice control unit, 235… Estimation unit, 236… Evaluation unit, 51… User area

Claims

1. A display information generation unit that performs image processing on an image of a user participating in a meeting conducted via a network so as to approach a predetermined standard, generates display information including the image of the user on which the image processing has been performed, and transmits the display information to a user terminal used by another user; An audio control unit that controls the audio in the meeting; An estimation unit that estimates the emotional state of the user based on the image of the user; Comprising: The display information generation unit is a conference control device that generates display information including an image showing a time-series change in the number of users in each emotional state for the users participating in the meeting.

2. The conference control device according to claim 1, wherein the predetermined standard is a standard related to the size of the image of the user's face.

3. The conference control device according to claim 1, wherein the predetermined standard is a standard related to the white balance or contrast of the image of the user's face.

4. The display information generation unit performs image processing on the image according to the emotional state of the estimation result in the estimation unit. The conference control device according to any one of claims 1 to 3.

5. A display information generation step of performing image processing on an image of a user participating in a meeting conducted via a network so as to approach a predetermined standard, generating display information including the image of the user on which the image processing has been performed, and transmitting the display information to a user terminal used by another user; An audio control step of controlling the audio in the meeting; An estimation step of estimating the emotional state of the user based on the image of the user; Having: In the display information generation step, for the users participating in the meeting, a conference control method for generating display information including an image showing a time-series change in the number of users in each emotional state.

6. A computer program for causing a computer to function as the conference control device according to any one of claims 1 to 4.

Images